Camera platform control apparatus

Abstract

To control the photographing direction of a camera based on the line-of-sight direction of a photographer.SOLUTION: A camera platform control device includes a face direction detection unit that detects a front direction of a face of a photographer, a line-of-sight detection unit that detects a line-of-sight direction of the photographer, and a calculation unit that calculates driving information for a driving unit of an automatic camera platform that moves a photographing direction of a photographing unit that photographs a subject, on the basis of the front direction of the face and the line-of-sight direction.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a camera head control device. [Background technology]

[0002] A helmet with a display equipped with a camera that photographs the eyeball is known (see Patent Document 1). In conventional technology, the gaze direction is detected from the direction of the pupil photographed by the camera, and an image processing device is used to create an image that appears as if the gaze direction had been photographed. Therefore, no consideration has been given to controlling the photographing direction so that the gaze direction is actually photographed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 5-208393 Summary of the Invention

[0004] A camera head control device according to a first aspect of the present invention comprises a face direction detection unit that detects the forward direction of the photographer's face, a gaze detection unit that detects the gaze direction of the photographer, and a calculation unit that calculates drive information for a drive unit of an automatic camera head that moves the shooting direction of a shooting unit that photographs a subject based on the forward direction of the face and the gaze direction. [Brief explanation of the drawings]

[0005] [Figure 1] 10A and 10B are schematic diagrams illustrating a scene in which the camera head control device controls the shooting direction of a camera for shooting. [Figure 2] 2 is a schematic diagram illustrating a situation in which the shooting direction of the camera in FIG. 1 is controlled. FIG. [Figure 3] FIG. 1 is a diagram illustrating an example of an outline of a photography system including an automatic camera platform. [Figure 4] FIG. 1 is a diagram illustrating a configuration of a camera for photographing according to an embodiment. [Figure 5]FIG. 2 is a diagram illustrating a configuration of a camera for gaze detection according to an embodiment. [Figure 6] FIG. 2 is a schematic diagram illustrating an example of a camera for detecting the photographer's eyeballs and line of sight. [Figure 7] FIG. 2 is a schematic diagram illustrating the shooting distance of a camera for shooting. [Figure 8] 10 is a flowchart illustrating the flow of processing of a program. [Figure 9] 10 is a flowchart illustrating the flow of processing of a program. [Figure 10] FIG. 1 is a schematic diagram illustrating an eyeball fitted with a contact lens. [Figure 11] FIG. 10 is a diagram illustrating program provision. DETAILED DESCRIPTION OF THE INVENTION

[0006] Hereinafter, an embodiment of the invention will be described with reference to the drawings. 1 is a schematic diagram illustrating an example of a scene in which the camera head control device according to an embodiment controls the shooting direction of a camera 1 for shooting. An automatic camera head is attached to a tripod or the like, and a scene of a sprint (e.g., 100 m) in track and field is shot with camera 1 attached to the automatic camera head. Participating athletes run from start line 11 toward finish line 12 on a sprint track 10 used for sprinting. Camera 1 is installed beside sprint track 10 and captures the sprinting athletes from the side.

[0007] A gaze detection camera 6, which is fixed to a tripod or the like separate from camera 1, captures an image of the pupils of photographer 5 and detects the gaze direction of photographer 5 based on the captured image. A head sensor 40 attached to the head of photographer 5 detects the direction in which the face of photographer 5 is facing. In this embodiment, the direction in which the face of photographer 5 is facing is defined as the direction in which the face of photographer 5 is facing.

[0008] The gaze detection camera 6 of the embodiment functions as a camera head control device together with the head sensor 40. The camera 6 functioning as a camera head control device controls the shooting direction of the camera 1 (in other words, controls the camera head to rotate the base of the automatic camera head to which the camera 1 is attached) based on the gaze direction of the photographer 5 and the front direction of the photographer's 5 face. The head sensor 40 may be any sensor that changes its orientation with the movement of the head of the photographer 5 and can detect the front direction of the face of the photographer 5, and its shape is not limited to the shape shown in the figure, and it may be a helmet type, a glasses type, or an earmuff type.

[0009] FIG. 2 is a schematic diagram illustrating a situation in which the shooting direction of camera 1 in FIG. 1 is controlled. In FIG. 2, the cameraman 5 and camera 6 are not shown. When the athlete starts, cameraman 5 follows the athlete with his eyes as he runs from right to left in FIG. 2. Cameraman 5 may also change the direction of his face while following the athlete with his eyes. When cameraman 5 follows, for example, athlete R1 with his eyes, camera 6, functioning as a camera head control device, controls the shooting direction of camera 1 so that athlete R1 is kept captured approximately in the center of the shooting screen. When cameraman 5 is set to take continuous shots or to shoot video, simply by following athlete R1 with his eyes, the shooting direction of camera 1 capturing athlete R1 is automatically controlled from the state indicated by dashed line 1a to the state indicated by dashed line 1b. In addition, camera 6 in this embodiment also detects the distance from camera 1 to athlete R1, with respect to athlete R1 being followed by cameraman 5. An overview of such an imaging system will be described with reference to FIG.

[0010] 3 is a diagram illustrating an example of an outline of a photography system including the automatic camera platform. The photography system includes a camera 1 for photography, an automatic camera platform 50 for the camera 1, a head sensor 40 that functions as a camera platform control device, and a camera 6 for gaze detection. The camera 1 for photography includes at least a wireless communication unit 270 and a lens driving unit 330. The camera 6 for detecting the line of sight includes at least a line of sight detection unit 241, a photographing distance calculation unit 244, a camera head driving amount calculation unit 243, and a wireless communication unit 270. The head sensor 40 includes a face direction detection unit 42 , a posture / shake detection unit 43 , and a wireless communication unit 41 . The automatic camera platform 50 includes a wireless communication unit 51, a control unit 52, a drive unit / detection unit 53, and an operation member .

[0011] <Camera for shooting> Camera 1 for photography will be described in detail with reference to Figure 4. Figure 4 is a diagram illustrating the configuration of camera 1 according to an embodiment. Camera 1 is attached to the base of automatic camera platform 50, which is fixed to a tripod (not shown). Camera 1 may be configured with a detachable photographic lens (interchangeable lens) attached to the camera body, or may be configured with the camera body and photographic lens integrated together. The shooting direction of the camera 1 is controlled by the automatic camera platform 50. In the example of Fig. 4, the shooting direction of the camera 1 is the left direction along the optical axis O of the shooting lens 360.

[0012] Camera 1 includes control unit 230, imaging unit 260, wireless communication unit 270, operation member 280, photographing lens 360, lens driving unit 370, aperture driving unit 380, and lens control unit 330. Control unit 230 is connected to imaging unit 260, wireless communication unit 270, operation member 280, and lens control unit 330.

[0013] The imaging unit 260 includes an imaging element 261 and a signal processing unit 262. Light from a subject that passes through the photographing lens 360 is guided to the imaging element 261 of the imaging unit 260. The imaging element 261 is, for example, a solid-state imaging element such as a CMOS image sensor. Based on a control signal from the control unit 230, the imaging element 261 captures an image of the subject formed on an imaging surface 261S by the photographing lens 360 and outputs a signal. The imaging unit 260 is capable of capturing moving images and still images. In addition to recording moving images, the moving image capturing function also includes continuous shooting and capturing live view images for continuously displaying images for monitoring on a display unit (not shown).

[0014] The image sensor 261 has a photoelectric conversion unit for image generation and a photoelectric conversion unit for focus detection. Image capturing pixel signals based on charges generated in the image generating photoelectric conversion unit are used to generate image data in the signal processing unit 262. The signal processing unit 262 performs predetermined image processing on the image capturing pixel signals to generate image data. The image capturing pixel signals are further used by the control unit 230 for exposure calculation. A focus detection pixel signal based on the electric charge generated in the focus detection photoelectric conversion unit is sent to the control unit 230 via the signal processing unit 262.

[0015] The control unit 230 is composed of a microcomputer and its peripheral circuits, and executes control programs and the like to control each unit of the camera 1. The control unit 230 includes a photometry unit 231, an exposure control unit 232, a lens drive instruction unit 233, an information storage unit 234, and a recording control unit 235.

[0016] The photometry unit 231 performs photometry based on the imaging pixel signals. For example, it detects brightness information (Bv value) of the subject based on the imaging pixel signals output from the imaging unit 260. The exposure control unit 232 determines the aperture value (Av value), shutter speed (Tv value), and sensitivity (Sv value) based on the Bv value and program chart information stored in the information storage unit 234, for example.

[0017] The lens drive instruction unit 233 calculates the defocus amount (the amount of deviation between the imaging position of the photographing lens 360 and the imaging surface 261S of the image sensor 261) by, for example, performing focus detection processing using a phase difference detection method using focus detection pixel signals output from the imaging unit 260. The imaging position is the position where an image is formed when the focusing lens 361 of the photographing lens 360 is in its current position. The lens drive instruction unit 233 calculates the drive amount Δd for moving the focusing lens 361 of the photographing lens 360 from its current position to the in-focus position based on the calculated defocus amount, and outputs a drive instruction to the lens control unit 330.

[0018] The lens drive instruction unit 233 is also configured to be able to acquire shooting distance information indicating the shooting distance L from an external device, calculate a drive amount Δd for moving the position of the focusing lens 361 of the shooting lens 360 in the direction of the optical axis O from the current position to a focused position, and output a drive instruction to the lens control unit 330. The shooting distance L corresponds to the distance from the camera 1 (more precisely, the imaging surface 261S) to the main subject. For example, as information on the optical characteristics of the photographing lens 360, the position of the focusing lens 361 in the direction of the optical axis O corresponding to the focal length and the shooting distance L is stored in advance in the information storage unit 241. When the lens drive instruction unit 233 acquires shooting distance information from an external device, it refers to the information storage unit 241 to determine the position of the focusing lens 361 in the direction of the optical axis O (focus position) that will bring the focusing lens 361 into focus on a subject that is away by the acquired shooting distance L, then calculates a drive amount Δd for moving the focusing lens 361 from its current position to the focus position, and outputs a drive instruction to the lens control unit 330.

[0019] The information storage unit 234 is a memory in which the recording and reading of data and information is controlled by the control unit 230. The information storage unit 241 stores, for example, information indicating the optical characteristics of the photographing lens 360 and information on a program diagram for exposure control.

[0020] The recording control unit 235 controls the recording and reading of data and information onto a recording medium 500 such as a memory card. The recording control unit 235 converts image data output from the imaging unit 260 into a predetermined file format and records the data onto the recording medium 500 such as a memory card. The recording control unit 235 can also read image data recorded on the recording medium 500 and send it to an external device via the wireless communication unit 270. The recording medium 500 is detachable from a slot (not shown) provided in the camera 1 .

[0021] Operation members 280, including a release button, operation switches, etc., are provided on the exterior surface of camera 1. Operation members 280 send operation signals in response to user operations to control unit 230. By operating operation members 280, the user can switch the operation mode of camera 1 and set shooting conditions (exposure settings, etc.).

[0022] The wireless communication unit 270 transmits and receives data and information to and from external devices that have wireless communication functions. The wireless communication unit 270 is controlled by the control unit 230 to transmit and receive data and information. In this embodiment, communication is performed with a camera 6 for detecting the line of sight as an external device, and information indicating the shooting distance L from the camera 1 to the main subject (for example, player R1) is obtained from the camera 6.

[0023] The photographing lens 360 includes a focusing lens 361, an aperture member 362, a lens driver 370, and an aperture driver 380. The lens controller 330 is connected to the lens driver 370, the aperture driver 380, and the controller 230.

[0024] The lens control unit 330 is composed of a microcomputer and its peripheral circuits, etc. The lens control unit 330 executes a control program stored therein and controls each part of the photographic lens 360. The lens control unit 330 also periodically communicates with the control unit 230.

[0025] Through communication between the lens control unit 330 and the control unit 230, instructions to drive the photographing optical system 360 and the like are sent from the control unit 230 to the lens control unit 330. In addition, information such as the driving state of the photographing lens 360 (for example, the position of the focusing lens 361 and the aperture diameter of the diaphragm member 362) is sent from the lens control unit 330 to the control unit 230. Additionally, information indicating the optical characteristics of the photographing lens 360 can also be transmitted from the lens control unit 330 to the control unit 230.

[0026] The photographing lens 360 includes lenses such as a focusing lens 361 and an aperture member 362, and guides subject light to the imaging surface 261S of the imaging section 260. The focusing lens 361 is configured to be movable in the direction of the optical axis O by the lens driving unit 370. By moving the focusing lens 361, the position of the subject image formed by the photographing optical system 360 in the direction of the optical axis O is changed. This allows for a focusing operation. The position of the focusing lens 361 in the direction of the optical axis O can be detected by the lens driving unit 370.

[0027] The diaphragm member 362 adjusts the amount of light incident on the imaging unit 260. The diaphragm member 362 is configured so that the aperture diameter (aperture value) can be changed by driving the diaphragm blades using the diaphragm driving unit 380 or manually. The aperture diameter of the diaphragm member 362 is configured so that it can be detected by the diaphragm driving unit 380.

[0028] Driving instructions such as the direction, amount, and speed of movement of the focusing lens 361 are issued by the control unit 230. Note that instructions may also be issued from the lens control unit 330 based on information from the control unit 230.

[0029] <Head sensor> As described above, the head sensor 40 is worn on the head of the photographer 5. The head sensor 40 also includes a wireless communication unit 41, a face direction detection unit 42, and a posture / shake detection unit 43. The wireless communication unit 41 is composed of a microcomputer (not shown), its peripheral circuits, a communication circuit, etc. The wireless communication unit 41 transmits and receives data and information to and from an external device equipped with a wireless communication function. In this embodiment, the wireless communication unit 41 communicates with a gaze detection camera 6, which is an external device, and transmits information indicating the direction in which the face of the photographer 5 is facing to the camera 6. The microcomputer of the wireless communication unit 41 executes a control program stored therein and controls each part of the head sensor 40 .

[0030] The face direction detection unit 42 detects geomagnetism using, for example, a three-axis geomagnetic sensor and calculates an azimuth angle based on the detected information. The posture / shake detection unit 43 detects gravity using, for example, a three-axis acceleration sensor and calculates the tilt angle of gravity based on the detected information. The calculated tilt angle of gravity corresponds to the tilt of the head of the photographer 5, in other words, the tilt of the head sensor 40 worn on the head. In addition, the posture / shake detection unit 43 repeatedly detects tilt information of the head sensor 40 at predetermined time intervals and detects the shake of the head of the photographer 5, in other words, the shake information of the head sensor 40 worn on the head.

[0031] In this embodiment, face direction detection unit 42 estimates the attitude of the geomagnetic sensor based on tilt information and shake information of the head of photographer 5 acquired by attitude / shake detection unit 43, and corrects the shake, and detects the azimuth as the direction in which the face of photographer 5 is facing based on the estimated attitude and the detection information of the geomagnetic sensor. The detected direction has been corrected for the influence of head shake.

[0032] <Automatic head> As described above, the automatic camera head 50 is attached to a tripod or the like for the camera 1. The automatic camera head 50 also includes a wireless communication unit 51, a control unit 52, a drive unit / detection unit 53, and an operation member 54. The wireless communication unit 51 is composed of a communication circuit and the like, and transmits and receives data and information to and from an external device equipped with a wireless communication function. In this embodiment, it communicates with a camera 6 used for line of sight detection, which is an external device, and transmits information indicating the current base position of the base to which the camera 1 is fixed to the camera 6, and receives drive information for driving and rotating the base from the camera 6. The drive information includes a drive amount ΔQ and a drive direction for moving the automatic camera head 50 from the current base position to a target base position.

[0033] The control unit 52 is made up of a microcomputer (not shown) and its peripheral circuits, etc. The microcomputer of the control unit 52 executes a control program stored therein and controls each part of the automatic camera platform 50.

[0034] The driving unit / detecting unit 53 includes a driving mechanism that uses the rotation of a motor to rotate the base to which the camera 1 is fixed, and a detecting mechanism that detects the rotation angle (amount of rotation) of the base as viewed from a reference position as the base position. When the driving information for rotating the base is received by the wireless communication unit 51, the driving unit / detecting unit 53 drives the base in the specified direction by the specified driving amount ΔQ.

[0035] The operation member 54 includes known screws and handles. The screws include camera screws for fixing the camera 1 to the base, and the handles include pan handles and tilt handles. When preparing to shoot, photographer 5 first aligns the reference position of automatic camera head 50 with a predetermined direction (e.g., north) and then secures camera 1 to the base of automatic camera head 50. Then, photographer 5 adjusts the pan and tilt of automatic camera head 50 by operating the pan handle and tilt handle. In the scene illustrated in FIGS. 1 and 2, the sprinting runner moves horizontally, so the shooting direction of camera 1 during shooting only needs to move horizontally (left and right) from facing start line 11 of sprint track 10 to facing finish line 12. Therefore, automatic camera head 50 is installed and adjusted so that camera 1 is panned horizontally when drive unit / detector unit 53 rotates the base of camera 1.

[0036] <Gaze detection camera> The camera 6 for line of sight detection will be described in detail with reference to Fig. 5. Fig. 5 is a diagram illustrating the configuration of the camera 6 according to an embodiment. The camera 6 is fixed to a tripod (not shown), and its shooting direction is directed toward the eyes of the photographer 5 who is gazing in the direction of the short-distance track 10. The tripod for the camera 6 does not need to have an automatic pan head. 5, components similar to those in camera 1 are assigned the same reference numerals as in FIG. 4, and descriptions thereof will be omitted. As in the case of camera 1, camera 6 may be configured with a detachable photographing lens (interchangeable lens) attached to a camera body, or may be configured with the camera body and photographing lens integrated. As described above, the photographing direction of camera 6 is fixed so as to face the eyes of photographer 5. In the example of FIG. 5, the photographing direction of camera 6 is the left direction along the optical axis P of photographing lens 360.

[0037] Camera 6 includes control unit 240, imaging unit 260, wireless communication unit 270, operation member 280, photographing lens 360, lens driving unit 370, aperture driving unit 380, and lens control unit 330. Control unit 240 is connected to imaging unit 260, wireless communication unit 270, operation member 280, and lens control unit 330.

[0038] The control unit 240 is composed of a microcomputer and its peripheral circuits, etc., and executes control programs, etc. to control each unit of the camera 6. In addition to the configuration of the control unit 230 of the camera 1, the control unit 240 includes, for example, a line of sight detection unit 241, an eye focal length detection unit 242, a pan head drive amount calculation unit 243, a shooting distance calculation unit 244, a recording control unit 245, a focus detection unit 246, and a lens drive amount calculation unit 247.

[0039] The gaze detection unit 241 detects the gaze direction of the photographer 5 based on the image data output from the imaging unit 260. The shooting direction of the camera 6 is assumed to be adjusted in advance to a direction that captures the pupil of the photographer 5's eye. 6 is a schematic diagram illustrating the eyeballs of the photographer 5 and the camera 6. The eyeball e shown by the solid line corresponds to the eyeball of the photographer 5 when the photographer 5 faces a predetermined direction (for example, north) and looks at a point at infinity (not shown) located directly in front of his or her face. The eyeball e' shown by the dashed line corresponds to the eyeball of the photographer 5 when the photographer 5 looks at an arbitrary position (in this embodiment, player R1 as the subject of photography). If the origin is the center of the pupil when the eyeball e is photographed by the camera 6, the gaze direction displacement angle θ is expressed by the following equation (1). sinθ=x / d …(1) The gaze direction displacement angle θ corresponds to the displacement angle from the front direction of the face of the photographer 5. The symbol d corresponds to the distance from the cornea to the retina of the eyeball e. The symbol x corresponds to the displacement of the pupil center when the photographer 5 is looking at an arbitrary position (the displacement of the pupil center between the eyeball e and the eyeball e').

[0040] The gaze detection unit 241 detects the horizontal and vertical gaze direction displacement angles θ for both the left and right eyes, respectively, to determine the horizontal component θLx and vertical component θLy of the gaze direction displacement angle of the left eye, and the horizontal component θRx and vertical component θRy of the gaze direction displacement angle of the right eye. In this embodiment, the direction of the point at infinity corresponds to the front direction of the face of the photographer 5 detected by the head sensor 40.

[0041] 7 is a schematic diagram illustrating the shooting distance L from the imaging plane 261S of the shooting camera 1 to the object to be photographed (player R1). For ease of understanding, FIG. 7 illustrates only the horizontal component of the line of sight direction displacement angle. 7, eyeball Le corresponds to the left eyeball of photographer 5, and eyeball Re corresponds to the right eyeball of photographer 5. Furthermore, angle θLx corresponds to the horizontal component of the gaze direction displacement angle of the left eye, and angle θRx corresponds to the horizontal component of the gaze direction displacement angle of the right eye.

[0042] The focal length D1 of the eyeball is expressed as a function of the above-mentioned gaze direction displacement angle and the interocular distance a by the following equation (2): D1=f(θLx, θLy, θRx, θRy, a) (2) Here, the horizontal component of the gaze direction displacement angle of the left eye is θLx, the vertical component is θLy, and the horizontal component of the gaze direction displacement angle of the right eye is θRx, the vertical component is θRy. The distance between the eyes, a, is approximately 6.5 cm for an adult human.

[0043] In the two-dimensional case illustrated in FIG. 7, the focal length D1 can be calculated by the following equation (3). D1×(tanθLx+tanθRx)=a ……(3) The eyeball focal length detection unit 242 calculates the eyeball focal length D1 using the method described above.

[0044] 7, distance D2 corresponds to the distance from the retina of the eyeball Re (or Le) of the photographer 5 to the imaging plane 261S of the camera 1. Distance D2 is determined based on the positional relationship between the photographer 5 and the camera 1 (for example, 0.5 m). The photographing distance calculation unit 244 determines the photographing distance L to be the distance obtained by adding the distance D2 to the focal length D1 of the eyeball.

[0045] The camera platform drive amount calculation unit 243 calculates the amount of camera platform drive so as to keep player R1, as the subject to be photographed (main subject), in approximately the center of the shooting screen of the camera 1, based on the direction in which the face of the photographer 5 is facing (the direction in front of the face of the photographer 5) and the line-of-sight displacement angle θ of the eyeball (the displacement angle from the direction in front of the face of the photographer 5). For example, using the direction in front of the face of the photographer 5 as a reference, the unit calculates drive information for moving the shooting direction of the camera 1 from the current direction to the line-of-sight direction of the photographer 5; in other words, for moving the automatic camera platform 50 from the current base position to a target base position. The base position may be represented by the rotation angle of the rotationally driven base from a reference position, or the number of control pulses sent to the motor used as the drive source. The control unit 240 transmits the drive information and drive instructions for the base calculated by the camera platform drive amount calculation unit 243 to the automatic camera platform 50 via the wireless communication unit 270 . Furthermore, the control unit 240 transmits the shooting distance L calculated by the shooting distance calculation unit 244 and a driving instruction for the focusing 361 to the camera 1 via the wireless communication unit 270.

[0046] The recording control unit 245 controls the recording and reading of data and information to and from a memory (not shown) in the control unit 240. The recording control unit 245 can store in memory, for example, image data of the pupils of the eyes of the photographer 5 when the photographer 5 is looking at a point (not shown) at infinity located in front of his or her face. In addition, the recording control unit 245 can also store data on the gaze direction displacement angle detected by the gaze detection unit 241, data on the shooting distance L calculated by the shooting distance calculation unit 244, data on the base drive amount ΔQ calculated by the camera head drive amount calculation unit 243, etc. in memory together with time information.

[0047] The focus detection unit 246 calculates the defocus amount (the amount of deviation between the imaging position of the photographing lens 360 and the imaging surface 261S of the image sensor 261) by performing, for example, focus detection processing using a phase difference detection method using focus detection pixel signals output from the imaging unit 260. The imaging position is the position where an image is formed when the focusing lens 361 of the photographing lens 360 is in its current position.

[0048] Based on the defocus amount calculated by the focus detection unit 246, the lens drive amount calculation unit 247 calculates the drive amount Δd for moving the focusing lens 361 of the photographing lens 360 from the current position to the in-focus position. The control unit 240 outputs the drive amount Δd calculated by the lens drive amount calculation unit 247 and a drive instruction for the focusing 361 to the lens control unit 330. In this way, a focusing operation is performed on the eyeball of the photographer 5.

[0049] <Explanation of the flowchart> The flow of processing by the programs executed by head sensor 40 and gaze detection camera 6 will be described with reference to Fig. 8. The flowchart on the left side of Fig. 8 is a flowchart illustrating the flow of processing by the program executed by head sensor 40. The flowchart on the right side of Fig. 8 is a flowchart illustrating the flow of processing by the program executed by gaze detection camera 6.

[0050] (head sensor) When, for example, a power switch (not shown) of head sensor 40 is turned on and power supply to head sensor 40 begins, wireless communication unit 41 of head sensor 40 starts the process shown in FIG. In step S400, the head sensor 40 detects tilt information and shake information of the head of the photographer 5 using the posture / shake detection unit 43, and the process proceeds to step S410.

[0051] In step S410, the head sensor 40 detects the direction in which the face of the photographer 5 is facing (the front direction of the face of the photographer 5) using the face direction detection unit 42, and the process proceeds to step S420. In step S420, the head sensor 40 generates a signal indicating the front direction of the face of the photographer 5 detected by the face direction detection unit 42, and the process proceeds to step S430.

[0052] In step S430, the wireless communication unit 41 of the head sensor 40 communicates with the gaze detection camera 6 as an external device, and transmits a signal indicating the front direction of the face of the photographer 5 to the camera 6, and the process proceeds to step S440.

[0053] In step S440, head sensor 40 determines whether or not to end the process. If the power switch of head sensor 40 has been turned off, head sensor 40 makes an affirmative decision in step S440 and ends the process. If the power switch of head sensor 40 has not been turned off, head sensor 40 makes a negative decision in step S440, returns to step S400, and repeats the above-described process.

[0054] (Gaze detection camera) When, for example, a power switch (not shown) is turned on and the camera 6 is switched to the line-of-sight detection mode, the control unit 240 of the camera 6 starts the process shown in FIG. In step S600, control unit 240 receives signals from head sensor 40 and automatic camera platform 50 via wireless communication unit 270, and the process proceeds to step S610.

[0055] In step S610, the gaze detection unit 241 of the control unit 240 detects the gaze direction displacement angles of both the left and right eyes with reference to the front direction of the face of the photographer 5. When the gaze detection unit 241 detects the horizontal component θRx and vertical component θRy of the gaze direction displacement angle of the right eye and the horizontal component θLx and vertical component θLy of the gaze direction displacement angle of the left eye, the process proceeds to step S620.

[0056] In step S620, the gaze detection unit 241 of the control unit 240 calculates the gaze direction of the photographer 5 (for example, the gaze direction when the photographer 5 follows the player R1 with his eyes) based on the forward direction of the face of the photographer 5, and proceeds to step S630. In step S630, the tripod head drive amount calculation unit 243 of the control unit 240 calculates drive information for moving the shooting direction of the camera 1 from the current direction to the line of sight of the photographer 5, in other words, for moving the base of the automatic tripod head 50 from the current base position to the target base position, based on the front direction of the face of the photographer 5, and proceeds to step S640.

[0057] In step S640, the eyeball focal length detection section 242 of the control section 240 calculates the eyeball focal length D1 of the photographer 5, and the process proceeds to step S650. In step S650, the control unit 240 calculates the shooting distance L by adding the focal length D1 of the eyeball and the distance D2 based on the positional relationship between the photographer 5 and the camera 1 in the shooting distance calculation unit 244, and proceeds to step S660.

[0058] In step S660, the control unit 240 transmits the drive information of the base calculated by the head drive amount calculation unit 243 to the automatic head 50 via the wireless communication unit 270, and also transmits a signal indicating the shooting distance L calculated by the shooting distance calculation unit 244 to the camera 1 via the wireless communication unit 270, and proceeds to step S670.

[0059] In step S670, the control unit 240 determines whether or not to end the process. For example, if an operation to switch to an operation mode other than the gaze detection mode has been performed, the control unit 240 makes a positive decision in step S670 and ends the process. If an operation to switch to an operation mode other than the gaze detection mode has not been performed, the control unit 240 makes a negative decision in step S670 and returns to step S600 to repeat the above-described process.

[0060] Next, the processing flow of the programs executed by the automatic camera head 50 and the photographing camera 1 will be described with reference to Fig. 9. The flowchart on the left side of Fig. 9 is a flowchart illustrating the processing flow of the program executed by the automatic camera head 50. The flowchart on the right side of Fig. 9 is a flowchart illustrating the processing flow of the program executed by the photographing camera 1.

[0061] (Automatic head) The control unit 52 of the automatic camera platform 50 starts the process shown in FIG. 9 when, for example, a power switch (not shown) is turned on and the mode is switched to the line of sight detection mode. In step S500, the control unit 52 of the automatic pan head 50 detects the current base position with the drive unit / detection unit 53, and the process proceeds to step S510.

[0062] In step S510, the wireless communication unit 51 of the automatic camera head 50 communicates with the camera 6 for line of sight detection as an external device, and transmits a signal indicating the detected base position to the camera 6, and the process proceeds to step S520.

[0063] In step S520, the wireless communication unit 51 of the automatic camera head 50 communicates with the camera 6 for gaze detection as an external device, receives a signal from the camera 6, in this embodiment a signal indicating driving information for turning the shooting direction of the camera 1 from its current direction to the gaze direction of the photographer 5, and proceeds to step S530.

[0064] In step S530, the drive section / detection section 53 of the automatic pan head 50 drives the base by the instructed drive amount ΔQ in the instructed direction based on the drive information, and the process proceeds to step S540. In step S540, the control unit 52 of the automatic camera platform 50 determines whether or not to end the process. If an operation to switch to an operation mode other than the gaze detection mode has been performed, the control unit 52 makes an affirmative decision in step S540 and ends the process. If an operation to switch to an operation mode other than the gaze detection mode has not been performed, the control unit 52 makes a negative decision in step S540, returns to step S500, and repeats the above-described process.

[0065] (camera for filming) 9 when, for example, a power switch (not shown) is turned on and the camera 1 is switched to a shooting mode based on gaze detection. In this embodiment, an example will be described in which exposure calculation and driving of the focusing lens 361 are repeatedly performed in order to immediately start shooting when a shooting instruction is received. In step S100, the control unit 230 communicates with the gaze detection camera 6 as an external device via the wireless communication unit 270, receives a signal from the gaze detection camera 6, and proceeds to step S110. In this embodiment, a signal indicating the shooting distance L from the camera 1 to the player R1 is received.

[0066] In step S110, the control unit 230 performs exposure control. As described above, the photometry unit 231 performs photometry based on the image pickup pixel signals, and the exposure control unit 232 determines the aperture value (Av value), shutter speed (Tv value), and sensitivity (Sv value) based on the photometry results and program diagram information, and the process proceeds to step S120.

[0067] In step S120, the control unit 230 drives the focusing lens 361 based on the shooting distance L acquired in step S100, and proceeds to step S130. As described above, the lens drive instruction unit 233 calculates the drive amount Δd of the focusing lens 361 to focus on a subject that is the shooting distance L away, and outputs a drive instruction to the lens control unit 330.

[0068] In step S130, control unit 230 starts shooting based on the shooting instruction, and the process proceeds to step S140. The shooting instruction is issued by control unit 230, for example, when preparations for capturing the next frame are complete during continuous shooting, when shooting distance L satisfies a preset condition (distance range), or at the start of the next frame during video shooting.

[0069] In step S140, the control unit 230 determines whether or not to end the process. If a switching operation to a shooting mode other than the shooting mode based on gaze detection has been performed, the control unit 230 makes an affirmative decision in step S140 and ends the process. If a switching operation to a shooting mode other than the shooting mode based on gaze detection has not been performed, the control unit 230 makes a negative decision in step S140, returns to step S100, and repeats the above-described process.

[0070] According to the embodiment described above, the following advantageous effects can be obtained. (1) The head sensor 40 as a tripod control device and the camera 6 for gaze detection are equipped with a face direction detection unit 42 that detects the front direction of the face of the photographer 5, a gaze detection unit 241 that detects the gaze direction of the photographer 5, and a tripod drive amount calculation unit 243 that calculates drive information for the drive unit / detection unit 53 of the automatic tripod head 50 that moves the shooting direction of the camera 1 for shooting the subject based on the front direction of the face of the photographer 5 and the gaze direction of the photographer 5. With this configuration, for example, the shooting direction of the camera 1 can be appropriately controlled to match the line of sight of the photographer 5 who is following the eyes of a runner R1 sprinting in a sprint race. Furthermore, since the camera 6 for detecting the line of sight is not provided on the head of the photographer 5, the annoyance felt by the photographer 5 can be reduced compared to when the camera 6 is provided near the head. Furthermore, since a head sensor 40 (face direction detection unit 42) is provided to detect the direction in which the face of the photographer 5 is facing, the gaze direction of the photographer 5 can be detected based on the orientation of the face of the photographer 5 even if a camera 6 for gaze detection is not provided on the head of the photographer 5.

[0071] (2) The camera platform drive amount calculation unit 243 calculates drive information for the drive unit / detection unit 53 of the automatic camera platform 50 in order to move the shooting direction of the camera 1 toward the line of sight of the photographer 5. With this configuration, the automatic camera platform 50 can be driven so that the shooting direction of the camera 1 appropriately moves toward the line of sight of the photographer 5.

[0072] (3) The gaze detection unit 241 detects the gaze displacement angle θ of the photographer 5 who follows the sprinting player R1 with his / her eyes, based on the forward direction of the face of the photographer 5 detected by the face direction detection unit 42, and calculates the gaze direction of the photographer 5 based on the gaze displacement angle θ. With this configuration, the gaze direction of the photographer 5 can be appropriately detected based on the direction of the face of the photographer 5, without providing a gaze detection camera 6 on the head of the photographer 5.

[0073] (4) The camera platform drive amount calculation unit 243 calculates drive information for the drive unit / detection unit 53 of the automatic camera platform 50 based on the front direction of the face detected by the face direction detection unit 42. With this configuration, the automatic camera platform 50 can be driven appropriately based on the direction of the face of the photographer 5.

[0074] (5) The face direction detection unit 42 detects the head orientation of the photographer 5 and detects the front direction of the face of the photographer 5 based on the detected orientation information. With this configuration, the front direction of the face of the photographer 5 can be detected more appropriately than when the head orientation is not taken into consideration.

[0075] (6) The face direction detection unit 42 detects head shake of the photographer 5 and corrects the front direction of the face based on the detected shake information. With this configuration, the front direction of the face of the photographer 5 can be detected more appropriately than when head shake is not taken into consideration.

[0076] (7) The camera 6 includes a wireless communication unit 270 that transmits drive information calculated by the camera head drive amount calculation unit 243 to the automatic camera head 50. With this configuration, drive information for the drive unit / detection unit 53 can be appropriately transmitted to the automatic camera head 50.

[0077] (8) The camera 6 is equipped with a photography processing calculation unit 244 that calculates the photography distance L from the camera 6 to the player R1 located in the line of sight of the photographer 5. With this configuration, the photography distance L to the player R1 located in the line of sight of the photographer 5 can be appropriately detected.

[0078] (9) Wireless communication unit 270 transmits information indicating shooting distance L calculated by shooting distance calculation unit 244 to camera 1. With this configuration, shooting distance L can be properly communicated to camera 1.

[0079] (10) The gaze detection unit 241 detects the gaze displacement angle θ based on the position of a mark 102 attached to a predetermined position on a contact lens 101 worn on the eye of the photographer 5. This configuration makes it possible to appropriately determine the center position S of the pupil g. Therefore, the accuracy of calculating the displacement x in the above formula (1) can be improved compared to when the center position of the pupil g is estimated from the area ratio of the white part of the eye surrounding the pupil g to the pupil g.

[0080] (Variation 1) In the above-described embodiment, an example has been described in which the gaze detection camera 6 functions as a camera head control device, but the function of the camera head control device may be transferred to the automatic camera head 50. For example, calculation of drive information for moving the automatic camera head 50 from the current base position to the target base position, which was performed by the camera head drive amount calculation unit 243 in the gaze detection camera 6 in the embodiment, may be performed by the automatic camera head 50.

[0081] In the first variant of the embodiment, a signal indicating the gaze direction displacement angle θ of the eyeball (displacement angle from the front direction of the face of the photographer 5) detected by the gaze detection unit 241 and the shooting distance L calculated by the shooting distance calculation unit 244 is transmitted from the gaze detection camera 6 to the automatic pan head 50. In the first modification of the embodiment, the information transmitted from the wireless communication unit 41 of the head sensor 40 and indicating the direction in which the face of the photographer 5 is facing is also received by the wireless communication unit 51 of the automatic camera platform 50 .

[0082] In the first variant of the embodiment, the control unit 52 of the automatic pan head 50 calculates drive information for rotating the base based on the direction in which the face of the photographer 5 is facing (the direction in front of the face of the photographer 5) and the eye gaze direction displacement angle θ (the displacement angle from the direction in front of the face of the photographer 5) so as to keep the player R1, as the subject to be photographed (main subject), in approximately the center of the shooting screen of the camera 1. When the driving information for driving the base to rotate is received by the wireless communication unit 51, the driving unit / detecting unit 53 drives the base in the instructed direction by the instructed driving amount ΔQ. As described above, the first modification of the embodiment in which the control unit 52 of the automatic camera head 50 functions as a camera head control device can also achieve the same effects as the embodiment.

[0083] (Variation 2) In the above-described embodiment, an example has been described in which the camera 1 for photographing and the camera 6 for gaze detection are different. Specifically, the camera 1 includes the control unit 230, and the camera 6 includes the control unit 240. In the second modification of the embodiment, the same camera may be used, with the operation mode switched.

[0084] In the second modification of the embodiment, for example, a camera whose power switch is turned on and whose mode is switched to a shooting mode based on gaze detection is operated as a shooting camera 1. Also, a camera whose power switch is turned on and whose mode is switched to a gaze detection mode is operated as a gaze detection camera 6. As described above, variant 2 of the embodiment in which multiple identical cameras are prepared and one camera is operated as a camera 1 for photography and the other camera is operated as a camera 6 for gaze detection can also obtain the same effects as the embodiment. At least one of camera 1 and camera 6 may be configured as a smartphone.

[0085] (Variation 3) In this embodiment, the position of the pupil center of the eyeball e of the photographer 5 is estimated based on image data. For example, the position of the pupil center is estimated from the area ratio of the white part of the eye surrounding the pupil to the pupil itself. The displacement x of the pupil center in the above equation (1) is calculated based on the estimated pupil center position. In the third modification of the embodiment, a marked contact lens is worn on the eye of the photographer 5 in order to improve the accuracy of the displacement x.

[0086] 10 is a schematic diagram illustrating the eyeball e of the photographer 5 wearing a contact lens 101. Although one eyeball e of both the left and right eyes is shown, the contact lens 101 is worn on both the left and right eyes.

[0087] The pupil g is an opening surrounded by the iris f. The symbol S indicates the position of the center of the pupil g. The marks 102 are provided at approximately equal intervals in a total of four positions on the contact lens 101 radially outward from the position corresponding to the pupil g. Two lines connecting two of the four marks 102 that face each other across the pupil g are configured to intersect at the position S, the center of the pupil g.

[0088] Based on the image data output from the imaging unit 260, the gaze detection unit 241 of the gaze detection camera 6 calculates the displacement x of the pupil center in the above equation (1) by taking the intersection of two lines connecting two of the four marks 102 attached to the contact lens 101 that face each other across the pupil g as the position S of the center of the pupil g.

[0089] As described above, in the third modification of the embodiment, the center position S of the pupil g can be appropriately determined by providing markings on the contact lens 101. Therefore, the accuracy of calculating the displacement x in the above formula (1) can be improved compared to when the center position of the pupil g is estimated from the area ratio of the white part of the eye surrounding the pupil g to the pupil g.

[0090] <Program> 8 and 9 may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be supplied to camera 1 and camera 6 via, for example, personal computer 800. "Computer-readable recording medium" refers to portable recording media such as flexible disks, magneto-optical disks, optical disks, and memory cards, as well as storage devices such as hard disks built into computer systems. A computer system, like the illustrated personal computer 800, includes an OS (Operating System) and peripheral hardware.

[0091] Furthermore, the term "computer-readable recording medium" may include a medium that dynamically stores a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, or a medium that stores a program for a certain period of time, such as a volatile memory within a computer system that serves as a server or client in such a case. The above-mentioned program may be one that realizes part of the above-mentioned functions, or may realize the above-mentioned functions in combination with a program already stored in the computer system.

[0092] 11 is a diagram illustrating the supply of a program to cameras 1 and 6. Cameras 1 and 6 can receive the program recorded on, for example, a CD-ROM 953 via a personal computer 800 that can be connected by short-range wireless communication or the like. Cameras 1 and 6 can also receive the program via communication line 900 without going through personal computer 800. Computer 952 is a server computer that provides the above program, and stores the program on a recording medium 954 such as a hard disk. Communication line 900 is a communication line such as the Internet or personal computer communication, or a dedicated communication line, wireless communication line, etc. Computer 952 reads the program from recording medium 954 and transmits the program to personal computer 800 via communication line 900. That is, the program is carried as a data signal by a carrier wave and transmitted via communication line 900. Camera 1 and camera 6 receive the program from personal computer 800 via short-range wireless communication, etc. Alternatively, they receive the program via a mobile phone line as communication line 900. In this way, the program can be supplied as a computer-readable computer program product in various forms such as a recording medium or a carrier wave.

[0093] Although various embodiments and modifications have been described above, the present invention is not limited to these. Aspects in which the configurations shown in the embodiments and modifications are used in combination are also included within the scope of the present invention. Other aspects conceivable within the scope of the technical concept of the present invention are also included within the scope of the present invention. [Explanation of symbols]

[0094] 1...camera for photographing, 5...photographer, 6...camera for gaze detection, 40...head sensor, 41, 51, 270...wireless communication unit, 42...face direction detection unit, 43...posture / shake detection unit, 50...automatic platform, 52...control unit, 53...drive unit / detection unit, 54...operation member, 240...control unit, 241...gaze detection unit, 242...eye focal length detection unit, 243...platform drive amount calculation unit, 244...shooting distance calculation unit

Claims

[Claim 1] a face direction detection unit that detects the front direction of the photographer's face; a gaze detection unit for detecting a gaze direction of the photographer; a calculation unit that calculates drive information for a drive unit of an automatic platform that moves a shooting direction of a shooting unit that shoots an object based on the front direction of the face and the line of sight; A camera head control device.

Images