Imaging device, control method for imaging device, and program

Abstract

To solve the problem in which camera data is not associated with a still image obtained by stopping a moving image that is being output.SOLUTION: An imaging device includes output means for outputting an image generated capturing, by imaging means, an image of a subject and camera data including at least data indicating an imaging direction of the imaging means, and control means for updating the camera data to generate and output the camera data, if the image to be output is a moving image, and for outputting the camera data without at least updating the data indicating the imaging direction in the camera data, if the image is a still image obtained by stopping the moving image.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to an imaging device, a control method for an imaging device, and a program. [Background technology]

[0002] Cameras used for distributing images including videos and still images are classified into fixed types that fix the entire imageable field of view and capture images, and PTZ (pan-tilt-zoom) types that capture images while appropriately switching or circulating multiple subjects within the imageable field of view. In PTZ type cameras, the image capturing angle of view for important subjects is registered in advance. Registering the image capturing angle of view is generally called presetting. There is known a camera that transmits still images while the PTZ type camera is driven and controlled by preset registration information. Patent Document 1 discloses a technology that transmits still images acquired from captured images while the camera is driven and controlled to a preset PTZ position.

[0003] On the other hand, in recent years, a technology has been known that transmits camera data corresponding to an image captured by a camera to an external device in real time. The camera data mainly includes the position, attitude, and lens information of the camera, and is used to convey the state of the camera when capturing an image to an external device. For example, Patent Document 2 discloses a technology in which camera data is input from a live-action camera equipped with a means for detecting camera operation information based on a user's operation to a CG (Computer Graphics) image generating device, and a CG image in which the camera operation information is reflected in a viewpoint in a virtual space is repeatedly synthesized with the live-action image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2009-188975 A [Patent Document 2] JP 2011-35638 A Summary of the Invention [Problem to be solved by the invention]

[0005] The camera data, such as the camera position, the camera imaging direction, and lens information, affect the imaging range and imaging effect of the camera. The camera data must be data corresponding to each frame image of the video. However, as in the example of Patent Document 1, even while a still image is being transmitted by pausing the video, the camera data is updated according to the operation of the camera driven by the preset registration information. Therefore, there is a problem that the camera data does not correspond to the still image by pausing the video being output. [Means for solving the problem]

[0006] In order to solve this problem, for example, an imaging device of the present invention has the following configuration: That is, an output unit that outputs an image generated by an imaging unit by imaging a subject and camera data including at least data indicating an imaging direction of the imaging unit; and a control means for updating the camera data, generating and outputting the camera data, when the image to be output is a moving image, and for outputting the camera data without updating at least data indicating the imaging direction in the camera data, when the image to be output is a still image obtained by stopping the moving image. Effect of the Invention

[0007] According to the present invention, it is possible to output camera data corresponding to a still image obtained by stopping a moving image being output by an imaging device. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing the configuration of an imaging system. [Diagram 2] FIG. 2 is a diagram showing the hardware configuration of the imaging apparatus. [Diagram 3] FIG. 2 is a hardware configuration diagram of an information processing device. [Figure 4]FIG. 2 is a functional block diagram of the imaging apparatus. [Diagram 5] An example of camera data. [Figure 6] FIG. 2 is a functional block diagram of an information processing device. [Figure 7] 5 is a flowchart showing the operation of the imaging apparatus in the first embodiment. [Figure 8] 10 is a flowchart showing the operation of an imaging apparatus according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0010] First Embodiment The first embodiment will be described below. FIG. 1 is a diagram showing an example of the configuration of an imaging system according to this embodiment. In the imaging system 100 of this embodiment, when the imaging device 102 outputs an image captured by the imaging device 102 to the information processing device 103, it determines whether the image being output is output as a moving image or as a still image by pausing the moving image, and simultaneously outputs camera data corresponding to the image. Specifically, when the imaging device 102 outputs a moving image, it updates and outputs camera data corresponding to each frame image of the moving image. On the other hand, when the imaging device 102 outputs a still image, it outputs a part of the camera data corresponding to the still image without updating it. Note that the camera data of this embodiment is output for the purpose of controlling the viewpoint of a virtual space when generating CG (Computer Graphics) in a virtual space linked to the operation of the imaging device 102 on the information processing device 103, for example.

[0011] (System Configuration) The imaging system 100 includes a network 101 , an imaging device 102 , and an information processing device 103 .

[0012] The network 101 includes at least one of the Internet, a wired or wireless LAN (Local Area Network), and a WAN (Wide Area Network). The network 101 has, for example, a plurality of routers, switches, and cables that comply with the Ethernet (registered trademark) communication standard. Note that the network 101 may be of any communication standard, scale, or configuration as long as it is configured to enable communication between the imaging device 102 and the information processing device 103.

[0013] The imaging device 102 is, for example, a digital camera that captures an image of a subject, converts an optical signal into an electrical signal, generates an image including a moving image and a still image, and can output the image to an external device. The imaging device 102 captures an image of a subject in real space and generates image data of a moving image or a still image. In response to a request from the information processing device 103, the imaging device 102 outputs the generated image data to the information processing device 103 via the network 101. In the following description, "image" is used as a term including an image and image data. The imaging device 102 may actively output an image, etc. to the information processing device 103 even without a request from the information processing device 103. The imaging device 102 outputs at least a part of the camera information related to the imaging device 102, including a current position, an attitude, a zoom magnification, a focus position, and a pan angle or a tilt angle, as camera data to the information processing device 103. The camera data includes control data required for the information processing device 103 to perform a synthesis process of an image to be described later. The control data is data capable of controlling a viewpoint in a virtual space. The control data may include information on at least one of the imaging range (imaging direction) and imaging effect of the imaging device 102 as data capable of controlling the viewpoint of the virtual space. The camera information included in the camera data may be changed as necessary. The camera data may include at least information indicating the imaging direction in which the imaging device 102 is imaging, as data capable of controlling the viewpoint of the virtual space. The information indicating the imaging direction is, for example, a pan angle which is an angle in the pan direction (horizontal direction) in the imaging direction in which the imaging device 102 is imaging. Also, the information indicating the imaging direction is, for example, a tilt angle which is an angle in the tilt direction (vertical direction) in the imaging direction in which the imaging device 102 is imaging. As an example, the pan angle is an angle represented by a positive numerical value at a position moved to the right from a reference position, and a negative numerical value at a position moved to the left. As an example, the pan angle is an angle represented by a positive numerical value at a position moved up from a horizontal position, and a negative numerical value at a position moved down. As described above, the information indicating the imaging direction can be expressed by the pan angle and tilt angle, but it does not necessarily have to be an angle, and may be expressed by two-dimensional coordinates with a predetermined position as the reference.

[0014] In addition, using the information indicating the imaging direction to control the viewpoint of the virtual space is just one example, and the information indicating the imaging direction may be used for other purposes. For example, it may be used to change the position at which predetermined information is displayed on the captured image. At least in each embodiment, it is sufficient that the camera data (particularly the information indicating the imaging direction) corresponds to a still image obtained by stopping the video output by the imaging device, and various methods of using the camera data are conceivable.

[0015] The information processing device 103 sets and controls a viewpoint in a virtual space based on camera data received from the imaging device 102 via the network 101. The information processing device 103 generates a virtual space image (CG) in which the virtual space is viewed from the set viewpoint. The information processing device 103 generates a composite image by synthesizing the generated virtual space image with an image received from the imaging device 102 via the network 101. In this way, the information processing device 103 enables image display as if a subject in real space exists in the virtual space. The information processing device 103 is, for example, a computer such as a personal computer, a tablet terminal, a smartphone, etc. The information processing device 103 can also operate an object in the virtual space based on the received camera data. For example, the information processing device 103 may operate the viewpoint in the virtual space as an object of the imaging device 102 based on control data included in the camera data.

[0016] (Device configuration) 2 is a diagram showing an example of a hardware configuration of the imaging device according to the present embodiment. The imaging device 102 includes an imaging optical system 201, an imaging unit 202, an image processing unit 203, an encoder 204, a CPU (Central Processing Unit) 205, a RAM (Random Access Memory) 206, a ROM (Read Only Memory) 207, a network I / F (Interface) 208, an image I / F 209, and a bus 210. The imaging optical system 201, the imaging unit 202, the image processing unit 203, the encoder 204, the CPU 205, the RAM 206, the ROM 207, the network I / F 208, and the image I / F 209 are connected to each other via the bus 210 so as to be able to transmit and receive data. The CPU 205, the RAM 206, the ROM 207, and the like also function as a so-called computer.

[0017] The imaging optical system 201 is a lens that forms an image of light from a subject on an imaging plane of the imaging unit 202, and is composed of, for example, a zoom lens, a focal lens, and a blur correction lens. In this embodiment, the imaging optical system 201 is configured integrally with the imaging device 102, but may be detachable like an interchangeable lens. The imaging optical system 201 operates the lens to change the magnification of the optical zoom based on a user's operation and adjust the focus position according to the distance of the subject.

[0018] The imaging unit 202 is an image sensor that captures an image of a subject. The imaging unit 202 is, for example, a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The imaging unit 202 captures the image of the subject by converting light from the subject through the imaging optical system 201 into an electrical signal and outputting the electrical signal, thereby generating an image.

[0019] The image processing unit 203 converts the electrical signal from the imaging unit 202 into a digital signal by A / D conversion. The image processing unit 203 performs various image processing including color interpolation processing, demosaicing processing, white balance processing, gamma processing, etc. on the image converted into the digital signal.

[0020] The encoder 204 encodes the processed image into a file format such as MotionJpeg, H264, or H265, and outputs the encoded image. The encoded image data is output to the information processing device 103 via the network 101.

[0021] The CPU 205 is a central processing unit that performs overall control of the image capturing apparatus 102. The CPU 205 reads out a program loaded from the ROM 207 to the RAM 206 to execute at least some of the functions of the image capturing apparatus 102, which will be described later.

[0022] The RAM 206 provides a work area used when the CPU 205 executes a program, and functions as a frame memory and a buffer memory.

[0023] The ROM 207 stores programs and image data for the CPU 205 to control the imaging device 102. Note that the imaging device 102 may include a storage such as a hard disk drive (HDD) and a solid state drive (SSD) instead of the ROM 207 or in addition to the ROM 207.

[0024] The network I / F 208 is connected to the information processing device 103 via the network 101. The CPU 205 outputs images and camera data via the network I / F 208 in response to a request from the information processing device 103.

[0025] The image I / F 209 is an image output terminal, and is, for example, an SDI (Serial Digital Interface) or an HDMI (registered trademark: High-Definition Multimedia Interface), etc. The CPU 205 may output an image signal to a display device or the like via the image I / F 209.

[0026] 3 is a diagram showing an example of a hardware configuration of the information processing device 103 according to this embodiment. The information processing device 103 has a CPU 301, a RAM 302, a ROM 303, an input I / F 304, an output I / F 305, a network I / F 306, and a bus 307. The CPU 301, the RAM 302, the ROM 303, the input I / F 304, the output I / F 305, and the network I / F 306 are connected via the bus 307 so as to be able to transmit and receive data to and from each other.

[0027] The CPU 301 is a central processing unit that performs overall control of the information processing device 103. The CPU 301 reads out a program loaded from the ROM 303 to the RAM 302, thereby executing at least a part of the functions of the information processing device 103, which will be described later.

[0028] The RAM 302 provides a work area used when the CPU 301 executes a program, and functions as a frame memory and a buffer memory.

[0029] The ROM 303 stores programs, image data, and the like for the CPU 301 to control the information processing device 103. The information processing device 103 may have a storage device such as an HDD or SSD instead of the ROM 303 or in addition to the ROM 303.

[0030] The input I / F 304 is an interface for receiving input from a user, such as a keyboard or a mouse. The user may input, for example, the coordinates of a viewpoint position in the virtual space, a viewing angle, a viewpoint direction, and the like, via the input I / F 304.

[0031] The output I / F 305 is an interface for outputting an image to be displayed on a display. In accordance with an instruction from the CPU 301, the output I / F 305 outputs an image captured by the imaging device 102, an image of a virtual space (CG), or a composite image of an image captured by the imaging device 102 and a virtual space image to the display for display.

[0032] The network I / F 306 is connected to the image capturing device 102 via the network 101. The network I / F 306 receives images and camera data output from the image capturing device 102.

[0033] (Functional configuration) The functional configuration of the imaging device according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram showing an example of the functional configuration of the imaging device 102 according to this embodiment. The imaging device 102 has an imaging optical system 201, an imaging unit 202, a storage unit 401, a control unit 402, an input unit 403, and an output unit 404.

[0034] The functions performed by the imaging optical system 201 and the imaging unit 202 are the same as those described in the device configuration, and therefore will not be described here. Furthermore, among the functional blocks, the functions realized by software are realized by a program stored in the storage unit 401 such as the ROM 207.

[0035] The CPU 205 reads a program from the ROM 207 to the RAM 206 and executes it to realize each function of the control unit 402, the input unit 403, the output unit 404, etc. The functions realized by the hardware are realized, for example, by automatically generating a dedicated circuit on the FPGA from a program for realizing the function of each functional block using a predetermined compiler. FPGA is an abbreviation for Field Programmable Gate Array. Each function may be realized as hardware by forming a gate array circuit in the same manner as the FPGA. Each function may be realized by an ASIC (Application Specific Integrated Circuit). Note that the configuration of the functional blocks shown in FIG. 4 is an example, and a plurality of functional blocks may constitute one functional block, or any functional block may be divided into blocks performing a plurality of functions.

[0036] The storage unit 401 stores captured images, current camera data, etc. The current camera data is set and updated in response to the operation of the imaging device 102 calculated based on data detected by a sensor installed in the imaging device 102. The storage unit 401 stores a setting for pausing a moving image in order to generate a still image during a preset operation.

[0037] The control unit 402 is responsible for the overall control of the imaging device 102. The control unit 402 controls the imaging optical system 201 and the imaging unit 202 based on the user's operation. Specifically, the control unit 402 controls the pan angle, tilt angle, zoom position, focus position, etc. of the imaging optical system 201, and sets or changes parameters such as exposure time and gain related to imaging by the imaging unit 202. The control unit 402 controls input and output of images and camera data. For example, the control unit 402 stores captured images and camera data in the storage unit 401. The control unit 402 acquires control commands such as preset operation commands input by the user from the input unit 403. The control unit 402 outputs the images and camera data stored in the storage unit 401 to the information processing device 103 via the output unit 404.

[0038] When outputting a moving image as an image, the control unit 402 reads camera data including control data for controlling the viewpoint in the virtual space from the stored camera information, and updates and outputs the camera data each time it outputs a frame image. When outputting a still image as an image, the control unit 402 outputs the control data for controlling the viewpoint in the virtual space without updating it. Note that the control unit 402 may update and output the camera data other than the control data even when outputting a still image.

[0039] Information is input to the input unit 403 via the network 101. In this embodiment, a control command for the imaging device 102 is input to the input unit 403. The control command includes a preset operation command and a command for changing a setting for pausing a moving image and turning it into a still image during a preset operation. The preset operation command includes preset identification information, PTZ (pan-tilt-zoom) coordinate information, and control information for parameters such as exposure time and gain related to imaging by the imaging unit 202.

[0040] The output unit 404 outputs images and camera data under the control of the control unit 402. The output unit 404 outputs camera data including control data for controlling a viewpoint in a virtual space including the attitude and position of the imaging device 102 and the zoom position and focus position of the imaging optical system 201, for example.

[0041] An example of camera data will be described with reference to FIG. 5. FIG. 5 is a diagram showing an example of camera data. The camera data is, for example, data based on the free-d protocol or the HF protocol. Specifically, the camera data includes a camera ID, a pan angle, a tilt angle, a roll angle, an X position, a Y position, a Z position, a zoom, a focus, and any other information. In this manner, the camera data includes control data for controlling a viewpoint in a virtual space. The camera data includes at least one of the position and attitude (information indicating an imaging direction) of the imaging device 102, and the zoom position and the focus position of the imaging optical system 201 as control data related to the viewpoint in the virtual space.

[0042] The camera ID of the camera data is an ID unique to the image capture device 102 or an ID of the image capture device 102 arbitrarily set by the user.

[0043] The pan angle, tilt angle, and roll angle of the camera data are pan angle, tilt angle, and roll angle that represent the attitude of the imaging device 102. Here, the pan angle, tilt angle, and roll angle of the camera information stored in a storage unit such as the ROM 207 are updated by data acquired by a sensor or the like attached to the imaging device 102. When a video is being output, the pan angle, tilt angle, and roll angle of the camera data are updated by reading the pan angle, tilt angle, and roll angle of the camera information from the storage unit, so that they become camera information corresponding to each frame image of the video. On the other hand, when a still image in which the video is paused is being output, the pan angle, tilt angle, and roll angle of the camera data are not read from the storage unit and are not updated, so that they become camera information corresponding to the still image.

[0044] The X, Y, and Z positions of the camera data are coordinate values ​​indicating the position of the imaging device 102 acquired by a sensor or the like attached to the imaging device 102. Here, the X, Y, and Z positions of the camera information stored in a storage unit such as ROM 207 are updated by data acquired by a sensor or the like attached to the imaging device 102. When a video is being output, the X, Y, and Z positions of the camera data are updated by reading out the X, Y, and Z positions of the camera information from the storage unit, so that the camera information corresponds to each frame image of the video. On the other hand, when a still image is output by pausing the video, the X, Y, and Z positions of the camera data are not read out from the storage unit and are not updated, so that the camera information corresponds to the still image.

[0045] The zoom and focus of the camera data are information indicating the zoom position and focus position of the imaging optical system 201. Here, the zoom position and focus position of the camera information stored in a storage unit such as the ROM 207 are updated by data acquired by a sensor or the like attached to the imaging device 102. When a video is being output, the zoom and focus of the camera data are updated by reading out the zoom position and focus position of the camera information from the storage unit, so that the camera information corresponds to each frame image of the video. On the other hand, when a still image in which the video is paused is being output, the zoom and focus of the camera data are not read out from the storage unit and are not updated, so that the camera information corresponds to the still image.

[0046] The user definition of the camera data is any information set in the image capture device 102. The user definition may include, for example, information on the time related to the output of a still image. The time related to the output of a still image may be, for example, information such as the remaining pause time when a video is paused and a still image is output.

[0047] The control data of the camera data described above is an example, and may include data that affects at least one of the imaging range and imaging effect of the imaging device 102 as data capable of controlling the viewpoint of the virtual space. Specifically, the control data may include information on the iris, white balance, gain, lens distortion correction parameters, and imaging condition settings of the imaging device 102. This allows the information processing device 103 to control the depth of field, white balance, brightness, and distortion of the image in the virtual space based on the control data.

[0048] The functional configuration of the information processing device according to the present embodiment will be described with reference to FIG. 6. FIG. 6 is a diagram showing an example of the functional configuration of the information processing device 103 according to the present embodiment. The information processing device 103 has a receiving unit 601, a setting unit 602, a generating unit 603, and a display control unit 604. A program for realizing the function of each functional block shown in FIG. 6 is stored in a storage unit such as the ROM 303. The CPU 301 reads the program stored in the ROM 303 into the RAM 302 and executes it to realize the function. The function realized by the hardware may be realized by automatically generating a dedicated circuit on the FPGA from a program for realizing the function of each functional block, for example, by using a predetermined compiler. In addition, each function may be realized as hardware by forming a gate array circuit in the same manner as the FPGA. In addition, each function may be realized by an ASIC. In addition, a plurality of functional blocks may be configured as one functional block, or any functional block may be divided into blocks performing a plurality of functions.

[0049] The receiving unit 601 receives information output from the imaging device 102. The receiving unit 601 receives images including moving images and still images output from the output unit 404, and camera data that is camera information related to the viewpoint in the virtual space.

[0050] The setting unit 602 sets and updates information relating to the viewpoint in the virtual space based on the camera data received by the receiving unit 601 .

[0051] The generation unit 603 generates a virtual space image based on information related to the viewpoint in the virtual space by the setting unit 602. The generation unit 603 also generates a composite image in the virtual space, as if a subject in the real space were present, by synthesizing the image received by the receiving unit 601 with the generated virtual space image.

[0052] A display control unit 604 controls the output I / F 305 so as to display at least one of the composite image generated by the generating unit 603 or the image acquired by the receiving unit 601 on a display.

[0053] (Operation description) The operation of the imaging device according to this embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the operation of the imaging device 102 according to this embodiment. The operation described in the flowchart shown in Fig. 7 is started when the CPU 205 of the imaging device 102 loads a program stored in the ROM 207 into the RAM 206 and executes the program.

[0054] In S701, the control unit 402 reads all current camera information constituting the camera data from the storage unit 401, and generates camera data.

[0055] In S702, the control unit 402 waits until the timing comes for the image captured and generated by the imaging unit 202 to be output from the output unit 404.

[0056] In S703, the control unit 402 outputs the frame image generated by the imaging unit 202 from the output unit 404 to the information processing device 103 via the network 101.

[0057] In S704, the control unit 402 determines whether or not a still image is being output from the output unit 404. If the image being output from the output unit 404 is a moving image, the control unit 402 executes S705, and if the image is a still image obtained by pausing the moving image, the control unit 402 executes S706.

[0058] In S705, the control unit 402 updates all camera data. Specifically, the control unit 402 reads all current camera information constituting the camera data from the storage unit 401 and updates the camera data. In other words, the control unit 402 generates the latest camera data regarding the attitude, position, zoom, and focus of the imaging device 102, which is control data for controlling the viewpoint in the virtual space that is updated by information from sensors, etc. As a result, when a video is being output, the control unit 402 generates camera data corresponding to each frame image of the video.

[0059] In S706, the control unit 402 updates the camera data other than the control data. Specifically, the control unit 402 reads out current camera information that is not related to controlling the viewpoint in the virtual space from the storage unit 401 among the camera information constituting the camera data, and updates the camera data other than the control data. The current camera information that is not related to controlling the viewpoint in the virtual space is, for example, a camera ID. The current camera information that is not related to controlling the viewpoint in the virtual space may include a frame ID, a time stamp, and a remaining pause time when the video is paused. The control unit 402 does not read out camera information that is related to controlling the viewpoint in the virtual space from the storage unit 401, so that the control data included in the camera data is not updated, and the camera data corresponds to a still image. It is only necessary to stop updating at least the information (pan angle and tilt angle) indicating the imaging direction.

[0060] In S707 , the control unit 402 outputs the camera data to the information processing device 103 via the output unit 404 .

[0061] The control unit 402 determines whether or not to end the image capture, and if the image capture is to be ended, ends the operation of this flowchart, and if not, returns to S702.

[0062] According to this embodiment, when the imaging device 102 outputs an image to the information processing device 103, the control unit 402 updates the control data and outputs the camera data if the image is output as a video. This makes it possible for this embodiment to output camera data including control data corresponding to the video. On the other hand, this embodiment does not update the control data for controlling the viewpoint of the virtual space included in the camera data if a still image produced by pausing the video is output as an image, so that it is possible to output camera data including control data corresponding to the still image. This makes it possible for this embodiment to output appropriate camera data corresponding to the image regardless of whether a video or a still image is output, so that it is possible to generate an appropriate viewpoint of the virtual space.

[0063] In this embodiment, the camera data is updated and generated so as to correspond to each frame image of the video. This makes it possible for the embodiment to output camera data corresponding to each frame image and generate a viewpoint in a virtual space corresponding to each frame image.

[0064] In this embodiment, control data including iris information and the like is generated, so that the output destination can generate a more accurate viewpoint in the virtual space.

[0065] In this embodiment, camera data including information on the time related to the output of a still image is generated, so that the remaining time of the still image, etc. can be communicated to the output destination.

[0066] In this embodiment, camera data other than the control data is updated even when a still image is being output, so that the viewpoint of the virtual space can be generated with high accuracy while the camera data other than the control data is kept up to date.

[0067] <Second embodiment> In the first embodiment, a case has been described in which all interfaces that output images in the imaging device 102 work together to pause the output image or output a moving image. In this embodiment, a case will be described in which the control of whether to output a moving image or to output a still image by pausing the moving image can be changed for each interface in the imaging device 102. The device configuration of this embodiment is omitted since it is similar to that of the first embodiment.

[0068] (Function Description) The functional configuration according to this embodiment will be described below. Differences from the first embodiment will be described below, and descriptions of functions similar to those of the first embodiment will be omitted.

[0069] Information is input to the input unit 403 via the network 101. In this embodiment, the information input to the input unit 403 includes camera data output destination information and compatible line information. The camera data output destination information includes an IP address and a port number on the network 101 of the information processing device 103. The camera data compatible line information is information that links the camera data to an image distribution interface provided in the imaging device 102, and is, for example, an IP stream, SDI (Serial Digital Interface), etc. Furthermore, information for setting whether or not to output a still image by pausing a moving image for each image output interface may be input to the input unit 403.

[0070] The storage unit 401 stores at least a part of the output destination information and corresponding line information of the camera data input to the input unit 403. The storage unit 401 also stores images, camera information input from the sensor, and the like.

[0071] The control unit 402 acquires the above-mentioned camera data output destination information and corresponding line information, etc., via the input unit 403. The control unit 402 determines the image line through which the image is output from the corresponding line information of the camera data, etc. The control unit 402 determines whether the image being output is a video or a still image obtained by pausing the video, based on the determined image line. If the control unit 402 determines that a video is being output, it updates and outputs the camera data including the control data (particularly, information indicating the imaging direction). If the control unit 402 determines that a still image is being output, it does not update the control data (particularly, information indicating the imaging direction) for controlling the viewpoint of the virtual space among the camera data, and outputs the camera data in which the data other than the control data has been updated.

[0072] The output unit 404 outputs the image and camera data to the information processing device 103 via the network I / F 208 based on elements included in the output destination information of the camera data stored in the storage unit 401. The output unit 404 also outputs the image via the image line based on information that sets whether or not to output a still image in which the image is paused for each image line connected to the image output I / F.

[0073] (Operation description) The operation of the imaging device 102 in this embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the operation of the imaging device 102 in this embodiment.

[0074] In S801, the control unit 402 acquires the output destination information and the corresponding line information of the camera data input to the input unit 403 from the storage unit 401. When the corresponding line information of the camera data is not stored in the storage unit 401, the control unit 402 may determine the corresponding line information of the camera data based on the usage status of the image line of the imaging device 102 and the output destination information of the camera data. For example, when the IP address of the output destination information of the camera data is the same as the IP address of the output destination of the IP stream image, the control unit 402 may determine the corresponding line information of the camera data to be an IP stream. When the image is not distributed by an IP stream, the control unit 402 may determine the corresponding line information of the camera data to be SDI or the like.

[0075] S802 to S804 are similar to S701 to S703 in the flowchart of the first embodiment, so the description will be omitted.

[0076] In S805, the control unit 402 determines whether or not a still image obtained by pausing a video is being output from the corresponding image line. Specifically, the control unit 402 determines the image line to be output from the corresponding line information of the camera data acquired in S801. The control unit 402 determines whether or not a still image is being output from the determined image line. For example, when the corresponding line information of the camera data is SDI, the control unit 402 determines whether or not the image of the SDI line is a still image. When the corresponding line information of the camera data is IP stream, the control unit 402 determines whether or not the image of the IP stream line is a still image. When a still image is not being output from the corresponding image line, the control unit 402 executes S806, and when a still image is being output from the image line, the control unit 402 executes S807.

[0077] In S806, the control unit 402 updates all the camera data. That is, the control unit 402 updates all the camera data when the corresponding image line is outputting a video.

[0078] In S807, the control unit 402 updates the camera data other than the control data (particularly, the information indicating the imaging direction). That is, when the corresponding image line outputs a still image, the control unit 402 updates the data other than the control data without updating the control data (particularly, the information indicating the imaging direction) to generate camera data.

[0079] In S808, the control unit 402 outputs the camera data generated for each image line via the output unit 404.

[0080] In this embodiment, since a moving image or a still image can be output individually for each image line, an image desired by the user can be output for each image line.

[0081] In this embodiment, whether or not to output a still image by pausing a video can be changed for each image line, and whether or not to update the control data of the camera data is determined depending on whether or not a still image is being output. This allows the imaging device 102 to appropriately update the camera data for each image line according to the image output from the image I / F and image line desired by the user, and output the camera data including the control data according to the image.

[0082] In this embodiment, the image line is determined by comparing the output destination of the image with the output destination of the camera data, so that the image line to which the camera data is output can be accurately determined and appropriate camera data can be generated for each image line.

[0083] (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0084] The disclosure of this specification includes the following imaging device, imaging device control method, and program. (Item 1) an output means for outputting an image generated by the imaging means by imaging a subject and camera data including at least data indicating an imaging direction of the imaging means; and a control means for updating the camera data, generating and outputting the camera data, when the image to be output is a moving image, and for outputting the camera data without updating at least data indicating the imaging direction in the camera data, when the image to be output is a still image obtained by stopping the moving image. (Item 2) 2. The imaging device according to item 1, wherein, when the image is a video, the control means updates and generates the camera data so as to correspond to each frame image of the video. (Item 3) 3. The imaging device according to claim 1, wherein the camera data includes information regarding at least one of an imaging range and an imaging effect as control data capable of controlling a viewpoint in a virtual space. (Item 4) The imaging device according to any one of items 1 to 3, wherein the control means generates the camera data including at least one of information regarding a position, an attitude, a zoom, a focus, an iris, a white balance, a gain, a distortion correction parameter, and an imaging condition setting of the imaging device. (Item 5) 5. The imaging device according to claim 1, wherein the control means generates the camera data including time information related to the output of the still image. (Item 6) 4. The imaging device according to item 3, wherein the control means updates and outputs the camera data other than the control data when the image to be output is a still image. (Item 7) 7. The imaging device according to claim 1, wherein the control means outputs the image as the moving image or the still image for each image line that outputs the image. (Item 8) 8. The imaging device according to item 7, wherein the control means generates the camera data in response to the image output from the image line. (Item 9) 8. The imaging device according to item 7, wherein the control means compares the output destination of the image with the output destination of the camera data to determine the image line. (Item 10) an output step of outputting an image generated by the imaging means by imaging a subject and camera data including at least data indicating an imaging direction of the imaging means; a control step of updating the camera data, generating and outputting the camera data when the image to be output is a moving image, and outputting the camera data without updating at least data indicating the imaging direction in the camera data when the image to be output is a still image obtained by stopping the moving image. (Item 11) A program for causing a computer to execute each step of the control method according to item 10 by reading and executing the program by the computer.

[0085] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0086] 100: imaging system, 102: imaging device, 103: information processing device, 201: imaging optical system, 202: imaging section, 402: control section, 404: output section.

Claims

1. an output means for outputting an image generated by the imaging means by imaging a subject and camera data including at least data indicating an imaging direction of the imaging means; and a control means for updating the camera data, generating and outputting the camera data, when the image to be output is a moving image, and for outputting the camera data without updating at least data indicating the imaging direction in the camera data, when the image to be output is a still image obtained by stopping the moving image.

2. 2. The imaging device according to claim 1, wherein, when the image is a moving image, the control means updates and generates the camera data so as to correspond to each frame image of the moving image.

3. 2. The imaging device according to claim 1, wherein the camera data includes information on at least one of an imaging range and an imaging effect as control data capable of controlling a viewpoint in a virtual space.

4. The imaging device according to claim 1, characterized in that the control means generates the camera data including at least one of information regarding the position, orientation, zoom, focus, iris, white balance, gain, distortion correction parameters, and imaging condition settings of the imaging device.

5. 2. The image pickup apparatus according to claim 1, wherein the control means generates the camera data including time information related to the output of the still image.

6. 4. The image pickup apparatus according to claim 3, wherein said control means updates said camera data other than said control data and outputs said updated image when said image to be output is a still image.

7. 2. The imaging apparatus according to claim 1, wherein the control means controls the image output means to output the moving image or the still image separately for each image line that outputs the image.

8. 8. The imaging device according to claim 7, wherein said control means generates said camera data in response to said image output from said image line.

9. 8. The imaging device according to claim 7, wherein said control means determines said image line by comparing an output destination of said image with an output destination of said camera data.

10. an output step of outputting an image generated by the imaging means by imaging a subject and camera data including at least data indicating an imaging direction of the imaging means; a control step of updating the camera data, generating and outputting the camera data when the image to be output is a moving image, and outputting the camera data without updating at least data indicating the imaging direction in the camera data when the image to be output is a still image obtained by stopping the moving image.

11. A program for causing a computer to execute each step of the control method according to claim 10 by reading and executing the program by the computer.

Images