Imaging device, control method for imaging device, and program

Abstract

To provide an imaging device, a control method for the imaging device, and a program that outputs additional information indicating a position or range within an image corresponding to a still image when the still image obtained by stopping an image is being output.SOLUTION: In a system in which an imaging device 1000 and a client device 2000 are connected via a network 3000 in a state in which they can communicate data with each other, an output control unit of the imaging device 1000 has an image output unit that acquires an image generated by imaging, by imaging means, a subject, and outputs either a moving image or a still image obtained by stopping the moving image, and an information output unit that, when outputting the moving image, updates and outputs additional information including at least one of the position and range of the subject, and, when outputting the still image, stops updating the additional information and outputs the additional information.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an imaging device, a control method for an imaging device, and a program. [Background technology]

[0002] In recent years, with the expansion of the video distribution market, the number of filming systems using cameras that can be remotely controlled and filmed over a network to film weddings and lectures is increasing. Some of these cameras have a preset function that allows you to set and store (register) combination patterns of pan-tilt-zoom (PTZ) positions that correspond to a specific imaging field of view, and then call up and set the PTZ information of this registered pattern by performing a specific operation.

[0003] When the camera uses the preset function to move the PTZ to a preset angle of view registered in the camera, images are captured during the movement, but depending on the application, such as when the PTZ movement speed is high, images may not be displayed. If you do not want to display images during such movement, a technique such as Patent Document 1 has been disclosed in which a still image immediately before the pan / tilt operation is transmitted. Patent Document 2 also discloses a technique in which a sub-preset is prepared midway when moving from preset position to preset position, and images are not transmitted at positions other than the start position, stop position, and sub-preset position. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2009-188975 A [Patent Document 2] JP 2013-30929 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the conventional methods disclosed in Patent Documents 1 and 2, still images are output during PTZ movement, but information on the position and range of objects in the image, such as faces based on face detection coordinates, is updated and output in accordance with the video, posing a problem that updated additional information that does not correspond to the still image when the video is stopped is output. [Means for solving the problem]

[0006] In order to solve this problem, for example, the imaging device of the present invention has the following configuration: That is, the imaging device is characterized by having an image output means which acquires an image generated by imaging a subject and outputs either a moving image or a still image obtained by stopping the moving image, and an information output means which, when outputting the moving image, updates and outputs additional information including at least one of the position and range of the subject, and, when outputting the still image, stops updating the additional information and outputs the additional information. Effect of the Invention

[0007] According to the present invention, when a still image is output by stopping a moving image, additional information indicating a position or range within the image corresponding to the still image can be output. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of the external configuration of an imaging apparatus. [Diagram 2] FIG. 1 is a diagram showing an example of a configuration of an imaging system. [Diagram 3] FIG. 2 is a diagram showing an example of the configurations and functions of an imaging apparatus and a client apparatus. [Figure 4] FIG. 2 is a block diagram of an output control unit according to the first embodiment. [Diagram 5] 5A to 5C are diagrams showing examples of image output during panning according to the first embodiment. [Figure 6] 5A to 5C are diagrams showing an example of transition of a face frame during panning according to the first embodiment. [Figure 7]5 is a flowchart showing a processing procedure of an output control unit according to the first embodiment. [Figure 8] FIG. 11 is a block diagram of an output control unit according to a second embodiment. [Figure 9] 13A to 13C are diagrams showing transition of the angle of view when a still image setting of the image IF according to the second embodiment is set to be output. [Figure 10] 13A and 13B are diagrams showing transition of the angle of view when the still image setting of the image IF according to the second embodiment is set to not output. [Figure 11] 13A and 13B are diagrams showing output control of detection coordinates in accordance with a still image output transition when a still image setting of an image IF according to the second embodiment is set to be output; [Figure 12] 13A and 13B are diagrams showing output control of detection coordinates in accordance with a still image output transition when the still image setting of the image IF according to the second embodiment is set to not output. [Figure 13] 10 is a flowchart showing a processing procedure of an output control unit according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0010] 1 is a diagram showing an image capturing apparatus 1000 according to an embodiment of the present invention. The image capturing apparatus 1000 includes a pan drive mechanism 1001, a tilt drive mechanism 1002, and a zoom mechanism 1003.

[0011] A pan drive mechanism 1001 can change the imaging area of ​​the imaging device 1000 in the pan direction. A tilt drive mechanism 1002 can change the imaging area of ​​the imaging device 1000 in the tilt direction. A zoom mechanism 1003 changes the angle of view of the imaging device 1000.

[0012] 2 is a diagram showing a system configuration including an imaging device 1000. The system includes the imaging device 1000 and a client device 2000. The client device 2000 is an example of an external device. The imaging device 1000 and the client device 2000 are connected to each other via a network 3000 so as to be able to communicate data with each other. The client device 2000 transmits control commands relating to angle of view control and image quality control to the imaging device 1000. The imaging device 1000 transmits responses to the angle of view control and image quality control commands to the client device 2000.

[0013] FIG. 3 is a diagram showing an example of the configuration and functions of the imaging device 1000 and the client device 2000.

[0014] The imaging device 1000 is, for example, a digital camera capable of converting an optical signal into an electrical signal. The imaging device 1000 includes an imaging unit 1011, an image processing unit 1012, a lens driving unit 1013, a zoom control unit 1014, a pan driving unit 1015, a tilt driving unit 1016, a pan / tilt control unit 1017, a storage unit 1018, a control system 1019, a communication unit 1020, and an IF control unit 1021.

[0015] The imaging unit 1011 captures an image of a subject, converts an optical signal into an electrical signal by photoelectric conversion, and generates an electrical signal corresponding to the image of the subject. Note that, although a person's face is used as an example of the subject, other subjects may be used. For example, the entire human body or a vehicle may be used. As long as the subject is of interest, various subjects can be supported according to the use case.

[0016] The image processing unit 1012 performs image processing and compression encoding processing on the electrical signal generated by the imaging unit 1011 to generate image data. Note that the number of image data generated is not limited to one, and multiple image data with different resolutions and image qualities may be generated simultaneously. The image processing unit 1012 transmits the generated image data to the control system 1019. In the following description, "image" is used as a term indicating both image and image data.

[0017] The lens driver 1013 has a drive system for the focus lens and the zoom lens. The operation of the lens driver 1013 is controlled by the zoom controller 1014.

[0018] The zoom control unit 1014 controls the lens driving unit 1013 based on instructions transmitted from the control system 1019. The zoom control unit 1014 transmits focus movement information and zoom movement information to the control system 1019.

[0019] The pan driving unit 1015 has a mechanical driving system for performing panning operation and a motor as a driving source. The operation of the pan driving unit 1015 is controlled by a pan / tilt control unit 1017.

[0020] The tilt driving unit 1016 has a mechanical driving system for performing tilt operation and a motor as a driving source. The operation of the tilt driving unit 1016 is controlled by a pan / tilt control unit 1017.

[0021] The pan / tilt control unit 1017 controls the pan driving unit 1015 and the tilt driving unit 1016 based on instructions transmitted from the control system 1019. The pan / tilt control unit 1017 transmits pan movement information and zoom movement information to the control system 1019.

[0022] The storage unit 1018 has a main storage device such as a random access memory (RAM) and an auxiliary storage device such as a hard disk drive (HDD) and a solid state drive (SSD). The storage unit 1018 provides, for example, a work area when executing a program, functions as a frame memory, and functions as a buffer memory. The storage unit 1018 stores images, various setting commands, parameters required for executing commands, and the like.

[0023] The control system 1019 is responsible for the overall control of the imaging device 1000. The control system 1019 is a processor including, for example, any one of a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), and a quantum processing unit (QPU). The control system 1019 realizes various functions by reading and developing a program stored in the storage unit 1018. For example, the control system 1019 analyzes a transmitted camera control command and performs processing according to the command. The control system 1019 instructs the image processing unit 1012 to adjust image quality, instructs the zoom control unit 1014 to control zoom and focus, and instructs the pan-tilt control unit 1017 to perform pan-tilt operations. The control system 1019 acquires an image generated by the image processing unit 1012 and transmits it to the communication unit 1020 and the IF control unit 1021. The control system 1019 realizes the function of the output control unit 1022 by reading a program.

[0024] The output control unit 1022 interprets a camera control command related to PTZ (pan-tilt-zoom) that the communication unit 1020 receives from the client device 2000 via the network 3000. Based on the interpretation, the output control unit 1022 executes output control of still images and output control of detection coordinates according to the operation status of the PTZ. The output control unit 1022 will be described as a block in the control system 1019 of the imaging device 1000, but may be included in the image processing unit 1012 or may be possessed by the client device 2000. The output control unit 1022 transmits images and detection coordinate information to the communication unit 1020 and the IF control unit 1021.

[0025] The communication unit 1020 is connected to the network 3000, and mediates communication between the control system 1019 and external devices such as the client device 2000. The communication unit 1020 transmits images transmitted from the control system 1019 to the client device 2000. The communication unit 1020 receives various setting commands and camera control commands transmitted from the client device 2000, and transmits them to the control system 1019. The communication unit 1020 transmits responses of the imaging device 1000 to commands transmitted from the client device 2000 to the client device 2000.

[0026] The IF control unit 1021 outputs images transmitted from the control system 1019 to the outside via each IF, such as HDMI (registered trademark), 3G-SDI, or 12G-SDI, connected to the imaging device 1000. The IF control unit 1021 outputs connectable IF information to the control system 1019.

[0027] A general-purpose computer such as a personal computer, or a mobile terminal such as a tablet is used as the client device 2000. The client device 2000 includes a display unit 2001, an input unit 2002, a system control unit 2003, and a communication unit 2004.

[0028] The display unit 2001 is a display device that displays images, such as a liquid crystal projector, a liquid crystal monitor, etc. The display unit 2001 displays images acquired from the imaging device 1000 and a graphic user interface (hereinafter referred to as GUI) for controlling the camera.

[0029] The input unit 2002 is, for example, a keyboard, or a pointing device such as a mouse or a touch panel. A user of the client device 2000 operates the GUI via the input unit 2002.

[0030] The communication unit 2004 is connected to an external device such as the imaging device 1000 via the network 3000 so as to be able to transmit and receive data to and from the external device. The communication unit 2004 transmits various setting commands and camera control commands transmitted from the system control unit 2003 to the imaging device 1000. The communication unit 2004 transmits images transmitted from the imaging device 1000 and responses of the imaging device 1000 to commands transmitted from the client device 2000 to the system control unit 2003.

[0031] The system control unit 2003 is realized by, for example, a processor. The system control unit 2003 generates various setting commands and camera control commands in response to GUI operations by a user. The system control unit 2003 transmits the generated commands to the imaging device 1000 via the communication unit 2004. The system control unit 2003 receives responses from the imaging device 1000 to the transmitted various setting commands and camera control commands via the communication unit 2004. The system control unit 2003 causes the display unit 2001 to display images received from the imaging device 1000 via the communication unit 2004.

[0032] The client device 2000 executes various camera controls of the imaging device 1000 via a network 3000 .

[0033] First Embodiment The configuration of the first embodiment of the present invention will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the functional configuration of an output control unit 1022A according to this embodiment.

[0034] The output control unit 1022A interprets the camera control command of the PTZ that the communication unit 1020 receives from the client device 2000 via the network 3000, and executes control of still image output and output control of detection coordinates according to the operation status of the PTZ. In this embodiment, the output control unit 1022A is described as a block in the control system 1019 of the imaging device 1000, but it may be included in the image processing unit 1012 or may be in the client device 2000. The output control unit 1022A has a moving state acquisition unit 101A, a face detection unit 102A, an image output unit 103A, and an information output unit 104A. The functions of the moving state acquisition unit 101A, the face detection unit 102A, the image output unit 103A, and the information output unit 104A are realized by the control system 1019 reading a program.

[0035] The movement status acquisition unit 101A determines whether the PTZ movement status is "moving" or "stopped". For example, if the pan / tilt control unit 1017 is controlling either the pan driving unit 1015 or the tilt driving unit 1016, or if the zoom control unit 1014 is controlling the lens driving unit 1013, the movement status acquisition unit 101A determines the PTZ movement status to be "moving", and otherwise determines the PTZ movement status to be "stopped". The movement status acquisition unit 101A outputs the determined PTZ movement status to the image output unit 103A.

[0036] The face detection unit 102A detects a face from the image data input from the image processing unit 1012. The face detection process may be a technique provided by OSS (Open Source Software) such as OpenCV. The face detection process to be performed by the face detection unit 102A is not particularly specified. The face detection unit 102A obtains face coordinate information from a moving image or a still image, and outputs it to the information output unit 104A. Note that, although face detection is taken as an example in this embodiment, any information that indicates a position on an image, such as human body detection, object detection, manual focus guide, or autofocus frame, may be used.

[0037] The image output unit 103A acquires an image, for example a video, from the image processing unit 1012, and outputs either a video or a still image obtained by stopping the video. For example, the image output unit 103A may select and output either a video or a still image depending on whether the angle of view of the imaging unit 1011 has been changed. Specifically, the image output unit 103A may determine whether the angle of view has been changed depending on the PTZ movement state input from the movement state acquisition unit 101A. When the PTZ movement state indicates movement, the image output unit 103A determines that the angle of view has been changed, and outputs the updated video as an image. On the other hand, when the PTZ movement state indicates stop, the image output unit 103A determines that the angle of view has not been changed, and outputs a still image obtained by stopping the video as an image. The method of updating and controlling the output of the image by the image output unit 103A will be described in detail with reference to FIG. 5.

[0038] FIG. 5 is a diagram showing an example of image output during pan movement. The image 200A, 201A, 202A have a transition in the angle of view in this order. Each of the images 200A, 201A, 202A includes an image of a human body 203A, 204A, 205A. The center of the angle of view 206A, 207A, 208A represents the center of the image 200A, 201A, 202A. The coordinate information of the corresponding positions of the center of the angle of view 206A, 207A, 208A in the real space is (0,0), (0,0), (-60,0). The origin position in the real space may be the center position of the image at a specific time, such as when the image capture is started.

[0039] Image 201A is panning, but a still image is being output, so that the view angle center 207A is maintained at the (0,0) position. Image 202A has its imaging direction directed to the left by panning, so that the x coordinate of view angle center 208A is a negative value (here, -60). When a pan movement command is received at the view angle of image 200A, the movement state acquisition unit 101A changes the PTZ movement state to "moving" and starts panning. Since the PTZ movement state is "moving" at the view angle of image 201A, the image output unit 103A stops updating the image and continues to output a still image. When a stop command is received at the view angle position of image 202A, the movement state acquisition unit 101A changes the PTZ movement state to "stopped". As a result, the image output unit 103A resumes updating the image and outputs a moving image. Then, the image output unit 103A outputs the image and the control information including the PTZ movement state to the information output unit 104A.

[0040] The information output unit 104A generates and outputs additional information including at least one of the position and range of the subject in the image. Here, when the information output unit 104A is outputting a moving image, the information output unit 104A outputs the additional information while updating it. As a result, the information output unit 104A outputs additional information corresponding to each frame image of the moving image. On the other hand, when the information output unit 104A is outputting a still image with the moving image stopped, the information output unit 104A stops updating the additional information and outputs the additional information. As a result, the information output unit 104A outputs additional information corresponding to the still image. For example, the information output unit 104A generates and outputs frame information indicating the position and range of a frame surrounding a face as additional information based on face coordinate information input from the face detection unit 102A and image control information input from the image output unit 103A. The position of the frame included in the frame information may be the coordinates of a relative position in the image. In addition, the information output unit 104A may generate and output coordinate information in real space of the center of the angle of view of the image as additional information. The control of the information output unit 104A will be described in detail with reference to FIG. 6.

[0041] FIG. 6 shows the transition of the face frame during pan movement. The images 300A, 301A, and 302A have their angle of view transitioned in this order. The images 300A, 301A, and 302A include images of human bodies 303A, 304A, and 305A. The angle of view centers 306A, 307A, and 308A represent the centers of the images 300A, 301A, and 302A. The coordinate information of the angle of view centers 306A, 307A, and 308A in the real space is (0,0), (0,0), and (-60,0). The angle of view center 307A is panning, but is maintained at the (0,0) position because a still image is output. The frames 309A and 310A are frames surrounding the faces of the detected human bodies 303A and 305A. The frames 309A and 310A are specified by additional information including the plane coordinates of the frame position and the size including the width and height of the frame based on the face detection coordinate information. For example, the additional information of the frames 309A and 310A is expressed as a reference position (x, y) indicating the center position of the frames 309A and 310A, and a value (x, y, width, height) including the width and height. If the top left of the image is (0, 0) and the size of the image is 1920 x 1080, the additional information indicating the position and range of the frames 309A and 310A is, for example, (320, 540, 640, 480) and (1600, 540, 640, 480), respectively. The coordinates of the reference position indicating the center position of the frames 309A and 310A are the coordinates of the relative position of the subject in the image with the top left of the image as the origin.

[0042] When a pan movement command is received in the angle of view of image 300A, movement state acquisition unit 101A changes the PTZ movement state to "moving" and starts moving the angle of view by panning. In order to stop updating the coordinate information of frame 309A that was output at the timing when the movement started, image output unit 103A outputs additional information with a value of 0 (0,0,0,0) or null values ​​(, , , ). Since the PTZ movement state in the state of image 301A is "moving", image output unit 103A outputs a still image without updating the image. Since information output unit 104A does not update the additional information of frame 309A, the frame surrounding human body 304A in image 301A is not displayed.

[0043] When a command to stop pan movement is received at the angle of view position of image 302A, the movement status acquisition unit 101A changes the PTZ movement status to "stopped". As a result, the image output unit 103A resumes updating the image, and a moving image is displayed as in image 302A. The information output unit 104A resumes updating the detected coordinate information as in frame 310A according to the resumed image, and generates additional information (1600, 540, 640, 480). The information output unit 104A outputs the additional information of frame 310A to the communication unit 1020 and the IF control unit 1021. In this embodiment, an example has been given in which the coordinate information of the additional information is a planar coordinate in real space or an image, but the coordinate information may be any of a relative coordinate from the central angle of view, an absolute coordinate, and a spherical coordinate in real space or an image.

[0044] Here, a procedure for output control of detected coordinates corresponding to a PTZ position designated by PTZ movement will be described with reference to the flowchart of Fig. 7. Fig. 7 is a flowchart showing the processing procedure of the output control unit.

[0045] In S101, the movement status acquisition unit 101A determines whether the PTZ movement status is "moving" or "stopped." The movement status acquisition unit 101A may determine the PTZ movement status based on whether the pan / tilt control unit 1017 is controlling the pan driving unit 1015 and the tilt driving unit 1016, or whether the zoom control unit 1014 is controlling the lens driving unit 1013. If the movement status acquisition unit 101A determines that the PTZ movement status is "moving" (YES in S101), it outputs the acquired PTZ movement status to the image output unit 103A and proceeds to S102. If the movement status acquisition unit 101A determines that the PTZ movement status is "stopped" (NO in S101), it repeats S101.

[0046] In S102, the image output unit 103A stops updating the image input from the image processing unit 1012 based on the PTZ movement state indicating "moving" input from the movement state acquisition unit 101A, and outputs a still image and control information including the PTZ movement state to the information output unit 104A. Then, the process proceeds to S103.

[0047] In S103, the information output unit 104A stops updating the additional information of the face frame based on the PTZ movement state indicating "moving" included in the control information, and then the process proceeds to S104.

[0048] In S104, the movement status acquisition unit 101A judges whether the PTZ movement status is "moving" or "stopped." If the movement status acquisition unit 101A judges that the PTZ movement status is "stopped" (YES in S104), that is, if the PTZ movement has reached the target position and ended, it outputs the PTZ movement status to the image output unit 103A and proceeds to S105. If the movement status acquisition unit 101A judges that the PTZ movement status is "moving" (NO in S104), that is, if the PTZ movement is continuing, it repeats S104.

[0049] In S105, the image output unit 103A resumes updating of the image input from the image processing unit 1012 based on the PTZ movement state indicating "stopped" input from the movement state acquisition unit 101A, and outputs the video and control information including the PTZ movement state to the information output unit 104A. Then, the process proceeds to S106.

[0050] In S106, the information output unit 104A resumes updating of the additional information using the face coordinate information output by the face detection unit 102A based on the PTZ movement state indicating "stopped" included in the control information, and outputs the information to the communication unit 1020 and the IF control unit 1021. Then, the process proceeds to S107.

[0051] In S107, the output control unit 1022A judges whether or not the user has input an end of the output control process. If the output control unit 1022A judges that the end has not been input (NO in S107), it proceeds to S101, and if the output control unit 1022A judges that the end has been input (YES in S107), it ends the output control process.

[0052] As described above, in this embodiment, when a still image is output, the updating of the additional information is stopped and the additional information corresponding to the still image is output. Also, in this embodiment, when a video is output, the additional information is updated and output, so that the additional information corresponding to each frame image of the video can be output.

[0053] In this embodiment, when the PTZ movement state is a moving state and the angle of view of the imaging unit 1011 is changed, a still image is output, so that images that you do not want to display during PTZ movement are not displayed and additional information corresponding to the still image can be output.

[0054] In this embodiment, additional information including at least one of the relative position and range within the image of the subject human face is output, so that a frame within the image can be easily generated and displayed.

[0055] In this embodiment, additional information including coordinates indicating the position in real space of the center of the angle of view, which is the center of the image, is output, so that movement of the angle of view in real space, etc., can be easily determined.

[0056] This embodiment outputs additional information including at least one of the relative position and range within the image of the subject human face, i.e., the position and range corresponding to the coordinates within the image displayed on the destination client device 2000, so that a frame within the image can be easily generated and displayed on the client device 2000.

[0057] The embodiment can more flexibly accommodate the requests of the client device 2000 by outputting the position and range corresponding to the spherical coordinates in the image.

[0058] Second Embodiment The configuration of the second embodiment of the present invention will be described with reference to Fig. 8. Fig. 8 is a block diagram showing the functional configuration of an output control unit 1022B according to this embodiment.

[0059] The output control unit 1022B acquires the type of image interface (hereinafter referred to as IF) capable of image output from the IF control unit 1021 and sets still image output control for each distribution IF. The output control unit 1022B interprets the PTZ camera control command received from the client device 2000 for each distribution IF according to the setting. Based on the interpretation, the output control unit 1022B executes control of output of still images according to the PTZ movement state and control of output of additional information including frame information by face detection. The output control unit 1022B is described as a block in the control system 1019 of the imaging device 1000, but may be included in the image processing unit 1012 or may be in the client device 2000. The output control unit 1022B has a movement state acquisition unit 101A, a face detection unit 102A, an image output unit 103B, an information output unit 104B, and an IF output unit 105B.

[0060] The IF output unit 105B sets the image output of each image IF based on the type of image IF that can be delivered input from the IF control unit 1021. The IF output unit 105B sets, for example, whether or not to output a still image during PTZ movement as output information (hereinafter, "necessity setting"). The IF output unit 105B outputs the necessity setting that is set in association with each image IF to the image output unit 103B. The types of image IF include, but are not limited to, HDMI, 3G-SDI, 12G-SDI, etc.

[0061] The image output unit 103B performs update control of the output of image data according to the image output setting of the distribution IF input from the IF output unit 105B, the PTZ movement state input from the movement state acquisition unit 101A, and the PTZ movement state from the image data input from the image processing unit 1012. The update control of the output of the image output unit 103B will be described in detail with reference to Figs. 9 and 10.

[0062] FIG. 9 shows the operation when the type of image IF is set to output a still image during PTZ movement by HDMI. The image 200B, 201B, and 202B have their angles of view transitioned in this order. The images 200B, 201B, and 202B include human bodies 203B, 204B, and 205B as subjects. The image angle centers 206B, 207B, and 208B represent the centers of the images 200B, 201B, and 202B. The coordinate information of the real space of the image angle centers 206B, 207B, and 208B is (0,0), (0,0), and (-60,0). The image 201B is panning, but the image angle center 207B is a still image at the coordinates (0,0). When a pan movement command is received at the image angle of 200B, the movement state acquisition unit 101A changes the PTZ movement state to "moving" and starts the movement. Since the PTZ movement status is "moving" in the angle of view of image 201B, image output unit 103B stops updating the image data based on the necessity setting, and outputs a still image and control information including the necessity setting, the PTZ movement status, and the type of each image IF to information output unit 104B. When a command to stop pan movement is received at the angle of view position of image 202B, movement status acquisition unit 101A changes the PTZ movement status to "stopped", and image output unit 103B resumes updating the image and outputs the video together with control information including the necessity setting, the PTZ movement status, and the type of each image IF to information output unit 104B.

[0063] 10 shows the operation when the type of image IF is HDMI and the necessity setting is set to not output still images during PTZ movement. The image angles of view of images 300B, 301B, and 302B change in this order. The images 300B, 301B, and 302B include human bodies 303B, 304B, and 305B as subjects. The image angle centers 306B, 307B, and 308B represent the centers of the images 300B, 301B, and 302B. The coordinate information of the real space of the image angle centers 306B, 307B, and 308B are (0,0), (-30,0), and (-60,0). When a pan movement command is received at the image angle of image 300B, the movement state acquisition unit 101A changes the PTZ movement state to "moving" and starts the movement. In the angle of view of image 301B, the PTZ movement status is "moving", but image output unit 103B continues updating the image based on the necessity setting and outputs the video being captured. When a command to stop pan movement is received in the angle of view position of image 302B, movement status acquisition unit 101A changes the PTZ movement status to "stopped". Image output unit 103B outputs the video being updated and control information including the necessity setting, the PTZ movement status, and the type of each image IF to information output unit 104B.

[0064] The information output unit 104B outputs additional information including the coordinates of a frame surrounding a human face, based on the face coordinate information input from the face detection unit 102A and the control information including the necessity setting set for each image IF input from the image output unit 103B. The information output unit 104B will be described in detail with reference to Figs. 11 and 12.

[0065] FIG. 11 shows the operation when the type of image IF is set to output still image output during PTZ movement with HDMI. Images 400B, 401B, and 402B have their angles of view transitioned in this order. Images 400B, 401B, and 402B include human bodies 403B, 404B, and 405B as subjects. Centers of angle of view 406B, 407B, and 408B represent the centers of images 400B, 401B, and 402B. The coordinate information of the centers of angle of view 406B, 407B, and 408B in real space is (0,0), (0,0), and (-60,0). Image 407B is a still image during panning movement, but the coordinates of the center of angle of view are at (0,0). Frames 409B and 410B are frames surrounding the faces of detected human bodies 403B and 405B. Frames 409B and 410B are specified by additional information of plane coordinates and size based on the face detection coordinate information. For example, the additional information of frames 409B and 410B is expressed as a reference position (x, y) indicating the center position of frames 409B and 410B, and a value (x, y, width, height) including width and height. If the top left of the image is (0, 0) and the image size is 1920 x 1080, the additional information of frames 409B and 410B is, for example, (320, 540, 640, 480) and (1600, 540, 640, 480), respectively.

[0066] When a pan movement command is received in the angle of view of image 400B, movement state acquisition unit 101B changes the PTZ movement state to "moving" and starts moving the angle of view by panning. In order to stop updating the coordinate information of frame 409B that was output at the timing when the movement started, image output unit 103B outputs additional information with a value of 0 (0,0,0,0) or null values ​​(, , , ). Since the PTZ movement state in the state of image 401B is "moving", image output unit 103B outputs a still image without updating the image. Since information output unit 104B does not update the additional information of frame 409B, the frame surrounding human body 404B in image 401B is not displayed.

[0067] When a command to stop pan movement is received at the angle of view position of image 402B, movement status acquisition unit 101B changes the PTZ movement status to "stopped." This causes image output unit 103B to resume updating of the image, and a moving image is displayed as in image 402B. Information output unit 104B resumes updating of the detected coordinate information as in frame 410B in accordance with the resumed image, and generates additional information (1600, 540, 640, 480).

[0068] 12 shows the operation when the type of image IF is set to HDMI and not to output still images during PTZ movement. Images 500B, 501B, and 502B have transitions in the angle of view in this order. Images 500B, 501B, and 502B include human bodies 503B, 504B, and 505B as subjects. Centers of angle of view 506B, 507B, and 508B represent the centers of images 500B, 501B, and 502B. The real space coordinate information of centers of angle of view 506B, 507B, and 508B is (0,0), (-30,0), and (-60,0).

[0069] Frames 509B, 510B, and 511B are frames surrounding the faces of the detected human bodies 503B and 505B. Frames 509B, 510B, and 511B are specified by additional information of plane coordinates and size based on face detection coordinate information. For example, the additional information of frames 509B, 510B, and 511B is expressed as a reference position (x, y) indicating the center position of frames 509B, 510B, and 511B, and a value (x, y, width, height) including width and height. If the top left of the image is (0, 0) and the image size is 1920x1080, the additional information of frames 509B, 510B, and 511B is, for example, (320, 540, 640, 480), (1600, 540, 640, 480), and (1280, 540, 640, 480), respectively.

[0070] When a pan movement command is received at the angle of view of image 500B, movement status acquisition unit 101B changes the PTZ movement status to "moving" and starts moving the angle of view by panning. Even after the movement starts, image output unit 103B updates the image and outputs a video. Information output unit 104B continues updating the additional information of frame 511B, and outputs (1280, 540, 640, 480) as the additional information.

[0071] When a command to stop pan movement is received at the angle of view position of image 502B, movement status acquisition unit 101B changes the PTZ movement status to "stopped." Since image output unit 103B continues updating the image, the moving image of image 502B is displayed. Information output unit 104B updates and generates additional information (1600, 540, 640, 480) in accordance with the image being updated. Information output unit 104B outputs the additional information of frame 510B to communication unit 1020 and IF control unit 1021.

[0072] The other blocks are similar to those in the first embodiment, and therefore the description thereof will be omitted.

[0073] The output of additional information during PTZ movement will be described with reference to the flowchart in FIG.

[0074] In S201, the IF output unit 105B performs a setting as to whether or not to output a still image in association with the type of deliverable image IF input from the IF control unit 1021. The IF output unit 105B outputs the setting as to whether or not to output each image IF to the image output unit 103B, and proceeds to S202.

[0075] In S202, the movement status acquisition unit 101A determines whether the PTZ movement status is "moving" or "stopped" depending on whether the pan / tilt control unit 1017 is controlling the pan driving unit 1015 and the tilt driving unit 1016, or whether the zoom control unit 1014 is controlling the lens driving unit 1013. If the movement status acquisition unit 101A determines that the PTZ movement status is "moving" (YES in S202), it outputs the PTZ movement status to the image output unit 103B and proceeds to S203. If the movement status acquisition unit 101A determines that the PTZ movement status is "stopped" (NO in S202), it repeats S202.

[0076] In S203, the output control unit 1022B repeats the processes from S204 to S207 the number of times corresponding to the number of types of image IF that can be delivered and are input from the IF control unit 1021.

[0077] In S204, the image output unit 103B determines whether or not a still image is to be output during PTZ movement based on the necessity setting set in the IF output unit 105B for the image IF to be determined. If the image output unit 103B determines that a still image is to be output during PTZ movement (YES in S204), the process proceeds to S205. If the image output unit 103B determines that a still image is not to be output during PTZ movement (NO in S204), the process proceeds to S207 without stopping the update of the image and additional information.

[0078] In S205, the image output unit 103B stops updating the image, and outputs the still image together with control information including the PTZ movement state, necessity setting, and type of image IF associated with the image IF to the information output unit 104B.

[0079] In S206, the information output unit 104B stops updating the additional information of the frame surrounding the face based on the information indicating that a still image is to be output, which is indicated by the necessity setting included in the control information.

[0080] In S207, the output control unit 1022B repeats the process the number of times corresponding to the number of types of deliverable image IFs input in S203 from the IF control unit 1021. After repeating the process the necessary number of times, the output control unit 1022B proceeds to S208.

[0081] In S208, the movement status acquisition unit 101A determines whether the PTZ movement status is "moving" or "stopped" depending on whether the pan / tilt control unit 1017 is controlling the pan driving unit 1015 and the tilt driving unit 1016, or whether the zoom control unit 1014 is controlling the lens driving unit 1013. If the movement status acquisition unit 101A determines that the PTZ movement status is "stopped" (YES in S208), it outputs the PTZ movement status to the image output unit 103B and proceeds to S209. If the PTZ movement status is "moving" (NO in S208), the movement status acquisition unit 101A repeats S208.

[0082] In S209, the output control unit 1022B repeats the processes from S210 to S213 the number of times corresponding to the number of types of image IF that can be delivered and are input from the IF control unit 1021.

[0083] In S210, if the necessity setting indicating the necessity of outputting a still image during PTZ movement, which is set in association with the type of image IF to be determined in the IF output unit 105B, is to output a still image (YES in S210), the image output unit 103B proceeds to S211. If the necessity setting is not to output a still image (NO in S210), the image output unit 103B proceeds to S213.

[0084] In S211, the image output unit 103B resumes updating of the stopped image to generate a moving image. The image output unit 103B outputs the moving image and control information including the PTZ movement state, necessity setting, and type of image IF associated with the image IF, to the information output unit 104B. Then, the process proceeds to S212.

[0085] In S212, the information output unit 104B resumes updating of the additional information using the face coordinate information output by the face detection unit 102A based on the PTZ movement state indicating "stopped" included in the control information, and outputs the information to the communication unit 1020 and the IF control unit 1021. Then, the process proceeds to S213.

[0086] In S213, the output control unit 1022B repeats the process the number of times corresponding to the number of types of deliverable image IFs input in S210 from the IF control unit 1021. After repeating the process the necessary number of times, the output control unit 1022B proceeds to S214.

[0087] In S214, the output control unit 1022B determines whether or not a command to end the output control process has been input by a user operation. If the output control unit 1022B determines that the command to end has not been input (NO in S214), it proceeds to S202, and if the output control unit 1022B determines that the command to end has been input (YES in S214), it ends the output control process.

[0088] As described above, in this embodiment, either a moving image or a still image is output based on the necessity setting associated with each of the multiple image IFs that output images. This allows the embodiment to flexibly and appropriately output a moving image or a still image for each image IF.

[0089] In this embodiment, since it is determined whether or not the additional information needs to be updated based on the necessity setting, it is possible to more appropriately update the additional information.

[0090] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention.

[0091] For example, in the configuration of the first embodiment, while updating of the additional information is stopped in the information output unit 104A, instead of outputting the additional information (0, 0, 0, 0), the additional information (320, 540, 640, 480) before the PTZ movement state became “moving” may continue to be output.

[0092] (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0093] For example, in the above embodiment, the coordinates in real space of the center of the angle of view in the image are included in the additional information. However, the range in real space indicating the range of the entire image may be included in the additional information.

[0094] In the above embodiment, the additional information including frame information related to the face detection result is exemplified, but the additional information may include information based on at least one of the pupil detection result, autofocus, and manual focus. This allows the embodiment to handle various types of additional information.

[0095] In the above embodiment, the additional information including frame information related to the face detection result is exemplified, but the additional information may include information related to the angle of view of the imaging unit such as a PTZ movement state, etc. This allows the additional information to provide information related to the angle of view to a client device, etc., and the client device can generate a viewpoint in a virtual space, etc.

[0096] The disclosure of this specification includes the following imaging device, imaging device control method, and program. (Item 1) an image output means for acquiring an image generated by the imaging means by imaging a subject, and outputting either a moving image or a still image obtained by stopping the moving image; and an information output means for, when the video is being output, updating and outputting additional information including at least one of the position and range of the subject, and, when the still image is being output, stopping the updating of the additional information and outputting the additional information. (Item 2) 2. The imaging device according to item 1, wherein the image output means outputs the still image when the angle of view of the imaging means is changed. (Item 3) The imaging device according to item 1 or 2, wherein the image output means outputs either the moving image or the still image based on output information associated with each of a plurality of interfaces that output the image. (Item 4) The imaging device according to any one of items 1 to 3, characterized in that the information output means determines whether or not it is necessary to update the additional information when the still image is being output, based on output information associated with each of a plurality of interfaces that output the image. (Item 5) 5. The imaging device according to claim 1, wherein the information output means outputs the additional information including at least one of a relative position and a range of the subject within the image. (Item 6) 6. The imaging device according to claim 1, wherein the information output means outputs the additional information including at least one of a position and a range corresponding to the image in real space. (Item 7) The imaging device according to any one of items 1 to 6, characterized in that the information output means outputs the additional information including at least one of a position and a range corresponding to coordinates in the image displayed at an output destination. (Item 8) 8. The imaging device according to any one of claims 1 to 7, wherein the information output means outputs the additional information including at least one of a position and a range corresponding to the image in spherical coordinates of real space. (Item 9) 9. The imaging device according to claim 1, wherein the information output unit outputs the additional information corresponding to the still image when the still image is being output. (Item 10) The imaging device according to any one of items 1 to 9, wherein the information output means outputs the additional information including information based on at least one of a face detection result, an eye detection result, autofocus, and manual focus. (Item 11) 11. The imaging device according to claim 1, wherein the information output unit outputs the additional information including information related to an angle of view of the imaging unit. (Item 12) an image output step of acquiring an image generated by imaging a subject with an imaging means, and outputting either a moving image or a still image obtained by stopping the moving image; an information output process for, when the video is being output, updating and outputting additional information including at least one of the position and range of the subject, and, when the still image is being output, stopping the updating of the additional information and outputting the additional information. (Item 13) A program that, when read and executed by a computer, causes the computer to execute each step of the method for controlling an imaging device according to item 12.

[0097] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Therefore, the claims are attached to publicize the scope of the invention. Also, each embodiment may be combined. [Explanation of symbols]

[0098] 1000 ... imaging device, 1019 ... control system, 1022 ... output control section, 103A ... image output section, 104A ... information output section, 1022B ... output control section, 103B ... image output section, 104B ... information output section.

Claims

1. an image output means for acquiring an image generated by the imaging means by imaging a subject, and outputting either a moving image or a still image obtained by stopping the moving image; and an information output means for, when the video is being output, updating and outputting additional information including at least one of the position and range of the subject, and, when the still image is being output, stopping the updating of the additional information and outputting the additional information.

2. 2. The imaging apparatus according to claim 1, wherein the image output means outputs the still image when the angle of view of the imaging means is changed.

3. 2. The imaging device according to claim 1, wherein the image output unit outputs either the moving image or the still image based on output information associated with each of a plurality of interfaces for outputting the image.

4. The imaging device according to claim 1, characterized in that the information output means determines whether or not it is necessary to update the additional information when the still image is being output, based on output information associated with each of a plurality of interfaces that output the image.

5. 2. The imaging apparatus according to claim 1, wherein the information output means outputs the additional information including at least one of a relative position and a relative range of the subject within the image.

6. 2. The imaging device according to claim 1, wherein the information output means outputs the additional information including at least one of a position and a range corresponding to the image in real space.

7. 2. The imaging device according to claim 1, wherein the information output means outputs the additional information including at least one of a position and a range corresponding to coordinates in the image displayed on an output destination.

8. 2. The imaging device according to claim 1, wherein the information output means outputs the additional information including at least one of a position and a range corresponding to the image in spherical coordinates of real space.

9. 2. The imaging apparatus according to claim 1, wherein, when the still image is being output, the information output unit outputs the additional information corresponding to the still image.

10. 2. The image pickup apparatus according to claim 1, wherein the information output unit outputs the additional information including information based on at least one of a face detection result, an eye detection result, autofocus, and manual focus.

11. 2. The imaging apparatus according to claim 1, wherein the information output means outputs the additional information including information relating to an angle of view of the imaging means.

12. an image output step of acquiring an image generated by imaging a subject with an imaging means, and outputting either a moving image or a still image obtained by stopping the moving image; an information output process for, when the video is being output, updating and outputting additional information including at least one of the position and range of the subject, and, when the still image is being output, stopping the updating of the additional information and outputting the additional information.

13. A program for causing a computer to execute each step of the method for controlling an imaging apparatus according to claim 12 by reading and executing the program.

Images