Video output device and method, program and storage medium

Abstract

To select a priority area and also select a subject easily and speedily.SOLUTION: A video output device has: input means for inputting a video; detecting means for detecting an operation performed on the video; setting means for setting a frame indicating a partial video as part of the video according to the operation; display control means for displaying the video and the frame one over the other on display means; selecting means for selecting one of displayed frames according to the operation; output means for outputting a partial video of the frame selected by the selecting means; and control means for performing control corresponding to the selected subject according to the operation, wherein the selecting means selects the frame including the selected subject when a plurality of frames are set.SELECTED DRAWING: Figure 5C

Description

[Technical field]

[0001] The present invention relates to a video output device and method, a program, and a storage medium, and in particular to a technique for selecting and outputting a partial area of ​​a video captured by one imaging unit. [Background technology]

[0002] The crop function in video equipment is a function that cuts out a part of the entire input video and outputs it. By making the crop range changeable, you can change the composition and angle of view of the output video without performing pan / tilt or zoom operations on the input source camera.

[0003] Patent Document 1 discloses a digital camera that can simultaneously capture images of multiple regions from a captured image. Non-Patent Document 1 discloses a camera that can set up to three regions as crop candidate regions for an input image and crop one selected region from the three regions before outputting the image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2005-175683 A [Non-patent literature]

[0005] [Non-Patent Document 1] Panasonic UE-AW100 Instruction Manual (Japanese) Summary of the Invention [Problem to be solved by the invention]

[0006] Focus control in a camera is a process of controlling the lens so that the focus is set at a specified subject distance from the imaging unit. When multiple crop areas are set in an image captured by one imaging unit, it is not possible to focus on the main subject in all of those areas unless all of the main subjects are equidistant from the imaging unit.

[0007] Therefore, it is practical to select one of the multiple crop areas as a priority area and focus on the main subject in that area. However, when changing the priority area, even if the AF is controlled to follow the change in the priority area, for example, if the switched-to area contains multiple subjects, it is not always possible to correctly determine and focus on the subject desired by the user in the switched-to area. Therefore, after changing the priority area, the user is required to at least check the focus status of the switched-to area and perform further focus operations as necessary.

[0008] In order to minimize the time that the focus becomes unstable when the priority area is changed, the above operations must be performed quickly and in sequence, which places a heavy burden on the user. Also, in a system in which the imaging unit that controls the focus and the switcher unit that selects the priority area are separate, it is difficult to perform the above operations in sequence and in sequence.

[0009] The above problem occurs not only in focus control, but also in exposure control, white balance control, image processing control, and the like, in which the control content is determined based on the main subject.

[0010] The present invention has been made in consideration of the above problems, and an object of the present invention is to make it possible to easily and quickly select a priority area and a subject. [Means for solving the problem]

[0011] In order to achieve the above-mentioned object, the video output device of the present invention has an input means for inputting video, a detection means for detecting an operation performed on the video, a setting means for setting a frame indicating a partial image that is part of the video in response to the operation, a display control means for superimposing the video and the frame on a display means, a selection means for selecting one of the displayed frames in response to the operation, an output means for outputting the partial image of the frame selected by the selection means, and a control means for performing control in response to a subject selected in response to the operation, and when multiple frames are set, the selection means selects a frame that includes the selected subject. Effect of the Invention

[0012] According to the present invention, it is possible to easily and quickly select a priority area and a subject. [Brief description of the drawings]

[0013] [Figure 1] 1 is a block diagram showing the configuration of a camera according to a first embodiment of the present invention. [Diagram 2] FIG. 4 is a diagram showing an example of an image displayed on a screen in the first embodiment. [Diagram 3] FIG. 4 is a diagram showing an example of an image in the first embodiment. [Figure 4] 5 is a flowchart showing the operation of the camera in the first embodiment. [Figure 5A] 5 is a flowchart of a menu setting process according to the first embodiment. [Figure 5B] 5 is a flowchart of movement control of a cropping frame in the first embodiment. [Figure 5C] 6 is a flowchart of focus control and selection processing of a cropping frame to be output in the first embodiment. [Figure 6] FIG. 2 is a view showing an example of a menu display according to the first embodiment. [Figure 7] FIG. 11 is a block diagram showing the configuration of a switcher according to a second embodiment. [Figure 8]FIG. 11 is a conceptual diagram showing the operation of a switcher according to the second embodiment. [Figure 9] 10 is a flowchart showing the operation of a switcher in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0015] <First embodiment> FIG. 1 is a block diagram showing the configuration of a camera 100 as an example of an imaging device according to the first embodiment.

[0016] The lens 101 includes a focus mechanism, a zoom mechanism, an aperture, and an image stabilization mechanism, and performs lens control such as focus lens drive, zoom drive, aperture drive, and image stabilization lens drive based on control commands received from the system control unit 109. The lens 101 may be built into the camera 100, or may be an external lens attached to the camera 100 via a lens mount (not shown).

[0017] The imaging unit 102 includes an imaging sensor that converts an optical image collected by the lens 101 into an electrical signal, and an A / D converter that converts an analog signal into a digital signal. The imaging sensor captures images at a predetermined frame rate.

[0018] The image processing unit 103 inputs image data at a predetermined frame rate from the imaging unit 102 or the image input unit 107, and performs predetermined resizing, trimming, color conversion, subject distance calculation, and distortion correction. The image processing unit 103 then uses the volatile memory 116 as a VRAM to store the processed image data as a full-angle image. Alternatively, the RAW data before these image processes can be output to the volatile memory 116.

[0019] Furthermore, the image processing unit 103 performs a predetermined calculation process using the image data obtained from the imaging unit 102, and the system control unit 109 performs exposure control, focus adjustment control, vibration reduction control, and AWB (auto white balance) control based on the obtained calculation results. The predetermined calculation process includes processing for detecting subjects such as faces. This allows AF (autofocus) processing, AE (auto exposure) processing, vibration reduction processing, and AWB processing to be performed.

[0020] Furthermore, the video processing unit 103 can generate a partial video by cropping (cutting out) the full-angle video within a coordinate range based on an instruction from the system control unit 109. Furthermore, the cropped partial video can be resized so that it has a resolution suitable for output from the video output unit 104.

[0021] The OSD drawing unit 113 renders character strings and icons indicating the state and settings of the camera 100, menu screens, and the like, in the VRAM on the volatile memory 116. Characters and icons are stored in the non-volatile memory 117, and the OSD drawing unit 113 reads them out and renders them in the VRAM.

[0022] The video output unit 104 reads out the video data recorded in the volatile memory 116 and the data rendered by the OSD drawing unit 113 as necessary, superimposes them, converts them into a video signal, and outputs it to each of the output terminal 105, the display 106, and the communication control unit 118. Alternatively, the video output unit 104 can output the RAW data as it is to the output terminal 105. The video output unit 104 also converts the associated information received from the system control unit 109 into metadata, and outputs it to the output terminal 105 together with the video. The metadata can be linked to the video data for each frame, and can express the associated information for each frame, such as a time code.

[0023] The output terminal 105 is for outputting video to an external device, and can output a video signal to an external monitor or recording device connected via an HDMI (registered trademark) cable, SDI cable, etc. (not shown). Metadata output from the video output unit 104 can also be output to an external device via the respective cables.

[0024] The display 106 is composed of, for example, an LCD panel, and is display-controlled by the video output unit 104. The display 106 mainly displays an image in which data drawn by an OSD drawing unit 113 is superimposed on the video data processed by the video processing unit 103.

[0025] Video input unit 107 can input a video signal equivalent to the video captured by lens 101 and imaging unit 102 from the outside via input terminal 108. Therefore, by omitting lens 101 and imaging unit 102, camera 100 can be considered as an effecter device that only inputs, outputs, and processes video, and the present invention can be applied in this case as well. The input terminal 108 is for inputting video from an external device, and is configured by an HDMI terminal, an SDI terminal, or the like.

[0026] The system control unit 109 controls the entire camera 100. It reads out and executes a program recorded in the non-volatile memory 117 to control each unit and realize each operation of this embodiment. The system control unit 109 may include multiple CPU cores. In that case, the tasks described in the program can be shared and processed by the multiple CPU cores. It also includes a system timer for controlling the timing of each process.

[0027] The operation unit 110 includes a menu key, a cross key, a touch panel, and the like for operating a menu screen on an image displayed on the display 106 and for selecting a range of a cropped image (partial image) described later. The touch panel is configured integrally with the display 106, and can issue various instructions to subjects and display elements displayed on the display 106 by detecting user operations such as tapping and dragging. The operation input terminal 111 is for controlling the camera 100 from an external controller, and notifies the system control unit 109 of a control signal via the operation input terminal 111. The system control unit 109 detects the content of the operation based on the control signal, and can perform the same operation as that of the operation unit 110.

[0028] A power switch 112 is used to select the power state of the camera 100. Depending on the position of the switch, the power state is determined as either on or off, and notified to the system control unit 109.

[0029] A recording medium 115 such as an HDD or SSD is connected to the recording medium control unit 114, and reads data from and writes data to the recording medium 115. The recording medium 115 is typified by an HDD or SSD, and records video and associated metadata. The recording medium 115 may be connected to a removable non-volatile memory such as an SD card via a socket (not shown) or the like.

[0030] Volatile memory 116 is used as a dynamic memory area for video data or RAW data handled by video processing unit 103, video output unit 104, and OSD drawing unit 113, and for programs used by system control unit 109. Volatile memory 116 also has a storage capacity sufficient to store moving images for a predetermined period of time.

[0031] The non-volatile memory 117 is an electrically erasable and recordable memory, and may be, for example, an EEPROM. Constants, programs, and the like for the operation of the system control unit 109 are recorded in the non-volatile memory 117. The programs referred to here include programs for executing various flowcharts described later in this embodiment.

[0032] The communication control unit 118 is used to communicate with an external device. Either or both of the network connection terminal 119 and the wireless network connection unit 120 enable wireless / wired IP communication. The communication control unit 118 transmits video from the video output unit 104 to an external device, thereby enabling monitoring of the video via IP communication. In addition, by receiving a control signal generated by an operation of an external controller, the communication control unit 118 can notify the system control unit 109 of an operation request corresponding to an operation that can be performed by the operation unit 110. Alternatively, the communication control unit 118 can receive a setting change request for directly changing the setting of a setting item on a menu screen displayed on the display 106, and notify the system control unit 109 of the request.

[0033] The network connection terminal 119 is for performing wired network communication by the communication control unit 118, and the wireless network connection unit 120 is for performing wireless network communication by the communication control unit 118. The wireless network connection unit 120 may be configured to connect a detachable antenna such as a WiFi dongle via a socket or the like (not shown).

[0034] 2(a) shows an example of an image displayed on the display 106 or an external controller when a partial image is generated by cropping the full-angle image captured by the image processing unit 103 within a coordinate range based on an instruction from the system control unit 109. The full-angle image can be enlarged or reduced and displayed on the display 106 with a predetermined area as a reference.

[0035] Crop frame 201 indicates the image range of the cropped image (partial image) being output to output terminal 105. Crop frame 202 indicates that it is an object of operation, and the image range of crop frame 202 is not output to output terminal 105. The number of crop frames that can be set is not limited to two, and may be set arbitrarily by the user. When multiple crop frames are set, camera 100 can output a cropped image of the image range of at least one of the multiple crop frames to output terminal 105. At this time, the crop frame indicating the image range of the cropped image being output to output terminal 105 and the crop frame indicating the image range not being output to output terminal 105 may be displayed in different forms, such as solid and dotted lines, or by changing the frame color.

[0036] FIG. 2(b) shows an example of a cropped image output from the output terminal 105 when the cropping frame 201 and the cropping frame 202 are set in the display state shown in FIG. 2(a). FIG. 3 is a diagram showing the video data handled by the camera 100 when the video data 304 is output from the camera 100, and the video displayed on the display 106 or an external controller.

[0037] Full-angle video data 300 is input to the video processing unit 103 from the imaging unit 102 or the video input unit 107. Reference numeral 301 denotes the external controller or the display 106, and reference numeral 302 denotes an example of a video image displayed on the display unit.

[0038] The video data 303 is cropped video data that has been cropped and resized by the video processing unit 103 to be output to the output terminal 105 and stored in the VRAM of the volatile memory 116. The video data 303 stored in the VRAM of the volatile memory 116 can be output to the outside as a video output from the output terminal 105, the network connection terminal 119, and the wireless network connection unit 120. Furthermore, the video data 303 can be recorded on the recording medium 115.

[0039] The video data 304 is outputted to the outside as a video output from the output terminal 105, the network connection terminal 119, and the wireless network connection unit 120.

[0040] 4 is a flowchart showing an example of the operation of the camera 100 according to this embodiment. The processing of this flowchart and the processing of sub-flowcharts called from this flowchart are realized by the system control unit 109 controlling each unit of the camera 100 based on a program stored in the non-volatile memory 117. Furthermore, a user operation occurring in the processing of this flowchart and the sub-flowcharts occurs when the system control unit 109 detects an operation performed by the user on the operation unit 110. Alternatively, as shown in FIG. 3, the user operation may occur when the system control unit 109 detects a control signal received from the operation input terminal 111 or the communication control unit 118 using an external controller.

[0041] When the process starts, an image is input in S400. The image input here may be an image captured by the imaging unit 102 or an image input from the input terminal 108 via the image input unit 107.

[0042] Next, in S401, it is determined whether or not the menu key has been pressed. If the menu key has been pressed, the process proceeds to S402, and if not, the process proceeds to S403. In S402, menu settings are performed. Details of the process in S402 will be described later with reference to the flowchart shown in FIG.

[0043] In S403, the menu setting stored in the volatile memory 116 is read to determine whether the crop output is enabled. If the crop output is enabled, the process proceeds to S404, and if the crop output is not enabled, the process proceeds to S408. In S408, the video output unit 104 is controlled to output the full-angle video.

[0044] On the other hand, in S404, an image as shown in Fig. 2(a) is displayed. The image displayed here can be realized by superimposing data drawn in the VRAM on the volatile memory 116 by the OSD drawing unit 113 on all image images stored in the volatile memory 116 and displaying the superimposed data on the display 106. Alternatively, the logical information for displaying the image as shown in Fig. 2(a) may be configured to be displayed on an external display device via the network connection terminal 119 or the wireless network connection unit 120.

[0045] Then, in S405, the movement of the cropping frame is controlled. Details of the control in S405 will be described later with reference to the flowchart in FIG. In S406, focus control and selection control of a cropping frame to be output are performed. Details of the control in S406 will be described later with reference to the flowchart in FIG. 5C.

[0046] In S407, information on the cropping frame to be output selected in S406 is read from the volatile memory 116, and based on this, the video output unit 104 is controlled to crop and output the full-angle video output by the video processing unit 103 as shown in Fig. 2(b). However, if it is determined in S406 that the full-angle video is to be output, the full-angle video is output without performing cropping.

[0047] FIG. 5A is a flowchart showing the menu setting process performed in S402 of FIG. In S500, a menu item selection screen shown in Fig. 6(a) is displayed. The selection screen can be realized by displaying on the display 106 a menu screen drawn by the OSD drawing unit 113 in the VRAM on the volatile memory 116. Alternatively, the logical information for displaying the menu screen may be configured to be displayed on an external display device via the network connection terminal 119 or the wireless network connection unit 120. Reference numeral 601 denotes a menu display in which setting items are arranged in a row. Reference numeral 602 denotes a cursor display showing the selection state of a setting item, which can be moved to another setting item by up and down operation or touch panel operation on the operation unit 110.

[0048] In S501, it is determined whether or not the menu key has been pressed. If the menu key has been pressed, the process proceeds to S502, and if not, the process proceeds to S503. In S502, the menu item selection screen is closed and the process returns to FIG.

[0049] Meanwhile, in S503, it is determined whether or not "Crop Output" has been selected from among the menu items. "Crop Output" is an item for setting whether or not to use the crop output function in camera 100. The "Crop Output" setting consists of two options, "On" and "Off," and if it is "On," a crop frame is displayed on display 106. If "Crop Output" has been selected, the process proceeds to S504, and if not, the process proceeds to S507.

[0050] In S504, the "Crop output" setting screen shown in Fig. 6(b) is displayed. In Fig. 6(b), 603 indicates the item being set, and 604 indicates a cursor indicating the selection state of the setting. The cursor 604 can be moved to another setting by up / down operation on the operation unit 110 or by touch panel operation.

[0051] In S505, it is determined whether the "crop output" setting has been changed, and if the setting has been changed, the process proceeds to S506, and if not, the process returns to S500. In S506, the result changed in S505 is stored in the volatile memory 116, and the process returns to S500.

[0052] In S507, it is determined whether or not "cropping frame priority order" has been selected from among the menu items. If "cropping frame priority order" has been selected, the process proceeds to S508, and if not, the process proceeds to S511. "Cropping frame priority order" sets the method for determining the priority order of cropping frames when a subject is selected in the process described below and the subject exists within multiple cropping frames. By providing this setting, the user can select the method for determining the priority order according to the shooting conditions. Possible selectable items are, for example, as follows:

[0053] "By cropping frame number": Determines the priority order based on the number pre-assigned to each cropping frame. With this setting, the priority cropping frame is pre-determined, making it easier for the user to predict the selection results. "Selection History": Prioritize the selection of the most recently used cropping frame. This setting allows you to select and output the most desirable frame based on your cropping history. "By distance": The cropping frame whose center coordinates are closest to the coordinate values ​​obtained by the subject selection operation is given priority for selection. This setting allows you to determine the cropping frame to be output based on the priority order that you would like to use, based on distance.

[0054] In S508, a setting screen for "cropping frame priority" is displayed. Here, for example, options are displayed indicating the method of determining the priority of cropping frames, such as the above-mentioned "cropping frame number order," "selection history order," and "distance order." The user can select the desired determination method by moving the cursor by up and down operations on the operation unit 110 or by operating the touch panel.

[0055] In S509, it is determined whether the "cropping frame priority order" setting has been changed, and if the setting has been changed, the process proceeds to S510, and if not, the process returns to S500. In S510, the result changed in S509 is stored in the volatile memory 116, and the process returns to S500.

[0056] In S511, it is determined whether or not "Action when subject is reselected" is selected from among the menu items. If "Action when subject is reselected" is selected, the process proceeds to S512, and if not, the process proceeds to S515. "Action when subject is reselected" is an item for setting the action to be taken when a subject that exists within multiple cropping frames is selected once and then the same subject is selected again. Possible selectable items are, for example, the following:

[0057] "Do not change cropping frame": The cropping frame for the output will not change even if a reselection operation is performed. For ambiguous operations such as selecting a subject that exists within multiple cropping frames, you can prioritize maintaining the current state without arbitrarily switching the cropping frame for the output. "Change cropping frame": When a reselection operation is performed, the cropping frame for output is changed to the frame with the next highest priority. This proactively captures the ambiguous operation of selecting a subject that exists within multiple cropping frames, and prioritizes changing the cropping frame for output.

[0058] In S512, a setting screen for "Action when reselecting subject" is displayed. Here, for example, the above-mentioned options of "Do not change cropping frame" and "Change cropping frame" are displayed. The user can select one of them by moving the cursor up and down using the operation unit 110 or by operating the touch panel.

[0059] In S513, it is determined whether the setting of "action when reselecting subject" has been changed, and if it has been changed, the process proceeds to S514, and if it has not been changed, the process returns to S500. In S514, the result changed in S513 is stored in the volatile memory 116, and the process returns to S500.

[0060] In S515, it is determined whether or not "Action when outside crop frame is selected" has been selected from the menu items. If "Action when outside crop frame is selected" has been selected, the process proceeds to S516, and if not, the process returns to S500. "Action when outside crop frame is selected" is an item that sets the action to be taken when a subject that is not included in any crop frame is selected. By providing "Action when outside crop frame is selected," the user can select whether to prioritize prevention of erroneous operation or to prioritize switching to a full-angle image. Possible selectable items are, for example, as follows:

[0061] "Do not change": The cropping frame or focus control criteria for the output target will not change even if a selection operation is performed. This prevents the output image from being in an undesired state when a selection operation is performed on an area outside the cropping frame by mistake. "Output full angle of view": The full angle of view is the output target. The operation to switch to full angle of view can be done naturally in a similar way to the operation to change the crop frame of the output target.

[0062] In S516, a setting screen for "Action when selecting outside crop frame" is displayed. Here, for example, the above-mentioned options of "Do not change" and "Output full angle of view" are displayed. The user can select one of them by moving the cursor by up and down operation on the operation unit 110 or by operating the touch panel.

[0063] In S517, it is determined whether the setting of "Action when selecting outside crop frame" has been changed, and if it has been changed, the process proceeds to S518, and if it has not been changed, the process returns to S500. In S518, the result of the change made in S517 is saved in the volatile memory 116, and the process returns to S500.

[0064] FIG. 5B is a flowchart of the cropping frame movement control performed in S405 of FIG. In S520, it is determined whether or not a cropping frame movement operation has been performed. A movement operation is generated, for example, by operation of a cross key included in operation unit 110 or a drag operation on a touch panel. If a movement operation has been performed, the process proceeds to S521, and if not, the process returns to the process in FIG.

[0065] In S521, it is determined whether the object moved in S520 is a cropping frame to be output. In the example shown in Fig. 2(a), this corresponds to cropping frame 201. If the move operation is for cropping frame 201, the process proceeds to S522, where cropping frame 201 is moved in a direction corresponding to the move operation, and then the process returns to Fig. 4.

[0066] If the movement operation is not directed to the cropping frame 201, then in S523 the cropping frame 202 is moved in a direction corresponding to the movement operation, and the process returns to the process in FIG.

[0067] FIG. 5C is a flowchart of the focus control and the selection process of the cropping frame to be output, which are performed in S406 of FIG.

[0068] In S530, it is determined whether or not a subject has been selected. The selection of a subject occurs, for example, when the system control unit 109 detects a tap operation on the subject on a touch panel included in the operation unit 110. If a subject has been selected, the process proceeds to S531, and if not, the process returns to the process in FIG.

[0069] In S531, it is determined whether the subject selected in S530 is within the cropping frame of the output target for which the cropped image is to be output. Whether the selected subject is within the cropping frame of the output target can be determined by comparing the coordinate values ​​obtained by the subject selection operation with the coordinates of the display position of the cropping frame of the output target. If the subject is within the cropping frame of the output target, the process proceeds to S532, and if not, the process proceeds to S538.

[0070] In S532, it is determined whether the subject selected in S530 is included in an overlapping cropping frame of the output target and another cropping frame. If it is included in an overlapping frame, the process proceeds to S533, and if it is not included in an overlapping frame, the process proceeds to S537.

[0071] In S533, the content set by the processes of S511 to S514 in FIG. 5A, i.e., the content set as "action when subject is reselected", "do not change cropping frame" or "change cropping frame", is read from volatile memory 116. Then, it is determined whether "do not change cropping frame" is set. If "do not change cropping frame" is set, the process proceeds to S537, and if "do not change cropping frame" is not set, i.e., if "change cropping frame" is set, the process proceeds to S535.

[0072] In S535, it is determined whether or not the same subject as the subject selected in S530 has been reselected within a predetermined time since the most recent subject was selected. If the same subject has not been reselected within the predetermined time in S535, the process proceeds to S537.

[0073] On the other hand, if the same subject is reselected within a predetermined time, the process proceeds to S536. In S536, based on the priority order of the cropping frames set by the processes of S507 to S510, the cropping frame having the next highest priority as the crop to be outputted is determined as the new crop to be outputted and stored in the volatile memory 116, and the process returns to S535. Note that if the cropping frame to be outputted currently has the lowest priority, the cropping frame with the highest priority is determined as the new cropping frame to be outputted.

[0074] In S537, the lens 101 is controlled so as to focus on the subject selected in S530, and then the process returns to the process in FIG.

[0075] If the subject selected in S530 is not within the cropping frame to be output, then in S538 it is determined whether the subject selected in S530 is within a cropping frame that is not to be output. If it is within a cropping frame that is not to be output, then the process proceeds to S539, and if it is not within a cropping frame that is not to be output, then the process proceeds to S548.

[0076] In S539, it is determined whether the subject selected in S530 is included in a cropping frame other than the cropping frame determined in S538. If it is not included in the cropping frame, the process proceeds to S540, and if it is included in the cropping frame, the process proceeds to S541.

[0077] In S540, the cropping frame including the selected subject is provisionally determined as a new output target, and is stored in the volatile memory 116, and the process proceeds to S544.

[0078] On the other hand, in S541, the cropping frame to be output is provisionally determined based on the priority order of the cropping frames set in the processes of S507 to S510, and is saved in the volatile memory 116.

[0079] Next, in S542, it is determined whether or not the same subject as the subject selected in S530 has been reselected within a predetermined time since the most recent subject was selected. If the same subject has not been reselected within the predetermined time in S542, the process proceeds to S544.

[0080] On the other hand, if the same subject is reselected within a predetermined time, the process proceeds to S543. In S543, based on the priority order of cropping frames set by the processes of S507 to S510, a cropping frame having the next highest priority as the crop to be outputted is provisionally determined as the new crop to be outputted and stored in the volatile memory 116, and the process returns to S542. Note that if the cropping frame to be outputted currently has the lowest priority, a cropping frame with the highest priority is provisionally determined as the new cropping frame to be outputted.

[0081] In S544, the movement time when the focus lens is moved from the current focus lens position to focus on the subject selected in S530 is predicted. The movement time can be predicted, for example, from the distance to the subject to be focused on at the current focus lens position, the distance to the subject selected in S530, and the driving performance of lens 101. As a simpler method, the average focus movement time when the subject is changed in many shooting scenes may be recorded in advance in non-volatile memory 117 and used as the predicted value.

[0082] In S545, the lens 101 is controlled to start moving the focus lens so as to focus on the subject selected in S530.

[0083] In S546, it is determined whether a predetermined time, such as 1 / 2 of the predicted movement time in S544, has elapsed since the movement of the focus lens was started in S545. The time is measured by a system timer included in the system control unit 109. When it becomes 1 / 2 of the predicted movement time, the process proceeds to S547.

[0084] In S547, the cropping frame of the output target that was provisionally determined in S541 or S543 is determined as the cropping frame of the new output target, and is stored in the volatile memory 116, and the process returns to the process of FIG.

[0085] In S548, the content set by the processing of S515 to S518, i.e., the content set as the "action when selecting outside the crop frame", "do not change" or "output the entire angle of view", is read from the volatile memory 116. Then, it is determined whether "output the entire angle of view" is set. If "output the entire angle of view" is set, the process proceeds to S549, but if "output the entire angle of view" is not set, i.e., if "do not change" is set, the crop frame to be output is not changed and the process returns to the process of FIG. 4. In S549, the output target is determined to be the full angle of view, and stored in the volatile memory 116, and the process returns to the process of FIG.

[0086] As described above, according to the first embodiment, in camera 100, it is possible to appropriately control focus and the like based on the selection operation of a subject, and at the same time select a cropping frame that includes that subject as a new cropping area and output the image. Therefore, it is possible to minimize the time during which an image with an unstable focus is output when the frame for crop output is changed, without forcing the user to perform complex composite operations.

[0087] In particular, in the control of S544 to S547, by changing the crop frame to be output when, for example, half of the predicted movement time of the focus lens has elapsed, the effect of a blurred image being output before and after the change in the crop frame to be output can be made as inconspicuous as possible.

[0088] In the first embodiment, a case where focus control is performed based on the selection of a subject has been described, but a control different from focus control may be performed, or multiple types of control may be performed simultaneously. For example, exposure control, white balance control, image processing control, etc. may be performed. In either case, the control of a function applied based on the selection operation of a subject is appropriately performed, and at the same time, a cropping frame including the subject can be selected as a new cropping area and the image output.

[0089] Furthermore, even when an undesirable cropping frame is prioritized due to a selection operation of a subject that exists in an area where cropping frames overlap, a different frame can be quickly and smoothly selected through the controls of S535 to S536 and S542 to S543.

[0090] Furthermore, in this embodiment, two cropping frames are displayed simultaneously, but the present invention is also applicable to cases where three or more cropping frames are displayed simultaneously.

[0091] <Second embodiment> Next, a second embodiment of the present invention will be described, which relates to a switcher that does not perform image capture, inputs video from outside, and processes the input video.

[0092] FIG. 7 is a block diagram showing the configuration of a switcher 700 according to the second embodiment.

[0093] The input terminal 701 is for inputting video from an external device, and is configured by an HDMI terminal, an SDI terminal, etc. The input terminal 701 can input multiple videos simultaneously. The video input unit 702 can input at least two lines of video signals from the outside via the input terminal 701 .

[0094] The video processing unit 703 receives video data from the video input unit 702 at a predetermined frame rate, and performs predetermined resizing, trimming, color conversion, and distortion correction processes. The video processing unit 703 then uses the volatile memory 712 as a VRAM to store the processed video data.

[0095] The OSD drawing unit 711 renders character strings and icons representing the state and settings of the switcher 700, and a menu screen, in the VRAM on the volatile memory 712. Characters and icons are stored in the non-volatile memory 713, and the OSD drawing unit 711 reads them out and renders them in the VRAM.

[0096] The video output unit 704 reads out the video data recorded in the volatile memory 712 and the data rendered by the OSD drawing unit 711 as necessary, selects or synthesizes them, and outputs them to the output terminal 705, the display 706, and the communication control unit 714. The video output unit 704 also converts the associated information received from the system control unit 707 into metadata, and outputs it together with the video to the output terminal 705. The metadata can be linked to the video data for each frame, and can express the associated information for each frame, such as a time code.

[0097] The output terminal 705 is for outputting video to an external device, and can output a video signal to an external monitor or recording device connected via an HDMI cable, SDI cable, etc. (not shown). The metadata output from the video output unit 704 can also be output to an external device via the respective cables.

[0098] The display 706 is configured, for example, with an LCD panel, and is display-controlled by the video output unit 704. It is mainly used for simultaneously displaying multiple videos stored in the VRAM of the volatile memory 712 for monitoring purposes.

[0099] The system control unit 707 controls the entire switcher 700. It reads out and executes a program recorded in the non-volatile memory 713 to control each unit and realize each operation of this embodiment. The system control unit 707 may include multiple CPU cores. In that case, the tasks described in the program can be shared and processed by the multiple CPU cores.

[0100] The operation unit 708 includes menu keys, cross keys, a touch panel, and the like for operating a menu screen on the image displayed on the display 706 and for selecting an image to be output from the output terminal 705 .

[0101] The operation input terminal 709 is for controlling the switcher 700 from an external controller, and notifies a control signal to the system control unit 707 via the operation input terminal 709. The system control unit 707 detects the content of the operation based on the control signal, and can perform the same operation as that of the operation unit 708.

[0102] The power switch 710 is for selecting the power state of the switcher 700, and determines whether the power is on or off depending on the position of the switch, and notifies the system control unit 707 of this.

[0103] The volatile memory 712 is used as a dynamic memory area for video data handled by the video processing unit 703, the video output unit 704, and the OSD drawing unit 711, and for programs used by the system control unit 707. The volatile memory 712 also has a storage capacity sufficient to store moving images for a predetermined period of time.

[0104] The non-volatile memory 713 is an electrically erasable and recordable memory, and may be, for example, an EEPROM. Constants, programs, and the like for the operation of the system control unit 707 are recorded in the non-volatile memory 713. The programs referred to here include programs for executing the flowcharts described later in this embodiment.

[0105] The communication control unit 714 is used to communicate with an external device. Either or both of the network connection terminal 715 and the wireless network connection unit 716 are used to enable wireless / wired IP communication. The communication control unit 714 transmits video from the video output unit 704 to an external device, thereby enabling monitoring of the video via IP communication. In addition, by receiving a control signal generated by the operation of an external controller, the communication control unit 714 can notify the system control unit 707 of an operation request corresponding to an operation that can be performed by the operation unit 708. Alternatively, the communication control unit 714 can receive a setting change request for directly changing the setting of a setting item on a menu screen displayed on the display 706, and notify the system control unit 707 of the request.

[0106] The network connection terminal 715 is for performing wired network communication by the communication control unit 714, and the wireless network connection unit 716 is for performing wireless network communication by the communication control unit 714. The wireless network connection unit 716 may be configured to connect a detachable antenna such as a WiFi dongle via a socket or the like (not shown).

[0107] FIG. 8 is a diagram showing an example of video data captured by an external camera and video data handled by the switcher 700 when the switcher 700 receives a plurality of cropped videos and selects a subject.

[0108] Reference numeral 800 denotes an external camera having a configuration capable of generating a plurality of cropped images from a captured full-angle image and transmitting them simultaneously to the switcher 700. Reference numeral 801 denotes full-angle image data captured by the camera 800 before cropping.

[0109] Reference numeral 802 denotes an example of cropped image A output from camera 800 and input to switcher 700, and is composed of an image range obtained by cropping a partial area of ​​full-angle image data 801 by camera 800.

[0110] 803 shows an example of cropped image B output from camera 800 and input to switcher 700, and is composed of an image range in which a partial area different from cropped image A 802 of full-angle image data 801 has been cropped by camera 800.

[0111] Reference numeral 804 denotes an example of a video output from the output terminal 705 by the switcher 700, which is generated based on at least one of the cropped video A 802 and the cropped video B 803.

[0112] Reference numeral 805 denotes video data selection information, which includes, for example, coordinate information of a subject selected within the received cropped video A 802 or cropped video B 803.

[0113] 9 is a flowchart showing an example of the operation of the switcher 700 according to this embodiment. The processing of this flowchart is realized by the system control unit 707 controlling each unit of the switcher 700 based on a program stored in the non-volatile memory 713. Furthermore, a user operation occurring in the processing of this flowchart occurs when the system control unit 707 detects an operation performed by the user on the operation unit 708. Alternatively, the system control unit 707 may detect a control signal received from the operation input terminal 709 or the communication control unit 714 using an external controller.

[0114] When the process starts, in S900, cropped image A 802 and cropped image B 803 are input from the input terminal 701 via the image input unit 107.

[0115] Next, in S901, it is determined whether or not subject a included in the cropped image A 802 has been selected. Selection of subject a occurs, for example, when the system control unit 707 detects a tap operation on the subject on a touch panel included in the operation unit 708. If subject a has been selected, the process proceeds to S903, and if not, the process proceeds to S902.

[0116] In S902, it is determined whether or not subject b included in cropped image B 803 has been selected. If subject b has been selected, the process proceeds to S907, and if not, the process proceeds to S911.

[0117] In S903, the coordinate information of subject a obtained by the selection operation of subject a is transmitted to the input source of the cropped image A802 via the network connection terminal 715 or the wireless network connection unit 716.

[0118] In S904, it is determined whether the image currently being output from the output terminal 705 is the cropped image A802. If it is the cropped image A802, the process proceeds to S911, and if it is not the cropped image A802, the process proceeds to S905.

[0119] In S905, it is determined whether an output change timing notification has been received from the input source of the cropped image A802, and if an output change timing notification has been received from the input source of the cropped image A802, the process proceeds to S906. In S906, the image to be output from the output terminal 705 is changed to the cropped image A802, which is then stored in the volatile memory 712, and the process proceeds to S911.

[0120] In S907, the coordinate information of subject b obtained by the selection operation of subject b is transmitted to the input source of the cropped image B 803 via the network connection terminal 715 or the wireless network connection unit 716.

[0121] In S908, it is determined whether or not the image currently being output from the output terminal 705 is the cropped image B803. If it is the cropped image B803, the process proceeds to S911, and if it is not the cropped image B803, the process proceeds to S909.

[0122] In S909, it is determined whether an output change timing notification has been received from the input source of the cropped video B803, and if an output change timing notification has been received from the input source of the cropped video B803, the process proceeds to S910. In S910, the image to be output from the output terminal 705 is changed to the cropped image B 803, and stored in the volatile memory 712, and the process proceeds to S911.

[0123] In S911, either the cropped image A 802 or the cropped image B 803 is output from the output terminal 705 based on the information stored in the volatile memory 712 in S906 or S910.

[0124] As described above, according to the second embodiment, in switcher 700, it is possible to notify a single camera that is the input source of multiple cropped images of subject coordinate information, and to select an input image containing that subject as a new output target and output the image. Therefore, by having the camera that receives the subject selection information appropriately control focus and the like based on the notification, it is possible to minimize the time during which an image with an unstable focus is output when changing the partial area to be cropped and output, without forcing the user to perform complex composite operations.

[0125] Furthermore, according to the second embodiment, in a system in which a camera that controls focus, etc. and a switcher that selects the priority area are separate, the above-mentioned effects can be achieved without having to perform each operation on a separate device.

[0126] In particular, by changing the cropping frame to be output at the output change timing notified from the input source camera based on the control of S905 and S909, the effect of a blurred cropped image being output before and after the change in the cropping frame to be output can be made as inconspicuous as possible.

[0127] In this embodiment, two lines of cropped images are input simultaneously, but the present invention is also applicable to the case where more than two lines of cropped images are input simultaneously.

[0128] <Other embodiments> The present invention may be applied to a system made up of a plurality of devices, or to an apparatus made up of a single device.

[0129] The present invention can also be realized by supplying a program for implementing one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0130] <Summary> The disclosure of this embodiment includes the following configuration.

[0131] (Item 1) An input means for inputting a video image; A detection means for detecting an operation performed on the image; a setting means for setting a frame indicating a partial image that is a part of the image in response to the operation; a display control means for displaying the image and the frame in a superimposed manner on a display means; a selection means for selecting one of the displayed frames in response to the operation; an output means for outputting a partial image of the frame selected by the selection means; a control unit that performs control according to the subject selected in response to the operation, When a plurality of frames are set, the selection means selects a frame including the selected subject. (Item 2) 2. The video output device according to item 1, wherein the control means performs at least one of focus adjustment control, exposure control, white balance control, and image processing control based on the selected subject. (Item 3) When a plurality of slots are set, the setting means prioritizes the plurality of slots according to a predetermined method; 3. The video output device according to item 1 or 2, characterized in that, when the selected subject is included in multiple frames, the selection means selects a frame with a higher priority from among the multiple frames. (Item 4) The video output device described in item 3, characterized in that the predetermined method is one of a method of determining based on numbers previously assigned to the plurality of frames, a method of determining based on a history of the plurality of frames being selected by the selection means, and a method of determining based on the distance from the center of the image to the plurality of frames. (Item 5) 5. The video output device according to item 3 or 4, wherein the predetermined method can be selected from a plurality of predetermined methods. (Item 6) The video output device described in any one of items 3 to 5, characterized in that when the subject is reselected within a predetermined time after the subject is selected, the selection means changes the frame to be selected from among the multiple frames containing the subject, in order from the frame with the highest priority. (Item 7) The video output device described in any one of items 3 to 5 is characterized in that, when the subject is selected again within a predetermined time after the subject is selected, it is possible to select whether or not to change the frame selected by the selection means. (Item 8) 8. The video output device according to any one of items 1 to 7, characterized in that, when the subject is not included in any of the frames, the selection means does not change the selected frame. (Item 9) The video output device described in any one of items 1 to 7, characterized in that when the subject is not included in any of the frames, the selection means does not select the frame, and the output means outputs the video input by the input means. (Item 10) A video output device according to any one of items 1 to 7, characterized in that, when the subject is not included in any of the frames, it is possible to choose whether to not change the frame or to output the video. (Item 11) The video output device described in any one of items 1 to 10, characterized in that when the frame selected by the selection means is changed from a first frame to a second frame, the output means switches the partial image to be output from the partial image of the first frame to the partial image of the second frame at a predetermined timing between the start and end of control by the control means. (Item 12) An input means for inputting a plurality of images; a display control means for displaying the plurality of images on a display means; a detection means for detecting an operation performed on the plurality of images displayed on the display means; a selection means for selecting an image including a subject selected by the operation from among the plurality of images displayed; an output means for outputting the image selected by the selection means; a notification means for notifying an input source of the selected image of information of the selected subject; A video output device comprising: (Item 13) The video output device described in item 12 is characterized in that when the video selected by the selection means is changed from a first video to a second video, a timing for switching to the second video is received from an input source of the second video, and the output means switches from the first video to the second video based on the timing. (Item 14) an input step of inputting a video; a detection step of detecting an operation performed on the image; a setting step of setting a frame indicating a partial image that is a part of the image in response to the operation; a display control step of displaying the image and the frame in a superimposed manner on a display means; a selection step of selecting one of the displayed frames in response to the operation; an output step of outputting a partial image of the frame selected in the selection step; a control step of performing control according to the subject selected in response to the operation, A video output method, characterized in that, when a plurality of frames are set, a frame including the selected subject is selected in the selection step. (Item 15) an input step of inputting a plurality of videos; a display control step of displaying the plurality of images on a display means; a detection step of detecting an operation performed on the plurality of images displayed on the display means; a selection step of selecting an image including the subject selected by the operation from among the plurality of images displayed; an output step of outputting the video selected in the selection step; a notification step of notifying an input source of the selected image of the subject; A video output method comprising: (Item 16) A program for causing a computer to function as each of the means of the video output device according to any one of items 1 to 13. (Item 17) Item 17. A computer-readable storage medium storing the program according to item 16.

[0132] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0133] 100: camera, 101: lens, 102 imaging section, 103,703: image processing section, 104,704: image output section, 105,705: output terminal, 106,706: display, 107,702: image input section, 108,701: input terminal, 109,707: system control section, 110,708: operation section, 111,709: operation input terminal, 112,710: power switch, 113,711: OSD drawing section, 116,712: volatile memory, 117,713: non-volatile memory, 118,714: communication control section, 119,715: network communication terminal, 120,716: wireless network connection section, 700: switcher

Claims

1. Setting means capable of setting multiple frames that indicate a partial image which is a part of the input video, A selection means for selecting a subject in response to an operation performed on the input video, An output means for outputting video, The system includes a control means that performs control according to the subject selected by the selection means, The video output device is characterized in that, when a subject is selected while multiple frames are set by the setting means, the output means outputs a portion of the frame containing the selected subject from among the multiple set frames.

2. The video output device according to Claim 1, further comprising a display control means for superimposing and displaying the video and the frame.

3. The video output device according to claim 1, characterized in that the control means performs at least one of the following based on the selected subject: focus adjustment control, exposure control, white balance control, and image processing control.

4. The setting means, when multiple frames are set, assigns priority to the multiple frames in a predetermined manner, The video output device according to claim 1, characterized in that, when the selected subject is included in the plurality of frames, the selection means selects the frame with the higher priority from among the plurality of frames.

5. The video output device according to claim 4, characterized in that the predetermined method is one of the following: a method of determining based on numbers predetermined for the plurality of frames; a method of determining based on the history of the plurality of frames being selected by the selection means; or a method of determining based on the distance from the center of the image to the plurality of frames.

6. The video output device according to claim 4, characterized in that the predetermined method can be selected from a plurality of predetermined methods.

7. The video output device according to claim 4, characterized in that if the subject is re-selected within a predetermined time after the subject has been selected, the selection means changes the frame selected from among a plurality of frames containing the subject, starting with the frame with the highest priority.

8. The video output device according to claim 4, characterized in that, if the subject is re-selected within a predetermined time after the subject has been selected, it is possible to select whether or not to change the frame selected by the selection means.

9. The video output device according to claim 1, characterized in that if the subject is not included in any of the frames, the selection means does not change the selected frame.

10. The video output device according to claim 1, characterized in that, if the subject is not included in any of the frames, the selection means does not select a frame, and the output means outputs the input video.

11. The video output device according to claim 1, characterized in that, if the subject is not included in any of the frames, the selection means can choose not to change the frame or to output the video.

12. The video output device according to claim 1, characterized in that when the frame selected by the selection means is changed from the first frame to the second frame, the output means switches the output portion of the video from the portion of the first frame to the portion of the second frame at a predetermined timing between the start and end of control by the control means.

13. Display means comprising display control means for displaying a plurality of input video images, A selection means for selecting a subject in response to an operation performed on the input video, An output means that outputs an image including the subject selected by the selection means, A notification means that notifies the input source of the selected video of the information of the subject selected by the selection means. A video output device characterized by having the following features.

14. The video output device according to claim 13, characterized in that when the video selected by the selection means is changed from the first video to the second video, the output means receives the timing for switching to the second video from the input source of the second video, and switches from the first video to the second video based on that timing.

15. A setting step of setting at least one frame that shows a partial image which is a part of the input video, A selection step is performed to select a subject in accordance with the operation performed on the input video, The output process for outputting video, The system includes a control step that performs control according to the subject selected in the selection step, In the setting step, if the subject is selected while multiple frames are set, the output step is characterized in that it outputs a portion of the frame containing the selected subject from among the multiple set frames.

16. A display control step for displaying a plurality of input images in a display means, A selection step is performed to select a subject in accordance with the operation performed on the input video, An output step which outputs an image including the subject selected in the selection step, A notification step in which information about the subject selected in the selection step is notified to the input source of the selected video; A video output method characterized by having the following features.

17. A program for causing a computer to function as one of the means of a video output device according to any one of claims 1 to 14.

18. A computer-readable storage medium storing the program described in claim 17.

Images