Video clipping device, clipping control device, and program

Abstract

A video clipping device (2) is provided with a clipping means (21) for clipping, out of an input video as a partial video, a video in which the coordinate values and / or the magnitude at a prescribed position changes momently. The coordinate values and / or the magnitude at the prescribed position changes linearly with respect to a prescribed function, and the prescribed function has properties including sections in which the absolute value of the jerk is equal to or less than a prescribed value with respect to the passage of time.

Description

Video extraction device, extraction control device and program

[0001] The present invention relates to a video clipping device that clips out a partial video from a video, a clipping control device that has a graphical user interface that controls clipping out a partial video from a video, and a program.

[0002] Conventionally, there are known techniques for extracting a partial image from a video. For example, a technique has been disclosed in which a partial image is extracted from an image captured by an omnidirectional camera and displayed by specifying coordinates on a screen (see, for example, Patent Document 1).

[0003] Also, a technology has been disclosed in which a partial image is cut out from a spherical image by operating (swiping) a touch sensor provided on a screen (see, for example, Patent Document 2).

[0004] Furthermore, a technology has been disclosed in which, when cutting out a panoramic image and generating a desired image stream, an image stream of desired frames including an area of ​​interest is selected using a fixed frame that is set in advance, a virtual moving camera that tracks an object in the scene, or a virtual moving camera that performs a predetermined movement, and image transitions between the desired frames are applied (see, for example, Patent Document 3).

[0005] JP 2005-192057 A JP 2022-129799 A Japanese Patent No. 6432029 A

[0006] The technology of the aforementioned Patent Document 1 allows a cutout area to be selected from multiple options when specifying the cutout area on an image, but it is not possible to smoothly transition between the selected cutout areas.

[0007] Furthermore, the technology of the aforementioned Patent Document 2 changes the cut-out area depending on the amount of user operation, but it is not possible to automatically transition this cut-out area between areas that involve a predetermined path or speed change.

[0008] Furthermore, although the technology of Patent Document 3 realizes image transition between regions of interest, it does not prescribe a specific transition method such as acceleration / deceleration, or a specific method for specifying a virtual camera to be the transition target.

[0009] Therefore, the present invention has been made to solve the above-mentioned problems, and its purpose is to provide a video cutting-out device, a cutting-out control device, and a program that can achieve cutting-out that is in line with human intuition when cutting out areas that change from moment to moment from a video as partial images.

[0010] In order to solve the above problem, the video extraction device of claim 1 is a video extraction device that extracts a partial video from an input video, and is equipped with extraction means that extracts from the input video an video in which either or both of coordinate values ​​and size of a predetermined position change from moment to moment as the partial video, wherein either or both of the coordinate values ​​and size of the predetermined position change linearly with respect to a predetermined function, and the predetermined function has the property of including a section in which the absolute value of the jerk is equal to or less than a predetermined value over time.

[0011] According to the first aspect of the present invention, it is possible to transition the partial video smoothly and with less shock, thereby reducing discomfort when viewing the partial video.

[0012] Further, a video extraction device according to claim 2 is the video extraction device according to claim 1, characterized in that the predetermined function has, with respect to the passage of time, an interval A having a positive uniform jerk and positive acceleration, an interval B having a negative uniform jerk and positive acceleration, an interval C of a constant speed, an interval D having a negative uniform jerk and negative acceleration, and an interval E having a positive uniform jerk and negative acceleration, wherein durations of the intervals A, B, C, D and E are TA, TB, TC, TD and TE, respectively, and TA>0, TB>0, TC≧0, TD≧0 and TE≧0, or TA≧0, TB≧0, TC≧0, TD>0 and TE>0, and the curves in the intervals A, B, C, D and E are connected in a continuous and second-order differentiable state over all intervals.

[0013] According to the second aspect of the present invention, the partial video can be transitioned more smoothly and with less shock, and discomfort felt when viewing the partial video can be further reduced.

[0014] Furthermore, the cutting-out control device of claim 3 is a cutting-out control device that controls a video cutting-out device that cuts out a partial video from an input video, and comprises: a control means that generates dynamic region information for the video cutting-out device to cut out the partial video; an input means that generates cutting-out information by specifying one or more regions of interest in accordance with an operator's operation, specifying one region from the one or more regions of interest as a destination region, and specifying the region from which the partial video is cut out as a cutting-out region, and specifying the type and timing of transition when the cutting-out region transitions from one of the one or more regions of interest to the destination region; a drawing means that draws either one or both of the one or more regions of interest and the dynamic region information as a picture; and a display means that presents the picture drawn by the drawing means to the operator, and is characterized in that the control means generates the dynamic region information based on the cutting-out information generated by the input means.

[0015] According to the invention of claim 3, video switching can be performed with the same operability as operating a video switcher, by specifying the area of ​​interest, transition destination, type of transition, and timing on the input means, thereby realizing operability that matches the intuition of the operator producing the video.

[0016] The extraction control device according to claim 4 is the extraction control device according to claim 3, characterized in that the input means specifies an arbitrary rectangular area as the transition destination area in accordance with an operation by the operator.

[0017] According to the fourth aspect of the present invention, it is possible to specify any rectangular area as the transition destination area regardless of the area set as the area of ​​interest, thereby improving convenience.

[0018] Furthermore, the cut-out control device of claim 5 is characterized in that, in the cut-out control device of claim 3, the input means specifies, in accordance with the operator's operation, as the type of transition, a panning transition in which the cut-out area moves, expands, or shrinks from the area of ​​interest to the area to which it is to be transitioned from time to time.

[0019] According to the fifth aspect of the present invention, a panning transition can be executed with an effect that is as if it were realized by panning and zooming a camera.

[0020] Furthermore, the extraction control device of claim 6 is the extraction control device of claim 3, characterized in that the input means specifies a rectangular area for each of the one or more regions of interest by the operator's drag or swipe operation.

[0021] According to the sixth aspect of the present invention, it is possible to intuitively edit the region of interest with operability similar to that of a graphics drawing application.

[0022] Furthermore, the cutting control device of claim 7 is characterized in that, in the cutting control device described in claim 3, the input means specifies the one or more regions of interest, the region to which the transition will occur, and the type and timing of the transition in accordance with the operator's operation on a touch panel, and the display means presents the drawing to the operator via the touch panel.

[0023] According to the seventh aspect of the present invention, the region of interest, the region to which the user wishes to transition, etc. can be specified by directly touching the image displayed on the screen of the touch panel, thereby enabling more intuitive operation.

[0024] The cut-out control device of claim 8 is characterized in that, in the cut-out control device of claim 5, the input means specifies the panning transition as the type of transition in accordance with the operation of the operator, and specifies the timing of the transition so that the transition amount of the position and size of the cut-out area transitioning from the area of ​​interest to the area to which the transition is made with respect to time follows a transition curve including a constant jerk curve.

[0025] According to the invention of claim 8, acceleration and deceleration at the start and end of the transition enable a smooth transition with less shock, thereby reducing discomfort when viewing the cut-out partial video.

[0026] The cutting control device according to claim 9 is the cutting control device according to claim 8, characterized in that the input means changes the transition curve in accordance with an operation by the operator.

[0027] According to the invention of claim 9, it is possible to adjust the agility and smoothness of the transition, making it possible to produce a variety of effects.

[0028] Furthermore, the cutting control device of claim 10 is the cutting control device of claim 6, characterized in that when the input means specifies the rectangular area for each of the one or more regions of interest, the size of the rectangular area is corrected so that the aspect ratio of the rectangular area becomes a predetermined ratio.

[0029] According to the tenth aspect of the present invention, it is possible to cut out an image while maintaining its aspect ratio.

[0030] Furthermore, the cut-out control device of claim 11 is characterized in that, in the cut-out control device described in claim 6, when the input means specifies a rectangular area for each of the one or more regions of interest, the position of the rectangular area is corrected so that the rectangular area does not include an area outside the range of the input image.

[0031] According to the eleventh aspect of the present invention, it is possible to prevent the area of ​​the cut-out partial video from being set outside the input video.

[0032] Furthermore, the cutting control device of claim 12 is characterized in that, in the cutting control device described in claim 3, the input means specifies at least one region of interest from the one or more regions of interest in accordance with instructions from an external device.

[0033] According to the twelfth aspect of the present invention, the setting of the region of interest can be automated.

[0034] Furthermore, a program according to claim 13 causes a computer to function as the video extraction device according to claim 1.

[0035] A program according to a fourteenth aspect of the present invention is characterized in that it causes a computer to function as the cutting control device according to the third aspect of the present invention.

[0036] As described above, according to the present invention, when an area that changes from moment to moment is extracted from a video as a partial video, extraction that matches human intuition can be achieved.

[0037] FIG. 1 is a schematic diagram showing an example of the overall configuration of a video cutting system including a cutting control device and a video cutting device according to an embodiment of the present invention. FIG. 2 is a block diagram showing an example of the configuration of a cutting control device according to an embodiment of the present invention. FIG. 3 is a flowchart showing an example of a processing example of an immediate transition operation. FIG. 4 is a flowchart showing an example of a processing example of a cut transition operation. FIG. 5 is a flowchart showing an example of a processing example of a panning transition operation. FIG. 6 is a diagram showing an example of an area selection operation. FIG. 7 is a diagram explaining an example of a transition curve. FIG. 8 is a diagram showing an example of implementation of an input means and a display means in a setting mode. FIG. 9 is a diagram showing examples of an input video and a CH1 output video, etc. FIG. 10 is a diagram showing examples of a PGM output video and a NEXT output video. FIG. 11 is a diagram showing an example of a display on the display means when a panning transition has started in a performance mode. FIG. 12 is a diagram showing an example of a display on the display means when a panning transition is halfway through in a performance mode. FIG. 13 is a diagram showing an example of a PGM output video due to a panning transition operation. FIG. 14 is a block diagram showing an example of the configuration of a video cutting device according to an embodiment of the present invention.

[0038] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] The following describes in detail the embodiments of the present invention with reference to the accompanying drawings. [Video Extraction System] First, the overall configuration of the video extraction system will be described. Fig. 1 is a schematic diagram showing an example of the overall configuration of a video extraction system including an extraction control device and a video extraction device according to an embodiment of the present invention.

[0039] The video clipping system 10 comprises a clipping control device 1 , a video clipping device 2 , and a resolution conversion device 3 .

[0040] The extraction control device 1 is connected to a video extraction device 2 that extracts a partial video from an input video, and controls the video extraction device 2. The extraction control device 1 receives video whose resolution has been converted from a resolution conversion device 3, generates a control signal for the video extraction device 2 to extract a partial video in accordance with an operator's operation, and outputs the control signal to the video extraction device 2. The control signal includes area information for extracting the partial video, i.e., the extraction position and size of the area.

[0041] 1, since the video cutout device 2 outputs video for multiple channels, the cutout control device 1 generates a control signal including the cutout position and size of the region for each channel. For example, the cutout control device 1 generates a control signal including the upper left coordinate and the lower right coordinate of the region for each channel.

[0042] The video cutting-out device 2 inputs a video (hereinafter referred to as "input video") and also inputs a control signal from the cutting-out control device 1, cuts out a partial video from the input video based on the cutting-out position and size of the area indicated by the control signal, and outputs the cut-out partial video.

[0043] 1 , the video extraction device 2 extracts a partial video for each channel from the input video based on the extraction position and size of the region for each channel indicated by the control signal. The video extraction device 2 then sets the extracted partial video for each channel as a PGM (program) output video, a NEXT output video, a CH1 output video, a CH2 output video, a CH3 output video, a CH4 output video, a CH5 output video, and a CH6 output video, and outputs these videos. In other words, the position and size of the partial video extracted from the input video by the video extraction device 2 are controlled by the control signal input from the extraction control device 1.

[0044] Here, the PGM output video output by the video cutting device 2 is the main line video cut out by the video cutting device 2 and is the subject of the immediate transition operation, cut transition operation and panning transition operation described later by the cutting control device 1.

[0045] The resolution conversion device 3 receives an input video, converts the resolution of the input video, and outputs the video with converted resolution to the extraction control device 1. The resolution conversion device 3 may also perform format conversion such as frame rate conversion and progressive / interlace conversion on the input video.

[0046] [Extraction Control Device 1] Next, the extraction control device 1 shown in Fig. 1 will be described. Fig. 2 is a block diagram showing an example of the configuration of the extraction control device 1 according to an embodiment of the present invention. This extraction control device 1 includes a control unit 11, an input unit 12, a drawing unit 13, and a display unit 14.

[0047] The control means 11 receives cutout information (described later) from the input means 12, and generates a control signal for causing the video cutout device 2 to cut out the video based on the cutout information. The control means 11 then outputs the control signal to the video cutout device 2 and outputs the cutout information to the drawing means 13.

[0048] The control signal includes the cut-out position and size of the region cut out by the video cut-out device 2 for each channel.

[0049] The input means 12 specifies one or more preset regions (regions of interest) in accordance with the operation of the operator using the extraction control device 1, and specifies one region from the one or more preset regions as the next region (region to which the transition will be made).

[0050] In this case, the input means 12 generates cut-out information including identifiers of channels corresponding to one or more preset regions and next regions, and the designated region, and outputs the cut-out information to the control means 11 .

[0051] In the example of Fig. 1, the channels corresponding to the multiple presets are the channels for output video CH1, CH2, CH3, CH4, CH5, and CH6. The channel corresponding to the next region is the channel for output video NEXT.

[0052] The input means 12, in accordance with the operator's operation, specifies the type and timing of the transition when the cut-out area transitions from one of one or more preset areas to the next area, generates cut-out information including an identifier indicating the channel of the PGM output video, the type of transition, timing, cut-out area, etc., and outputs the cut-out information to the control means 11.

[0053] When the control means 11 receives the cutting information from the input means 12, it sequentially generates control signals including dynamic area information whose cutting position and size change for the channels of the PGM output video cut out by the video cutting device 2 in accordance with the change.

[0054] The drawing means 13 inputs the resolution-converted image from the resolution conversion device 3 and also inputs the cropping information from the control means 11. The drawing means 13 then draws each area, such as a preset area, indicated by the cropping information as a picture together with the input image. For example, the drawing means 13 draws one or more preset areas and / or dynamic area information as a picture together with the image. The drawing means 13 outputs the drawn picture to the display means 14.

[0055] The drawing means 13 may perform predetermined scaling on the areas (one or more preset areas and dynamic area information) indicated by the cutout information input from the control means 11 so that the size of the areas corresponds to the size of the video. The drawing means 13 draws a picture by superimposing the area whose size has been changed by scaling on the video. This allows the operator to check the settings of the preset area, next area, etc., and the transition state while referring to the video, improving operability.

[0056] The display means 14 receives the drawing from the drawing means 13 and displays it on the screen, thereby presenting the drawing to the operator.

[0057] The input means 12 and the display means 14 may be configured to function as a touch panel, which allows the operator to specify the preset area and the next area by directly touching the image displayed on the screen, allowing for more intuitive operation.

[0058] <Types of Transition> There are three types of transitions when the cut-out area of ​​the PGM output video transitions: immediate transition, cut transition, and panning transition, as shown below.

[0059] The input means 12 specifies the type and timing of the transition shown below in accordance with the transition operation by the operator, and generates cut-out information. Then, the control means 11 generates a control signal including the cut-out position and size of the area for the channel of the PGM output video based on the cut-out information, and outputs the control signal to the video cut-out device 2. In other words, the control means 11 generates and outputs a control signal corresponding to one of the type and timing of the transition in accordance with the transition operation shown below for the area of ​​the PGM output video cut out by the video cut-out device 2, thereby updating the cut-out area in the video cut-out device 2.

[0060] The operations performed by the transition operations are as follows: (1) By the "instant transition operation," an area is set (for example, by pressing any button in the instant transition operation button row (PGM button row) 1206 shown in FIG. 9 , which will be described later), and then the cut-out area of ​​the video cut-out device 2 is immediately updated. (2) By the "cut transition operation," a next area is set (for example, by pressing any button in the next area button row (NEXT button row) 1202, which will be described later), and then the cut-out area of ​​the video cut-out device 2 is updated from the current cut-out area to the next area by pressing a button (for example, by pressing the CUT button 1203, which will be described later). (3) By the "panning transition operation," a next area is set (for example, by pressing any button in the next area button row 1202, which will be described later), and then the cut-out area of ​​the video cut-out device 2 is updated so as to move, enlarge, or reduce from time to time from the current cut-out area to the next area, triggered by pressing a button (for example, by pressing any button in the pan button row 1204, which will be described later). (4) By the "transition stop operation", pressing a button (for example, the stop button 1217 described later) during the momentary panning transition stops the momentary transition in the cut-out area of ​​the video cut-out device 2, and the cut-out area at the time of the stop is maintained.

[0061] <Area Setting Method> Next, the setting of a preset area and a next area using the input unit 12 will be described. (1) The "Preset Area Selection" operation selects one or more preset areas. (2) The "Arbitrary Area Selection" operation selects an arbitrary area using a button, mouse, touch panel, numerical input, or the like. (3) The "Previous Area Selection" operation selects the cut-out area before the start of the transition of the "Cut Transition Operation" or "Panning Transition Operation" as the previous area. (4) The "Resize" operation (e.g., pressing one of the buttons in the resize button row 1209, described later) sets an area in which the size of the currently set area is changed to a predetermined size while maintaining the center coordinates of the area. (5) The "Centering" operation (e.g., pressing the centering button (center alignment button) 1210, described later) sets an area in which the center position of the currently set area is moved to a predetermined position (e.g., the center position of the input video) while maintaining the size of the area. (6) The "File Load" operation (e.g., pressing the Area File Load button 1215, described later) sets an area recorded in a file. (7) By operating "remote designation" (for example, by pressing the reconnect button 1223 of the AI ​​cutout communication port described later), an area notified by communication from an external device is set. For example, the input unit 12 sets at least one area out of one or more preset areas in accordance with instructions from the external device. This makes it possible to automate the setting of the preset area.

[0062] Hereinafter, an identifier for identifying a plurality of preset areas will be referred to as a "preset channel."

[0063] The input unit 12 sets a preset area after specifying a preset channel in accordance with the operator's operation. (1) "Select Default Area" sets an area with a preset center position and size (e.g., the entire screen area, an area in the center of the screen with each side being 1 / 2 of the full screen, etc.). (2) "Copy Preset Area" sets an area with the same center coordinates and size as another preset area. (3) "Select Arbitrary Area" sets an area to be arbitrarily specified by operating a button, mouse, touch panel, numerical input, etc. (4) "Resize" sets an area by changing the size of the area currently set as the preset area while maintaining the center coordinates. (5) "Centering" sets an area by moving the center position of the area currently set as the preset area to a predetermined position (e.g., the center position of the input video) while maintaining the size. (6) "Load File" sets an area recorded in a file. (7) "Remote Designation" sets an area notified by communication from an external device.

[0064] 9 is a diagram showing an example of the implementation of the input means 12 and the display means 14 in the setting mode. The setting mode will be described later. In this example, part of the functions of the input means 12 (in addition to the functions shown in FIG. 9, the input means 12 is also provided by a mouse, touch panel, or keyboard) and the display means 14 are implemented as an integrated unit.

[0065] The extraction control device 1 has two operation modes: a setting mode and a production mode, which will be described later. The behavior of the input means 12 and the display means 14 changes depending on the operation mode. In particular, in the example of Fig. 9, the display content of the viewer 1207, the behavior when the operator touches within the viewer 1207, and the behavior when operating the keyboard change.

[0066] The setting mode is an operation mode in which preset areas are set, and the performance mode is an operation mode in which the next area is set and transition operations are performed. Referring to Fig. 9, the operation mode is switched to the setting mode by pressing the setting mode button 1211. On the other hand, the operation mode is switched to the performance mode by pressing the performance mode button 1212.

[0067] The setting mode and the production mode may be alternately switched by pressing a predetermined key on the keyboard (for example, the TAB key or the ESC key).

[0068] <Preset Channel Designation> Next, a description will be given of designating a preset channel using the input means 12. The input means 12 designates a preset channel in accordance with an operation by an operator.

[0069] The preset button row (PSET button row) 1201 is a button row for specifying a preset channel. In this example, the input means 12 selectively specifies one of preset channels 0 to 9. Note that the input means 12 may switch the operation mode to a setting mode when any button in the preset button row 1201 is pressed.

[0070] <Specifying Preset Area> Next, a description will be given of specifying a preset area using the input unit 12. The input unit 12 specifies a preset area in accordance with an operation by an operator.

[0071] The next area button row 1202 is a button row for setting the next area by specifying a preset area. In response to an operator pressing any button in the next area button row 1202, the input unit 12 sets the preset area corresponding to the preset channel numbered on the button as the next area.

[0072] The input unit 12 may set the preset region corresponding to the preset channel of a specific number as the next region in accordance with the operation of pressing a specific key on the keyboard (for example, any of the numeric keys 0 to 9) in the setting mode instead of pressing the next region button column 1202. Furthermore, the input unit 12 may set the next region in accordance with the operation of touching or clicking a mouse on a preset region frame (1208-1 to 1208-3 or 1208-6 in this example) presented in the viewer 1207 in the setting mode. Furthermore, the input unit 12 may switch the operation mode to the production mode when any of the buttons in the next region button column 1202 is pressed.

[0073] This allows any rectangular area to be specified as the next area, so that any rectangular area can be set as the destination cut-out area regardless of the area set as the preset area, thereby increasing convenience.

[0074] <Resizing> Next, a description will be given of the process of resizing the preset area or the next area by the input means 12. The input means 12 resizes the preset area or the next area in accordance with the operation of the operator.

[0075] When the operation mode is the setting mode, the input means 12 changes the size of the preset area of ​​the selected preset channel to a predetermined size while maintaining the current center coordinates, in response to the operator pressing any button in the resize button row (area edit button row) 1209.

[0076] When the operation mode is the production mode, the input means 12 changes the size of the next area to a predetermined size while maintaining the current center coordinates in response to the operator pressing any button in the resize button row 1209 .

[0077] In addition, the input means 12 may be configured to perform the same processing as when any button in the resize button column 1209 is pressed, in response to a specified key operation on the keyboard (for example, simultaneous pressing of the Ctrl key and the f key, simultaneous pressing of the Ctrl key and the h key, simultaneous pressing of the Ctrl key and the q key, or simultaneous pressing of the Ctrl key and the m key).

[0078] <Centering> Next, a description will be given of the processing of centering the preset area or the next area by the input means 12. The input means 12 centers the preset area or the next area in accordance with the operation of the operator.

[0079] When the operating mode is the setting mode, the input means 12, in response to the operator pressing the centering button 1210, changes the center coordinates of the preset area of ​​the selected preset channel to the center coordinates of the image input by the extraction control device 1 while maintaining the current size of the preset area.

[0080] When the operating mode is the production mode, the input means 12 changes the center coordinates of the next area to the center coordinates of the image input by the extraction control device 1 while maintaining the current size of the next area in accordance with the operator's operation of pressing the centering button 1210.

[0081] The input unit 12 may perform the same processing as when the centering button 1210 is pressed, in response to a predetermined key operation on the keyboard (for example, pressing the Ctrl key and the c key simultaneously).

[0082] <Deleting Preset Area> Next, a description will be given of the process of deleting a preset area by the input unit 12. The input unit 12 deletes a preset area in accordance with an operation by the operator.

[0083] When the operation mode is the setting mode, the input means 12 deletes the preset area information of the selected preset channel in response to the operator pressing the area deletion button 1213 .

[0084] The input unit 12 may perform the same processing as when the area delete button 1213 is pressed, in response to a predetermined key operation on the keyboard (for example, pressing the Delete key).

[0085] Furthermore, when the operation mode is the setting mode, the input means 12 deletes the information of the preset areas of all the preset channels in response to the operator pressing the delete all areas button 1214 .

[0086] The input unit 12 may perform the same processing as when the Delete All Areas button 1214 is pressed, in response to a predetermined key operation on the keyboard (for example, pressing the Shift key and the Delete key simultaneously).

[0087] <File Reading and File Saving> Next, a description will be given of file reading and file saving processes performed by the input unit 12. The input unit 12 reads a file and sets a preset area in accordance with an operator's operation. The input unit 12 also saves the set preset area in a file in accordance with the operator's operation.

[0088] In response to the operator pressing the area file read button 1215, the input means 12 reads the area information recorded in the file and sets the area indicated by the area information as the preset area of ​​all preset channels or the preset area of ​​the preset channel specified by the operator's operation.

[0089] In response to the operator pressing the area file save button 1216, the input unit 12 saves the information on the preset areas of all the preset channels or the information on the preset area of ​​the preset channel designated by the operator's operation in a file.

[0090] <Immediate Transition Operation> Next, we will explain the processing of the immediate transition operation by the input means 12 and the control means 11. The input means 12 generates cut-out information including the type and timing of transition indicating the immediate transition in accordance with the immediate transition operation by the operator. The control means 11 generates a control signal including the cut-out position and size of the area of ​​the partial video, which is the PGM output video, cut out by the video cut-out device 2 based on the cut-out information, and outputs the control signal to the video cut-out device 2.

[0091] 3 is a flowchart illustrating an example of processing for an immediate transition operation. The immediate transition operation button column 1206 shown in FIG. 9 is a button column for executing an immediate transition operation. The input unit 12 determines whether or not the operator has pressed any button in the immediate transition operation button column 1206 (step S301).

[0092] When the input unit 12 determines that no button in the immediate transition operation button column 1206 has been pressed (step S301: N), the input unit 12 waits until the button is pressed.

[0093] When the input means 12 determines that any button in the immediate transition operation button row 1206 has been pressed (step S301: Y), the input means 12 uses the button press as a trigger to generate cut-out information including the identifier of the channel of the PGM output video (identifier of the PGM channel), the type of transition indicating immediate transition and the timing indicating immediate, and the preset area of ​​the preset channel numbered with the button (position and size of the preset area) (step S302).Then, the input means 12 outputs the cut-out information to the control means 11.

[0094] The control means 11 receives the cutout information from the input means 12 and sets the position and size of the preset region to the cutout position and size of the region in the PGM channel based on the cutout information. Then, the control means 11 generates a control signal including the cutout position and size of the region in the PGM channel and outputs the control signal to the video cutout device 2 (step S303).

[0095] As a result, the video clipping device 2 clips an area having the clipping position and size indicated by the control signal from the input video as a clipping area, and outputs the clipped partial video as a PGM output video.

[0096] Therefore, when any button in the immediate transition operation button row 1206 is pressed in the cutting control device 1, the preset area corresponding to that button is immediately reflected as the cutting area by the image cutting device 2, and the partial image of that cutting area is output as the PGM output image.

[0097] The input unit 12 may be configured to switch the operation mode to the production mode when any button in the immediate transition operation button row 1206 is pressed.

[0098] <Cut transition operation> Next, we will explain the processing of cut transition operations by the input means 12 and the control means 11. The input means 12 generates cut-out information including the type and timing of transition indicating the cut transition in accordance with the cut transition operation by the operator. The control means 11 generates a control signal including the cut-out position and size of the area of ​​the partial video, which is the PGM output video, cut out by the video cut-out device 2 based on the cut-out information, and outputs the control signal to the video cut-out device 2.

[0099] 4 is a flowchart illustrating an example of a process for a cut transition operation. The cut button 1203 shown in FIG. 9 is a button for executing a cut transition operation. The input unit 12 determines whether or not the cut button 1203 has been pressed by the operator (step S401).

[0100] When it is determined that the cut button 1203 has not been pressed (step S401: N), the input unit 12 waits until the cut button 1203 is pressed.

[0101] When the input means 12 determines that the cut button 1203 has been pressed (step S401: Y), the input means 12 generates, using the button press as a trigger, cut-out information including a PGM channel identifier, a transition type indicating a cut transition and a timing indicating an immediate cut, and a next region (position and size of the next region) (step S402).Then, the input means 12 outputs the cut-out information to the control means 11.

[0102] The control means 11 receives the cut-out information from the input means 12, and sets the position and size of the next region to the cut-out position and size of the region in the PGM channel based on the cut-out information. Then, the control means 11 generates a control signal including the cut-out position and size of the region in the PGM channel, and outputs the control signal to the video cut-out device 2 (step S403).

[0103] As a result, the video clipping device 2 clips an area having the clipping position and size indicated by the control signal from the input video as a clipping area, and outputs the clipped partial video as a PGM output video.

[0104] Therefore, when the cut button 1203 is pressed in the cutout control device 1, the next region is immediately reflected as a cutout region by the video cutout device 2, and a partial video of the cutout region is output as a PGM output video.

[0105] The input unit 12 may switch the operation mode to the live mode when the cut button 1203 is pressed. The input unit 12 may also perform the same processing as when the cut button 1203 is pressed, in response to a predetermined key operation on the keyboard (for example, pressing the c key).

[0106] <Panning Transition Operation> Next, the processing of the panning transition operation by the input means 12 and the control means 11 will be described. The input means 12 generates cut-out information including the type and timing of the panning transition, the cut-out area, etc., in accordance with the panning transition operation by the operator. The control means 11 generates a control signal including the cut-out position and size of the area of ​​the partial video, which is the PGM output video, cut out by the video cut-out device 2 based on the cut-out information, and outputs the control signal to the video cut-out device 2. In this case, the cut-out information and the control signal change from moment to moment from the start to the end of the panning transition. The cut-out area at the start of the panning transition is the area of ​​the current PGM output video, and the cut-out area at the end of the panning transition is the next area.

[0107] Fig. 5 is a flowchart illustrating an example of a process for a panning transition operation. The pan button row 1204 shown in Fig. 9 is a button row for executing a panning transition operation. The input unit 12 determines whether or not the operator has pressed any button in the pan button row 1204 (step S501).

[0108] When the input unit 12 determines that no button in the pan button column 1204 has been pressed (step S501: N), the input unit 12 waits until the button is pressed.

[0109] When the input means 12 determines that any button in the pan button row 1204 has been pressed (step S501: Y), the input means 12 uses the button press as a trigger to generate cut-out information including the identifier of the PGM channel, the type and timing of the transition indicating the panning transition, and the area (position and size of the area) that automatically moves, expands, or contracts from the current cut-out area (area of ​​the PGM output video) to the next area from time to time (step S502).The input means 12 then outputs the cut-out information to the control means 11.

[0110] Each button in the pan button row 1204 is set with a time required for the transition (5 seconds, 10 seconds, 15 seconds, and 20 seconds in the example of FIG. 9 ), and the time from the start to the end of the transition from the current cropped area to the next area is determined depending on which button is pressed. The time required for the transition may also be specified by a transition time designation slider 1205. The pan slider designation buttons included in the pan button row 1204 determine the cropped area based on the time designated by the transition time designation slider 1205 and the transition curve designated by the transition curve setting slider 1222, and perform a panning transition.

[0111] The control means 11 receives the cut-out information from the input means 12 and sets the position and size of the transition area to the cut-out position and size of the area in the PGM channel based on the cut-out information. Then, the control means 11 generates a control signal including the cut-out position and size of the area in the PGM channel and outputs the control signal to the video cut-out device 2 (step S503).

[0112] As a result, the video clipping device 2 clips an area having the clipping position and size indicated by the control signal from the input video as a clipping area, and outputs the clipped partial video as a PGM output video.

[0113] Therefore, when any button in the pan button row 1204 is pressed in the cut-out control device 1, the area that transitions over a time period corresponding to the button is reflected as a cut-out area by the video cut-out device 2, and a partial image of the cut-out area is output as a PGM output image. In other words, the PGM output image transitions from the area of ​​the PGM output image at the time the button was pressed to the next area by constantly moving, enlarging, or reducing, and when the panning transition is complete, the image of the next area is output as the PGM output image, making it possible to execute a panning transition with an effect that is just as if it were achieved by panning and zooming a camera.

[0114] The input unit 12 may switch the operation mode to the performance mode when any button in the pan button row 1204 is pressed. The input unit 12 may also perform the same processing as when any button in the pan button row 1204 is pressed, in response to a predetermined key operation on the keyboard (for example, pressing any of the f key, m key, s key, v key, or space key).

[0115] 12 is a diagram showing an example of a display on the display unit 14 when a panning transition starts in the live mode. This viewer 1207 shows the display content when any button in the pan button row 1204 is pressed, when the preset region of the preset channel number 1 is set as the cut-out region (region of the cut-out region frame 1220) and the preset region of the preset channel number 3 is set as the next region (region of the next region frame 1221).

[0116] At the time when the transition starts when the button is pressed, the video of the cut-out area of ​​the cut-out area frame 1220 is output from the video cut-out device 2 as a PGM output video.

[0117] 13 is a diagram showing an example of the display on the display means 14 when a panning transition is in progress in the production mode, showing the panning transition from preset area number 1 to preset area number 3. This viewer 1207 shows display content including the cut-out area (area of ​​cut-out area frame 1220) when the panning transition is in progress. When the panning transition is in progress, the video of the cut-out area of ​​cut-out area frame 1220 is output from the video cut-out device 2 as a PGM output video.

[0118] <Panning transition cancellation> Next, a description will be given of the processing for canceling a panning transition by the input unit 12 and the control unit 11. When the stop button 1217 for canceling a panning transition is pressed by the operator while a panning transition operation is being performed, the input unit 12 maintains the cut-out information generated at the time of the panning transition cancellation operation and outputs the cut-out information to the control unit 11. The control unit 11 maintains the control signal generated based on the cut-out information at the time of the panning transition cancellation operation and outputs the control signal to the video cut-out device 2.

[0119] As a result, the transition is stopped, and the PGM output video cut out by the video cutout device 2 is maintained in the state it was in when the button was pressed.

[0120] The input means 12 may be configured to perform the same processing as when the stop button 1217 is pressed, in response to a predetermined key operation on the keyboard (for example, pressing the x key).

[0121] <Swapping of next region and previous region at each transition> Here, the cut-out region immediately before the start of the above-mentioned cut transition or panning transition is referred to as the "previous region." The input means 12 may set the previous region as the new next region immediately after the end of a cut transition or panning transition (setting the position and size of the previous region as the position and size of the next region as they are). In other words, the previous region, which is the region immediately before the start of the cut transition or panning transition, is set as the next region immediately after the end of a cut transition or panning transition.

[0122] <Arbitrary Area Selection> Next, a process will be described in which the input unit 12 sets the position and size of an area in accordance with a predetermined mouse operation or touch panel operation (hereinafter referred to as "area selection operation") on the viewer 1207 by the operator.

[0123] When the operation mode is the setting mode, the input means 12 sets (and modifies) the preset region of the currently specified preset channel in accordance with the region selection operation. When the operation mode is the production mode, the input means 12 sets (and modifies) the next region in accordance with the region selection operation.

[0124] 6 and 7 are diagrams showing examples of an area selection operation. (Setting mode: arbitrary area selection: new diagonal designation operation) Referring to Fig. 6(1), the processing of a new diagonal designation operation when the operation mode is the setting mode will be described. If the start position of a drag (swipe) with the mouse is away from any area frame (e.g., 1208-6 in the example of Fig. 6(1)) of the preset area frames (e.g., 1208-1 to 1208-3 or 1208-6 in Fig. 9) by a predetermined distance or more, the input means 12 determines that the mouse cursor is in a position unrelated to the preset area frame and is in an unselected state.

[0125] The input unit 12 sets a preset area of ​​the specified preset channel as a rectangular area having diagonal corners defined by the drag (swipe) start position and the drag (swipe) end position, thereby setting a preset area frame. The input unit 12 deletes the previous preset area frame of the specified preset channel.

[0126] In this case, the input unit 12 may expand the preset area in the vertical or horizontal direction to a minimum extent so that the set preset area has a predetermined aspect ratio (for example, a horizontal:vertical ratio of 16:9). That is, the input unit 12 corrects the size of the rectangular area by expanding it in either the vertical or horizontal direction so that it includes the original set preset area, and sets a new preset area with a predetermined aspect ratio. The same applies to the operations in Figures 6 (2) and (3) and Figures 7 (4) and (5) described below.

[0127] This allows the video extraction device 2 to extract the video while maintaining the aspect ratio of the video.

[0128] (Setting mode: Arbitrary area selection: New 100% magnification designation operation) The processing of the new 100% magnification designation operation when the operation mode is the setting mode will be described with reference to Fig. 6 (2). When the mouse click position is away from any area frame (e.g., 1208-6 in the example of Fig. 6 (2)) of the preset area frames (e.g., 1208-1 to 1208-3 or 1208-6 in Fig. 9) by a predetermined distance or more, the input means 12 determines that the mouse cursor is in a position unrelated to the preset area frame and is in an unselected state.

[0129] The input unit 12 sets a preset area of ​​the specified preset channel as a high-resolution rectangular area centered on the mouse cursor position, which is the click position, and sets a preset area frame. The input unit 12 deletes the previous preset area frame of the specified preset channel.

[0130] (Setting mode: arbitrary area selection: parallel movement operation) The processing of the parallel movement operation when the operation mode is the setting mode will be described with reference to Fig. 6 (3). When the start position of a drag (swipe) with the mouse is within a predetermined distance (excluding within a predetermined distance from both end points or near the center of the side) from one side of any of the preset area frames (for example, 1208-1 to 1208-3 or 1208-6 in Fig. 9), the input means 12 determines that the mouse cursor is near the side and is in a frame selection state.

[0131] The input unit 12 sets the preset channel to the number of the preset area frame, and moves the preset area of ​​the preset channel in parallel using a vector from the drag (swipe) start position to the drag (swipe) end position to set the preset area frame. In this case, the size of the preset area is fixed.

[0132] The input means 12 may be configured to perform translational movement by operating keys on a keyboard (for example, cursor keys) instead of (or in addition to) dragging with a mouse.

[0133] (Setting mode: Arbitrary area selection: Zoom in / out operation) The processing of the zoom in / out operation (opposite side fixed) when the operation mode is the setting mode will be described with reference to Fig. 7(4). When the start position of the drag (swipe) with the mouse is within a predetermined distance from a predetermined position near the center of one side of any of the preset area frames (e.g., 1208-1 to 1208-3 or 1208-6 in Fig. 9) (1208-6 in the example of Fig. 7(4)), the input means 12 determines that the mouse cursor is near the midpoint of the side and is in a side selection state.

[0134] The input means 12 sets the preset channel to the number of the preset area frame, and sets the preset area of ​​the preset channel as the smallest rectangular area that includes the center of the opposite side of the one side and the drag (swipe) end position and maintains the aspect ratio, thereby setting the preset area frame. In this case, the opposite side of the preset area is fixed.

[0135] 7(5) will be used to explain the process of zooming in and out (fixing the diagonal vertices) when the operation mode is the setting mode. When the start position of a drag (swipe) with the mouse is within a predetermined distance from one vertex of one of the preset area frames (e.g., 1208-1 to 1208-3 or 1208-6 in FIG. 9 ), the input unit 12 determines that the mouse cursor is near the vertex and is in a vertex selection state.

[0136] The input means 12 sets the preset channel to the number of the preset area frame, and sets the preset area of ​​the preset channel as the smallest rectangular area that includes the diagonal vertex position of the one vertex and the drag (swipe) end position and maintains the aspect ratio, thereby setting the preset area frame. In this case, the diagonal vertex of the preset area is fixed.

[0137] In addition, the input means 12 may be configured to change the size of the area (enlarge or reduce it) by operating keys on the keyboard (e.g., the PageUp key and PageDown key) instead of (or in addition to) dragging with the mouse (however, the center position of the area is fixed).

[0138] (Main mode: arbitrary area selection: new diagonal designation operation) The processing of a new diagonal designation operation when the operation mode is the main mode will be described with reference to Fig. 6 (1). When the start position of a drag (swipe) by the mouse is a predetermined distance or more away from the next area frame (for example, 1221 in Fig. 12 described later), the input means 12 determines that the mouse cursor is in a position unrelated to the next area frame and is in an unselected state.

[0139] The input unit 12 sets a next region as a rectangular region having diagonal corners formed by the drag (swipe) start position and the drag (swipe) end position, and sets a next region frame. The input unit 12 deletes the previous next region frame.

[0140] In this case, the input unit 12 may expand the set next region in the vertical or horizontal direction to a minimum extent so that the set next region has a predetermined aspect ratio (for example, 16:9). That is, the input unit 12 expands the set next region in either the vertical or horizontal direction so that it includes the set original next region, thereby setting a new next region with a predetermined aspect ratio. The same applies to the operations in Figures 6 (2) and (3) and Figures 7 (4) and (5), which will be described later.

[0141] (Production mode: Arbitrary area selection: New same-size designation operation) The processing of the new same-size designation operation when the operation mode is production mode will be described with reference to Fig. 6 (2). When the mouse click position is away from the next area frame (for example, 1221 in Fig. 12 described later) by a predetermined distance or more, the input means 12 determines that the mouse cursor is in a position unrelated to the next area frame and is in an unselected state.

[0142] The input means 12 sets the next region as a high-resolution region (rectangular region) centered on the mouse cursor position, which is the click position, and sets a next region frame. The input means 12 deletes the previous next region frame.

[0143] (Production mode: arbitrary area selection: parallel movement operation) The processing of the parallel movement operation when the operation mode is production mode will be described with reference to Fig. 6 (3). When the start position of the drag (swipe) by the mouse is within a predetermined distance from one side of the next area frame (for example, 1221 in Fig. 12 described later) (excluding within a predetermined distance from both end points or near the center of the side), the input means 12 determines that the mouse cursor is near the side and is in a frame selection state.

[0144] The input unit 12 translates the next region along a vector from the drag (swipe) start position to the drag (swipe) end position, and sets a frame for the next region. In this case, the size of the next region is fixed.

[0145] The input means 12 may be configured to perform translational movement by operating keys on a keyboard (for example, cursor keys) instead of (or in addition to) dragging with a mouse.

[0146] (Production mode: arbitrary area selection: zoom in / out operation) The process of zoom in / out operation (opposite side fixed) when the operation mode is production mode will be described with reference to Fig. 7 (4). When the start position of dragging (swiping) with the mouse is within a predetermined distance from a predetermined position near the center of one side of the next area frame (for example, 1221 in Fig. 12 described later), the input means 12 determines that the mouse cursor is near the midpoint of the side and is in a side selection state.

[0147] The input unit 12 sets the next region as the smallest rectangular region that includes the center of the opposite side of the one side and the drag (swipe) end position and maintains the aspect ratio, and sets the next region frame. In this case, the opposite side of the next region is fixed.

[0148] 7 (5), the process of the zoom operation (fixed diagonal vertices) when the operation mode is the production mode will be described. When the start position of the drag (swipe) by the mouse is within a predetermined distance from one vertex of the next region frame (for example, 1221 in FIG. 12 described later), the input means 12 determines that the mouse cursor is near the vertex and is in a vertex selection state.

[0149] The input unit 12 sets the next region as a smallest rectangular region that includes the diagonal vertex position of the one vertex and the drag (swipe) end position and maintains the aspect ratio, and sets the next region frame. In this case, the diagonal vertex of the next region is fixed.

[0150] In addition, the input means 12 may be configured to change the size of the area (enlarge or reduce it) by operating keys on the keyboard (e.g., the PageUp key and PageDown key) instead of (or in addition to) dragging with the mouse (however, the center position of the area is fixed).

[0151] In this way, the operator can set the region of interest and the next region by performing a region selection operation (e.g., a drag or swipe operation), which allows intuitive editing of the region of interest, etc., with the same operability as operating a graphics drawing application.

[0152] Furthermore, when setting the region of interest and the next region, the input means 12 may correct the positions of the region of interest and the next region by translating them as little as possible so as not to include areas outside the range of the area of ​​the video input from the resolution conversion device 3. This makes it possible to prevent the area of ​​the partial video to be cut out from being set outside the input video.

[0153] <Processing of Display Unit 14> Next, a description will be given of processing of the display unit 14 shown in Fig. 2. The display unit 14 displays (previews) the video input from the resolution conversion device 3 on the viewer 1207 shown in Fig. 9 and other figures.

[0154] (Setting Mode) When the operation mode is the setting mode, the display unit 14 displays a preset area frame as a rectangle superimposed on the preview display. The preset area frame may be marked with a character string or a symbol according to the number of the preset channel, and the frame color, brightness, pattern (solid line, dashed line, dotted line, chain line, multiple line, wavy line, etc.), and line width may be changed.

[0155] When the operation mode is the setting mode, the display means 14 may highlight and display the preset area frame of the currently specified preset channel (for example, by thickening the line width, increasing the brightness, changing the pattern, etc.).

[0156] The display means 14 may be configured to light an indicator light (tally lamp) near the button of the currently specified preset channel in the preset button row 1201 shown in FIG. 9 (in the example of FIG. 9, indicator light 1224 is lit).

[0157] Furthermore, when the operation mode is the production mode, the display unit 14 may not display the preset button row 1201 or may gray it out.

[0158] In addition, if there is a preset channel for which the same area as the next area is set as a preset area, the display means 14 may be configured to light up an indicator light (tally lamp) near the button for that preset channel among the buttons in the next area button row 1202 (in the example of Figure 9, indicator light 1225 is lit).

[0159] In addition, if there is no preset channel for which the same area as the next area is set as a preset area, the display means 14 may be configured to turn on an indicator light (tally lamp) 1218 indicating the next area.

[0160] Furthermore, if there is a preset channel for which the same area as the cut-out area (area of ​​the PGM output video) is set as the preset area, the display means 14 may be configured to light up an indicator light (tally lamp) near the button for that preset channel among the buttons in the immediate transition operation button row 1206 (in the example of Figure 9, indicator light 1226 is lit).

[0161] In addition, if there is no preset channel in which the same area as the cut-out area (which may be a cut-out area in transition) is set as a preset area, the display means 14 may be configured to turn on an indicator light (tally lamp) 1219 indicating other cut-out areas.

[0162] (Production Mode) When the operation mode is production mode, the display unit 14 displays the clipping area frame 1220 in a shape such as a rectangle superimposed on the preview display. To distinguish the clipping area frame 1220 from other frames, a specific character string or symbol (for example, character strings such as "PGM" or "LINE") may be added to the clipping area frame 1220, or the color, brightness, pattern (solid line, dashed line, dotted line, chain line, multiple line, wavy line, etc.) or line width of the frame may be changed, or a shape like parentheses may be displayed only near the four corners.

[0163] When the operation mode is the production mode, the display unit 14 displays a next area frame 1221 as a rectangle or the like, superimposed on the preview display. To distinguish the next area frame 1221 from other frames, a specific character string or symbol (for example, character strings such as "NEXT" or "PVW") may be added, or the color, brightness, pattern (solid line, dashed line, dotted line, chain line, multiple line, wavy line, etc.) or line width of the frame may be changed, or a figure in brackets may be displayed only near the four corners.

[0164] The colors of the tally lamp and frame may be different for the cut-out region (for example, red) and the next region (for example, green).

[0165] <Transition Curve> Next, a case where a cut-out area is determined in accordance with a transition curve in the panning transition operation described above will be described. As described above, when any button in the pan button row 1204 is pressed, the input unit 12 generates cut-out information including an identifier of the PGM channel, the type and timing of the transition indicating the panning transition, and the area (cut-out area) that automatically moves, expands, or contracts from the current cut-out area to the next area from time to time.

[0166] The timing of the transition is specified by the operator so that the transition amount of the position and size of the area undergoing the panning transition with respect to time follows a transition curve including, for example, a constant jerk curve.

[0167] In this case, when a button designated by a pan slider included in the pan button column 1204 is pressed, the input means 12 determines the cut-out area that transitions during the panning transition according to the time specified by the transition time specification slider 1205 and the transition curve specified by the transition curve setting slider 1222.

[0168] The input means 12 determines the cut-out region (center coordinates [x(t), y(t)], height v(t) and width h(t)) during the panning transition using the following formula.

[0169] Here, the center coordinates of the cutout region at the start of the transition are [x0, y0], the height is v0, and the width is h0; the center coordinates of the cutout region at the end of the transition are [x1, y1], the height is v1, and the width is h1; the total transition time from the start of the transition to the end of the transition is T, and the elapsed time from the start of the transition is t. The total transition time T is the time specified by the transition time specification slider 1205. The function f(τ) is a function that determines the transition curve, and is a monotonically increasing function in the broad sense with a domain 0≦τ<1 and a range 0≦f(τ)≦1.

[0170] The function f(τ) that defines the transition curve may be, for example, the following equation:

[0171] Furthermore, the function f(τ) that defines the transition curve may be expressed as follows:

[0172] Here, s is a parameter that changes the shape of the transition curve, and is set within the range of 0≦s≦1 by the transition curve setting slider 1222. The parameter s indicates the proportion of the total transition time T that is occupied by a section accompanied by acceleration / deceleration (a section that is not at a constant speed).

[0173] Fig. 8 is a diagram illustrating an example of a transition curve, showing the properties of the function f(τ) in equation (3). The lower part of Fig. 8 shows transition curves based on the function f(τ) when the parameter s is 0.00, 0.65, and 1.00. The horizontal axis represents time, and the vertical axis represents the position of the transition curve based on the function f(τ).

[0174] When the parameter s=0.00, the transition curve increases linearly over time, and it can be seen that the closer the parameter s is to 1, the steeper the slope of the S-shaped curve becomes. This transition curve can be changed using the transition curve setting slider 1222, and the operator can visually confirm the curve shape and the current transition state during panning transition (see the upper right of FIG. 8, FIG. 9, etc.).

[0175] In this way, the shape of the transition curve (transition curve) can be changed by the operation of the operator, so that the agility and smoothness of the transition can be adjusted, and a variety of effects can be produced.

[0176] The upper left side of Fig. 8 shows changes in jerk, acceleration, and velocity from transition start time ta to transition end time tb. With respect to the passage of time, section A has a positive uniform jerk and positive acceleration, section B has a negative uniform jerk and positive acceleration, section C has a constant velocity, section D has a negative uniform jerk and negative acceleration, and section E has a positive uniform jerk and negative acceleration.

[0177] In other words, the section 0<t<sT / 2 is a time section of positive acceleration, the section (T-sT / 2)<t<T is a time section of negative acceleration, and the section sT / 2<t<(T-sT / 2) is a time section of constant velocity.

[0178] More specifically, section A, 0<t<sT / 4, is a section of positive uniform acceleration and positive acceleration; section B, sT / 4<t<sT / 2, is a section of negative uniform acceleration and positive acceleration; section C, sT / 2<t<(T-sT / 2), is a constant speed section; section D, (T-sT / 2)<t<(T-sT / 4), is a section of negative uniform acceleration and negative acceleration; and section E, (T-ST / 4)<t<T, is a section of positive uniform acceleration and negative acceleration.

[0179] In this way, according to the transition curve of the function f(τ) of the above-mentioned formula (3), it is possible to suppress the jerk (false acceleration) within a constant value over the entire time period when s > 0. This makes it possible to make the acceleration curve continuous, and to provide a PGM output image that is less likely to cause a physical impact (shock) from before the transition starts to after the transition ends.

[0180] Therefore, the control means 11 can determine an extraction region that realizes a panning transition that is less shocking to the human eye, and as a result, the video extraction device 2 can realize extraction that suits human intuition. Furthermore, acceleration and deceleration at the start and end of the transition enable a smooth transition with less shock, thereby reducing discomfort when viewing the extracted partial video.

[0181] It should be noted that the larger the parameter s, the more likely it is that the driver will experience shaking due to acceleration or deceleration, so it is preferable to set the parameter s to 0.5≦s≦0.8.

[0182] Furthermore, the cut-out area during the panning transition determined by the input means 12 is set so that both its coordinate values ​​(center coordinates [x(t), y(t)]) and size (height v(t) and width h(t)) change linearly according to a single function f(τ), as in the above formula (1), but either the coordinate values ​​or the size may change linearly according to a single function f(τ).

[0183] Furthermore, the function f(τ) used by the input means 12 when determining the cut-out region during panning transition has the jerk characteristics shown in FIG. 8, but it is sufficient if it has characteristics that include a section in which the absolute value of the jerk is equal to or less than a predetermined value over time.

[0184] 8 is configured to include sections A, B, C, D, and E over time, but as long as both section A and section B exist, one or more of sections C, D, and E may not exist. Also, as long as both section D and section E exist, one or more of sections A, B, and C may not exist.

[0185] That is, if the durations of sections A, B, C, D, and E are TA, TB, TC, TD, and TE, respectively, the transition curves shown in Fig. 8 are TA > 0, TB > 0, TC > 0, TD > 0, and TE > 0. Alternatively, the transition curves may be TA > 0, TB > 0, TC ≥ 0, TD ≥ 0, and TE ≥ 0, or TA ≥ 0, TB ≥ 0, TC ≥ 0, TD > 0, and TE > 0, and the curves in sections A, B, C, D, and E may be continuous over the entire section and connected in a state where they are twice differentiable.

[0186] As described above, according to the cut-out control device 1 of the embodiment of the present invention, the input means 12 specifies one or more preset areas in accordance with the operation of the operator, and specifies one of these areas as a next area that is a transition destination area. Then, in accordance with the operation of the operator, the input means 12 specifies the type and timing of the transition when the cut-out area transitions from the transition source preset area to the transition destination next area, and generates cut-out information including an identifier indicating the channel of the PGM output video, the type and timing of the transition, the cut-out area, etc.

[0187] Based on the cut-out information, the control means 11 generates a control signal for the video cut-out device 2 to cut out a partial video from the input video, the control signal including the cut-out position and size of the cut-out area, and outputs it to the video cut-out device 2.

[0188] The drawing means 13 inputs the resolution-converted image from the resolution conversion device 3, and draws one or more areas selected from the preset area, the next area, and the area indicated by the control signal as a picture based on the cut-out information. The display means 14 displays the picture on the screen to present it to the operator.

[0189] The video clipping device 2 receives a control signal from the clipping control device 1, clips a partial video from the input video based on the control signal, and outputs the partial video.

[0190] In this way, the type and timing of transition when transitioning from a preset area as a transition source to a next area as a transition destination is specified as an area from which a partial video is cut out in accordance with an operation by the operator. This allows the video cutout device 2 to achieve cutout that matches human intuition when cutting out areas that change from moment to moment from the video as partial video.

[0191] [Video Extraction Device 2] Next, the video extraction device 2 shown in Fig. 1 will be described. Fig. 15 is a block diagram showing an example of the configuration of the video extraction device 2 according to an embodiment of the present invention. This video extraction device 2 includes an extraction unit 21.

[0192] The cut-out means 21 receives the input video and a control signal from the cut-out control device 1, cuts out the region from the input video based on the cut-out position and size of the region indicated by the control signal, and outputs the cut-out partial video. As described above, the control signal includes the cut-out position and size of the region for each channel.

[0193] For example, the cutting means 21 cuts out a partial image from the input image based on a control signal including the cutting position and size of an area corresponding to the channel of the PGM output image, sets the partial image as the PGM output image, and outputs the PGM output image. The same applies to the NEXT output image and the CH1 to CH6 output images.

[0194] Here, when the above-mentioned panning transition operation is performed in the cut-out control device 1, the cut-out means 21 inputs a control signal from the cut-out control device 1 including the cut-out position and size of the area corresponding to the channel of the PGM output video, which transitions from the first preset area at the transition source to the second preset area (next area) at the transition destination so as to move, expand or contract from moment to moment, and cuts out a partial video from the input video.

[0195] In other words, the cutting means 21 cuts out a partial image of an area that transitions from the input image by constantly moving, expanding or contracting from the first preset area that is the source of the transition to the second preset area (next area) that is the destination of the transition, and sets the partial image as the PGM output image and outputs it.

[0196] In this case, the area of ​​the partial image to be cut out may be determined by the function f(τ) shown in equations (1) to (3), and both the coordinate values ​​and the size may change linearly in accordance with the function f(τ), or either the coordinate values ​​or the size may change linearly in accordance with the function f(τ).

[0197] <Example of video output by video extraction device 2> Next, an example of video output by the video extraction device 2 will be described. Fig. 10 is a diagram showing examples of input video and CH1 output video, etc. Fig. 11 is a diagram showing examples of PGM output video and NEXT output video. Fig. 14 is a diagram showing an example of PGM output video by a panning transition operation.

[0198] The input video shown in the upper part of Fig. 10 corresponds to the display content of the viewer 1207 shown in Fig. 9. The CH1 output video, CH2 output video, CH3 output video, and CH6 output video shown in the lower part of Fig. 10 are videos that are cut out and output by the video cutout device 2 based on a control signal when the preset area frames 1208-1 to 1208-3 and 1208-6 shown in Fig. 9 are set.

[0199] The input video is, for example, 8K video (resolution 7680×4320 pixels), and the CH1 output video, CH2 output video, CH3 output video, and CH6 output video are, for example, 2K video (resolution 1920×1080 pixels).

[0200] 9, preset regions are not set for preset channel numbers 4 and 5 corresponding to the CH4 output video and CH5 output video. In this case, the CH4 output video and CH5 output video for which no preset region is set may be video obtained by down-converting the entire input video, or may be a predetermined video (for example, a video of a single color of a predetermined color (blue, black, white, gray, etc.), a color bar video, a video of predetermined characters or patterns, etc.).

[0201] Furthermore, for preset channels that exist in the extraction control device 1 but do not exist in the video extraction device 2, the video extraction device 2 does not output individual videos like the CH1 to CH6 output videos.

[0202] The PGM output video shown on the left side of Figure 11 is a video that is cut out and output by the video cutting-out device 2 based on a control signal when, for example, button number 1 in the immediate transition operation button row 1206 shown in Figure 9 is pressed, and the preset area in the preset area frame 1208-1 is set as the cut-out area corresponding to the area of ​​the PGM output video.

[0203] In addition, the NEXT output video shown on the right side of Figure 11 is a video that is cut out and output by the video cutting device 2 based on a control signal when, for example, button number 3 in the next area button row 1202 shown in Figure 9 is pressed, and the preset area of ​​the preset area frame 1208-3 is set to the next area.

[0204] The PGM output video shown in Figure 14 (1) is the video that is cut out and output by the video cutting device 2 based on the control signal when the cut-out area frame 1220 shown in Figure 12 is set and the panning transition begins.

[0205] The PGM output video shown in Figures 14(2) and 14(3) is video that is cut out and output by the video cutout device 2 based on a control signal during panning transition. The PGM output video in Figure 14(3) corresponds to the video of the cutout area frame 1220 shown in Figure 13.

[0206] The PGM output video shown in Fig. 14(4) is a video that is cut out and output by the video cutout device 2 based on the control signal when the panning transition is completed. This PGM output video corresponds to the video of the next area frame 1221 when the panning transition is completed in Figs. 12 and 13.

[0207] As described above, according to the video extraction device 2 of the embodiment of the present invention, when a panning transition operation is performed, the extraction means 21 extracts a partial video from the input video so that the partial video moves, expands, or shrinks from the source area to the destination area from time to time in accordance with the function f(τ).

[0208] This makes it possible to extract a partial video that achieves a panning transition that is less shocking to the human eye. Therefore, when extracting a region that changes from moment to moment from a video as a partial video, it is possible to achieve extraction that matches human intuition.

[0209] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments and can be modified in various ways without departing from the technical concept thereof.

[0210] A normal computer can be used as the hardware configuration of the cut-out control device 1 and the video cut-out device 2 according to the embodiment of the present invention. Each of the cut-out control device 1 and the video cut-out device 2 is configured by a computer including a CPU, a volatile storage medium such as RAM, a non-volatile storage medium such as ROM, an interface, etc.

[0211] The functions of the control means 11, input means 12, drawing means 13 and display means 14 provided in the cutting control device 1 are realized by causing the CPU to execute a program describing these functions.

[0212] Furthermore, the functions of the extraction means 21 provided in the video extraction device 2 are also realized by causing the CPU to execute a program describing these functions.

[0213] These programs are stored in the storage medium and are read and executed by the CPU. These programs can also be stored and distributed on storage media such as magnetic disks (floppy disks, hard disks, etc.), optical disks (CD-ROMs, DVDs, etc.), and semiconductor memories, and can also be transmitted and received via a network.

[0214] 1 Extraction control device 2 Video extraction device 3 Resolution conversion device 10 Video extraction system 11 Control means 12 Input means 13 Drawing means 14 Display means 21 Extraction means 1201 Preset button row 1202 Next area button row 1203 Cut button 1204 Pan button row 1205 Transition time designation slider 1206 Immediate transition operation button row 1207 Viewer 1208-1 to 1208-3, 1208-6 Preset area frame 1209 Resize button row 1210 Centering button 1211 Setting mode button 1212 Production mode button 1213 Delete area button 1214 Delete all areas button 1215 Load area file button 1216 Save area file button 1217 Stop button 1218, 1219, 1224, 1225, 1226 Indicator light 1220 Extraction area frame 1221 Next area frame 1222 Transition curve setting slider 1223 Reconnect button

Claims

1. An image extraction device for extracting a partial image from an input image, comprising: extraction means for extracting from the input image an image in which either or both of the coordinate values ​​and size of a predetermined position change from moment to moment as the partial image, wherein either or both of the coordinate values ​​and size of the predetermined position change linearly with respect to a predetermined function, and the predetermined function has the property of including a section in which the absolute value of jerk is equal to or less than a predetermined value over time.

2. An image extraction device as claimed in claim 1, wherein the predetermined function has, with respect to the passage of time, section A having a positive uniform jerk and positive acceleration, section B having a negative uniform jerk and positive acceleration, section C with a constant speed, section D having a negative uniform jerk and negative acceleration, and section E having a positive uniform jerk and negative acceleration, wherein the durations of section A, section B, section C, section D and section E are TA, TB, TC, TD and TE, respectively, such that TA>0, TB>0, TC≧0, TD≧0 and TE≧0, or TA≧0, TB≧0, TC≧0, TD>0 and TE>0, and wherein the curves in section A, section B, section C, section D and section E are connected in a continuous and second-order differentiable state over the entire section.

3. An extraction control device that controls an image extraction device that extracts a partial image from an input image, comprising: a control means for generating dynamic region information for causing the image extraction device to extract the partial image; an input means for generating extraction information by designating one or more regions of interest in accordance with an operator's operation, designating one region from the one or more regions of interest as a destination region, and designating the region from which the partial image is extracted as an extraction region and specifying the type and timing of a transition when the extraction region transitions from one of the one or more regions of interest to the destination region; a drawing means for drawing either one or both of the one or more regions of interest and the dynamic region information as a picture; and a display means for presenting the picture drawn by the drawing means to the operator, wherein the control means generates the dynamic region information based on the extraction information generated by the input means.

4. An extraction control device according to claim 3, wherein said input means specifies an arbitrary rectangular area as the transition destination area in accordance with an operation by said operator.

5. In the cut-out control device described in claim 3, the input means specifies, in accordance with the operator's operation, as the type of transition, a panning transition in which the cut-out area moves, expands or shrinks from the area of ​​interest to the destination area from time to time.

6. A cut-out control device according to claim 3, wherein the input means is adapted to specify a rectangular area for each of the one or more regions of interest by a drag or swipe operation by the operator.

7. An extraction control device as described in claim 3, characterized in that the input means specifies the one or more regions of interest, the region to which the transition is to be made, and the type and timing of the transition in accordance with the operator's operation on a touch panel, and the display means presents the image to the operator via the touch panel.

8. An extraction control device according to claim 5, characterized in that the input means, in accordance with the operation of the operator, specifies the panning transition as the type of transition, and specifies the timing of the transition so that the transition amount of the position and size of the extraction area transitioning from the area of ​​interest to the destination area with respect to time follows a transition curve including a constant jerk curve.

9. A cutting control device according to claim 8, wherein said input means changes said transition curve in accordance with an operation by said operator.

10. In the cut-out control device described in claim 6, the input means corrects the size of the rectangular region so that the aspect ratio of the rectangular region becomes a predetermined ratio when the rectangular region is specified for each of the one or more regions of interest.

11. In the cut-out control device described in claim 6, the input means is characterized in that, when specifying a rectangular area for each of the one or more areas of interest, the input means corrects the position of the rectangular area so that the rectangular area does not include an area outside the range of the input image.

12. The extraction control device according to claim 3, wherein the input means specifies at least one region of interest from the one or more regions of interest in accordance with an instruction from an external device.

13. A program for causing a computer to function as the video extraction device according to claim 1.

14. A program for causing a computer to function as the cutting control device according to claim 3.

Images