Editing processing device and editing processing program

Abstract

To reduce an operation burden of editing a moving image.SOLUTION: A change in an optional area in a moving image is identified and additional information is disposed on a portion meeting a disposition condition. The change may be identified on the basis of, information on motion tracking performed on the moving image. The additional information may be disposed in the moving image such that voice information is reproduced with timing meeting the disposition condition, in a case where the moving image is reproduced. The additional information may be voice information.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] At least one embodiment of the present invention relates to an editing processing device for video editing and an editing processing program. [Background technology]

[0002] Conventionally, various systems have been used to edit moving images.

[0003] Some such systems, for example, attempt to overlay a re-rendered digital content object onto a displayed animated video content object (see US Pat. No. 6,323,625). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-092557 Summary of the Invention [Problem to be solved by the invention]

[0005] However, with conventional systems, tasks such as placing sound effects in videos required a huge amount of work, as the user had to simply add sounds at the appropriate timing while watching the video. While there are audio trigger functions that use the results of motion-measuring controllers and video analysis, these are primarily designed for real-time sound generation and do not automatically place data. Furthermore, footstep timing auto-generation tools, known in 3D game development, also have a function for automatically placing data, but these use 3D bone coordinates and are not based on video analysis.

[0006] An object of at least one embodiment of the present invention is to solve the above-mentioned problems and reduce the workload of video editing. [Means for solving the problem]

[0007] According to a non-limiting aspect, an editing processing device according to one embodiment of the present invention is an editing processing device for video editing, characterized in that it includes: a specifying means for specifying a change in an arbitrary region of a video; an extraction function for referencing placement-related information in which placement conditions and additional information are associated with each other, and extracting a portion of the change specified by the specifying means that satisfies the placement conditions; and a placement means for placing the additional information in the portion that satisfies the placement conditions based on the extraction result by the extraction means.

[0008] From a non-limiting perspective, an editing processing program according to one embodiment of the present invention is an editing processing program for causing a computer to realize functions for editing moving images, and is intended to realize a specific process for identifying changes in any area of ​​a moving image, an extraction process for referring to placement-related information in which placement conditions and additional information are associated, and extracting a portion of the change identified by the specific function that satisfies the placement conditions, and an arrangement process for placing the additional information in a portion that satisfies the placement conditions based on the extraction results of the extraction function. [Effects of the Invention]

[0009] Each embodiment of the present application addresses one or more of the deficiencies. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing an example of the configuration of an editing processing device corresponding to at least one of the embodiments of the present invention. [Figure 2] 10 is a flowchart illustrating an example of the operation of the edit processing device in edit processing corresponding to at least one of the embodiments of the present invention. [Figure 3] 1 is a block diagram showing an example of the configuration of an editing processing device corresponding to at least one of the embodiments of the present invention. [Figure 4] 10 is a flowchart illustrating an example of an editing process according to at least one of the embodiments of the present invention. [Figure 5] FIG. 10 is an explanatory diagram illustrating an example of an editing screen according to at least one embodiment of the present invention. [Figure 6] FIG. 10 is an explanatory diagram illustrating an example of an editing screen according to at least one embodiment of the present invention. [Figure 7] FIG. 10 is an explanatory diagram illustrating an example of an editing screen according to at least one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, examples of embodiments of the present invention will be described with reference to the drawings. Note that the various components in the examples of the embodiments described below can be combined as appropriate to the extent that no inconsistencies or the like arise. Furthermore, the content described as an example of one embodiment may be omitted in other embodiments. Furthermore, the content of operations and processes unrelated to the characteristic parts of each embodiment may be omitted. Furthermore, the order of the various processes constituting the various flows described below may be in any order to the extent that no inconsistencies or the like arise in the processing content.

[0012] [First embodiment] FIG. 1 is a block diagram showing an example of the configuration of an editing processing device 10 according to an embodiment of the present invention. The editing processing device 10 is managed by a user of the editing processing device and has various functions for providing information related to various processes. In this example, the editing processing device 10 is configured as an information processing device and includes a storage medium for storing various types of information. The configuration of the editing processing device 10 is not particularly limited as long as it includes general components for performing various processes as a computer, such as a control unit and a communication unit. An example of the hardware configuration of the editing processing device 10 will be briefly described below.

[0013] Although not shown, the editing processing device 10 includes at least a CPU (Central Processing Unit), a memory, and a storage device.

[0014] The CPU is a central processing unit that performs various calculations and controls. If the editing processing device 10 is equipped with a GPU (Graphics Processing Unit), some of the calculations and controls may be performed by the GPU. The editing processing device 10 uses the CPU to perform various information processes required for video editing using data appropriately read into memory, and stores the obtained processing results in a storage device as needed.

[0015] The storage device functions as a storage medium for storing various types of information. The configuration of the storage device is not particularly limited, but it is preferable that the storage device be configured to be able to store all of the various types of information required for video editing. Examples of such a configuration include an HDD and an SSD. However, the storage unit that stores the various types of information only needs to have a storage area accessible by the editing processing device 10, and may be configured to have a dedicated storage area outside the editing processing device 10, for example.

[0016] The editing processing device 10 includes at least a specifying unit 11, an extracting unit 12, and a arranging unit 13.

[0017] The identification unit 11 has a function of identifying a change in an arbitrary area in a moving image.

[0018] The structure of the video is not particularly limited as long as it is trackable, and may be an animated video, a live-action video, or a video of a video game being played.

[0019] The arbitrary region means a region that is arbitrarily set in a video. The configuration for setting the arbitrary region is not particularly limited, but it is preferable that the arbitrary region can be set when tracking is performed. An example of such a configuration is a known application that reads and analyzes a captured image.

[0020] Furthermore, identifying a change means identifying how an arbitrary region changes when a video is played. The configuration for identifying the change is not particularly limited, and the change may be identified by selecting data from motion tracking performed in advance, or may be identified in real time in accordance with the playback of the video.

[0021] The extraction unit 12 has a function of extracting a part that satisfies the placement condition in the change identified by the identification unit 11, by referring to placement related information in which the placement condition and the additional information are associated with each other.

[0022] Here, the placement condition means a condition for placing the additional information. The content of the placement condition is not particularly limited, but it is preferable that the user can recognize the relationship with the additional information. Examples of placement conditions include moving the arbitrary area up and down within a predetermined range, or swinging the arbitrary area left and right within a predetermined range.

[0023] The additional information refers to information that can be added to a video. The content of the additional information is not particularly limited, but it is preferable that the user can recognize that it has been added to the video. Examples of additional information include audio information and image information.

[0024] The placement unit 13 has a function of placing additional information in a portion that satisfies a placement condition based on the extraction result by the extraction unit 12.

[0025] Here, the configuration for placing additional information in a video is not particularly limited, but it is preferable that the configuration does not require user operation. In other words, it is preferable that the placement in the video is performed automatically. An example of such a configuration is a configuration in which a predetermined amount of audio is automatically associated with an elapsed playback time that satisfies the placement condition.

[0026] Next, the operation of the editing processing device 10 of this example will be described.

[0027] 2 is a flowchart showing an example of editing processing executed by the editing processing device 10. In this example, the editing processing involves processing related to adding information to a moving image. The following describes an example in which the editing processing is executed by the editing processing device 10 alone.

[0028] The editing process is started, for example, when a user of the editing processing device 10 requests the editing process.

[0029] In the editing process, the editing processing device 10 first identifies a change in an arbitrary region (step S101). In this example, the editing processing device 10 reads data of motion tracking that has been performed in advance on the video, and identifies a change in the arbitrary region.

[0030] When the edit processing device 10 identifies the change in the arbitrary region, it extracts the portion that satisfies the placement condition (step S102). In this example, the edit processing device 10 extracts the portion that satisfies the placement condition by comparing the placement-related information with the motion tracking data.

[0031] When the editing processing device 10 extracts a portion that satisfies the placement condition, the editing processing device 10 places the additional information (step S103). In this example, the editing processing device 10 adds the additional information to the video by associating the elapsed playback time of the video with the additional information.

[0032] As described above, as one aspect of the first embodiment, the editing processing device 10 is configured to include an identification unit 11, an extraction unit 12, and a placement unit 13, so that it can identify changes in any area in a video, refer to placement-related information in which placement conditions and additional information are associated, extract parts of the identified changes that satisfy the placement conditions, and place the additional information in the parts that satisfy the placement conditions based on the extraction results, thereby reducing the workload of video editing.

[0033] That is, the user does not need to select the timing to add information while watching the video, which reduces the workload of editing the video.

[0034] [Second embodiment] 3 is a block diagram showing the configuration of an edit processing device 20, which is an example of the edit processing device 10. In this example, the edit processing device 20 includes at least an identification unit 21, an extraction unit 22, an arrangement unit 23, a storage unit 24, and a display unit 25.

[0035] The identification unit 21 has a function of identifying a change in an arbitrary area in a moving image.

[0036] Here, the configuration for identifying the change is not particularly limited, but it is preferable that the configuration allows the user to recognize the area defined as the arbitrary area. Below, we will explain an example in which changes in an arbitrary area of ​​a video are tracked using a motion tracking function such as a composite tool, and time-series key frame sequence information (KFS) is generated. It is preferable that the change include at least one of movement on the screen, movement of coordinates in virtual space, and deformation.

[0037] The extraction unit 22 has a function of referring to the placement related information in which the placement conditions and the additional information are associated with each other, and extracting a part that satisfies the placement conditions in the change identified by the identification unit 21.

[0038] Here, the configuration for identifying the part that satisfies the placement condition is not particularly limited, but it is preferable that the configuration does not require user judgment. Below, we will explain an example of analyzing the generated KFS and detecting the occurrence timing of some action defined as the placement condition.

[0039] The placement unit 23 has a function of placing additional information in a portion that satisfies a placement condition based on the extraction result by the extraction unit 22.

[0040] Here, the configuration for placing additional information is not particularly limited, but it is preferable that the placed additional information be recognizable by the user. The following describes an example in which assets and trigger information (audio / MIDI / video files, etc.) corresponding to the placement conditions are automatically placed at the timing detected by KFS analysis (motion detection). Note that the various information that can be placed here is not limited to audio, and it is preferable that the configuration include any information, such as images, videos, and compositor markers. In this case, it is also possible to provide placement-related information that includes multiple combinations of placement conditions and additional information.

[0041] The storage unit 24 has a function of storing various information required for editing processing.

[0042] Here, the configuration of the various information is not particularly limited, but it is preferable that the configuration include placement-related information. Below, an example will be described in which, in addition to the placement-related information, information for displaying an editing screen and information for playing a video in which additional information is placed are stored in the storage device 24.

[0043] The display unit 25 has a function of displaying various screens for editing.

[0044] Here, the configuration for displaying the various screens is not particularly limited, but it is preferable that the configuration allows the user to distinguish between the video and the additional information. The following describes an example in which an editing screen is displayed.

[0045] 4 is a flowchart showing an example of editing processing executed by the editing processing device 20. In this example, the editing processing involves processing related to adding information to a moving image. Each processing step will be explained below. The order of each processing step may be random as long as no inconsistencies occur in the processing content.

[0046] The editing process is started, for example, when a user of the editing processing device 20 requests the editing process.

[0047] In the editing process, the editing processing device 20 first displays an editing screen (step S201). In this example, the editing screen includes a display area for a moving image selected by the user.

[0048] Fig. 5 is an explanatory diagram for explaining an example of an editing screen. As shown in Fig. 5, the editing screen includes a video display area 500 and a time display area 510 that indicates the elapsed playback time of the video. Below, an example will be described in which additional information is placed on a video that depicts a character 501 moving toward the depth of the screen.

[0049] When the editing screen is displayed, the editing processing device 20 selects motion tracking (step S202). In this example, a case will be described where motion tracking that has been performed beforehand is selected from the editing screen.

[0050] Fig. 6 is an explanatory diagram for explaining an example of the editing screen. As shown in Fig. 6, the editing screen includes a tracking area image 601 that represents an arbitrary area in a moving image, and a tracking information display area 602 that represents information related to tracking. Below, an example will be explained in which the tracking area is provided on the back of a character.

[0051] When motion tracking is selected, the editing processing device 20 accepts an arrangement request (step S203). In this example, a case will be described where an operation to specify a predetermined option (or an operation button) provided on the editing screen is accepted.

[0052] When the editing processing device 20 receives the placement request, it detects a motion (step S204). In this example, a case will be described in which a motion that satisfies the placement condition is detected from the selected motion tracking.

[0053] When the editing processing device 20 detects a motion, it automatically arranges the asset (step S205). In this example, a case where a script (tracking auto SE script) that automatically adds a sound effect (SE) is executed will be described as an example.

[0054] Fig. 7 is an explanatory diagram for explaining an example of an editing screen. As shown in Fig. 7, the editing screen includes an additional information display area 700 in which additional information automatically placed by executing a tracking auto SE script is displayed at a position corresponding to the elapsed playback time of the video. Below, an example will be described in which an additional image 701 representing at least a part of the additional information is displayed in the additional information display area 700. In other words, the editing screen shown in Fig. 7 indicates a state in which SEs as additional information are placed in seven locations relative to the video.

[0055] As described above, as one aspect of the second embodiment, the editing processing device 20 is configured to include an identification unit 21, an extraction unit 22, and an arrangement unit 23, so that it can identify changes in any area in the video (for example, identify changes in an area in the video where a moving character is displayed), refer to arrangement-related information in which arrangement conditions and additional information are associated, extract a portion of the identified change that satisfies the arrangement conditions (for example, extract a portion where a change corresponding to footsteps has occurred), and, based on the extraction result, arrange the additional information in the portion that satisfies the arrangement conditions (for example, arrange footsteps), thereby reducing the workload of video editing.

[0056] In other words, it will be possible to realize applications (plug-ins, etc.) such as automatically arranging assets such as sound and video on the composite tool based on information on video material alone or motion tracking results.

[0057] Furthermore, in the example of the second embodiment described above, the editing processing device 20 is configured to identify changes based on information regarding motion tracking performed on the video (for example, by analyzing motion tracking performed on the video to be edited), so there is no need to analyze and edit the video simultaneously, which further reduces the workload of video editing.

[0058] Furthermore, in the example of the second embodiment described above, the additional information includes audio information, and the editing processing device 20 is configured to place the additional information in the video so that when the video is played back, the audio information is played back at a timing that satisfies the placement conditions (for example, footsteps are generated in conjunction with the movement of a character), thereby reducing the workload of editing the video.

[0059] Although not specifically mentioned in the example of the second embodiment described above, the editing processing device 20 may be configured to play the edited video and change the position of the additional information placed on the video being played back based on a change operation on the position of the additional information. In other words, the video in which the additional information has been automatically placed on the editing screen may be further manually edited. This configuration makes it possible to organize videos in a way that better suits the user's needs.

[0060] Furthermore, in the example of the second embodiment described above, the editing processing device 20 is configured to display changes in the identified arbitrary area in a manner that allows the user to distinguish them (e.g., in the tracking information display area 602), allowing the editor to easily grasp the relationship between the automatically arranged additional information and changes in the arbitrary area. In this case, the editing processing device 20 may be configured to accept a user's designation of the elapsed playback time of the video, and to identifiably display at least one of the state of the video at the accepted elapsed playback time, the state of the arbitrary area, and the arranged additional information. This configuration reduces the burden on the user of checking the information after automatic arrangement.

[0061] As described above, each embodiment of the present application solves one or more deficiencies. Note that the effects of each embodiment are non-limiting effects or examples of effects.

[0062] In the above-described embodiments, the edit processing devices 10 and 20 execute the above-described various processes in accordance with various control programs (for example, edit processing programs) stored in their own storage devices.

[0063] Furthermore, the configuration of the editing processing devices 10 and 20 is not limited to the configurations explained as examples of the above-mentioned embodiments, and may be realized by, for example, a terminal and a server. [Industrial Applicability]

[0064] According to one embodiment of the present invention, it is useful for reducing the workload of video editing. [Explanation of symbols]

[0065] 10,20 Editing processing device 11,21 Specific part 12,22 Extraction part 13,23 Placement part 24 Memory section 25 Display section

Claims

1. An editing processing device for video editing, A means for identifying a change in an arbitrary region in a video; an extraction means for extracting a portion of the change identified by the identification means that satisfies the placement condition by referring to placement-related information in which the placement condition is associated with additional information; a placement means for placing the additional information in a portion that satisfies the placement condition based on the extraction result by the extraction means; a display means for displaying the additional information arranged by the arrangement means and the change identified by the identification means in a manner that allows a user to distinguish the additional information and the change; a change means for changing the position of the additional information based on a change operation for the position of the additional information arranged by the arrangement means; 20. An editing processing device comprising:

2. The identification means identifies the change based on information about motion tracking performed on the video.

2. The editing processing device according to claim 1.

3. the additional information includes audio information; The arrangement means arranges the additional information on the video so that, when the video is played, the audio information is played at a timing that satisfies the arrangement condition.

3. The editing processing device according to claim 1.

4. An editing processing program for causing a computer to realize a function for video editing, A process for identifying a change in an arbitrary region in a video; an extraction process of extracting a portion that satisfies the placement condition in the change identified by the identification process, by referring to placement related information in which the placement condition is associated with additional information; a placement process for placing the additional information in a portion that satisfies the placement condition based on the extraction result of the extraction process; a display process for displaying the additional information arranged by the arrangement process and the change identified by the identification process in a manner that allows a user to distinguish the additional information and the change; a change process for changing the position of the additional information based on a change operation for the position of the additional information arranged by the arrangement process; An editing program to make this happen.

Images