Moving image editing method, moving image editing device, and moving image editing program

Abstract

To provide a moving image editing device and the like capable of supporting creation of a shortened moving image of a desired reproduction time by editing a moving image.SOLUTION: The moving image editing device comprises: a designated time acquisition unit which acquires a designated time concerning a reproduction time of a shortened moving image scheduled to be generated; a candidate extraction unit which extracts an editing point candidate from a moving image of an editing source; an editing point specifying unit which specifies a presentation editing point to present to a user based on the designated time acquired by the designated time acquisition unit and the editing point candidate extracted by the candidate extraction unit; and an editing point output unit which outputs a presentation editing point specified by the editing point specifying unit.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to a video editing method, a video editing device, and a video editing program. [Background technology]

[0002] A system is known that detects discontinuities (cut points) in a video, detects silent parts of the audio as breaks in the audio, and detects parts that are both breaks in the audio and cut points in the video as breaks in the video as a whole, and then selects representative frames for each part thus separated and displays them in a list (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 9-214879 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology as shown in Patent Document 1, when generating a shortened video by shortening the playback time of the original video, cuts that cannot be adopted depending on the playback time of the desired shortened video are also displayed as a list of editing points. For this reason, a user who wants to create a shortened video by editing a video must choose from many editing points displayed in a list, and the user may have to consider many editing points. In other words, sufficient consideration has not been given to making it easier for users to edit videos when shortening them to a desired playback time.

[0005] The present disclosure has been made to solve such problems, and has an object to provide a video editing method, a video editing device, and a video editing program that can assist in creating a video with a desired shortened playback time by editing moving images. [Means for solving the problem]

[0006] The video editing method according to the present disclosure is a video editing method executed by a computer of an information processing device, and performs video editing to generate a shortened video by shortening the playback time of an original video, and includes a designated time acquisition step of acquiring a designated time related to the playback time of the shortened video to be generated, a candidate extraction step of extracting editing point candidates from the original video, an editing point identification step of identifying suggested editing points to be presented to a user based on the designated time acquired in the designated time acquisition step and the editing point candidates extracted in the candidate extraction step, and an editing point output step of outputting the suggested editing points identified in the editing point identification step.

[0007] A video editing program according to the present disclosure is for causing a computer of an information processing device to execute the above video editing method.

[0008] The video editing device according to the present disclosure is a video editing device that performs video editing to generate a shortened video by shortening the playback time of an original video, and includes a designated time acquisition unit that acquires a designated time related to the playback time of the shortened video to be generated, a candidate extraction unit that extracts editing point candidates from the original video, an editing point identification unit that identifies suggested editing points to be presented to a user based on the designated time acquired by the designated time acquisition unit and the editing point candidates extracted by the candidate extraction unit, and an editing point output unit that outputs the suggested editing points identified by the editing point identification unit. Effect of the Invention

[0009] The video editing method, video editing device, and video editing program according to the present disclosure have the effect of assisting in creating a video with a desired shortened playback time by editing moving images. [Brief description of the drawings]

[0010] [Figure 1] 1 is a diagram showing an overall configuration of a video editing system according to a first embodiment. [Diagram 2]2 is a block diagram showing a configuration of a terminal of the video editing system according to the first embodiment. FIG. [Diagram 3] 1 is a block diagram showing a configuration of a server (information processing device) of the video editing system according to the first embodiment. [Figure 4] 4 is a block diagram showing a configuration of a control unit of the server according to the first embodiment. FIG. [Diagram 5] 4 is a flowchart showing an example of a processing operation of a server (information processing device) of the video editing system according to the first embodiment. [Figure 6] 1 is a diagram showing an example of displaying suggested editing points and generating a shortened video by the video editing system according to the first embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The embodiments for implementing the video editing method, video editing device, and video editing program according to the present disclosure will be described with reference to the accompanying drawings. In each drawing, the same or corresponding parts are given the same reference numerals, and duplicated descriptions are appropriately simplified or omitted. In the following description, for convenience, the positional relationship of each structure may be expressed based on the illustrated state. Note that the present disclosure is not limited to the following embodiments, and the embodiments may be freely combined, any component of each embodiment may be modified, or any component of each embodiment may be omitted, within the scope of the gist of the present disclosure.

[0012] Embodiment 1 A first embodiment of the present disclosure will be described with reference to Figs. 1 to 6. Fig. 1 is a diagram showing an overall configuration of a video editing system. Fig. 2 is a block diagram showing a configuration of a terminal of the video editing system. Fig. 3 is a block diagram showing a configuration of a server (information processing device) of the video editing system. Fig. 4 is a block diagram showing a configuration of a control unit of the server. Fig. 5 is a flowchart showing an example of a processing operation of the server (information processing device) of the video editing system. Fig. 6 is a diagram showing an example of displaying suggested editing points and generating a shortened video by the video editing system.

[0013] As shown in Fig. 1, a video editing system 1 according to this embodiment includes a server 100 and a terminal 200. In the video editing system, the server 100 and the terminal 200 are communicably connected via a network (not shown). The server 100 provides video editing services and the like to the terminal 200 owned by a user via the network. The number of terminals 200 connected to the network 300 is not limited to one, and may be two or more. The video editing system 1 may or may not include the terminal 200.

[0014] The network serves to connect one or more terminals 200 and one or more servers 100. In other words, the network refers to a communication network that provides a connection path so that the terminals 200 can transmit and receive data after connecting to the server 100. One or more parts of the network may or may not be a wired network or a wireless network.

[0015] The network may include, for example, an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), a portion of the Internet, a portion of the public switched telephone network (PSTN), a mobile phone network, an integrated service digital network (ISDN), a wireless LAN, a long term evolution (LTE), a code division multiple access (CDMA), Bluetooth (registered trademark), or a satellite communication, or a combination of two or more of these. The network may include one or more networks.

[0016] The terminal 200 may be any information processing terminal capable of implementing the functions of the embodiments of the present disclosure. The terminal 200 includes, for example, a smartphone, a mobile phone (feature phone), a computer (e.g., a desktop PC, a laptop PC, a tablet PC, etc.), a media computer platform (e.g., a cable or satellite set-top box, a digital video recorder), a handheld computer device (e.g., a PDA (Personal Digital Assistant), an email client, etc.), a wearable terminal (a glasses-type device, a watch-type device, etc.), or other types of computers or communication platforms. The terminal 200 may also be expressed as an information processing terminal.

[0017] The server 100 has a function of providing a predetermined service to the terminal 200. The server 100 may be any device as long as it is an information processing device capable of realizing the functions of the embodiment according to the present disclosure. The server 100 includes, for example, a server device, a computer (e.g., a desktop PC, a laptop PC, a tablet PC, etc.), a media computer platform (e.g., a cable or satellite set-top box, a digital video recorder), a handheld computer device (e.g., a PDA, an email client, etc.), or other types of computers or communication platforms. The server 100 may also be expressed as an information processing device. When it is not necessary to distinguish between the server 100 and the terminal 200, the server 100 and the terminal 200 may or may not be expressed as information processing devices.

[0018] Next, the configuration of the terminal 200 will be described with reference to Fig. 2. The terminal 200 includes a terminal control unit 230, a terminal storage unit 220, a terminal communication unit 210, an input / output unit 240, a display unit 250, a microphone 260, a speaker 270, and a camera 280. The hardware components of the terminal 200 are connected to each other via a bus, for example. It is not essential that the hardware configuration of the terminal 200 includes all the components described here. For example, the terminal 200 may or may not be configured such that individual components such as the camera 280 or multiple components are detachable.

[0019] The terminal communication unit 210 transmits and receives various data via the network 300. The communication may be performed either wired or wirelessly, and any communication protocol may be used as long as the communication between the terminal and the server 100 is possible. The terminal communication unit 210 has a function of communicating with the server 100 via the network 300.

[0020] The terminal communication unit 210 includes a terminal transmission unit 211 and a terminal reception unit 212. The terminal transmission unit 211 transmits various data to the server 100 according to instructions from the terminal control unit 230. The terminal reception unit 212 receives various data transmitted from the server 100 and transmits it to the terminal control unit 230. The terminal communication unit 210 may also be expressed as a terminal communication I / F (interface). Furthermore, when the terminal communication unit 210 is configured from a physically structured circuit, it may also be expressed as a terminal communication circuit.

[0021] The input / output unit 240 includes an input unit and an output unit. The input unit is a device for inputting various operations to the terminal 200. The output unit is a device for outputting the processing results processed by the terminal 200. The input / output unit 240 may be integrated with the input unit and the output unit, may be separated into the input unit and the output unit, or may not be.

[0022] The input unit is realized by any one or a combination of all types of devices that can receive input from a user and transmit information related to the input to the terminal control unit 230. The input unit can include, for example, a touch panel, a touch display, hardware keys such as a keyboard, a pointing device such as a mouse, and the like.

[0023] The output unit is realized by any one or a combination of all kinds of devices capable of outputting the processing results processed by the terminal control unit 230. The output unit may include, for example, a touch panel, a touch display, a speaker (audio output), a lens (e.g., 3D (Three Dimensions) output or hologram output), a printer, etc.

[0024] The display unit 250 is realized by any one of all kinds of devices capable of displaying according to the display data written in the frame buffer, or a combination thereof. The display unit 250 may include, for example, a touch panel, a touch display, a monitor (for example, a liquid crystal display or an OELD (Organic Electroluminescence Display)), a head mounted display (HDM), a projection mapping, a hologram, a device capable of displaying images, text information, and the like in the air (which may or may not be a vacuum). Note that these display units 250 may or may not be capable of displaying display data in 3D. Note that, when the input / output unit 240 has a touch panel, the input / output unit 240 and the display unit 250 may be disposed opposite each other with approximately the same size and shape.

[0025] The terminal control unit 230 has a circuit physically structured to execute the functions realized by the code or instructions included in the program, and is realized by, for example, a data processing device built into the hardware. Therefore, the terminal control unit 230 may or may not be expressed as a control circuit.

[0026] The terminal control unit 230 includes, for example, a central processing unit (CPU), a microprocessor, a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc.

[0027] The terminal storage unit 220 has a function of storing various programs and various data required for the operation of the terminal 200. The terminal storage unit 220 includes various storage media such as a hard disk drive (HDD), a solid state drive (SSD), a flash memory, a random access memory (RAM), and a read only memory (ROM). The terminal storage unit 220 may or may not be expressed as a memory.

[0028] Terminal 200 stores a program in terminal storage unit 220, and executes this program, causing terminal control unit 230 to execute processing as each unit included in terminal control unit 230. That is, the program stored in terminal storage unit 220 causes terminal 200 to realize each function executed by terminal control unit 230. In other words, a processor executes a program stored in memory in terminal 200, and the hardware and software of terminal 200 work together to realize the function of each unit included in terminal 200. Note that this program may or may not be expressed as a program module.

[0029] The microphone 260 is used to input audio data, the speaker 270 is used to output audio data, and the camera 280 is used to obtain moving image data and / or still image data.

[0030] Next, the configuration of the server 100 will be described with reference to Fig. 3. As shown in the figure, the server 100 includes a server communication unit 110, a server storage unit 120, and a server control unit 130. The hardware components of the server 100 are connected to each other, for example, via a bus. It is not essential that the hardware configuration of the server 100 includes all the components described here.

[0031] The server communication unit 110 transmits and receives various data via a network. The communication may be performed either wired or wirelessly, and any communication protocol may be used as long as the communication between the two devices is possible. The server communication unit 110 has a function of communicating with the terminal 200 and the like via the network.

[0032] The server communication unit 110 includes a server transmission unit 111 and a server reception unit 112. The server transmission unit 111 transmits various data to the terminal 200, etc., in accordance with instructions from the server control unit 130. The server reception unit 112 receives various data transmitted from the terminal 200, etc., and transmits it to the server control unit 130. The server communication unit 110 may also be expressed as a communication I / F (interface). Furthermore, when the server communication unit 110 is configured from a physically structured circuit, it may also be expressed as a communication circuit.

[0033] The server control unit 130 has a circuit that is physically structured to execute the functions realized by the code or instructions included in the program, and is realized by, for example, a data processing device built into hardware. Therefore, the server control unit 130 may or may not be expressed as a control circuit.

[0034] The server control unit 130 is typically a central processing unit (CPU), and may or may not be a microprocessor, a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc. In the present disclosure, the server control unit 130 is not limited to these.

[0035] The server storage unit 120 has a function of storing various programs and various data required for the operation of the server 100. The server storage unit 120 is realized by various storage media such as a hard disk drive (HDD), a solid state drive (SSD), a flash memory, a random access memory (RAM), and a read only memory (ROM). However, in the present disclosure, the server storage unit 120 is not limited to these. Furthermore, the server storage unit 120 may or may not be expressed as a memory.

[0036] Server 100 stores a program in server storage unit 120, and executes this program, causing server control unit 130 to execute processing as each unit included in server control unit 130. That is, the program stored in server storage unit 120 causes server 100 to realize each function executed by server control unit 130. In other words, a processor executes the program stored in memory in server 100, and the hardware and software of server 100 work together to realize the function of each unit included in server 100. Note that this program may or may not be expressed as a program module.

[0037] The server control unit 130 of the server 100 may or may not realize each process by a CPU having a control circuit, or by a logic circuit (hardware) formed in an integrated circuit (IC (Integrated Circuit) chip, LSI (Large Scale Integration)), or a dedicated circuit. These circuits may be realized by one or more integrated circuits, and the multiple processes shown in the embodiments of the present disclosure may or may not be realized by one integrated circuit. LSIs may also be called VLSIs, super LSIs, ultra LSIs, etc., depending on the degree of integration.

[0038] In addition, the programs (e.g., software programs, computer programs, or program modules) of the embodiments of the present disclosure may or may not be provided in a state stored in a computer-readable storage medium. The storage medium can store the programs in a "non-transitory tangible medium." In addition, the programs may or may not be for realizing part of the functions of the embodiments of the present disclosure. Furthermore, the programs may or may not be what can realize the functions of the embodiments of the present disclosure in combination with a program already recorded in the storage medium, that is, a so-called difference file (difference program).

[0039] The storage medium may include one or more semiconductor-based or other integrated circuits (ICs) (e.g., field programmable gate arrays (FPGAs) or application specific ICs (ASICs)), hard disk drives (HDDs), hybrid hard drives (HHDs), optical disks, optical disk drives (ODDs), magneto-optical disks, magneto-optical drives, floppy diskettes, floppy disk drives (FDDs), magnetic tapes, solid state drives (SSDs), RAM drives, secure digital cards, or drives, any other suitable storage media, or any suitable combination of two or more of these. The storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile, where appropriate. It should be noted that the storage medium is not limited to these examples and may be any device or medium capable of storing a program. The storage medium may or may not be referred to as a memory.

[0040] Furthermore, the program of the present disclosure may or may not be provided to the server 100 via any transmission medium capable of transmitting the program (such as a communication network or broadcast waves). When the program is provided via a transmission medium, the server 100 can realize the functions of the multiple functional units shown in each embodiment by executing a program downloaded via the Internet or the like.

[0041] Moreover, the embodiment according to the present disclosure may be realized in the form of a data signal embedded in a carrier wave in which a program is embodied by electronic transmission. Moreover, at least a part of the processing in the server 100 may or may not be realized by cloud computing configured by one or more computers. Moreover, at least a part of the processing in the terminal 200 may or may not be configured to be performed by the server 100. Moreover, a part or all of the processing in the server 100 may or may not be configured to be performed by the terminal 200. In this case, a part or all of the processing of each functional unit of the server control unit 130 of the server 100 may or may not be configured to be performed by the terminal 200.

[0042] When the terminal 200 performs a part or all of the processes described in the embodiment according to the present disclosure, the program in which the processes are described and executed by the terminal 200 may be a program that is preinstalled in the terminal 200 in advance, that is, a program that is preinstalled, or a program that is installed from an external recording medium such as a CD (Compact Disc), an SD (Secure Digital) card, or a USB (Universal Serial Bus) memory, or a program that is downloaded from an external server including a cloud server to the terminal 200 via the Internet or the like and installed. Specifically, for example, when the terminal 200 is a smartphone, the program of the present disclosure may be at least a part of a program included in an APP (application) that is installed in the smartphone by a user via the Internet. In another example, when the terminal 200 is a PC, the program of the present disclosure may be at least a part of a program included in software that is installed in the PC by a user via the Internet. In addition, when a part or all of the processes described in the embodiment according to the present disclosure are executed by a system accessed from the terminal 200 such as a smartphone or a PC via a web browser, the program of the present disclosure may be at least a part of a program that is temporarily cached in the terminal 200 by the web browser.

[0043] The program of the present disclosure may be implemented using, for example, a scripting language such as ActionScript or JavaScript (registered trademark), an object-oriented programming language such as Objective-C or Java (registered trademark), or a markup language such as HTML5.

[0044] The video editing system 1 according to the present disclosure performs video editing to generate a shortened video by shortening the playback time of an original video. In the video editing system 1 according to this embodiment, as shown in FIG. 3, the server storage unit 120 of the server 100 includes a video storage unit 121. The original video described above is stored in the video storage unit 121. The original video may not be stored permanently by the video storage unit 121 but may be stored temporarily. For example, the original video may be temporarily saved in a cache for video editing. Alternatively, for example, a user may temporarily upload a video to the server 100 for editing, and when the editing is completed, the original video may be deleted from the video storage unit 121.

[0045] In the video editing system 1 according to this embodiment, as shown in FIG. 3 and FIG. 4, the server 100 has, as functions realized by the server control unit 130, a specified time acquisition unit 131, a candidate editing point extraction unit 132, a proposed editing point identification unit 133, and a proposed editing point output control unit 134.

[0046] The designated time acquisition unit 131 acquires the designated time. The designated time is related to the playback time of the shortened video to be generated. The designated time is, for example, a value that designates the playback time of the shortened video to be generated. The designated time is, for example, input by the user. In this case, the user of the video editing system 1 inputs the value of the designated time by operating the keyboard, touch panel, etc. of the input / output unit 240 of the terminal 200. In this case, the value of the designated time may be input directly, or may be input by selecting a desired value from a plurality of options (for example, 10 minutes, 5 minutes, 3 minutes, 1 minute, etc.) set in advance. The input value of the designated time is transmitted from the terminal transmission unit 211 of the terminal 200 to the server 100 via the network. The server reception unit 112 of the server 100 receives the value of the designated time transmitted from the terminal 200. Then, the designated time acquisition unit 131 acquires the value of the designated time received by the server reception unit 112.

[0047] Alternatively, the user may input desired conditions for the shortened video by operating the terminal 200 or the like, and the designated time acquisition unit 131 may set and acquire the designated time according to the input conditions. As an example, the conditions for the shortened video input at this time may include a time range designation. In this case, the conditions may include a time designation such as "between 8 minutes and 10 minutes". The conditions may not directly indicate the designated time, i.e., the playback time of the shortened video to be generated, but may be related to the purpose of use of the shortened video to be generated. The purpose of use of the shortened video designated as the condition may be, for example, sharing (uploading) the generated shortened video on a video hosting service, SNS, or the like. In this case, when a specific video hosting service name or the like to which the shortened video is to be uploaded is input as a condition, the designated time acquisition unit 131 identifies the video hosting service from the input information. Then, the designated time acquisition unit 131 identifies a recommended playback time for the video to be uploaded to the video hosting service from information such as the length, resolution, and size of the video that can be uploaded to the identified video hosting service, and sets and acquires this as the designated time. Information on the length, resolution, size, etc. of videos that can be uploaded to video hosting services may be stored in advance for each service in the server storage unit 120, or may be obtained by accessing the server of the video hosting service when necessary. Also, once information on a video hosting service has been obtained, it may be stored in the server storage unit 120.

[0048] As another example, the designated time acquisition unit 131 may determine the designated time based on the original video. In this case, the designated time acquisition unit 131 acquires, for example, the video stored in the video storage unit 121 of the server storage unit 120. Then, the designated time acquisition unit 131 determines the designated time according to the playback time of the acquired original video, that is, the length of the video. In this case, the designated time according to the length of the video may be set in advance, for example, the designated time may be set to 5 minutes if the length of the video is 60 minutes, the designated time may be set to 1 minute if the length of the video is 10 minutes, etc. Alternatively, the ratio of the designated time to the length of the video may be set in advance. In addition, the designated time may be determined according to, for example, the genre, content, etc. of the video. As an example of this case, if a distribution video is one in which people are talking, it is often shortened to 5 minutes, so if the content of the video is one in which people are talking, the designated time is determined to be 5 minutes. The designated time acquisition unit 131 acquires the designated time determined in this manner.

[0049] The editing point candidate extraction unit 132 extracts editing point candidates from the original video. First, the editing point candidate extraction unit 132 acquires, for example, video stored in the video storage unit 121 of the server storage unit 120. The editing point candidate extraction unit 132 may acquire the original video from a source other than the video storage unit 121, for example, from outside the server 100 via the server communication unit 110. In this case, the video acquired from outside the server 100 may or may not be stored in the video storage unit 121 of the server storage unit 120. The server 100 may include a means for confirming with a user whether or not to store the original video acquired from outside in the video storage unit 121. When acquiring a video from outside the server 100, the original video may be acquired by specifying the URL of the video on the Internet. In this case, however, it is preferable that the server 100 has one or both of an illegality confirmation unit and a similarity determination unit, which will be described later.

[0050] Then, the editing point candidate extraction unit 132 extracts candidates for editing points to be presented to the user for the obtained original video based on one or more of the following: - Content of speech in the original video -Tension of speech in the original video - Estimation of emotions of characters in the original video Text information of the original video

[0051] When extracting editing point candidates based on the speech content in the original video, the editing point candidate extraction unit 132 extracts editing point candidates, for example, as follows. First, the editing point candidate extraction unit 132 divides the speech sentences in the video. Next, the editing point candidate extraction unit 132 extracts a topic distribution for each group of divided speech sentences. Then, the editing point candidate extraction unit 132 extracts a point where the distribution shape of the topic changes significantly as an editing point candidate. Specific examples of a method for dividing the speech sentences include the following methods: dividing at a constant time interval set in advance, dividing at a constant amount of speech set in advance, dividing when a silent state continues for a preset time, dividing by detecting a change in camera angle, and the like. Other known methods can also be used.

[0052] As another example of extracting editing point candidates based on the speech content in the original video, one or both of the start and end points of a scene may be detected based on the speech content in the video, and the detected start and end points of the scene may be extracted as editing point candidates. In this case, as a method for detecting the start and end points of a scene, for example, a specific speech sentence set in advance (e.g., greetings such as "Hello," "Goodbye," etc.) may be used, or a scene classification (e.g., a conversation scene, etc.) may be estimated from the speech content by machine learning. In this case, automatic speech recognition (ASR / Automated Speech Recognition), acoustic feature extraction, etc. may be used.

[0053] When extracting editing point candidates based on speech tension in the original video, the editing point candidate extraction unit 132 estimates the speech tension in the video. The speech tension can be estimated based on, for example, the speech speed, pitch, and volume. The speaker's movements and gestures can also be used to estimate the speech tension. If the speech speed is fast, the pitch is high, the voice is loud, and the speaker's movements and gestures are active, the speech tension can be estimated to be high. Then, the editing point candidate extraction unit 132 predicts the flow of the scene based on the estimated speech tension, i.e., identifies one or both of the start and end points of the scene, and extracts editing point candidates.

[0054] When extracting editing point candidates based on the estimation result of the emotions of the characters in the original video, the editing point candidate extraction unit 132 first estimates the emotions of the characters in the original video. The emotions of the characters in the video can be estimated, for example, from the facial expressions, tone of voice, etc. Then, based on the emotion estimation result, the editing point candidate extraction unit 132 extracts, for example, starting points of changes in emotions as editing point candidates.

[0055] When extracting editing point candidates based on text information of the original video, the text information of the video referred to here is telop, subtitles, etc. included in or accompanying the video. In this case, the editing point candidate extraction unit 132 extracts, for example, a point where the color or size of the subtitles or telop of the video changes as an editing point candidate. Alternatively, the editing point candidate extraction unit 132 may estimate a change in scene or topic from the text of the subtitles or telop, and extract the point where the scene or topic changes as an editing point candidate.

[0056] The presented editing point identification unit 133 identifies a presented editing point based on the specified time and the editing point candidate. The specified time acquired by the specified time acquisition unit 131 is used. The editing point candidate is extracted by the editing point candidate extraction unit 132. The presented editing point is an editing point to be presented to the user among the editing point candidates.

[0057] For example, the presented editing point identification unit 133 identifies a set of editing point candidates among the editing point candidates such that the interval between the editing point candidates is equal to or less than the designated time. Then, each of the editing point candidates constituting the identified set is identified as a presented editing point. Alternatively, the presented editing point identification unit 133 may identify, as the presented editing point, an editing point candidate capable of constituting a set of editing point candidates such that the difference between the interval between the editing point candidates and the designated time is within a preset tolerance. In addition, when the number of presented editing points identified as above is large, the presented editing points may be narrowed down until the number of presented editing points is equal to or less than a desired value. In this case, for example, the presented editing points are identified in descending order of the difference between the interval between the editing point candidates and the designated time. As another example, the presented editing point identification unit 133 may exclude, from the editing point candidates, an editing point candidate whose interval to the editing point candidate before and after the editing point candidate exceeds the designated time, and identify the remaining editing point candidates as the presented editing points.

[0058] The proposed editing point identification unit 133 may also identify, as the proposed editing points, a set of candidate editing points that specify clipping from two or more locations of the original video such that the total length of the video is close to the specified time. In this case, it is assumed that a start point and an end point of one clipped video are specified by a pair of two editing points, and a shortened video is generated by linking multiple clipped videos. That is, multiple pairs, in other words, 2n editing points (n is a natural number), are identified as a set of proposed editing points. The proposed editing point identification unit 133 may then identify two or more sets of such 2n proposed editing points. As a specific example, when the specified time is 10 minutes, a set of six editing points, namely editing points 1 and 2 which specify the start and end points, respectively, of an approximately 3 minute cut-out video A, editing points 3 and 4 which specify the start and end points, respectively, of an approximately 5 minute cut-out video B, and editing points 5 and 6 which specify the start and end points, respectively, of an approximately 2 minute cut-out video C, are identified as a set of editing point candidates, and further a set of four editing points, namely editing points 7 and 8 which specify the start and end points, respectively, of an approximately 4 minute cut-out video D, and editing points 9 and 10 which specify the start and end points, respectively, of an approximately 6 minute cut-out video E, are identified as another set of editing point candidates.

[0059] Note that the acquisition of the designated time by the designated time acquisition unit 131 and the extraction of the editing point candidates by the editing point candidate extraction unit 132 may occur first, as long as they are performed before the identification of the proposed editing point by the proposed editing point identification unit 133. The editing point candidate extraction unit 132 may take the designated time into consideration when extracting the editing point candidates. In this case, for example, the editing point candidate extraction unit 132 may extract editing point candidates within a range in which the interval between the editing point candidates and the already extracted editing point candidates is within the designated time.

[0060] The proposed editing point output control unit 134 controls an operation of outputting the proposed editing point identified by the proposed editing point identification unit 133. For example, the proposed editing point output control unit 134 generates a display control signal for displaying the proposed editing point, and outputs the generated display control signal to the server communication unit 110. The server transmission unit 111 transmits the display control signal to the user's terminal 200. The terminal reception unit 212 of the terminal 200 receives the display control signal transmitted from the server 100. Then, the terminal control unit 230 of the terminal 200 causes the display unit 250 of the terminal 200 to display the proposed editing point in accordance with the display control signal received by the terminal reception unit 212. In this way, the proposed editing point is presented to the user who is editing the video.

[0061] In this manner, the server 100 outputs a display control signal for displaying the presented editing point identified by the presented editing point identification unit 133, by the presented editing point output control unit 134 and the server transmission unit 111. Here, the display control signal includes information on the presented editing point identified by the presented editing point identification unit 133. Therefore, outputting a display control signal for displaying the presented editing point is one example of outputting the presented editing point. In addition, the server transmission unit 111 may transmit information on the presented editing point identified by the presented editing point identification unit 133, rather than the display control signal, to the terminal 200, and the terminal 200 may display the presented editing point on the display unit 250 of the terminal 200 using the received information on the presented editing point. In this manner, the server 100, which is a video editing device, outputs the presented editing point.

[0062] Displaying the proposed editing points on the display unit 250 of the terminal 200 also corresponds to outputting the proposed editing points by the video editing system 1. Note that the method of presenting the proposed editing points is not limited to the above-described display on the display unit 250 of the terminal 200. As another example, the server transmission unit 111 may transmit a display control signal or information related to the proposed editing points to a display device (not shown) such as a display provided outside the server 100, and display the proposed editing points on the display device to present them to the user.

[0063] According to the video editing system 1 configured as described above, it is possible to present editing points according to the length (playback time) of a shortened video desired by a user, and to assist the user in editing moving images to create a shortened video with the desired playback time.

[0064] The display control signal generated by the proposed editing point output control unit 134 may be used to display the proposed editing point on a timeline of the original video on the display unit 250 of the terminal 200. In this way, the user can easily grasp the position of the proposed editing point in the original video and the contents before and after the proposed editing point.

[0065] The user may, for example, operate a keyboard, a mouse, a touchpad, or the like of the input / output unit 240 of the terminal 200 to select one or more of the proposed editing points displayed on the display unit 250 or the like of the terminal 200. In this case, the proposed editing point output control unit 134 may highlight other proposed editing points whose intervals from the selected proposed editing point are equal to the specified time on the timeline of the source video image displayed on the display unit 250 or the like of the terminal 200.

[0066] As described above, when the presented editing point identification unit 133 identifies a set of editing point candidates whose interval is equal to or less than the specified time as the presented editing points, the presented editing point output control unit 134 may display the set of presented editing points on the display unit 250 or the like of the terminal 200. Alternatively, as described above, when the presented editing point identification unit 133 identifies a set of editing point candidates whose difference between the interval between the editing point candidates and the specified time is within the tolerance as the presented editing points, the presented editing point output control unit 134 may display the set of presented editing points on the display unit 250 or the like of the terminal 200. Furthermore, when the presented editing point identification unit 133 identifies a set of 2n editing point candidates that specify cutouts from two or more locations of the original video such that the total length of the video is close to the specified time as the presented editing points, the presented editing point output control unit 134 may display the set of presented editing points on the display unit 250 or the like of the terminal 200. In addition, when displaying a set of suggested editing points, when a suggested editing point included in a certain set is selected by a user, the other suggested editing points included in the same set may be highlighted or otherwise displayed so that the user can easily identify them.

[0067] The suggested editing point output control unit 134 may create a scene label indicating the content of each scene separated by the suggested editing points, and output a display control signal for displaying the created scene label on the display unit 250 of the terminal 200, etc. In this case, the suggested editing point output control unit 134 can create a scene label by using the content of the scene estimated by the editing point candidate extraction unit 132, for example.

[0068] The suggested editing point output control unit 134 may specify the flow of the story within each scene separated by the suggested editing points. The flow of the story within a scene may be, for example, an introduction, development, twist, conclusion, or climax. The suggested editing point output control unit 134 may output a display control signal for displaying the flow of the story for each scene separated by the suggested editing points on the display unit 250 of the terminal 200 as, for example, text, icons, or the like. In this case, it is preferable to display the flow of the story for each scene separated by the suggested editing points on the timeline of the original video.

[0069] Furthermore, the proposed editing point output control unit 134 may take into consideration the flow of the story for each scene separated by the proposed editing points, create a recommended combination of proposed editing points, and display it on the display unit 250 or the like of the terminal 200. The proposed editing point output control unit 134 determines the recommended combination of proposed editing points so that, for example, a structure is formed in which the beginning, development, twist, and conclusion are each included within a specified time, or a structure is formed in which an exciting scene is included within a specified time.

[0070] Machine learning may be used for at least one of the processes of extracting editing point candidates by the editing point candidate extraction unit 132, identifying the presented editing points by the presented editing point identification unit 133, and determining a combination of presented editing points recommended by the presented editing point output control unit 134. When extracting editing point candidates by machine learning, the editing point candidate extraction unit 132 performs inference using, for example, a trained model for outputting editing point candidates from an input video, and outputs the editing point candidates as the inference results. When identifying the presented editing points by machine learning, the presented editing point identification unit 133 performs inference using, for example, a trained model for outputting presented editing points from an input designated time and editing point candidates, and outputs the presented editing points as the inference results. When determining a combination of presented editing points recommended by machine learning, the presented editing point output control unit 134 performs inference using, for example, a trained model for outputting a combination of presented editing points recommended from an input video and presented editing points, and outputs the combination of presented editing points as the inference results.

[0071] An example of the process flow in the video editing system 1 configured as above will be described with reference to Fig. 5. The series of processes shown in Fig. 5 is executed by a computer of the server 100, which is an information processing device. The series of processes shown in Fig. 5 is for editing a video to generate a shortened video by shortening the playback time of the original video.

[0072] First, in step S1, the designated time acquisition unit 131 acquires a designated time related to the playback time of the shortened video to be generated. Then, in step S2, the editing point candidate extraction unit 132 extracts editing point candidates from the original video. As described above, the processes of steps S1 and S2 may be performed in any order, or may be performed simultaneously in parallel. After the processes of steps S1 and S2 are completed, the server control unit 130 of the server 100 next performs the process of step S3.

[0073] In step S3, the proposed editing point identification unit 133 identifies a proposed editing point to be presented to the user based on the specified time acquired in step S1 and the editing point candidates extracted in step S2. Then, in step S4, the proposed editing point output control unit 134, the server transmission unit 111, the display unit 250, etc. output the proposed editing point identified in step S3.

[0074] The video editing system 1 according to this embodiment may further include a shortened video generating unit that automatically generates a shortened video. The shortened video generating unit is provided in, for example, the server control unit 130. The shortened video generating unit generates a shortened video of the original video based on the proposed editing points selected by the user. In addition, when a combination of recommended proposed editing points is created by the proposed editing point output control unit 134, the shortened video generating unit may generate a shortened video of the original video based on the recommended combination of proposed editing points.

[0075] Alternatively, the shortened video generation unit may create a summary of the contents of the original video and automatically generate a shortened video based on the created summary. In this case, the shortened video generation unit first automatically recognizes the voice of a target part in the original video and acquires a sentence of the spoken content of the target part. Next, the shortened video generation unit creates a summary of the acquired spoken content. The length of the summary to be created is determined according to a specified time. The summary is created by deleting a part of the original sentence. Then, the shortened video generation unit generates a shortened video by extracting and concatenating a relevant part of the original video based on the summary.

[0076] The shortened video generating unit may also generate a description, a title, a thumbnail image, etc. of the generated shortened video. The description and title of the shortened video can be generated, for example, by using automatic speech recognition of the shortened video, the above-mentioned summary, etc. The shortened video generating unit generates a thumbnail image from an image of the most characteristic or representative scene using a video analysis result for the corresponding part of the shortened video or the original video image. In addition, the shortened video generating unit may add decorations such as subtitles to the thumbnail image.

[0077] Next, an example of displaying suggested editing points and generating a shortened video in the video editing system 1 configured as above is shown in Fig. 6. The figure shows an example of a video of a triathlon race. In this example, the transition points of each scene of the start of the race, the swim (swimming), the bike (bicycle road race), the run (long-distance running), and the finish are presented as suggested editing points. In particular, the start and end points of a scene in the run where a top group has overtaken are presented as suggested editing points to cut through the exciting scene.

[0078] In the example shown in the figure, the shortened video generation unit automatically generates a shortened video, and also generates a description, title, and thumbnail image of the generated shortened video. As shown in the figure, artificial intelligence (AI) using machine learning may be used to generate the shortened video, description, title, and thumbnail image by the shortened video generation unit. In this case, the shortened video generation unit performs inference using a trained model for outputting the shortened video, description, title, and thumbnail image from the input video image, and outputs the shortened video, description, title, and thumbnail image that are the inference results.

[0079] The video editing system 1 according to this embodiment may further have a function of supporting video editing in compliance with intellectual property laws such as the Copyright Act. In this case, for example, in order to realize this function, the server control unit 130 may further include an illegality checking unit (not shown). When the suggested editing point output control unit 134 displays the suggested editing points, the illegality checking unit may also display a message calling attention to whether or not the original video image can be edited and uploaded to an Internet video distribution service or the like without problems under intellectual property laws.

[0080] Alternatively, when the editing point candidate extraction unit 132 extracts an editing point candidate, the fraud confirmation unit may display a message to confirm whether the original video is illegal under intellectual property laws such as the Copyright Act. Also, when the server control unit 130 acquires the original video, the fraud confirmation unit may display a message to confirm whether the original video is illegal under intellectual property laws. Specifically, for example, this may confirm with the user whether generating a shortened video from the original video will infringe copyright. Then, editing of the original video may be prohibited until it is confirmed that the original video is not illegal.

[0081] The server control unit 130 may further include a similarity determination unit (not shown) that searches through video distribution services on the Internet and determines whether there is a video whose characteristics are substantially similar to the original video. The similarity determination unit may use, for example, AI when determining the similarity. In this case, the AI ​​engine that determines the similarity of the video may be provided in the server 100 or may be located outside the system. When using an AI engine outside the system, the similarity determination unit transmits, together with the acquired input video, a determination request as to whether there is a video similar to the input video on the Internet to the AI ​​engine outside the system. The AI ​​engine outputs a determination result in response to the determination request. The similarity determination unit obtains the determination result output from the AI ​​engine to make a final determination as to whether there is a video whose characteristics are substantially similar to the original video. When determining that there is a video similar to the original video, the similarity determination unit may display a message that calls attention. At this time, the similarity determination unit may also cause the reason for the warning to be presented. Furthermore, the similarity determination unit may display information about the video determined to be similar to the original video, or a message indicating that generating and publishing a shortened video based on the original video may infringe on intellectual property rights (copyright, etc.) related to a video similar to the original video. In this way, video editing that complies with intellectual property laws such as the Copyright Act can be promoted.

[0082] In addition, in the present disclosure, the embodiments and configuration examples may be combined in any manner without departing from the spirit of the present disclosure.

[0083] INDUSTRIAL APPLICABILITY The present disclosure can be used in a video editing method, a video editing device, and a video editing program for editing a video to generate a shortened video by shortening the playback time of an original video image. [Explanation of symbols]

[0084] 1. Video editing system 100 Servers 110 Server Communication Department 111 Server transmission unit 112 Server Reception Unit 120 Server memory unit 121 Video storage unit 130 Server control unit 131 Specified Time Acquisition Unit 132 Editing point candidate extraction unit 133 Presentation editing point identification unit 134 Presentation editing point output control unit 200 terminals 210 Terminal communication unit 211 Terminal transmitter 212 Terminal receiving unit 220 Terminal memory unit 230 Terminal Control Unit 240 Input / output section 250 Display section 260 Mike 270 Speaker 280 Camera 300 Network

Claims

1. A video editing method for generating a shortened video by shortening a playback time of an original video, the method being executed by a computer of an information processing device, the method comprising: A designated time acquisition step of acquiring a designated time related to a playback time of the shortened video to be generated; a candidate extraction step of extracting editing point candidates from the original video; an editing point identification step of identifying a proposed editing point to be presented to a user based on the designated time acquired in the designated time acquisition step and the editing point candidates extracted in the candidate extraction step; and outputting the suggested editing points identified in the editing point identification step.

2. 2. The video editing method according to claim 1, The designated time acquisition step acquires the designated time input by a user.

3. 2. The video editing method according to claim 1, The designated time acquisition step determines and acquires the designated time based on the original video.

4. A video editing method according to any one of claims 1 to 3, The candidate extraction step extracts editing point candidates based on the contents of speech in the original video.

5. A video editing method according to any one of claims 1 to 3, The candidate extraction step extracts editing point candidates based on the tension of speech in the original video.

6. A video editing method according to any one of claims 1 to 3, The candidate extraction step estimates emotions of the characters in the original video, and extracts editing point candidates based on the emotion estimation results.

7. A video editing method according to any one of claims 1 to 3, The candidate extraction step extracts editing point candidates based on the emotion estimation result and on text information of the original video.

8. A video editing method according to any one of claims 1 to 3, The editing point output step includes outputting display control information for displaying the proposed editing points on a timeline of the original video.

9. A video editing method according to any one of claims 1 to 3, The editing point output step outputs display control information for displaying the contents of each scene divided by the suggested editing points.

10. A video editing method according to any one of claims 1 to 3, The method further includes a step of confirming with a user whether or not generating a shortened video from the original video image will result in copyright infringement.

11. A video editing program for causing a computer of the information processing device to execute the video editing method according to any one of claims 1 to 3.

12. A video editing device that performs video editing to generate a shortened video by shortening a playback time of an original video, comprising: a designated time acquisition unit that acquires a designated time related to a playback time of the shortened video to be generated; a candidate extraction unit that extracts editing point candidates from the original video; an editing point identification unit that identifies a proposed editing point to be presented to a user based on the designated time acquired by the designated time acquisition unit and the editing point candidates extracted by the candidate extraction unit; and an editing point output unit that outputs the suggested editing point identified by the editing point identification unit.

Images