Automatic non-linear editing style transfer

Abstract

The invention relates to automatic non-linear editing style transfer. The present disclosure provides systems, methods, and computer program products for performing automatic non-linear editing style transfer. A computer-implemented method may include determining one or more shot boundaries in a video; analyzing the identified content in each of the one or more shots in the video based on performing the object detection; determining an editing style for a respective shot based at least in part on measuring motion across a frame within each of the one or more shots in the video; determining a content segment to be adjusted from the target content set based on analyzing the target content set in consideration of the identified content and the determined editing style from the shot of the video, and automatically adjusting the content segment based at least in part on modifying the content segment from the target set of content with the determined editing style from the shot of the video.

Description

[0001] Description of the case

[0002] This application is a divisional application of Chinese invention patent application No. 202080102660.4, with a filing date of November 13, 2020. Technical Field

[0003] The present disclosure relates generally to computer vision and automatic processing of visual content. More specifically, the present disclosure relates to systems, methods, and computer program products for performing automatic non-linear editing style transfer. Background Art

[0004] Advances in computer vision and machine learning have improved the detection and classification of objects within visual content. For example, many of these improvements have been targeted at perception applications in autonomous vehicles, robotics, and security. However, such applications often focus on people and other objects in a scene without specifically considering how cameras depict various scenes.

[0005] Many websites, including content sharing and social networking sites, allow users to store and share images and video content with other users. In addition, smartphones with one or more high-quality digital cameras, large amounts of storage space, and mobile broadband have made it easy for users to capture and distribute visual content from anywhere. However, while the amount of newly generated visual content continues to increase over time, most users do not have professional editing skills, access to post-production editing software, or the time to perform complex edits across large amounts of raw visual content. Summary of the invention

[0006] Aspects and advantages of embodiments of the present disclosure will be set forth in part in the following description, or may be learned from the description, or may be learned through practice of the embodiments.

[0007] One example aspect of the present disclosure is directed to a system for performing automatic non-linear style transfer from source content to target content, for example, by determining one or more shot boundaries in a video based on analyzing a video, analyzing identified content in each of the one or more shots in the video based on performing object detection on the corresponding shots, determining an editing style of the corresponding shots based at least in part on measuring motion across frames within each of the one or more shots in the video, determining content segments to be adjusted from a target content set based on analyzing the target content set taking into account the identified content and the determined editing style of the shots from the video, and automatically adjusting content segments from the target content set based at least in part on modifying the content segments using the determined editing style of the shots from the video.

[0008] Other aspects of the present disclosure are directed to various devices, non-transitory computer-readable media, computer-implemented methods, user interfaces, and electronic devices. These and other features, aspects, and advantages of various embodiments of the present disclosure will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, are used to explain the relevant principles. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] A detailed discussion of embodiments for those of ordinary skill in the art is set forth in the specification with reference to the accompanying drawings, in which:

[0010] Figure 1 Depicted is a block diagram of an example system for performing automatic non-linear editing style transfer according to an example embodiment of the present disclosure.

[0011] Figure 2 Depicted is a flow chart of an example method for performing automatic non-linear editing style transfer according to an example embodiment of the present disclosure.

[0012] Figure 3 Depicted is a flow chart of an example method for performing automatic non-linear editing style transfer from source content to user-provided target content according to an example embodiment of the present disclosure.

[0013] Figure 4 Depicted is a diagram of an example match performed between source content and target content according to an example embodiment of the present disclosure.

[0014] Figure 5 An illustration of an example keypoint tracking for automatic non-linear editing style transfer is depicted in accordance with an example embodiment of the present disclosure.

[0015] Figure 6 An illustration of an example frame-to-frame keypoint tracking for measuring camera motion for automatic non-linear editing style transfer is depicted in accordance with an example embodiment of the present disclosure.

[0016] Figure 7 Depicted is a diagram depicting an example re-projection of a starting position of target content based on camera motion in source content, according to an example embodiment of the present disclosure.

[0017] Figure 8 A block diagram of an example computer system is depicted that may perform one or more example embodiments of the present disclosure. DETAILED DESCRIPTION

[0018] Overview

[0019] In general, the present disclosure is directed to improving various forms of content using automatic non-linear editing style transfer, including but not limited to visual content such as images and video content. Examples described in the present disclosure enable automatic detection and measurement of editing styles in one or more segments of content, for example, whether provided as part of the original content or introduced later using post-production software. In addition, the present disclosure also enables automatic transfer of editing styles from source content to other sets and segments of target content. In addition, examples of the present disclosure provide improvements in the automatic processing and manipulation of visual content compared to existing methods.

[0020] The editing style used to capture and present visual content is important for conveying the emotion, mood, message, and / or other aspects of the storytelling process. For example, editing attributes are often considered to be the building blocks of editing style. Example editing attributes may include framing, camera movement, focus, transitions, playback speed, audio, volume, color, lighting, etc. When combined together, such editing attributes may leave a significant stylistic impression on the audience.

[0021] Although raw footage itself can capture events and tell stories to convey messages, non-linear editing can be used to change or enhance such content. Non-linear editing generally refers to a form of post-production editing of audio, video and / or image content in which the original content is saved and an editing decision list is compiled to keep track of the edits. The edited content can then be reconstructed from the original content and from the associated editing decision list. Traditional non-linear editing is a manual, time-consuming process that requires certain editing skills and access to specialized editing software. As a result, non-linear editing is not usually performed on user-generated content due to the associated complexity and expense. As such, most user-generated content remains unedited, otherwise such content could be enhanced and improved at this time.

[0022] The present disclosure provides an example of performing automatic detection, measurement, and transfer of editing styles from a segment of source content to a set of one or more segments of target content, as an alternative to conventional methods. In an example of the present disclosure, a computer system performs automatic non-linear editing style transfer from source video content to a set of target content by: determining one or more shot boundaries in the video based on analyzing the video, analyzing the identified content in the corresponding shot based on performing object detection on each of the one or more shots in the video, determining the editing style of the corresponding shot based at least in part on measuring motion across frames within each of the one or more shots in the video, determining content segments to be adjusted from the target content set based on analyzing the target content set taking into account the identified content and the determined editing style of the shots from the video, and automatically adjusting the content segments based at least in part on modifying the content segments from the target content set using the determined editing style of the shots from the video.

[0023] The systems, methods, and computer program products described herein provide a number of technical effects and benefits. As an example, the embodiments described in the present disclosure perform automatic analysis of source content and target content to automatically transfer editing styles from source content to target content more efficiently and with fewer computing resources (e.g., less processing power, less memory usage, less power consumption, etc.) than, for example, conventional and manual post-production editing.

[0024] Referring now to the drawings, example embodiments of the present disclosure will be discussed in further detail.

[0025] An example system for performing automatic non-linear style transfer

[0026] Figure 1 Depicted is a block diagram of a system 100 for performing automatic non-linear editing style transfer according to an example embodiment of the present disclosure.

[0027] System 100 includes one or more user devices 102 , one or more server machines 110 , and data storage 170 communicatively coupled via a network 120 .

[0028] The network 120 may be a public network (e.g., the Internet), a private network (e.g., a local area network (LAN) or a wide area network (WAN)), or any combination thereof. In an example, the network 120 may include the Internet, one or more intranets, a wired network, a wireless network, and / or other suitable types of communication networks. The network 120 may also include a wireless telecommunication network (e.g., a cellular network) suitable for communicating with other communication networks such as the Internet. In addition, the network 120 may include one or more short-range wireless networks.

[0029] The user device 102 may typically be a personal computer (PC), a laptop computer, a mobile phone, a tablet computer, a server computer, a wearable computing device, or any other type of computing device (i.e., a client). The user device 102 may run an operating system (OS) that manages the hardware and software of the corresponding device. A browser application (not shown) may be running on the user device 102. The browser application may be a web browser that is capable of accessing content and services provided by the server machine 110 or a combination of server machines 110. Other types of computer programs and scripts may also be running on the user device 102.

[0030] The user device 102 includes one or more applications 104, a user interface 106, and a data store 108. The user device 102 may generally execute or run one or more applications 104 or client versions of the applications 104 that interoperate with one or more server machines 110. For example, the applications 104 may include or provide access to one or more automated non-linear editing systems 140 and services.

[0031] The applications 104 may each generally provide a user interface 106 that allows a user to submit user input and receive various text, graphical, and audio output associated with the respective application 104 running on the user device 102. For example, a user may generally provide user input to the application 104 via a user input component of the respective user device 102, such as a touch-sensitive component (e.g., a touch-sensitive display screen or touchpad) that is sensitive to the touch of a user input object (e.g., a finger or stylus), a virtual keyboard, a traditional keyboard, a mouse, a microphone, a traditional keyboard, or other means by which a user may provide input to the application 104 via a device. The user may also receive output from the application 104 via one or more user interfaces 106 provided by a display, a user device 102, a computing system, or any other type of device.

[0032] The applications 104 may include, for example, social networking, photo sharing, video sharing, storage services, and / or any other type of “app” running on the user device 102. In addition, such applications 104 may have corresponding websites, services, and / or APIs that may be used in association with or separately and separately from the applications 104. The data storage 108 may generally include any removable or non-removable storage device associated with the user device 102, which may also generally have access to various other storage (e.g., data storage 116, data storage 170) via the network 120.

[0033] The server machines 110 may each include one or more web servers 112 and / or one or more application servers 114. The web servers 112 may provide text, audio, image, video, or any other content from the server machines 110 or other sources (e.g., data stores 116, 170) and the user devices 102. The web servers 112 may also provide web-based application services, business logic, and interfaces to the server machines 110 and user devices. The web servers 112 may also send and receive text, audio, video, and image content to and from the user devices 102, which may be stored or provided by, for example, the data stores 108, 116, 170 for storage, sharing, and / or publication of content 130.

[0034] In an example, one or more Web servers 112 are coupled to one or more application servers 114 that provide application services, data, business logic, and / or APIs to the server machines 110 and user devices 102. In some examples, the application servers 114 independently provide one or more such services without using the Web servers 112. In an example, the Web servers 112 may provide the server machines 110 and user devices 102 with access to one or more application server 114 services associated with an automatic non-linear editing system (e.g., the automatic non-linear editing system 140). Such functionality may also be provided as part of one or more different Web applications, stand-alone applications, systems, plug-ins, Web browser extensions, application programming interfaces (APIs), and the like. In some examples, plug-ins and extensions may generally be referred to individually or collectively as "add-ons."

[0035] The server machine 110 includes a local data store 116 and has access to other data stores 170 to store and access various forms of content 130. In examples, the server machine 110 may provide, be associated with, or be used in conjunction with one or more cloud or web-based services and applications, such as a social networking site, a cloud storage provider, a content sharing site, an image sharing site, a video sharing site, and / or any other site, service, or application that stores, processes, and displays user-generated and / or other types of content 130. In various examples, such sites, services, and / or applications may be accessed by a user via one or more applications 104 running on a respective user device 102.

[0036] In examples, user devices 102 and server machines 110 can store and access various forms of content 130 (e.g., source content, target content) from data stores (e.g., data stores 108, 116, 170). In examples, content can generally refer to any text data, audio data, visual data, graphic data, graphics, images, video, multimedia, and / or any other data in various forms or formats for representing or describing any type of content of any dimension (e.g., 2D, 3D). In various examples, content 130 can be original or newly captured, pre-processed or partially edited, professionally edited, curated, and / or user-generated.

[0037] The data store 170 generally refers to persistent storage capable of storing various types of content 130, such as text, audio, video, and images. In some examples, the data store 170 may include a network-attached file server or cloud storage, while in other examples, the data store 170 may include other forms of persistent storage, such as object-oriented databases, relational databases, etc. The data store 170 may include user-generated content 130 uploaded by the user device 102 (e.g., user-generated images, videos, etc.) and / or content 130 provided by one or more other parties. Data may be added to the data store 170, for example, as discrete files (e.g., Moving Picture Experts Group (MPEG) files, Windows Media Video (WMV) files, Joint Photographic Experts Group (JPEG) files / Graphics Interchange Format (GIF) files, Portable Network Graphics (PNG) files, etc.) or as components of a single compressed file (e.g., a zip file).

[0038] Content 130 may generally include visual content, such as images and videos. Images may generally include any visual or graphical representation, such as a photograph or screenshot captured by a camera, computing device, or other device. Images may include, for example, portrait, square, panoramic, and / or any other type of image.

[0039] Video generally refers to a set of continuous image frames representing a scene in motion. For example, a series of continuous images can be captured continuously or reconstructed later to produce the effect of a moving image, which can include the movement of the camera and / or the content in the scene. Video content can be presented in various formats, including but not limited to analog, digital, two-dimensional and three-dimensional video. In addition, video content can include movies, video clips, or any collection of animated images to be displayed sequentially. Video data can include digital video with a sequence of static image frames that can also be stored as image data. Therefore, each image frame can represent a snapshot of a scene that has been captured according to a time interval.

[0040] In various examples, the user device 102 and / or the server machine 110 runs, executes, or otherwise utilizes the automated non-linear editing system 140 and the associated source content editing templates 150. For example, the user device 102 may include one or more applications 104 associated with a service (e.g., the automated non-linear editing system 140) provided by one or more server machines 110. For example, various types of computing devices (e.g., smart phones, smart TVs, tablet computers, smart wearable devices, smart home computer systems, etc.) may use specialized apps and / or APIs to access services provided by the server machines 110, issue commands to the server machines 110, and / or receive content from the server machines 110 without accessing or using a web page.

[0041] In an example, the functions performed by one or more server machines 110 may be performed in whole or in part by one or more other machines and / or user devices 102. The server machines 110 may be accessed as a service provided by a system or device via an appropriate application programming interface (API) and data feeds, and are therefore not limited to use with a website. In addition, the server machines 110 may be associated with and / or utilize one or more automatic non-linear editing system 140 services, for example, independently operated or provided by different providers.

[0042] In various examples, the automatic non-linear editing system 140 analyzes various types of visual content, such as images and video data, to detect, measure, and transfer an editing style from a collection of one or more segments of source content to another collection of one or more segments of target content. In examples, the automatic non-linear editing system 140 performs various automatic analyses on existing, edited video content to detect and measure the editing style present in the video content, and automatically applies the editing style to other edited and / or unedited content, such as clips from one or more videos, images, image sequences, and / or collections of images in the target content.

[0043] Editing style generally refers to a set of one or more style attributes associated with content 130, which may include attributes present in the original content and / or style attributes applied as part of the editing process. In some examples, content 130 may include one or more editing styles based on one or more combinations of attributes associated with the content and / or a specific portion of the content, such as a specific shot in video content. Editing style attributes generally may include, but are not limited to, framing, camera motion, focus, zoom, transitions, playback speed, color, lighting, audio, and / or text associated with content 130 or a specific portion of content 130.

[0044] In various examples, camera framing generally refers to the placement and position of a subject within a shot. Focus generally refers to the focus of a lens during a shot, and may include, for example, deep focus, shallow focus, soft focus, and / or other types of focus. Camera motion types generally may include a still camera, a pan-tilt-zoom (PTZ) camera motion, and / or a six-degree-of-freedom camera motion. Zoom generally refers to the change in the focal length of a zoom lens during a shot, which allows a change from a close-up view to a wide / full / long view, or vice versa, during a shot. Transitions generally refer to the manner in which two or more shots of visual content are combined or joined together. Example transitions generally include, but are not limited to, dissolves, fades, wipes, and various forms of cuts (e.g., L-cuts, jump cuts, cut-ins, cut-aways, match cuts, etc.).

[0045] Color generally refers to the color attributes or coloring schemes applied to visual content. For example, visual content may be presented in full color or various types of coloring schemes (vivid, warm, cool, etc.). Such content may also be presented in grayscale, black and white, silver tones, etc. Lighting generally refers to the lighting that is captured and / or applied to visual content. Various lighting aspects of visual content may include, but are not limited to, light fading, light filtering, glare, brightness, contrast, etc.

[0046] Playback speed generally refers to the speed at which content, such as visual content and / or audible content, is played back. The playback speed can be adjusted, for example, from a native real-time value (e.g., 1x) to a faster speed (e.g., 1.25x, 1.5x, 2x, etc.) or a slower speed (e.g., 0.75x, 0.5x, etc.). In addition to fixed speeds, the speed can be adjusted dynamically in a rigid pattern (e.g., speed ramping, piecewise linear transitions) or in a complex pattern where the playback speed changes irregularly.

[0047] Audio generally refers to audio content, such as background music, sounds, or audio-based speech associated with content 130. For example, audio present in source content may or may not be transferred to target content along with other editing style attributes. Similarly, text such as text overlays may be associated with content 130. Text associated with source content may or may not be transferred to target content with other editing style attributes. Various other editing style attributes may include, but are not limited to, content quality, content resizing, content orientation, resolution, etc.

[0048] In various examples, the automatic non-linear editing system 140 analyzes the content 130 to automatically detect and measure various editing style attributes present in or associated with the content. The automatic non-linear editing system 140 can generally use data and metadata to represent the editing style attributes detected and measured. For example, the automatic non-linear editing system 140 can generate one or more editing styles for a section of content, wherein each editing style, for example, represents a different grouping of editing style attributes. The automatic non-linear editing system 140 can also generate a single editing style that includes some or all of the editing style attributes detected and measured in the source content. The source content generally refers to the content 130 that is used as a model or source of the style of a collection of one or more other sections of other content 130 that is to be transferred to, which can be referred to as the target content.

[0049] In various examples, the automatic non-linear editing system 140 automatically detects and measures editing style attributes of source content, and records the information in a source content editing template 150 that can be used, and then reused to automatically transfer editing styles from source content to various target content sets from any number of users (1, 10, hundreds, thousands, millions, etc.). As such, editing style attributes can be automatically extracted and described as data and metadata that can be analyzed and used later at different times, and by various users to transfer editing styles from source content to various types of target content. The source content editing template 150 may include, for example, global editing style attributes that are consistent within the source content and / or may include local editing style attributes associated with specific portions, segments, or time periods within the source content. In some examples, the source content editing template 150 may include information about shot boundaries detected in the video, information about one or more shots in the video, information about content identified within corresponding shots of the video, and / or editing style attributes determined at various times across various frames from shots in the video.

[0050] Example method to perform automatic non-linear style transfer

[0051] Figure 2 Depicted is a flow chart of an example method for performing automatic non-linear editing style transfer according to an example embodiment of the present disclosure.

[0052] although Figure 2 For the purpose of illustration and discussion, the steps performed in a specific order are depicted as examples, but the method of the present disclosure is not limited to the specific order or arrangement shown. Various steps of method 200 can be omitted, rearranged, combined and / or adapted in various ways without departing from the scope of the present disclosure.

[0053] At 202, a computing system determines one or more shot boundaries in a video. In an example, the computing system receives one or more segments of source content to be used for automatic non-linear editing style transfer. For example, a user may provide a collection of one or more segments of original and / or edited content, such as images, a burst of photos, video clips, movie trailers, montages, music videos, high-energy moment videos, advertising videos, or any other type of content. In various examples, the computing system receives a collection of one or more segments of video content as source content to perform automatic non-linear editing style transfer on a collection of one or more segments of target content.

[0054] In an example, the automatic non-linear editing system 140 of the computing system analyzes the source video content to detect one or more shot boundaries present in the source video content. For example, the computing system may perform shot boundary detection analysis based on detecting color changes and / or identifying key point matches across video frames to identify shot boundaries and associated shots. Shots may generally be referred to as continuous views or a series of frames captured or recorded by a camera without cutting, jumping or other interruptions. In some examples, shots may be grouped together into groups that may be referred to as shots or scenes depicted within the video content. In some examples, shots may be constructed or reconstructed from various images that may or may not be associated with the video content.

[0055] In an example, the automatic non-linear editing system 140 performs color analysis across frames of the video content and signals a shot change based on detecting a significant change in color across the video frames. For example, color can be tracked across one or more video frames, and the occurrence of a shot change can be signaled when the color of the one or more frames differs from the color in one or more other comparison frames by more than a threshold. In some examples, the automatic non-linear editing system 140 calculates a color histogram for each of the one or more frames, compares the color histogram information to previous frames, and signals a shot change when the distribution of frame colors changes at a rate different from the color representation along a sliding history window.

[0056] In an example, the automatic non-linear editing system 140 determines shots, shot groups, scenes, and / or shot boundaries in the video content by performing keypoint matching in corresponding frames and matching the identified sets of keypoints across frames using an approximate nearest neighbor search and by applying weak geometric constraints in the form of a base matrix to obtain a set of spatially consistent keypoint matches. In some examples, shot boundaries can be identified based on detecting significant changes in keypoint matches occurring between frames. For example, a significant change in keypoints across frames exceeding a threshold level can be used to signal a shot change.

[0057] In some examples, color analysis and keypoint matching can be used together to identify shots, detect shot boundaries, and / or group shots into scenes. One or more other types of shot boundaries and shot identifications can be used alone or in combination with color analysis and / or keypoint matching. For example, one or more machine learning models can be used to automatically detect, predict, or confirm transitions between frames in video content.

[0058] At 204, the computing system analyzes the content identified in the corresponding shot based on performing object detection on the corresponding shot in the video. In an example, the automatic non-linear editing system 140 of the computing system analyzes the frames of the identified shot by performing object detection on the frames of the corresponding shot to classify the content and focus of the shot. In some examples, the automatic non-linear editing system 140 runs object detection on one or more frames of the shot to identify a set of one or more objects present in the shot. Then, the list of identified objects can be analyzed, for example, from largest to smallest. In an example, the union of the boxes of each object can be iteratively calculated until the frame coverage exceeds a threshold (e.g., 50%, 65%, 80%, or any percentage of frame coverage). Like this, the list of identified objects can be analyzed until the analyzed objects cover a point in the frame that allows for sufficient space for classification of the shot.

[0059] In some examples, the automatic non-linear editing system 140 identifies, analyzes, and determines features and details about the recognized faces and human poses as part of object detection, including, but not limited to, emotions, expressions, and activities. In some examples, the automatic non-linear editing system 140 may identify, analyze, and determine features and details about one or more objects, such as food, products, tools, animals, or other objects present in the visual content. In various examples, one or more objects, features, expressions, activities, and / or other aspects of a shot identified via object detection and analysis may, for example, be used, at least in part, to identify matching target content having similarity, consistency, and / or compatibility with the source content.

[0060] In an example, the automatic non-linear editing system 140 performs object detection and analysis on the visual source content, and then identifies the focus of a shot of the visual source content. In some examples, a shot that includes a small number of objects, such as defined by one or more thresholds, may be classified or labeled as a single-focus shot. For example, a single-focus shot may generally indicate a tendency for camera attention and / or motion to focus on one or more specific objects in or across a shot.

[0061] In some examples, shots that include a medium number of objects, such as defined by one or more thresholds, may be classified or labeled as medium focus shots. For example, medium focus shots may generally include an arrangement of a group or text overlay of one or more people and / or products. As such, medium focus shots may generally focus on multiple objects in the shot, rather than focusing on a specific object. In some examples, shots may be labeled or classified as having little or no focus on a specific object. For example, shots of crowds, landscapes, and / or general behind-the-scenes footage shots may not have a specific focus on a specific object.

[0062] Figure 4 A diagram 400 showing an example match that can be performed between source content and target content according to an example embodiment of the present disclosure is depicted. For example, a first example of source content 402 can be processed using object detection and classified as focusing on a single object on a small number of objects including a person's face. Then, the first example of source content 402 can be determined to be compatible and matched with a first example of target content 406, for example, also processed using object detection and classified as focusing on a small number of objects including a person's face. Similarly, a second example of source content 404 focusing on a close-up of a hand during eating can be determined to be compatible and matched with a second example of target content 408 focusing on a close-up of a hand during typing. In various examples, the results of the object detection and analysis performed on the source and target content can be used as a set of considerations and criteria when matching source content with target content when performing automatic non-linear editing style transfer.

[0063] At 206, the computing system determines an editing style for a corresponding shot based at least in part on measuring motion of frames across the corresponding shot in the video. In an example, the automatic non-linear editing system 140 of the computing system analyzes the shot from the source visual content to determine one or more editing style attributes of the editing style used in the corresponding shot. In an example, the automatic non-linear editing system 140 detects and measures camera motion in a shot or across a set of shots in the source visual content. For example, the automatic non-linear editing system 140 can detect and measure camera motion or how the camera moves across a set of frames in one or more shots.

[0064] In the example, the camera motion can be represented as a frame-to-frame reprojection of the image as a projective transformation: . In various examples, the automatic non-linear editing system 140 uses frame-to-frame keypoint matching and uses a random sample consensus (RANSAC) homography solver or another type of homography solver to calculate camera motion or movement, for example, in each of the source content and / or the target content, respectively. In some examples, keypoints are tracked on at least foreground objects, background objects, and human foreground objects. In some examples, a classifier or other tool is used to reject keypoints detected on the face or body to remove foreground motion from the homography estimate. In various examples, camera motion or movement detected across frames can be determined based on measuring the distance between corresponding keypoints across frames, where the current total reprojection is a left multiplication accumulation of previous projections: .

[0065] Figure 5 An illustration 500 of an example keypoint tracking for automatic non-linear editing style transfer according to an example embodiment of the present disclosure is depicted. In an example, keypoints may be tracked in a content frame such as a video frame 502. In an example, keypoints 504 may generally be tracked on foreground and / or background objects, human foreground keypoints 506 may also be tracked, and matching keypoints 508 may be identified across frames of video content to detect and measure camera motion. As described above, in some examples, human foreground keypoints 504 may not be considered when detecting and measuring camera motion.

[0066] Figure 6 An illustration 600 of an example frame-to-frame keypoint tracking for measuring camera motion for automatic non-linear editing style transfer according to an example embodiment of the present disclosure is depicted. Frames 602, 604, and 606 show keypoint tracking across a series of frames, where the camera spans a scene including two seated people. Using frame 602 as a starting position, the keypoints in frame 604 extend to the line showing the frame-to-frame keypoint tracking used to form a homography, and frame 606 generally represents the position where the camera movement ends in this example. Frames 608, 610, and 612 represent actual frames without keypoint tracking, where frame 608 corresponds to the starting position at frame 602, frame 610 corresponds to frame 604 showing movement from the starting position, and frame 612 corresponds to frame 606 after the camera movement is completed.

[0067] In various examples, such movement measured in the source content can be transferred to the target content, e.g., during a user session or as part of a batch process. In some examples, the movement in the source content can be measured and stored in one or more source content editing templates 150, e.g., based on the timing, location, and / or identification of associated video frames and using various data describing the movement occurring across frames. As such, for example, the source content editing templates 150 can be used and reused to perform automatic non-linear editing style transfer on different sets of one or more segments of the target content at different times, without the need to repeatedly reanalyze the source content for each request.

[0068] In various examples, the automatic non-linear editing system 140 analyzes the content to determine one or more editing style attributes associated with the content. For example, the content may be analyzed directly or with the aid of one or more general or specialized classifiers or models to identify editing style attributes such as framing, camera motion, focus, zoom, transitions, playback speed, color, lighting, audio, and / or text that are generally associated with the content and / or with a specific portion of the content. Such information may include data and metadata describing the editing style attributes and their relationship to the content or a specific portion of the content (e.g., a shot, frame, timing, sequence, etc.), which may be stored as part of the source content editing template 150.

[0069] At 208, the computing system determines content segments to be adjusted from the target content set based on analyzing the target content set in view of the identified content and the determined editing style of the footage from the video. In an example, the automatic non-linear editing system 140 of the computing system receives a set of one or more segments of target content that match the source content as part of performing automatic non-linear editing style transfer from the source content. (e.g., by transferring the style from the edited source content page to the target content)

[0070] In an example, the automatic non-linear editing system 140 receives a collection of one or more segments of user-generated content, which may include visual content, such as one or more images and or one or more videos provided by the user. In some examples, the user may provide or select source content that the automatic non-linear editing system 140 analyzes and processes to determine a source content editing style that will be transferred to target content, such as one or more segments of content created by the user. The user may also select or provide a source content editing template 150 that includes the source content editing style to be transferred, for example, the user's target content.

[0071] In an example, the automatic non-linear editing system 140 analyzes each of one or more segments of content in the target content set to determine which portions of the target content are compatible with the source content for automatic non-linear style transfer. For example, the automatic non-linear editing system 140 may perform shot boundary detection on the target video content to identify shots, perform object detection and analysis on shots and images to identify objects and shot focus, and identify one or more editing style attributes of the target content.

[0072] In an example, the automatic non-linear editing system 140 compares the editing style attributes from each of the one or more portions of the source content with the information determined for each of the one or more portions of the target content. For example, the automatic non-linear editing system 140 may compare one or more of the shot length, object content, shot focus, and / or one or more editing style attributes of a portion of the source content with each of the one or more segments or portions of the content within the target content set. Such a comparison may be performed, such as matching the source content and the target content for automatic non-linear style transfer based on one or more of the compatibilities, thereby allowing style transfer to occur between the source content and the target content, or matching similar objects, scenes, focus, and / or other attributes across the source content and the target content that are usually otherwise unrelated. In an example, the source content and the target content may be matched based on objects, emotions, focus, movement, scenes, editing styles, and other considerations to allow the editing style of the source content to be applied to the target content while maintaining the expression and meaning conveyed in the source content when being applied to the target content.

[0073] In an example, a model can be trained and used to analyze, compare, and match portions of source content and target content. In some examples, each of one or more portions or segments of the source content is compared and scored with one or more portions or segments of the target content. For example, shot lengths can be compared between shots of the source content and shots of the target content to determine whether the shot length of the target content is sufficient to perform style transfer to the target content.

[0074] In an example, source content is matched to target content based on the focus of shots where shots classified as single focus, medium focus, and / or little or no focus match in the source content and the target content. In some examples, objects, expressions, activities, and other content may be weighted more heavily when matching single focus shots to determine appropriate target segment shots compared to matching medium focus shots, which may generally be weighted based on the global behavior of such shots. Landscape segment shots in the source content may generally be identified as compatible and matched with target content that is also classified as having little or no focus.

[0075] In an example, the automatic non-linear editing system 140 matches one or more portions or segments of the source content with different portions or segments in the target content. For example, a source content video having eight shots may have one or more global and / or shot-specific editing style attributes, each shot having similar or different timing. The automatic non-linear editing system 140 may generally process each of the source video shots to identify compatible target content based on constraints (e.g., shot length, frame size), object recognition, shot focus, content similarity, and other aspects to match different segments of the target content (e.g., shots from one or more videos in the target content) to corresponding shots of the source content.

[0076] At 210, the computing system automatically adjusts the content segment from the target content based at least in part on modifying the content segment from the target content using the determined editing style of the shot from the first video. In an example, the automatic non-linear editing system 140 adjusts the shot of the target video content that has been matched with the shot of the source video content by transferring one or more editing style attributes extracted from the shot of the source video content to the matching shot of the target video content. In various examples, each of the one or more segments of the target content (e.g., a shot, an image, etc.) is matched with the source content (e.g., a shot, an image, etc.). Then, the editing style attributes of the segment source content are automatically transferred to the corresponding segment that matches the target content, for example, to generate and present the target content having the editing style attributes of the source content.

[0077] In the example, a collection of one or more segments of source content has been matched to corresponding corresponding segments of target content. Then, the editing style attributes detected in one or more segments of the source content are transferred to the corresponding corresponding segments of the target content. Such editing style attributes may include, but are not limited to, one or more information about framing, camera motion, focusing, zooming, transitions, playback speed, color, lighting, audio, or text. In addition, such editing style attributes can be measured, extracted, stored, and provided for use in source content editing template 150. In addition, such editing style attributes can be automatically transferred from source content to target content to correctly apply the same feeling, flow, effect, and / or impression from the source content to different target content provided by the user.

[0078] In an example, camera movement or motion is transferred from the source content to the target content. For example, frame-to-frame camera motion can be detected and measured in each of the source content and the target content. In an example, by back-projecting the target content The camera motion or movement in the target content may be canceled - for example, frame by frame - to stabilize the movement in the target content. Once or when the camera motion or movement is removed from the target content, the camera motion or movement from the corresponding source content may be applied to the target content. For example, transferring the camera motion or movement from the source content to the target content may include canceling the original motion, movement, or projection in the target content, and applying the camera motion or movement from the source content: In an example, reprojection for camera motion or movement may be performed based on concatenating homographies from source content to obtain a projection that is applied once to target content, e.g., instead of performing reprojection from source content to target content multiple times (e.g., for each frame in a video).

[0079] In an example, a starting position is determined in a segment of the target content based on the camera motion or movement being transferred from the source content. . For example, the starting position for re-projecting onto the target content may include an initial scale and offset. In an example, the starting position is determined based on analyzing the target content taking into account camera motion or movement of the segment being transferred to the target content. For example, the starting position can be determined so that the camera movement or movement projected onto the target content remains within the boundaries of the target content (e.g., remains within the frame and does not exceed the frame). As such, the starting position can be determined based on moving the starting position to a specific position within the frame of the target content and / or at a specific zoom level to allow camera movements or motions from the source content to be correctly performed based on the target content. In some examples, the target position is determined and / or adjusted, for example, to amplify or maximize the camera motion or movement being applied to the target content.

[0080] Figure 7 Depicted is a diagram 700 of a starting position for re-projecting target content based on camera motion of source content for automatic non-linear editing style transfer. In an example, images 702-706 represent positioning of source content movement within a rectangular frame boundary of target content such that the movement being applied from the source content remains within the boundary while maximizing the size and coverage of the shot content, as shown in images 708-712.

[0081] For example, image 702 shows the determined starting position of the source content movement within the rectangular boundary of the target content. Image 708 shows the starting position as viewed in the frame of the target content. Image 704 shows the start of the movement applied from the source content within the rectangular frame boundary of the target content. Image 710 shows how the source content movement corresponds to and appears within the frame of the target content. Image 706 shows how the applied source content movement continues and remains within the rectangular frame boundary of the target content. Image 712 shows the corresponding movement from image 706 within the frame of the target content.

[0082] In examples, any one or more editing style attributes of the source content can be transferred to the target content. For example, the playback speed detected and measured in the source content can be transferred and applied to the target content at the same rate as in the source content or at a different rate (e.g., faster, slower). In some examples, the playback of the source content and the target content is calculated as a constant value. In addition, frame resampling can be used to transfer the playback speed from the source content to the target content before reprojection. .

[0083] In an example, the lighting and brightness of the source content is transferred to the target content. In some examples, a brightness detector is used to detect and measure, for example, the lighting or brightness over time in the source content. For example, the brightness detector can average the values ​​of the lighting and / or pixels in the shots of the source content. Such values ​​can then be used to adjust the lighting and brightness in the target content, for example, to transfer stylistic lighting adjustments such as gradients and transitions from the source content to the target content.

[0084] In various examples, editing style attributes can be transferred from the same type of source content and target content and / or different types of source content and target content. For example, a shot from the target video content can be matched with various editing style attributes of a shot from the source video content via automatic editing style transfer and receive various editing style attributes from the shot of the source video content. Similarly, one or more target content images can be matched with various editing style attributes from different forms of source content (e.g., one or more source content shots and / or images) and receive various editing style attributes from different forms of source content. In one example, the target content image is matched with a shot of the source video content. Then, a video clip can be generated from the target content image, for example, by repeating the image as a frame over a period of time. Then, editing style attributes such as the measured movement in the source video content (e.g., a shot) can be transferred to the generated video clip for the matching target content image.

[0085] Figure 3 Depicted is a flow chart of an example method for performing automatic non-linear editing style transfer from source content to user-provided target content according to an example embodiment of the present disclosure.

[0086] although Figure 3 For the purpose of illustration and discussion, the steps performed in a specific order are depicted as examples, but the method of the present disclosure is not limited to the specific order or arrangement shown. Without departing from the scope of the present disclosure, various steps of method 300 can be omitted, rearranged, combined and / or adapted in various ways.

[0087] At 302, the computing system generates one or more editing styles from source content. In an example, the automatic non-linear editing system 140 receives one or more segments or fragments of content. For example, any user such as an end user, creator, administrator, or other person can provide one or more segments of source content. In some examples, the source content can be automatically selected or determined based on ratings, user feedback, machine learning analysis, content type, genre, user attributes, user preferences, and / or other ways of identifying source content for use by the automatic non-linear editing system 140.

[0088] In an example, the automatic non-linear editing system 140 generates one or more editing styles for each of one or more segments of source content. In various examples, the automatic non-linear editing system 140 analyzes and processes source content according to examples of the present disclosure to detect, measure and record editing style attributes of source content. Editing style attributes may include, for example, camera motion, focus, zoom, transition, playback speed, color, lighting, audio or text associated with the corresponding lens of the source video. In some examples, various aspects of the editing style may be determined, for example, during a user session and applied to target content. In other examples, a source content editing template 150 may be created and stored together with information about the editing style. Such information may then be used and reused across any number of users to automatically apply the editing style from the source content to various target contents.

[0089] At 304, the computing system provides the user with one or more editing styles for selection. In an example, the automatic non-linear editing system 140 provides the user with the option of selecting one or more source content items having an editing style. For example, the user can select one or more available editing styles to apply to the user's target content. For example, the user can select a single source content editing style to apply to the target content. The user can also generate different versions of the target content, each corresponding to a different source content editing style (e.g., for comparison to determine a favorite, etc.).

[0090] In various examples, the user is provided with an opportunity to preview the source content associated with the corresponding editing style, for example, using a thumbnail preview or other methods of providing access to the associated source content. In an example, the source content and the editing style can be searchable. The source content and the associated editing style can also be grouped together to be presented to the user, for example, based on themes, channels, influencers, creators and / or categories such as vacations, sports, hiking, cycling, cooking, etc. For example, a user can search for source content related to vacations, and be presented with several source content editing styles for selection. In some examples, users can also provide or select their own source content, including but not limited to video clips, music videos, original videos, pictures, photos, etc. Such content can be provided, for example, from one or more sites or services such as photo sharing, video sharing, social networking, etc.

[0091] At 306, the computing system receives the target content to be adjusted based on the selected editing style. In an example, the user provides one or more segments of the target content, which will be analyzed and matched to the source content to perform automatic editing style transfer. For example, the user can provide one or more segments of visual content, such as images and / or videos. Such content can be user-generated content associated with a user account. In some examples, the user can specify one or more specific segments of the target content to be used. The user can also provide a location that includes a collection of content that the user wishes to use as target content. In one example, the user provides a user account of a user associated with a photo sharing service, a video sharing service, a storage service, a social media, and / or one or more other services, and the automatic non-linear editing system 140 can access and analyze the user account to identify the user's various content that will be used as target content.

[0092] At 308, the computing system analyzes the target content in view of the selected editing style. In an example, the automatic non-linear editing system 140 of the computing system analyzes each of the one or more segments of the target content provided by or associated with the user. For example, data, metadata, images, and other aspects of the target content may be analyzed to identify and classify themes and objects associated within the respective segments of the target content.

[0093] In some examples, as described in 202 and in the present disclosure, shot boundary detection can be performed on the target video content to identify shots and shot boundaries in the video content. Similarly, as described in 204 and in the present disclosure, object detection and classification can be performed on the target visual content to identify one or more of objects, expressions, activities, shot focus, and / or other aspects of the visual content. For example, such detection and classification of the target visual content can be performed to assist in matching the source content with the target content when performing automatic editorial style transfer.

[0094] In an example, the automatic non-linear editing system 140 matches the analyzed target content with the corresponding segments of the source content. In one example, the source video content may include six shots that have been identified and analyzed for automatic editing style transfer based on the examples of the present disclosure. For each of the six shots in the source video content, the automatic non-linear editing system 140 compares various aspects of the shot, including but not limited to one or more of the shot length, shot focus, object, expression, activity, etc., with the various target contents that have been analyzed. For example, the automatic non-linear editing system 140 can compare and rate one or more segments or fragments of the target content that matches the source content. The matching target content can then be selected, for example, based on one or more criteria, which can be the highest-rated matching segment or fragment of the target content for each of the six shots in the source video content.

[0095] At 310, the computing system automatically adjusts the target content based on the selected editing style. In an example, the automatic non-linear editing system 140 of the computing system modifies each of the one or more segments or clips of the target content to transfer the editing style from the source content to the target content. In some examples, the automatic non-linear editing system 140 may modify the target content before transferring the editing style. For example, the length of the shot of the target video content may be adjusted, the target content orientation may be adjusted, and / or the target content quality may be automatically modified before performing the automatic editing style transfer.

[0096] At 312, the computing system provides the adjusted target content to the user. In an example, the automatic non-linear editing system 140 of the computing system automatically transfers the editing style from each of the one or more segments or sections of the source content to one or more matching segments or sections of the target content. In various examples, the automatic non-linear editing system 140 may generate a new segment of content, such as a new video including the target content having the editing style transferred from the source content.

[0097] In an example, one or more user interfaces 106 may be provided to a user to perform various operations involving the generated output, which includes target content having an editing style applied from source content. For example, a user may use the user interface 106 to store, modify, publish, regenerate, and / or discard the associated output. In one example, the user interface 106 may include a simplified editing interface that allows a user to include, exclude, and / or rearrange each of one or more shots in the visual output generated from the automatic non-linear editing system 140. For example, if a user does not like one shot or image in a generated output that includes twenty shots or images, the user may select the specific shot or image to be excluded and save the resulting output.

[0098] Example devices and systems

[0099] Figure 8 A diagram of an example machine in the form of a computer system 800 is illustrated, wherein an instruction set for causing the machine to perform any one or more of the operations discussed herein may be executed. In other examples, the machine may be connected (e.g., networked) to other machines in a LAN, an intranet, an extranet, or the Internet. The machine may operate in the capacity of a server or client machine in a client-server network environment, or as a peer machine in a peer (or distributed) network environment. The machine may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a wearable computing device, a Web appliance, a server, a network router, a switch or a bridge, or any machine capable of executing an instruction set (sequential or otherwise) of an action to be taken by the machine. In addition, although only a single machine is shown, the term "machine" should also be considered to include any collection of machines that execute an instruction set (or multiple instruction sets) individually or jointly to perform any one or more of the operations discussed herein.

[0100] The computer system 800 includes at least one processing device (processor 802), a main memory 804 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM), double data rate (DDR SDRAM) or DRAM (RDRAM)), etc.), a static memory 806 (e.g., flash memory, static random access memory (SRAM)), etc.), and a data storage device 818, which communicate with each other via a bus 830.

[0101] The processor 802 represents one or more general-purpose processing devices, such as a microprocessor, a central processing unit, etc. More specifically, the processor 802 may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor that implements other instruction sets or a processor that implements a combination of instruction sets. The processor 802 may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, etc. The processor 802 is configured to execute instructions 822 for performing the operations discussed herein.

[0102] The computer system 800 may also include a network interface device 808. The computer system 800 may further include a video display unit 810 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an alphanumeric input device 812 (e.g., a keyboard), a cursor control device 814 (e.g., a mouse), and a signal generating device 816 (e.g., a speaker).

[0103] The data storage device 818 may include a computer-readable storage medium 828 on which is stored one or more instruction sets 822 (e.g., software computer instructions) embodying any one or more of the examples described herein. The instructions 822 may also reside completely or at least partially within the main memory 804 and / or within the processor 802 during execution of the instructions by the computer system 800, the main memory 804 and the processor 802 also constituting computer-readable storage media. The instructions 822 may be transmitted or received over the network 820 via the network interface device 808.

[0104] In one example, instructions 822 include instructions for automatic non-linear editing system 140 (eg, Figure 1 The computer-readable storage medium 828 (machine-readable storage medium) is shown as an example as a single medium, but the term "computer-readable storage medium" should be considered to include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) storing the one or more instruction sets. The term "computer-readable storage medium" may also include any medium capable of storing, encoding, or carrying an instruction set executed by a machine and causing the machine to perform any one or more of the operations of the present disclosure. Therefore, the term "computer-readable storage medium" should be considered to include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0105] Many details are set forth in the foregoing description. However, those of ordinary skill in the art who benefit from the present disclosure will appreciate that the present disclosure can be practiced without these specific details. In some instances, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring the present disclosure.

[0106] Some portions of the detailed description have been presented in terms of processing and symbolic representations of operations on data bits within a computer memory. Here, a process is generally conceived to be a self-consistent sequence of steps leading to a desired result. The steps are those requiring physical manipulations of physical quantities. Usually, although not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. For reasons of common usage, it has proven convenient at times to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

[0107] It should be remembered, however, that all of these and similar terms will be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless otherwise stated, as will be apparent from the following discussion, discussions throughout the specification using terms such as "analyze," "determine," "identify," "adjust," "transmit," "receive," "process," etc. should be understood to refer to the actions and processes of a computer system or similar electronic computing device that manipulates data represented as physical (e.g., electronic) quantities within a computer system's registers and memories and transforms it into other data similarly represented as physical quantities in the computer system's memories or registers or other such information storage, transmission, or display devices.

[0108] Certain examples of the present disclosure also relate to an apparatus for performing the operations herein. The apparatus may be constructed for the intended purpose, or it may include a computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer-readable storage medium, such as, but not limited to, any type of disk including a floppy disk, an optical disk, a CD-ROM, and a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), an EPROM, an EEPROM, a magnetic card or an optical card, or any type of medium suitable for storing electronic instructions.

[0109] It should be understood that the above description is intended to be illustrative, rather than restrictive. After reading and understanding the above description, many other examples will be apparent to those skilled in the art. Therefore, the scope of the present disclosure should be determined with reference to the full scope of the equivalents authorized by the attached claims and such claims.

[0110] Additional public content

[0111] The technology discussed herein relates to servers, databases, software applications, and other computer-based systems, as well as the actions taken and the information sent to and from such systems. The inherent flexibility of computer-based systems allows for various possible configurations, combinations, and divisions of tasks and functions between and among components. For example, the processes discussed herein can be implemented using a single device or component or a combination of multiple devices or components. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0112] Although the subject matter has been described in detail with respect to various specific example embodiments of the subject matter, each example is provided by way of explanation, rather than limitation of the present disclosure. Those skilled in the art, after understanding the foregoing, can easily produce changes, variations, and equivalents to such embodiments. Therefore, the subject matter disclosure does not exclude such modifications, variations, and / or additions to the subject matter, which will be apparent to those of ordinary skill in the art. For example, a feature shown or described as part of one embodiment can be used together with another embodiment to produce yet another embodiment. Therefore, the present disclosure is intended to cover such changes, variations, and equivalents.

Claims

1. A computing system configured to decompose video data into a plurality of layers, the computing system comprising: one or more processors; as well as One or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the computing system to perform operations comprising: accessing, by the computing system, video data comprising a plurality of corresponding shots; accessing, via the computing system, a stored edit template; Processing the video data based on the editing template by the computing system; automatically transferring the editing style of the corresponding shot in a source video associated with the editing template to generate an updated output video; and The updated output video is provided by the computing system via an interactive graphical user interface.

2. The computing system of claim 1, wherein: The editing style includes at least one of: (i) framing, (ii) camera movement, (iii) focus, (iv) zoom, (v) transition, (vi) playback speed, (vii) color, (viii) lighting, (ix) audio, and (x) text.

3. The computing system of claim 1, wherein: The video data includes at least one of: (i) user-generated content, and (ii) third-party content.

4. The computing system of claim 1, wherein: The updated output video includes at least one of: (i) text data, (ii) audio data, (iii) graphic data, (iv) picture data, (v) image data, (vi) video data, and (vii) multimedia data.

5. The computing system of claim 1, wherein: The video data includes at least one of: (i) original content, (ii) newly captured content, (iii) pre-processed content, (iv) partially edited content, (v) curated content, and (vi) user-generated content.

6. The computing system of claim 1, wherein: The editing template includes global editing style attributes and local editing style attributes.

7. The computing system of claim 6, wherein: The local editing style attributes include attributes associated with a specific location, segment, or time period within the source content.

8. The computing system of claim 1, the operations comprising: The stored edit template is generated by: Based on analyzing the source video, determining one or more shot boundaries in the source video; analyzing the identified content in each of the one or more shots in the source video based on performing object detection on the corresponding shot; determining an editing style for each of the one or more shots in the source video based at least in part on measuring motion across frames within the corresponding shot; generating an editing template based on the editing style; as well as The edit template is stored.

9. The computing system of claim 8, wherein: Determining the one or more shot boundaries is based at least in part on detecting changes in frame color across frames of the source video.

10. The computing system of claim 8, wherein: Determining the one or more shot boundaries is based at least in part on analyzing keypoint matches across frames of the source video.

11. A computer-implemented method comprising: accessing, by a computing system, video data comprising a plurality of corresponding shots; accessing, via the computing system, a stored edit template; Processing the video data based on the editing template by the computing system; automatically transferring the editing style of the corresponding shot in a source video associated with the editing template to generate an updated output video; as well as The updated output video is provided by the computing system via an interactive graphical user interface.

12. The computer-implemented method of claim 11, wherein: The editing style includes at least one of: (i) framing, (ii) camera movement, (iii) focus, (iv) zoom, (v) transition, (vi) playback speed, (vii) color, (viii) lighting, (ix) audio, and (x) text.

13. The computer-implemented method of claim 11 , wherein: The video data includes at least one of: (i) user-generated content, and (ii) third-party content.

14. The computer-implemented method of claim 11, wherein: The updated output video includes at least one of: (i) text data, (ii) audio data, (iii) graphic data, (iv) picture data, (v) image data, (vi) video data, and (vii) multimedia data.

15. The computer-implemented method of claim 11, wherein: The video data includes at least one of: (i) original content, (ii) newly captured content, (iii) pre-processed content, (iv) partially edited content, (v) curated content, and (vi) user-generated content.

16. The computer-implemented method of claim 11, wherein: The editing template includes global editing style attributes and local editing style attributes.

17. The computer-implemented method of claim 16, wherein: The local editing style attributes include attributes associated with a specific location, segment, or time period within the source content.

18. The computer-implemented method of claim 11, comprising: The stored edit template is generated by: Based on analyzing the source video, determining one or more shot boundaries in the source video; analyzing the identified content in each of the one or more shots in the source video based on performing object detection on the corresponding shot; determining an editing style for each of the one or more shots in the source video based at least in part on measuring motion across frames within the corresponding shot; generating an editing template based on the editing style; as well as The edit template is stored.

19. The computer-implemented method of claim 18, wherein: Determining the one or more shot boundaries is based at least in part on detecting changes in frame color across frames of the source video.

20. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause a computing system to perform operations comprising: accessing, by the computing system, video data comprising a plurality of corresponding shots; accessing, via the computing system, a stored edit template; Processing the video data based on the editing template by the computing system; automatically transferring the editing style of the corresponding shot in a source video associated with the editing template to generate an updated output video; as well as The updated output video is provided by the computing system via an interactive graphical user interface.

Images