User interface for video editing applications on touch screen devices

Abstract

In one or more implementations, a computing device is configured to display a first frame of a media composition, the first frame corresponding to a first location of a movable playback head along a timeline. The first frame is displayed in a first display area of the GUI, while the timeline is concurrently displayed in a second display area of the GUI. The computing device also detects initiation of a hovering user input associated with a second location along the timeline. In response to detecting the initiation of the hovering user input, the computing device replaces display of the first frame with display of a second frame of the media composition in the first display area of the GUI. Thereafter, in response to detecting termination of the hovering user input, the computing device resumes display of the first frame of the media composition in the first display area of the GUI.

Description

[0001] A portion of the disclosure in this patent document contains material protected by copyright or photomask copyright. Because this patent document or the disclosure appears in the Patent and Trademark Office's patent files or records, the owner of the copyright or photomask copyright does not object to any reproduction of this patent document or the disclosure, but otherwise reserves all rights to the copyright or photomask copyright in any form.

[0002] Incorporated by reference; Disclaimer The following applications are hereby incorporated herein by reference: Application No. 18 / 314,110, filed May 8, 2023; and Application No. 63 / 500,897, filed May 8, 2023. The applicant hereby withdraws any disclaimers regarding the scope of the claims in the parent application or its examination history and informs the U.S. Patent and Trademark Office that the claims in this application may be broader than any claims in the parent application. Technical Field

[0003] This disclosure relates in its entirety to user interfaces for use with video editing applications on touchscreen devices. Background Technology

[0004] Changing the layout of the display area on a tablet or tablet computer can be difficult, where user input is typically received via a touchscreen display. When using fingertip touch input, accurately moving around an object on a touchscreen display is challenging because the user's fingertip is many times larger than the area being touched to induce the desired change. Furthermore, rapid movement through media is error-prone, and sometimes using touch input methods is impractical. Summary of the Invention

[0005] In some implementations, the computing device is configured to display a first video frame composed of media, the first video frame corresponding to a first position of a movable playhead along a timeline. The first video frame is displayed in a first display area of ​​a graphical user interface (GUI), while the timeline is concurrently displayed in a second display area of ​​the GUI. The computing device is configured to detect the initiation of hover user input associated with a second position along the timeline. In response to detecting the initiation of hover user input, the computing device is configured to replace the display of the first video frame in the first display area of ​​the GUI with the display of a second video frame composed of media. Subsequently, in response to detecting the termination of hover user input, the computing device is configured to resume the display of the first video frame composed of media in the first display area of ​​the GUI.

[0006] In one or more specific implementations, different types of hover user input can be detected by a computing device to trigger a preview of a second video frame. A first type of hover user input is positioned above a second position along the timeline. A second type of hover user input is positioned above a representation of the second video frame composed of media, which corresponds to the second position along the timeline.

[0007] In some implementations, a first video frame is displayed during playback of the media composition. When the media composition is played back, a specific video frame of the media composition is displayed based on the positioning of the movable playhead along the timeline at any given moment. Based on detected hover user input, the playback may be replaced with an indicated second video frame.

[0008] In one or more embodiments, the computing device is configured to detect the initiation of a subsequent hover user input associated with a third video frame. The subsequent hover user input may be of one of the following types: a third hover user input associated with the third video frame and positioned above a third location along the timeline; or a fourth hover user input positioned above a representation of a media clip corresponding to the third video frame. In response to detecting the initiation of the subsequent hover user input, the computing device is configured to replace the display of a second video frame with the display of the third video frame in a first display area of ​​the GUI.

[0009] In some implementations, the computing device may display a GUI comprising a set of user-adjustable display areas. Each user-adjustable display area in the set has a corresponding size including a height dimension and a width dimension. The computing device may receive a single touch input to adjust a first size of a first user-adjustable display area, and in response to receiving the single touch input, may make one or more adjustments to the GUI. The computing device may adjust the first size of the first user-adjustable display area, and adjust both the height and width dimensions of a second user-adjustable display area.

[0010] In one implementation, the set of user-adjustable display areas completely covers a specific area of ​​the graphical user interface. In this implementation, the computing device can calculate adjustments to the height and width dimensions of a second user-adjustable display area such that, after such adjustments, the specific area of ​​the GUI remains completely covered by the set of user-adjustable display areas.

[0011] In some implementations, the computing device may modify the layout of a set of user-adjustable display areas in response to receiving a single touch input. One such modification may include swapping the horizontal positioning of a second user-adjustable display area relative to a third user-adjustable display area. Another modification may include swapping the vertical positioning of a second user-adjustable display area relative to a third user-adjustable display area. Further modifications may include changing the layout from row-based to column-based, or from column-based to row-based.

[0012] This specific implementation offers at least the following advantages: Users can use touch-based user input to change the display layout and preview sections of the media composition with greater accuracy. The accuracy achievable using these touch-based user inputs is similar to that of input technologies available on desktop devices.

[0013] Details of one or more specific embodiments are set forth in the accompanying drawings and detailed description below. Other features, aspects, and potential advantages will become apparent from the detailed description, the drawings, and the claims. Attached Figure Description

[0014] Figure 1 This is a block diagram of an example system for manipulating a graphical user interface based on touch input.

[0015] Figures 2A to 2D An example user interface with a set of user-adjustable display areas is shown.

[0016] Figures 3A to 3B An example user interface with a set of user-adjustable display areas is shown.

[0017] Figures 4A to 4D An example user interface is shown, illustrating the hover user input for modifying content.

[0018] Figures 5A to 5D An example user interface is shown, illustrating the hover user input for modifying content.

[0019] Figure 6 This is a flowchart of an example process for modifying the display of content based on hover user input.

[0020] Figure 7 This is a flowchart of an example process for modifying the layout of a user interface based on touch input.

[0021] Figure 8 It is possible. Figures 1 to 7 A block diagram of an example computing device illustrating its features and processes.

[0022] Similar reference symbols in the various figures indicate similar elements. Detailed Implementation

[0023] System Architecture Figure 1 This is a block diagram of an example system 100 for manipulating a graphical user interface (GUI) based on touch input. System 100 includes an application engine 102 electrically coupled to at least one data store 122. The application engine 102 includes a set of modules and / or processes configured to perform one or more functions for capturing touch user input on a touchscreen display and to perform specific functions related to the display of information on the touchscreen display, which are described below.

[0024] In one or more methods, the user interface module 104 of application engine 102 is configured to create and / or construct one or more user interfaces 118 for providing information to user 120. The user interface 118 may be configured to be used on a touchscreen display, and the user interface module 104 may be configured to receive touch user input via the user interface 118 using the touchscreen display. In various embodiments, the user interface 118 may be dynamically updated based on user input received through the user interface 118.

[0025] In one embodiment, the user interface module 104 is configured to generate and display a GUI comprising a set of user-adjustable display areas via a touchscreen display. When displayed in the GUI, each user-adjustable display area is associated with a corresponding size. These sizes include a height dimension and a width dimension.

[0026] Each user-adjustable display area is configured to display certain information to user 120. For example, the user-adjustable display area may include a timeline for manipulating and displaying time positioning within media composition 128 and / or various media clips 126. In another example, the user-adjustable display area may include a collection of media clips 126 added to media composition 128.

[0027] According to one example, the user-adjustable display area may include a media clip information window that displays additional details about the media clip 126 and / or media composition 128, such as name, creation / modification date, length, media type, pictograms of the audio / video content therein, frame browser, etc.

[0028] In one example, the user-adjustable display area may include a playback window for viewing the current frame of the selected media source (e.g., media clip 126, media composition 128, etc.), and the playback window may include playback controls (play, stop, pause, fast forward, reverse, etc.).

[0029] In another example, the user-adjustable display area may include a view options window for selecting options on how things are displayed, such as options associated with the appearance and functionality of the GUI, view options associated with the playback of the selected media clip 126 and / or media composition 128 in the playback window, modification options associated with the selected media clip 126 and / or media composition 128, etc.

[0030] In various methods, one or more user interfaces 124, generated individually or in conjunction with one or more display areas 130 used by the user interface module 104, can be stored in the data store 122. The user interface 124 can be generated based on a user interface template, display area layout, and / or can be dynamically created. The generated user interface 124 can be stored in the data store 122 along with some associated identifiers for faster searching and retrieval when a specific type of user interface is requested for presentation to the user 120.

[0031] The touch input analysis module 106 of the application engine 102 is configured to analyze touch input provided by the user 120, which is received in the active user interface 118 displayed on the touchscreen display. This touch input may include finger touch input, stylus touch input, and / or hover input, where the user 120 hovers near the touchscreen display but does not actually touch it. Hover input can cause a different action compared to a single touch contact. Furthermore, swipe input and multiple taps can also be received via the touchscreen display and can cause a different action compared to a single touch contact.

[0032] In one implementation, a single touch input can be a click and drag user input that moves the edge of a specific user-reducible display area, either from the center outwards to indicate an increase in size, or from the center inwards to indicate a decrease in size. In another implementation, a single touch input can be a user input that selects a graphical interface element associated with a specific user-reducible display area, such as a button for closing the specific user-reducible display area, a control for expanding the specific user-reducible display area to full screen, a selector for minimizing the specific user-reducible display area, etc.

[0033] According to one method, a single touch input can be a swipe input. In one embodiment, the swipe input may begin on a portion of a specific user-reducible display area and end at the edge of the GUI or outside the area of ​​the GUI or touchscreen display. The swipe input may instruct the specific user-reducible display area to be minimized, reduced, removed, or turned off in various ways. In another embodiment, the swipe input may begin at the edge of the GUI or outside the area of ​​the GUI or touchscreen display and end at a location within the GUI. The swipe input may instruct the specific user-reducible display area to be turned on, displayed, maximized, or expanded in various ways.

[0034] In one or more embodiments, the touch input analysis module 106 is configured to detect a single touch input that adjusts the size of a specific user-adjustable display area from the set of user-adjustable display areas shown in the user interface 118. The touch input analysis module 106 analyzes this touch input that adjusts the size of the specific user-adjustable display area to determine which size has changed and by how much, as indicated by the single user touch input. This information is provided in real-time to the display area adjustment module 108 to calculate how to adjust the specific user-adjustable display area displayed to the user interface 118, as well as any other possible user-adjustable display areas, to maximize the use of the total display area of ​​the GUI based on the user's requested size change.

[0035] Display area adjustment module 108 is configured to modify display area 130 within active user interface 118 based on received touch input. In one embodiment, in response to receiving a single touch input, display area adjustment module 108 calculates in real time adjustments for both the height and width dimensions of another user-adjustable display area displayed on interface 118, as well as adjustments determined for the size of a specific user-adjustable display area manipulated by user 120. User interface module 104 uses these determined adjustments, alone or in conjunction with display area adjustment module 108, to modify user interface 118 to adjust both the height and width dimensions of another user-adjustable display area, while simultaneously adjusting the first size of the specific user-adjustable display area manipulated by user 120.

[0036] In one method, a set of user-adjustable display areas completely covers a specific area of ​​the GUI. In this method, a display area adjustment module 108 calculates adjustments to the height and width dimensions of another user-adjustable display area such that, after such adjustments, the specific area of ​​the GUI remains completely covered by the set of user-adjustable display areas.

[0037] In one or more embodiments, in response to receiving a single touch input, the display area adjustment module 108 may also modify the layout of the set of user-adjustable display areas to swap the horizontal positioning of at least two user-adjustable display areas (which may include or exclude swapping the horizontal positioning of a specific user-adjustable display area manipulated by user 120). In these cases, the vertical dimension of another user-adjustable display area differs from the vertical dimension of the user-adjustable display area that was horizontally swapped. Furthermore, the display area adjustment module 108 may modify the layout based on the difference in the vertical dimensions of the swapped user-adjustable display areas, because horizontally swapping these display areas allows for a more efficient display of information included in the set of display areas, rather than swapping other display areas or changing their size without performing a swap.

[0038] In some implementations, in response to receiving a single touch input, the display area adjustment module 108 can modify the layout of the set of user-adjustable display areas to swap the vertical positioning of at least two user-adjustable display areas (which may include or exclude swapping the vertical positioning of a specific user-adjustable display area manipulated by user 120). In these cases, the horizontal size of another user-adjustable display area differs from the horizontal size of the user-adjustable display area that was vertically swapped. Furthermore, the display area adjustment module 108 can modify the layout based on the difference in the horizontal size of the swapped user-adjustable display areas, because vertically swapping these display areas can achieve more efficient display of information included in the set of display areas or change the size without performing a swap.

[0039] According to one method, when the display area adjustment module 108 adjusts both the height and width of another user-adjustable display area, the size of the other user-adjustable display area can increase and expand. In other words, the percentage of the entire GUI area occupied by this other display area relative to other display areas in the set of display areas can increase and become larger.

[0040] According to one method, when the display area adjustment module 108 adjusts both the height and width dimensions of another user-adjustable display area, the other user-adjustable display area can be reduced in size. In other words, the percentage of the entire GUI area occupied by this other display area relative to other display areas in the set of display areas can be reduced and become smaller.

[0041] In another approach, when the display area adjustment module 108 adjusts both the height and width of another user-adjustable display area, the other user-adjustable display area is removed from the GUI. This removal may be the result of a request to minimize or close the other user-adjustable display area.

[0042] In some implementations, in response to receiving a single touch input, the display area adjustment module 108 can adjust the height, width, or both of the user-adjustable display area for a third user. In these implementations, a single touch input causes an adjustment to the size of three or more user-adjustable display areas within the GUI to achieve a more efficient display of information included in the set of display areas.

[0043] In one or more embodiments, in response to receiving a single touch input, the display area adjustment module 108 can modify the layout of the set of user-adjustable display areas from row-based to column-based, or from column-based to row-based. In some methods, portions of the GUI can be changed from row-based to column-based, or from column-based to row-based, to achieve more efficient display of information included in the set of display areas.

[0044] In one implementation, the content preview module 110 is configured to display a preview of a portion of media clip 126 or media composition 128 that is not currently playing in the main user-adjustable display area within the set of user-adjustable display areas. In some methods, the main user-adjustable display area may be used to play back media clip 126 and / or media composition 128 for user 120 to view via user interface 118.

[0045] In this implementation, the main user-adjustable display area displays a first video frame corresponding to a first position of the movable playhead along the timeline (e.g., shown in a separate user-adjustable display area) (e.g., media clip 126 and / or media composition 128). The touch input analysis module 106 detects the initiation of hover user input associated with a second position along the timeline. The hover user input indicates that the user 120 wants to preview a second video frame without actually adjusting the playback position of any content displayed in the main user-adjustable display area.

[0046] For example, hover user input could be of the first type, where user 120 hovers (e.g., with a finger or stylus) above a second location along the timeline. The second location is a position on the timeline associated with a second video frame, which is different from the first location associated with the currently displayed first video frame of media composition 128.

[0047] In another example, hover user input could be a second type of hover user input, where user 120 hovers over a representation of a second video frame comprising media, the second video frame corresponding to a second location along the timeline. The representation of the second video frame could be a thumbnail associated with the source of the second video frame, video played in a separate display area, etc.

[0048] According to another example, hover user input can be a third type of hover user input, where user 120 hovers over a pictograph associated with the source of the second video frame at a location associated with the second video frame.

[0049] In response to the detection of a hovering user input, the content preview module 110 causes the display of a second video frame composed of media to replace the display of the first video frame in the main display area of ​​the GUI. In one embodiment, the content preview module 110 communicates with one or more of the user interface module 104, the modification module 116, and / or the data storage interface 112 to cause the user interface 118 to display the second video frame composed of media in the main display area.

[0050] The touch input analysis module 106 can detect the termination of hovering user input. This termination can be caused by the user actually touching the touchscreen display (at the same location as the hovering input or elsewhere), moving beyond a threshold distance from the touchscreen display, or receiving an instruction to terminate by the user, etc.

[0051] In response to the detection of the termination of hovering user input, the content preview module 110 causes the display of the first video frame of the media composition to resume in the first display area of ​​the GUI. In one embodiment, the content preview module 110 communicates with one or more of the user interface module 104, the modification module 116, and / or the data storage interface 112 to cause the user interface 118 to resume the display of the first video frame of the media composition in the main display area.

[0052] In one method, a first video frame can be displayed during playback of the media composition. In this method, during playback, the first video frame is displayed based on the positioning of a movable playback head along a timeline at the moment corresponding to the first video frame within the media composition 128.

[0053] According to one or more methods, touch user input can be received while the playback of media composition 128 is presented on a touchscreen display of a computing device. Media composition 128 may include video, audio, images, moving images, animations, etc., or any combination thereof. In one embodiment, media composition generator 114 may generate media composition 128 based on available media content and / or user input.

[0054] Media composition generator 114 can generate media composition 128 based on one or more media clips 126. Modification module 116 is configured to allow user 120 to provide one or more modifications to media clip 126 and / or media composition 128. Modification module 116 receives user input to modify media clip 126 and / or media composition 128. In various methods, user input to modify media clip 126 and / or media composition 128 can be provided to modification module 116 by user interface module 104 or touch input analysis module 106. In response to user input to modify media clip 126 and / or media composition 128, modification module 116 can adjust and / or modify media clip 126 and / or media composition 128 according to user input.

[0055] In one implementation, the modified user input can indicate a new duration for media clip 126 and / or media composition 128. In response to the indicated new duration, the modification module 116 adjusts media clip 126 and / or media composition 128 to have the new duration. Following this modification, during each subsequent playback of media clip 126 and / or media composition 128, media clip 126 and / or media composition 128 are played for the new duration instead of the original duration.

[0056] In one implementation, user input can modify one or more media clips 126. For example, modifications may include, but are not limited to, trimming media clip 126 to remove at least a portion of media clip 126 (e.g., making media clip 126 shorter in duration and removing content from the end), shortening media clip 126 to compress the content of media clip during playback (e.g., shortening the playback duration without removing any content), and expanding media clip 126 to stretch the content of media clip 126 during playback (e.g., extending the playback duration without adding any content), etc.

[0057] In response to modifications to media clip 126, modification module 116 adjusts media clip 126 to generate a modified version showing the modified media clip. These modified media clips can also be stored in data storage 122.

[0058] Application engine 102 includes a data storage interface 112 for storing data in a data store 122 and for retrieving data from the data store 122. The data store 122 can be used to store information and / or data for application engine 102 and can be any type of storage unit and / or device for storing data (e.g., a file system, database, collection of tables, or any other storage mechanism). Additionally, the data store 122 may include multiple different storage units and / or devices. These multiple different storage units and / or devices may or may not be of the same type or located at the same physical site. The data store 122 can be implemented or executed on the same computing system as application engine 102. Alternatively or additionally, the data store 122 can be implemented or executed on a computing system separate from application engine 102. The data store 122 can be communicatively coupled to any device for sending and receiving data via a direct connection or via a network.

[0059] Example User Interface Figures 2A to 2D An example user interface with a set of user-adjustable display areas is shown. Figure 2A User interface 200 is shown, which includes the following user-adjustable display areas: main display area 202, timeline 204, playback controls 206, time indicator 208, view options 210, media clip library 212, and media clip details 214. Although these user-adjustable display areas are shown and described, any number and type of user-adjustable display areas may be included in various specific implementations. In one method, the main display area 202 displays and / or plays back media compositions that may be generated based on any number of individual media clips.

[0060] In one implementation, timeline 204 allows for easy manipulation of the current playback time by adjusting the positioning of the playback head indicator along the timeline. Timeline 204 also demonstrates how the media composition is assembled by showing the positioning of each media clip within the media composition along timeline 204 from the start time to the end time of the respective media clip. Furthermore, each of these clips can be repositioned within the media composition by moving it along timeline 204, such as via drag-and-drop touch input on a touchscreen display.

[0061] For example, clip A starts at time 0:00 and ends at time 0:21 (a span of 21 seconds), clip B starts at time 0:21 and ends at time 1:30 (a span of 69 seconds), clip C starts at time 0:10 and ends at time 0:30 (a span of 20 seconds), and clip D starts at time 1:30 and can extend beyond the current time mark shown on timeline 204.

[0062] In one or more implementations, the actual media content of clips A, B, C, and D may have originated from any source available in user interface 200. Furthermore, any media clip may represent an audio-only, video-only, or audio-video portion from a source media clip.

[0063] In one implementation, playback in the main display area 202 can be time-synchronized to the time associated with a playhead indicator that can move along the timeline 204. The user can manipulate the playhead indicator to select which portion and / or exact frame of the media to display in the main display area 202.

[0064] The playback control 206 may include selectable graphical elements for controlling the playback of media on the main display area 202, such as play, pause, stop, skip forward, skip backward, etc. In one approach, the user interface 200 may be implemented on a touchscreen display and may receive user input to the playback control 206 via finger touch input, stylus touch input, and / or hover input.

[0065] Hovering user input occurs when a user hovers near a touchscreen display within a threshold distance that the touchscreen display can detect, but does not actually touch the touchscreen display. Hovering user input can result in a different action compared to a touch contact. Additionally, swipe input and multiple tap inputs can also be received via the touchscreen display and can elicit a different action compared to a single touch contact.

[0066] The time indicator 208 shows the current playback time of the media shown in the main display area 202. This time indicator 208 can be synchronized with the playback head indicator shown in the timeline 204. In some embodiments, the time indicator 208 can be selectable to change the time displayed by the time indicator 208 between elapsed time, remaining time, total time, time associated with a particular media clip, etc.

[0067] View option 210 allows the user to select options for viewing media in the main display area 202 and / or how the interface 200 is displayed. Some example view options include, but are not limited to, full screen, minimize, maximize, color selection, background selection, size options, display priority, selection of one or more effects to be applied to the media displayed in the main display area 202, etc. In one approach, different view options for different parts of the user interface 200 may be displayed in different display areas.

[0068] Some example effects that can be applied include, but are not limited to, changing speed (e.g., slow motion or fast motion), applying filters (e.g., blur, black and white, drawing effects, color enhancement, color change or inversion, sharpening, softening, etc.), manipulating sound (e.g., enhancing sound, amplifying sound within a selected range, reducing sound within a selected range, changing loudness, etc.), jitter reduction, motion smoothing, removing unwanted objects, etc.

[0069] In one implementation, once a touch user input is received to change the size of one of the user-adjustable display areas, the display priority can specify which display area is shown in a preferred order in any given layout (e.g., when the main display area 202 has a higher priority than the media clip detail 214, the main display area 202 will be displayed instead of the media clip detail 214).

[0070] Media Clip Library 212 shows the available media clips that can be added to a media composition. Figure 2A In this configuration, media clip library 212 displays four media clips A, B, C, and D that can be added to a media composition. However, the number of media clips displayed and how they are displayed (e.g., thumbnails, widescreen, names, graphical representations, or icons) depends on the size of the display area for media clip library 212. In one approach, different media clip libraries can be displayed in different display areas.

[0071] Media clip details 214 display information about a media clip or media composition. In one approach, a user can select a media clip or media composition to display information about it. In another approach, media clip details 214 may display details of the media clip and / or media composition displayed in the main display area 202. In yet another approach, media clip details 214 may display details specific to the currently selected media clip and / or media composition. In one approach, media clip details for different media can be displayed in different display areas.

[0072] Some example details include, but are not limited to, name, date, length, mood, genre, size, pictograms of one or more aspects of the displayed media (audio level, video information, etc.), and type of media.

[0073] In one implementation, user input can modify one or more media clips along timeline 204. For example, modifications may include, but are not limited to, trimming a media clip to remove at least a portion of it (e.g., shortening the duration of the media clip and removing content from the end), shortening a media clip to compress its content during playback (e.g., shortening the playback duration without removing any content), and expanding a media clip to stretch its content during playback (e.g., extending the playback duration without adding any content), etc.

[0074] In response to modifications to a media clip, the modified version of the media clip indicating the change will be shown in timeline 204 instead of the previously shown versions.

[0075] According to one or more embodiments, touch user input can be received to modify a media clip on timeline 204. Example touch user input could involve touching, holding one end of a media clip, and dragging it toward the other end, thereby shortening the media clip in duration. Content will be removed from the media clip from the touched end. Alternatively, the content in the media clip can be compressed.

[0076] Another example of touch-based user input is the ability to touch, hold, and drag one end of a media clip away from the other end, thus extending the media clip in duration. If content is available, it is added to the media clip from the touched end. Otherwise, the content of the media clip can be stretched.

[0077] Another example is that touch user input allows the user to touch, hold, and drag the entire media clip in a specific direction (up, down, left, right). In one approach, such movement of the media clip can cause the media clip to exchange positions with another media clip in timeline 204. Alternatively, the movement can cause the media clip to move along the timeline to a new, designated position without exchanging positions. In yet another approach, the movement can cause one or more other media clips to slide up, down, left, or right in the same amount and direction as the media clip's movement.

[0078] In one example of touch input, a user can select a specific point in time within a media clip, which can cause the media clip to be split at that selected point in time. Furthermore, two media clips can be merged based on touch, drag, and release touch inputs that make the two media clips slide adjacent to each other.

[0079] Figure 2AA single touch input 216 is shown, which drags the corner of the media clip library 212 upwards and to the right toward the main display area 202. This single touch input instructs the resizing of the display area for the media clip library 212 (e.g., to increase its size). Based on the ending position of the touch input 216, this resizing will affect the positioning and / or placement of one or more other display areas of the interface 200. Furthermore, this resizing can change the content and amount of information displayed not only to the media clip library 212 but also to any other display areas of the interface 200 that are affected by the change.

[0080] exist Figure 2B The figure shows an example user interface 218 that can be generated by touch input 216. As shown, movement of touch input 216 has ended in user interface 218. In response to dragging media clip library 212 to the larger touch input 216, several example changes have been made to user interface 200 to generate user interface 218.

[0081] One example change is the size of the media clip library 212, which now extends its upper right corner to the end of the touch input 216. Another example change is the addition of space on the media clip library 212, allowing more media clips to be displayed (e.g., media clips A through D displayed in user interface 200, while media clips A through I are displayed in interface 218). Another example change made is the expansion of the view option 210, which extends the display area to the right towards the main display area 202.

[0082] Another example change is the reduction in the size of the main display area 202, which allows for the expansion of the media clip library 212. Additional example changes involve shifting the positioning of the media control 206 and the time indicator 208 to the right to allow for the expansion of the media clip library 212.

[0083] Although the aforementioned example changes were made to other display areas due to the change in the size of the media clip library 212 indicated by touch input 216, various methods can be used to modify the display area. Figure 2B The same, some, or different display areas may be modified.

[0084] exist Figure 2C In the user interface 218, another touch input 220 is shown, which reduces the size of the main display area 202 from the lower left corner in the upward and right directions.

[0085] Figure 2DAn example user interface 222 that can be generated by touch input 220 is shown. As shown, the movement of touch input 216 has ended in user interface 222. In response to touch input 216 dragging the corner of the main display area 202 inward to create a smaller display area, several example changes have been made to the display area shown in user interface 218 to generate user interface 222.

[0086] The example change that has been made is a reduction in the size of the main display area 202, decreasing the display area upwards and inwards from the lower left corner.

[0087] One example change is the increase in size and the display of additional information in media clip detail 214. Previously, in user interface 218, media clip detail 214 displayed a pictogram of the currently displayed portion of the media composition. In user interface 222, the pictogram is retained, but additional information about the currently displayed portion of the media composition is shown, such as name, size, date, and mood. Other possible information may be displayed in some approaches.

[0088] Another example change is the swapping of the positioning of view option 210 and media clip library 212. Additionally, an example change is the increase in the size of media clip library 212, allowing for a larger individual representation of the various media clips A through I. Other possible changes to the display area can be made as a result of touch input 216, and the changes shown are merely examples.

[0089] Figures 3A to 3B An example user interface with a set of user-adjustable display areas is shown. Figure 3A User interface 300 is shown, on which a single touch input 216 is received, dragging the lower left corner of the main display area 202 toward the media clip library 212 in the downward and leftward directions. This single touch input instructs the main display area 202 to be resized to increase in size, which will affect the positioning and / or placement of one or more other display areas of the interface 300 based on the position where the touch input 302 ends.

[0090] exist Figure 3B In this example, the user interface 304 can be generated by touch input 302. Movement of touch input 302 has already ended within user interface 304. In response to dragging the main display area 202 to a larger size touch input 302, several example changes have been made to user interface 300 to generate user interface 304.

[0091] One example change is the size of the media clip library 212, which has been reduced in size in the horizontal dimension. Another example change is the reduction in the size of the view options 210 in the horizontal dimension. Furthermore, the example change is a result of the reduced space on the view options 210, which results in fewer options being displayed for selection. Additionally, an example change has been made to expand the timeline 204, thus extending the display area to the left.

[0092] Several other example changes involve the removal of playback control 206 and time indicator 208 due to changes in the positioning of other display areas. Other possible changes to the display area can be made as a result of touch input 302, and the changes shown are merely examples.

[0093] Figures 4A to 4D An example user interface is shown, illustrating the hover user input for modifying content. Figure 4A User interface 400 is shown, which includes various display areas, including a main display area 202, a timeline 204, playback controls 206, a time indicator 208, view options 210, a media clip library 212, and media clip details 214. Although these display areas are shown, any number and type of display areas may be included in various specific implementations.

[0094] Timeline 204 displays four media clips: Media Clip A 402, Media Clip B 404, Media Clip C 406, and Media Clip D 408, which are arranged in the order shown on timeline 204 to form a media composition. Playback head indicator 410 indicates the current playback position of the media composition (e.g., it indicates the currently displayed frame in the main display area 202).

[0095] Figure 4B User interface 412 is shown, in which the user has provided hover input 414 above a second position along timeline 204. This second position corresponds to the second frame in the media composition. Figure 4C User interface 416 is shown, in which a second frame is displayed in the main display area 202 due to hover input 414 above a second position along timeline 204. In one embodiment, time indicator 208 can indicate the second frame in the media composition by displaying 1:20 and the corresponding position of playhead indicator 410 along timeline 204. In an alternative embodiment, one or both of playhead indicator 410 and time indicator 208 can continue to display the previous position and time corresponding to the first frame of the media composition, respectively.

[0096] Figure 4DThe user interface 418 is shown after the hover input 414 has ended. Because the hover input has ended, the main display area resumes displaying the media composition at the first frame, indicated by the 28:00 mark on the time indicator 208 and the position of the playhead indicator 410 in its previous position before receiving the hover input 414. In this way, the user can use the hover input to preview a portion of the media composition without actually adjusting the position of the playhead indicator 410 and changing the frame being viewed in the media composition.

[0097] Figures 5A to 5D An example user interface is shown, illustrating the hover user input for modifying content. Figure 5A User interface 500 is shown, which includes various display areas, including a main display area 202, a timeline 204, playback controls 206, a time indicator 208, view options 210, a media clip library 212, and media clip details 214. Although these display areas are shown, any number and type of display areas may be included in various specific implementations. In this example, the main display area 202 is playing a media composition, where the time is... Figure 5A The time was set to 28:00, and the playback controls were illuminated.

[0098] exist Figure 5B In the user interface 502, the user has provided a hover input 504 above a graphical representation of the second frame in the media composition (media clip A 402 shown in the media clip library 212). This hover input 504 has been initiated at 29:15 of the playback of the media composition, as indicated by the time indicator 208.

[0099] Figure 5C User interface 506 is shown, in which the second frame is displayed in the main display area 202 due to hover input 504 above a graphical representation of the second frame in the media composition. In one embodiment, time indicator 208 can indicate the second frame in the media composition by displaying 4:30 and the corresponding positioning of playhead indicator 410 along timeline 204. In an alternative embodiment, one or both of playhead indicator 410 and time indicator 208 can continue to display the previous positioning and time corresponding to the first frame of the media composition, respectively.

[0100] As long as the user keeps the hover input 504 above the graphic representation of the second frame in the media composition, the main display area 202 will continue to display the second frame. Furthermore, in some methods, moving the hover input 504 above a different location in the media composition will cause the display of the frame corresponding to that different location to replace the content shown in the main display area 202.

[0101] Figure 5D The user interface 508 is shown after the hover input 504 has ended. Because the hover input 504 has ended, the main display area resumes playback of the media composition after the first frame, indicated by the 29:16 mark on the time indicator 208 and the positioning of the playhead indicator 410 near its position before receiving the hover input 504. This allows the user to preview one or more frames of the media composition, or media clips available for the media composition, without actually adjusting the playhead indicator 410 and losing the previous playback positioning.

[0102] Example process To enable readers to gain a clear understanding of the technical concepts described herein, the following process describes specific steps performed in a particular order. However, one or more steps of a particular process may be rearranged and / or omitted while remaining within the intended scope of the technology disclosed herein. Furthermore, different processes and / or their steps may be combined, recombined, rearranged, omitted, and / or performed in parallel to create different processing flows also within the intended scope of the technology disclosed herein. Additionally, while for clarity, some details of the technology disclosed herein may be omitted or briefly summarized in the following processes, the details described in the preceding paragraphs can be combined with the process steps described below to obtain a more complete and comprehensive understanding of these processes and the technology disclosed herein.

[0103] Figure 6 This is a flowchart of an example process 600 for modifying content based on hover user input in one or more embodiments. In various approaches, process 600 may include more or fewer operations than those shown and described herein. Figure 6 The remainder of the description states that process 600 will be described as being executed by a computing device having at least one hardware processor for performing various operations.

[0104] In operation 602, the computing device displays a first video frame comprising media, the first video frame corresponding to a first position of a movable playback head along a timeline. The first video frame is displayed in a first display area of ​​the GUI, while the timeline is concurrently displayed in a second display area of ​​the GUI.

[0105] In one approach, a first video frame can be displayed during playback of the media composition. In this approach, while the media composition is being played back, a specific video frame of the media composition is displayed based on the positioning of the movable playhead along the timeline at any given moment.

[0106] In operation 604, the computing device detects the initiation of a hover user input associated with a second location along the timeline. The hover user input can be of any type that can be detected and associated with the second location along the timeline. For example, the hover user input can be a hover user input positioned above the second location along the timeline, or a hover user input positioned above a representation of a second video frame of the media composition corresponding to the second location along the timeline. This representation can be a thumbnail, icon, or some other indicator representing the second location within the media composition.

[0107] In one implementation, the initiation of hovering user input associated with a second location along the timeline can be detected based on the fact that the location of the user input location has not changed within a threshold time period (e.g., hovering over the location for a period of time without touching the touchscreen display).

[0108] In one approach, the initiation of hovering user input associated with a second positioning along a timeline can be detected based on the user's finger or touch input device being less than a threshold distance from the touchscreen display without touching it.

[0109] In operation 606, in response to the detection of a hovering user input, the computing device replaces the display of the first video frame with the display of a second video frame composed of media in the first display area of ​​the graphical user interface.

[0110] In one implementation, the computing device can detect the initiation of hovering user input associated with a second positioning along the timeline, based on the user hovering over a specific media clip from the media composition to select a specific media clip for preview in a first display area of ​​the GUI.

[0111] In one method, a second video frame can be displayed during playback of a portion of the media composition that begins with a second video frame. In this method, playback of the portion of the media composition that begins with the second video frame is initiated in response to detecting hover user input associated with a second positioning along the timeline.

[0112] In operation 608, in response to the detection of the termination of the hovering user input, the computing device resumes the display of the first video frame, which is composed of media, in the first display area of ​​the graphical user interface.

[0113] In one method, the computing device can restore the display of the first video frame by replaying the portion of the media composition after the first video frame and continuing to play the media composition backward from that point.

[0114] In one or more embodiments, the computing device can detect the initiation of a subsequent hover user input associated with a third video frame. The subsequent hover user input can be of any type indicating the third video frame. For example, the subsequent hover user input can be located above a third position along the timeline associated with the third video frame, above a representation of a media clip corresponding to the third video frame, etc. In response to detecting the initiation of the subsequent hover user input, the computing device will replace the display of the second video frame with the display of the third video frame in a first display area of ​​the GUI.

[0115] In one method, the representation of the second video frame is derived from a media clip displayed in a third display area of ​​the GUI. In this method, the media composition includes at least a portion of the media clip, and the user hovers over the representation of the media clip to select a frame in the media composition corresponding to the media clip.

[0116] Figure 7 This is a flowchart of an example process 700 for modifying a user interface layout based on touch input in one or more embodiments. In various approaches, process 700 may include more or fewer operations than those shown and described herein. Figure 7 The remainder of the description states that process 700 will be described as being executed by a computing device having at least one hardware processor for performing various operations.

[0117] In operation 702, the computing device displays a GUI comprising multiple user-adjustable display areas. Each user-adjustable display area is associated with a corresponding size including height and width dimensions.

[0118] In operation 704, the computing device receives a single touch input that adjusts a first size of a first user-adjustable display area among a plurality of user-adjustable display areas.

[0119] In one approach, a single touch input may include clicking and dragging user input, which moves the edge of a first user-adjustable display area to increase or decrease the size of the display area.

[0120] According to another approach, a single touch input can be a swipe input. In one embodiment, a swipe input may begin on a portion of a specific user-reducible display area and end at the edge of the GUI or outside the area of ​​the GUI or touchscreen display. The swipe input can instruct the specific user-reducible display area to be minimized, reduced, removed, or turned off in various ways. In another embodiment, a swipe input may begin at the edge of the GUI or outside the area of ​​the GUI or touchscreen display and end at a location within the GUI. The swipe input can instruct the specific user-reducible display area to be turned on, displayed, maximized, or expanded in various ways.

[0121] Operations 706 and 708 are performed in response to receiving a single touch input. In operation 706, the computing device adjusts a first size of a first user-adjustable display area.

[0122] In operation 708, the computing device adjusts both the height and width dimensions of the second user-adjustable display area.

[0123] In one implementation, multiple user-adjustable display areas completely cover a specific area of ​​the graphical user interface. In this implementation, the computing device calculates adjustments to the height and width dimensions of a second user-adjustable display area such that, after adjustment, the specific area of ​​the graphical user interface remains completely covered by the multiple user-adjustable display areas.

[0124] In one or more methods, the computing device can also modify the layout of multiple user-adjustable display areas in response to receiving a single touch input.

[0125] In one approach, the computing device may swap the horizontal positioning of a second user-adjustable display area with respect to a third user-adjustable display area. In this approach, the vertical dimensions of the second user-adjustable display area may differ from the vertical dimensions of the third user-adjustable display area. This difference in vertical dimensions may be a factor in determining whether to modify the layout.

[0126] In another approach, the computing device may swap the vertical positioning of the second user-adjustable display area with respect to the third user-adjustable display area. In this method, the horizontal dimensions of the second user-adjustable display area may differ from the horizontal dimensions of the third user-adjustable display area. This difference in horizontal dimensions may be a factor in determining whether to modify the layout.

[0127] According to one embodiment, the computing device can increase the size of the second user-adjustable display area by adjusting both its height and width. In another embodiment, adjusting both the height and width of the second user-adjustable display area can decrease its size. In yet another embodiment, adjusting both the height and width of the second user-adjustable display area can remove it from the display on the GUI, essentially making these dimensions zero.

[0128] In another implementation, the computing device may adjust the height, width, or both of the adjustable display area for a third user in response to receiving a single touch input.

[0129] In one or more embodiments, in response to receiving a single touch input, the computing device can modify the layout of a set of user-adjustable display areas from row-based to column-based, or from column-based to row-based. In some methods, portions of the GUI can be changed from row-based to column-based, or from column-based to row-based, to achieve a more efficient display of information included in multiple user-adjustable display areas.

[0130] In one approach, one or more display areas of the GUI may not be user-resizeable. In this approach, the computing device calculates layout changes for displaying user-resizeable display areas that avoid altering the positioning and size of any non-resizeable display areas displayed in the GUI.

[0131] Graphical User Interface This disclosure describes above various graphical user interfaces (GUIs) for implementing various features, processes, or workflows. These GUIs can be presented on a variety of electronic devices, including but not limited to laptops, desktop computers, computer terminals, television systems, tablets, e-book readers, and smartphones. One or more of these electronic devices may include a touch-sensitive surface. The touch-sensitive surface can process multiple simultaneous input points, including processing data related to pressure, degree, or location for each input point. Such processing can facilitate gestures using multiple fingers, including pinching and swiping.

[0132] When this disclosure refers to “selecting” a user interface element in a GUI, these terms are understood to include clicking or “hovering” over a user interface element using a mouse or other input device, or touching, tapping, or gesturing over a user interface element using one or more fingers or a stylus. User interface elements can be virtual buttons, menus, selectors, switches, sliders, brushes, knobs, thumbnails, links, icons, radio buttons, checkboxes, and any other mechanism used to receive input from or provide feedback to the user.

[0133] User-adjustable display area In various implementation schemes, touch input, as described below, can be used to manipulate the user-adjustable display area.

[0134] 1. A non-transitory computer-readable medium, the non-transitory computer-readable medium comprising one or more instruction sequences, the one or more instruction sequences causing the one or more processors to perform an operation when executed by one or more processors, the operation comprising: The display includes a graphical user interface comprising multiple user-adjustable display areas, each of which is associated with a corresponding size including a height dimension and a width dimension. Receive a single touch input that adjusts a first size of a first user-adjustable display area within the plurality of user-adjustable display areas; In response to receiving the single touch input: Adjust the first size of the first user-adjustable display area; and The second user can adjust both the height and width of the display area.

[0135] 2. The non-transitory computer-readable medium of claim 1, wherein the plurality of user-adjustable display areas completely cover a specific area of ​​the graphical user interface.

[0136] 3. The non-transitory computer-readable medium according to claim 2, wherein the operation further comprises: Calculate the adjustment of the height and width dimensions of the second user-adjustable display area such that, after the adjustment, the specific area of ​​the graphical user interface remains fully covered by the plurality of user-adjustable display areas.

[0137] 4. The non-transitory computer-readable medium of claim 1, wherein the operation further comprises: In further response to receiving the single touch input, the layout of the plurality of user-adjustable display areas is modified to swap the horizontal positioning of the second user-adjustable display area with respect to the third user-adjustable display area.

[0138] 5. The non-transitory computer-readable medium of claim 4, wherein the vertical dimension of the second user-adjustable display area is different from the vertical dimension of the third user-adjustable display area, and wherein the layout is modified based on the vertical dimensions of the second user-adjustable display area and the vertical dimensions of the third user-adjustable display area.

[0139] 6. The non-transitory computer-readable medium of claim 1, wherein the operation further comprises: In further response to receiving the single touch input, the layout of the plurality of user-adjustable display areas is modified to swap the vertical positioning of the second user-adjustable display area with respect to the third user-adjustable display area.

[0140] 7. The non-transitory computer-readable medium of claim 6, wherein the horizontal size of the second user-adjustable display area is different from the horizontal size of the third user-adjustable display area, and wherein the layout is modified based on the horizontal size of the second user-adjustable display area and the horizontal size of the third user-adjustable display area.

[0141] 8. The non-transitory computer-readable medium of claim 1, wherein adjusting both the height and width dimensions of the second user-adjustable display area increases the size of the second user-adjustable display area.

[0142] 9. The non-transitory computer-readable medium of claim 1, wherein adjusting both the height dimension and the width dimension of the second user-adjustable display area reduces the size of the second user-adjustable display area.

[0143] 10. The non-transitory computer-readable medium of claim 1, wherein the single touch input comprises clicking and dragging user input by moving the edge of the first user-adjustable display area.

[0144] 11. The non-transitory computer-readable medium of claim 1, wherein adjusting both the height and width dimensions of the second user-adjustable display area removes the second user-adjustable display area from the graphical user interface.

[0145] 12. The non-transitory computer-readable medium of claim 1, wherein the operation further comprises: In response to receiving the single touch input, at least one of the following is adjusted for the third user-adjustable display area: height and width.

[0146] 13. The non-transitory computer-readable medium of claim 1, wherein the operation further comprises: In response to receiving the single touch input, the layout of the plurality of user-adjustable display areas is changed from row-based to column-based.

[0147] 14. The non-transitory computer-readable medium of claim 1, wherein the operation further comprises: In response to receiving the single touch input, the layout of the plurality of user-adjustable display areas is changed from column-based to row-based.

[0148] 15. A system comprising: One or more processors; and A non-transitory computer-readable medium, the non-transitory computer-readable medium comprising one or more instruction sequences, the one or more instruction sequences causing the one or more processors to perform operations when executed by the one or more processors, the operations including: The display includes a graphical user interface comprising multiple user-adjustable display areas, each of which is associated with a corresponding size including a height dimension and a width dimension. Receive a single touch input that adjusts a first size of a first user-adjustable display area within the plurality of user-adjustable display areas; In response to receiving the single touch input: Adjust the first size of the first user-adjustable display area; and The second user can adjust both the height and width of the display area.

[0149] 16. A method comprising: The display includes a graphical user interface comprising multiple user-adjustable display areas, each of which is associated with a corresponding size including a height dimension and a width dimension. Receive a single touch input that adjusts a first size of a first user-adjustable display area within the plurality of user-adjustable display areas; In response to receiving the single touch input: Adjust the first size of the first user-adjustable display area; and The second user can adjust both the height and width of the display area.

[0150] privacy As described above, one aspect of this technology is the collection and use of data available from various sources to improve touch input analysis and display modification services. This disclosure anticipates that, in some instances, the collected data may include personal information that uniquely identifies or can be used to contact or locate specific individuals. Such personal information may include demographic data, location-based data, telephone numbers, email addresses, Twitter IDs, home addresses, data or records related to a user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, or any other identifying information or personal information.

[0151] This disclosure recognizes that the use of such personal information data in the techniques of this invention can be used to benefit users. For example, personal information data can be used for touch input analysis and display modification based on user preferences collected from the personal information data. Therefore, the use of such personal information data enables users to control how touch input analysis and display modification services alter the user experience. Furthermore, this disclosure also anticipates other uses of personal information data that benefit users. For example, health and fitness data can be used to provide insights into a user's overall health status or can be used as positive feedback for individuals using the technology to pursue health goals.

[0152] This disclosure anticipates that entities responsible for the collection, analysis, disclosure, transmission, storage, or other use of such personal information data will comply with robust privacy policies and / or privacy measures. Specifically, such entities should implement and maintain privacy policies and measures that are recognized as meeting or exceeding industry or governmental requirements for maintaining the privacy and security of personal information data. Such policies should be easily accessible to users and should be updated as the collection and / or use of data changes. Personal information from users should be collected for legitimate and reasonable entity purposes and should not be shared or sold outside of these legitimate purposes. Furthermore, such collection / sharing should be conducted only after receiving informed consent from users. Additionally, such entities should consider taking any necessary steps to protect and safeguard the right to access such personal information data and ensure that other entities with access to personal information data comply with the privacy policies and procedures of those other entities. Furthermore, such entities may subject themselves to third-party assessments to demonstrate their compliance with widely accepted privacy policies and privacy measures. Moreover, policies and measures should be adapted to the specific types of personal information data collected and / or accessed, and to applicable laws and standards, including considerations of specific jurisdictions. For example, in the United States, the collection or access to certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA); while in other countries, health data may be subject to other regulations and policies and should be handled accordingly. Therefore, different privacy measures should be advocated for different types of personal data in each country.

[0153] Regardless of the foregoing, this disclosure also anticipates implementation schemes that allow users to selectively block the use or access to personal information data. That is, this disclosure anticipates providing hardware and / or software components to prevent or block access to such personal information data. For example, with respect to touch input analysis and display modification services, the technology of this invention can be configured to allow users to opt-in or opt-out to participate in the collection of personal information data at any time during or after service registration. In another example, users can choose not to provide emotion-related data for touch input analysis and display modification services. In yet another example, users can choose to limit the length of time emotion-related data is retained, or completely prohibit the development of underlying emotional states. In addition to providing "opt-in" and "opt-out" options, this disclosure also anticipates providing notifications related to access to or use of personal information. For example, users can be notified when downloading an application that their personal information data will be accessed, and then reminded again just before the application accesses the personal information data.

[0154] Furthermore, the intent of this disclosure is that personal information data should be managed and processed in a manner that minimizes the risk of unintentional or unauthorized access or use. Once data is no longer needed, this risk can be minimized by restricting data collection and deleting data. Additionally, and where applicable, including in certain health-related applications, data deidentification can be used to protect user privacy. Deidentification can be facilitated, where appropriate, by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than the address level), controlling how data is stored (e.g., aggregating data among users), and / or other methods.

[0155] Therefore, while this disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, it is also contemplated that various embodiments can be implemented without access to such personal information data. That is, various embodiments of the present invention will not become inoperable due to the absence of all or part of such personal information data. For example, touch input analysis and display modification can be based on non-personal information data or an absolute minimum of personal information, such as content requested by a device associated with a user, other non-personal information available to the touch input analysis and display modification service, or publicly available information.

[0156] Example System Architecture Figure 8 It is possible. Figures 1 to 7The block diagram of a computing device 800 illustrates its features and processes. The computing device 800 may include a memory interface 802, one or more data processors, a graphics processor and / or a central processing unit 804, and a peripheral device interface 806. The memory interface 802, the one or more processors 804, and / or the peripheral device interface 806 may be separate components or may be integrated into one or more integrated circuits. The various components in the computing device 800 may be coupled via one or more communication buses or signal lines.

[0157] Sensors, devices, and subsystems can be coupled to peripheral interface 806 to facilitate multiple functionalities. For example, motion sensor 810, light sensor 812, and proximity sensor 814 can be coupled to peripheral interface 806 to facilitate orientation, illumination, and proximity functions. Other sensors 816 can also be connected to peripheral interface 806, such as Global Navigation Satellite System (GNSS) (e.g., GPS receiver), temperature sensors, biometric sensors, magnetometers, or other sensing devices to facilitate related functionalities.

[0158] The camera subsystem 820 and optical sensor 822 (e.g., an electro-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) optical sensor) can be used to facilitate camera functions such as recording photos and video clips. The camera subsystem 820 and optical sensor 822 can also be used to collect images of the user to be used during user authentication (e.g., by performing facial recognition analysis).

[0159] Communication functionality can be facilitated by one or more wireless communication subsystems 824, which may include radio frequency receivers and transmitters and / or optical (e.g., infrared) receivers and transmitters. The specific design and implementation of the communication subsystem 824 may depend on the communication network through which the computing device 800 intends to operate. For example, the computing device 800 may include subsystems designed to operate via GSM networks, GPRS networks, EDGE networks, Wi-Fi or WiMax networks, and Bluetooth. ™ A network-operated communication subsystem 824. Specifically, the wireless communication subsystem 824 may include a managed protocol that allows device 800 to be configured as a base station for other wireless devices.

[0160] The audio subsystem 826 can be coupled to the speaker 828 and microphone 830 to facilitate voice-enabled functions such as speaker recognition, voice copying, digital recording, and telephone functions. For example, the audio subsystem 826 can be configured to facilitate the processing of voice commands, voiceprints, and voice authentication.

[0161] I / O subsystem 840 may include touch surface controller 842 and / or other input controller 844. Touch surface controller 842 may be coupled to touch surface 846. Touch surface 846 and touch surface controller 842 may, for example, use any of a variety of touch-sensitive technologies (including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies) and other proximity sensor arrays or other elements for determining one or more contact points with touch surface 846 to detect contact and movement or interruption.

[0162] Other input controllers 844 may be coupled to other input / control devices 848, such as one or more buttons, rocker switches, thumbwheels, infrared ports, USB ports, and / or pointer devices such as styluses. One or more buttons (not shown) may include volume up / down buttons for speaker 828 and / or microphone 830.

[0163] In one implementation, pressing the button for a first duration unlocks the touch surface 846; and pressing the button for a second duration longer than the first duration turns the computing device 800 on or off. Pressing the button for a third duration activates a voice control or voice command module that allows the user to speak commands into the microphone 830 to have the device execute those commands. The user can customize the functionality of one or more buttons. For example, the touch surface 846 can also be used to implement virtual or soft buttons and / or a keyboard.

[0164] In some implementations, the computing device 800 may display recorded audio and / or video files, such as MP3, AAC, and MPEG files. In some implementations, the computing device 800 may include an MP3 player such as an iPod. ™ Functionality.

[0165] Memory interface 802 can be coupled to memory 850. Memory 850 may include high-speed random access memory and / or non-volatile memory, such as one or more disk storage devices, one or more optical storage devices, and / or flash memory (e.g., NAND, NOR). Memory 850 may store operating system 852, such as Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or embedded operating systems (such as VxWorks).

[0166] Operating system 852 may include instructions for handling basic system services and for performing hardware-related tasks. In some implementations, operating system 852 may be a kernel (e.g., a UNIX kernel). In some implementations, operating system 852 may include instructions for performing touch input analysis and display modification. For example, operating system 852 may implement instructions as described in the reference. Figures 1 to 7 The described touch input analysis and display modification features.

[0167] The memory 850 may also store communication instructions 854 that facilitate communication with one or more additional devices, one or more computers, and / or one or more servers. The memory 850 may include: graphical user interface instructions 856 that facilitate graphical user interface processing; sensor processing instructions 858 that facilitate sensor-related processing and functions; telephone instructions 860 that facilitate telephone-related processes and functions; electronic messaging instructions 862 that facilitate electronic messaging-related processes and functions; web browsing instructions 864 that facilitate web browsing-related processes and functions; media processing instructions 866 that facilitate media processing-related processes and functions; GNSS / navigation instructions 868 that facilitate GNSS and navigation-related processes and instructions; and / or camera instructions 870 that facilitate camera-related processes and functions.

[0168] Memory 850 can store software instructions 872 to facilitate other processes and functions, such as references Figures 1 to 7 The described touch input analysis and display modification process and functions.

[0169] The memory 850 may also store other software instructions 874, such as network video instructions that facilitate network video-related processes and functions; and / or network shopping instructions that facilitate network shopping-related processes and functions. In some specific implementations, the media processing instructions 866 are divided into audio processing instructions and video processing instructions, for use in facilitating audio processing-related processes and functions and video processing-related processes and functions, respectively.

[0170] Each of the instructions and applications identified above may correspond to a set of instructions for performing one or more functions described above. These instructions do not need to be implemented as a separate software program, process, or module. Memory 850 may include additional instructions or fewer instructions. Furthermore, various functions of computing device 800 may be implemented in hardware and / or software, including in one or more signal processing and / or application-specific integrated circuits.

[0171] In order to help the Patent Office and any reader of any patent issued for this application interpret the appended claims, the applicant wishes to note that they do not intend any of the appended claims or claim elements to invoke 35 U.SC 112(f) unless the words “part for…” or “step for…” are expressly used in a particular claim.

Claims

1. A non-transitory computer-readable medium, the non-transitory computer-readable medium comprising one or more instruction sequences, the one or more instruction sequences causing the one or more processors to perform operations when executed by one or more processors, the operations comprising: A first video frame comprising display media, the first video frame corresponding to a first position of a movable playhead along a timeline, the first video frame being displayed in a first display area of ​​a graphical user interface, the timeline being displayed in a second display area of ​​the graphical user interface; Detect the initiation of hover user input associated with a second location along the timeline, wherein the hover user input includes at least one of the following: The first hover user input above the second positioning along the timeline, or A second hover user input above the representation of a second video frame composed of the media, the second video frame corresponding to the second positioning along the timeline; In response to the initiation of the detected hovering user input, the display of the first video frame is replaced by the display of the second video frame composed of the media in the first display area of ​​the graphical user interface; and In response to the detection of the termination of the hovering user input, the display of the first video frame, which is composed of the media, is resumed in the first display area of ​​the graphical user interface.

2. The non-transitory computer-readable medium of claim 1, wherein the first video frame is displayed during playback of the media composition, wherein a particular video frame of the media composition is displayed along the timeline at any given moment according to the positioning of the movable playhead.

3. The non-transitory computer-readable medium of claim 1, wherein the second video frame is displayed during playback of the portion of the media composition beginning with the second video frame, and wherein playback of the portion of the media composition beginning with the second video frame is initiated in response to detecting the hovering user input associated with the second positioning along the timeline.

4. The non-transitory computer-readable medium of claim 1, wherein restoring the display of the first video frame comprises restoring playback of the portion of the media composition following the first video frame.

5. The non-transitory computer-readable medium of claim 1, wherein the initiation of detecting the hovering user input associated with the second location along the timeline comprises selecting a specific media clip from the media composition.

6. The non-transitory computer-readable medium according to claim 1, wherein the operation further comprises: Detecting the initiation of subsequent hover user input associated with the third video frame, wherein the subsequent hover user input includes at least one of the following: The third hover user input associated with the third video frame above the third position along the timeline, or The fourth hover user input above the media clip, which corresponds to the third video frame; as well as In response to the detection of the subsequent hovering user input, the display of the second video frame is replaced by the display of the third video frame in the first display area of ​​the graphical user interface.

7. The non-transitory computer-readable medium of claim 1, wherein the representation of the second video frame is derived from a media clip displayed in a third display area of ​​the graphical user interface, wherein the media composition includes at least a portion of the media clip.

8. The non-transitory computer-readable medium of claim 1, wherein the initiation of the hovering user input associated with the second location along the timeline is detected based on the user input location not changing within a threshold time period.

9. The non-transitory computer-readable medium of claim 1, wherein the initiation of the hovering user input associated with the second positioning along the timeline is detected based on the user's finger or touch input device being less than a threshold distance from the touchscreen display without touching the touchscreen display.

10. A system comprising: One or more processors; and A non-transitory computer-readable medium comprising one or more instruction sequences that, when executed by the one or more processors, cause the one or more processors to perform the operation according to any one of claims 1 to 9.

11. A method comprising the operation according to any one of claims 1 to 9.

12. A system comprising components for performing the operations according to any one of claims 1 to 9.

Images