Rhythm-based content creation

Abstract

This disclosure describes techniques for generating rhythm-based content, the techniques including obtaining a plurality of images including an object having motion within the plurality of images, determining whether at least one portion of the at least one portion of the object is aligned with a target image overlaid on the first image in a first image and whether the at least one portion is aligned with the target image in a time that is proximate to a first rhythmic point of playback of a selected piece of music, segmenting the at least one portion of the object from the first image in accordance with a determination that the at least one portion is aligned with the target image in a time that is proximate to the first rhythmic point of playback of the selected piece of music, and generating a first overlay based on the at least one portion of the object.

Description

[Background technology]

[0001] Increasingly, communication is conducted using Internet-based tools. Internet-based tools can be any software or platform. Existing social media platforms such as YouTube® and Facebook allow users to communicate with each other by sharing information such as images and videos through static applications or web pages. As communication devices such as mobile phones become more and more sophisticated, people continue to seek new ways of entertainment, social networking, and communication. [Brief description of the drawings]

[0002] The following detailed description can be better understood when read in conjunction with the accompanying drawings, in which: For purposes of illustration, there are shown in the accompanying drawings exemplary embodiments of various aspects of the disclosure; however, the invention is not limited to the specific methods and instrumentalities disclosed.

[0003] [Figure 1] FIG. 1 illustrates an exemplary system for distributing content.

[0004] [Diagram 2] FIG. 2 illustrates an exemplary content creation method executable by a user device in accordance with the present disclosure.

[0005] [Diagram 3] FIG. 1 illustrates another exemplary content creation method executable by a user device in accordance with the present disclosure.

[0006] [Figure 4] FIG. 1 illustrates another exemplary content creation method executable by a user device in accordance with the present disclosure.

[0007] [Diagram 5]FIG. 2 illustrates an exemplary user interface of a content delivery application in accordance with the present disclosure.

[0008] [Figure 6] FIG. 2 illustrates another exemplary user interface of a content delivery application in accordance with the present disclosure.

[0009] [Figure 7] FIG. 2 illustrates another exemplary user interface of a content delivery application in accordance with the present disclosure.

[0010] [Figure 8] FIG. 2 illustrates another exemplary user interface of a content delivery application in accordance with the present disclosure.

[0011] [Figure 9] FIG. 2 illustrates another exemplary user interface of a content delivery application in accordance with the present disclosure.

[0012] [Figure 10] FIG. 2 illustrates another exemplary user interface of a content delivery application in accordance with the present disclosure.

[0013] [Figure 11] FIG. 2 illustrates another exemplary user interface of a content delivery application in accordance with the present disclosure.

[0014] [Figure 12] FIG. 1 illustrates an example computing device that can be used to perform any of the methods disclosed herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] 1 illustrates an example system 100 for delivering content. The system 100 may include a cloud network 102 and a number of client devices 104a-d. The cloud network 102 and the number of client devices 104a-d may communicate with each other via one or more networks 120.

[0016] The cloud network 102 may be located in a data center, such as a single building, or may be distributed across different geographic locations (e.g., several buildings). The cloud network 102 may provide services through one or more networks 120. The networks 120 may include various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices, and / or similar devices. The networks 120 may include coaxial cable links, twisted pair cable links, fiber optic links, combinations thereof, and the like. The networks 120 may include wireless links, such as cellular links, satellite links, Wi-Fi links, and the like.

[0017] The cloud network 102 may include multiple computing nodes 118 hosting various services. In one embodiment, the nodes 118 host a short video service 112. The short video service 112 may include a content streaming service, such as an Internet Protocol video streaming service. The short video service 112 may be configured to distribute content via various transmission technologies. The short video service 112 is configured to provide content 116, such as video, audio, text data, combinations thereof, etc. The content 116 may include content streams (e.g., video streams, audio streams, information streams), content files (e.g., video files, audio files, text files), and / or other data. The content 116 may be stored in a database 114. For example, the short video service 112 may include a video sharing service, a video hosting platform, a content distribution platform, a collaborative gaming platform, etc.

[0018] In one embodiment, the content 116 delivered or provided by the short video service 112 includes short videos. A short video may have a duration up to a predetermined time limit, such as one minute, five minutes, or other predetermined minutes. By way of example and not limitation, a short video may include at least one and not more than four 15-second segments linked together. A short video duration may provide viewers with entertainment in rapid succession, allowing users to watch a large amount of video within a short time frame. Such rapid succession entertainment may be popular on social media platforms.

[0019] A Short Video may include a pre-recorded audio overlay, such as music or sounds from a television show or movie. When a Short Video includes a pre-recorded audio overlay, the Short Video may feature one or more people lip-syncing, dancing, or otherwise moving their bodies along with the pre-recorded audio. For example, a Short Video may feature a "dance challenge" that an individual completes to a hit song, or a Short Video may feature two people participating in a lip-sync or two-person dance. As another example, a Short Video may feature an individual completing a challenge that requires the individual to move their body to correspond to the pre-recorded audio overlay, e.g., to the beat or rhythm of a pre-recorded song that is featured by the pre-recorded audio overlay. Other Short Videos may not include a pre-recorded audio override. For example, these Short Videos may feature individuals playing sports, playing pranks, or giving advice, such as beauty and fashion advice, cooking tips, home decor tips, and the like.

[0020] In one embodiment, the content 116 may be output to different client devices 104 via the network 120. The content 116 may be streamed to the client devices 104. The content stream may be a stream of short videos received from the short video service 112. Multiple client devices 104 may be configured to access the content 116 from the short video service 112. In one embodiment, the client devices 104 may include a content application 106. The content application 106 outputs (e.g., displays, renders, presents) the content 116 to a user associated with the client device 104. The content may include video, audio, comments, text data, etc.

[0021] The client devices 104 may include any type of computing device, such as a mobile device, a tablet device, a laptop computer, a desktop computer, a smart television or other smart device (e.g., a smart watch, a smart speaker, smart glasses, a smart helmet), a gaming device, a set-top box, a digital streaming device, a robot, etc. The client devices 104 may be associated with one or more users. A user may access the cloud network 102 using one or more of the client devices 104. The client devices 104 may travel to different locations and access the cloud network 102 using different networks.

[0022] The short video service 112 may be configured to receive input from a user. The user may be registered as a user of the short video service 112 or may be a user of a content application 106 running on the client device 104. The user input may include a short video created by the user, a user comment associated with the short video, or a "like" associated with the short video. The user input may include a connection request and user input data, such as text data, digital image data, or user content. The connection request may include a request from the client device 104a-d to connect to the short video service 112. The user input data may include information that a user connected to the short video service 112 wants to share with other connected users of the short video service 112, such as a short video and / or a user comment.

[0023] The short video service 112 may be able to receive different types of input from users using different types of client devices 104. For example, a user using a content application 106 on a first user device, such as a mobile phone or tablet, may be able to create and upload short videos using the content application 106. A user using the content application 106 on a different mobile phone or tablet may be able to view, comment on, and "like" short videos or comments written by other users. In another example, a user using the content application 106 on a smart TV, laptop, desktop, or gaming device may not be able to create and upload short videos or comment on short videos using the content application 106. Instead, a user using the content application 106 on a smart TV, laptop, desktop, or gaming device may only be able to view short videos, view comments left by other users, and "like" short videos using the content application 106.

[0024] In one embodiment, a user can use a content application 106 on a client device 104 to create and upload a short video to the cloud network 102. The client device 104 can access an interface 108 of the content application 106. The interface 108 can include input elements. For example, the input elements may be configured to allow a user to create a short video. To create a short video, a user may give the content application 106 permission to access an image capture device, such as a camera or a microphone, of the client device 104. Using the content application 106, a user can select the duration of the short video or set the speed of the short video, for example, "slow motion" or "speed up."

[0025] A user can edit a short video using the content application 106. A user can add one or more text, filters, sounds, or effects, such as beauty effects, to a short video. To add a pre-recorded audio overlay to a short video, a user can select a song or sound clip from a sound library of the content application 106. The sound library may include different songs, sound effects, or audio clips from movies, albums, and television shows. In addition to or instead of adding a pre-recorded audio overlay to a short video, a user can add a narration to a short video using the content application 106. The narration may be a sound recorded by a user using a microphone of the client device 104. A user can add a text overlay to a short video and can specify when they want the text overlay to appear in the short video using the content application 106. A user can assign a caption, a location tag, and one or more hashtags to a short video to indicate the subject of the short video. The content application 106 may prompt the user to select a frame of the short video to use as a "cover image" for the short video.

[0026] After a user creates a short video, the user can use the content application 106 to upload the short video to the cloud network 102 and / or store the short video locally on the user device 104. When a user uploads a short video to the cloud network 102, the user can select whether the short video is viewable by all other users of the content application 106 or only by a subset of users of the content application 106. The short video service 112 can store the uploaded short videos and any metadata associated with the short videos in one or more databases 114.

[0027] In one embodiment, a user can provide input on a short video using a content application 106 on a client device 104. The client device 104 can access an interface 108 of the content application 106 that allows the user to provide input associated with the short video. The interface 106 can include an input element. For example, the input element can be configured to receive input from a user, such as a comment or a "like" associated with a particular short video. If the input is a comment, the content application 106 can allow the user to set an emoji to be associated with their input. The content application 106 can determine time information about the input, such as when the user wrote the comment. The content application 106 can transmit the input and associated metadata to the cloud network 102. For example, the content application 106 can transmit the comment, an identifier of the user who wrote the comment, and time information about the comment to the cloud network 102. The short video service 112 can store the input and associated metadata in a database 114.

[0028] The short video service 106 may be configured to output the uploaded short videos and user inputs to other users. A user may be a user of a content application 106 running on a client device 104. The content application 106 may output (display, render, present) the short videos and user comments to a user associated with the client device 104. The client device 104 may access an interface 108 of the content application 106. The interface 108 may include an output element. The output element may be configured to display information about different short videos so that a user can select and view the short videos. For example, the output element may be configured to display multiple cover images, captions, or hashtags associated with the short videos. The output element may also be configured to arrange the short videos according to a category associated with each short video.

[0029] In one embodiment, user comments associated with the short video can be output to other users viewing the same short video. For example, all users accessing the short video can see the comments associated with the short video. The short video service 112 can output the short video and the comments associated with it simultaneously. The comments can be output by the short video service 112 in real-time or near real-time. The content application 106 can display the short video and the comments on the client device 104 in various ways. For example, the comments can be displayed in an overlay on top of the content or next to the content. As another example, a user who wants to see other users' comments associated with the short video may need to select a button to view the comments. The comments can be animated when displayed. For example, the comments can be scrolled across the short video or across an overlay.

[0030] The computing nodes 118 may process tasks associated with the short video service 112. The computing nodes 118 may be implemented as one or more computing devices, one or more processors, one or more virtual computing instances, combinations thereof, and the like. The computing nodes 118 may be implemented by one or more computing devices. The computing devices may include virtualized computing instances. The virtualized computing instances may include virtual machines, such as emulations of computer systems, operating systems, servers, and the like. The virtual machines may be loaded by the computing devices based on a virtual image and / or other data that defines the specific software for the emulation (e.g., operating system, dedicated application, server). As the demand for different types of processing services changes, different virtual machines may be loaded and / or terminated on one or more computing devices. To manage the use of different virtual machines on the same computing device, a hypervisor may be implemented.

[0031] As described above, a user may use a content application 106 on a client device 104 to create and upload short videos to the cloud network 102. In one embodiment, a short video created by a user via a content application 106 on a client device 104 may be a short video featuring a user moving body parts in accordance with the rhythm or beat of a pre-recorded audio overlay (e.g., a pre-recorded audio overlay as described above). For example, a short video created by a user may be a short video featuring a user moving their body parts (e.g., the user's head and / or eyes) to correspond to the rhythm or beat of a song or song clip. As another example, a short video created by a user via a content application 106 on a client device 104 may be a short video featuring a user moving different user body parts (e.g., different users' heads and / or eyes) to correspond to the rhythm or beat of a song or song clip. In another example, a short video includes an object including at least one part having movement within multiple images of the short video.

[0032] The client device 104 can access an interface 108 of the content application 106. The interface 108 can include input elements. By way of example and not limitation, the input elements can be configured to allow a user to create a short video featuring an object moving one of its parts (e.g., the user or another user moving a body part) to the rhythm or beat of a pre-recorded audio overlay. For example, to create a short video featuring a user moving a body part to the rhythm or beat of a pre-recorded audio overlay, the user can give the content application 106 permission to access an image capture device such as a camera or microphone of the client device 104.

[0033] The client device 104 may use the feed captured by the image capture device to detect (e.g., locate) one or more parts of an object, such as one or more body parts of a user. For example, the client device 104 may use the feed captured by the image capture device to detect the user's head or face. The one or more body parts detected by the camera may include a body part that the user moves to the rhythm or beat of a pre-recorded audio overlay while creating a short video. For example, if the user moves his / her head / face to the rhythm or beat of a pre-recorded audio overlay while creating a short video, the client device 104 may use the feed captured by the image capture device to detect the user's head or face.

[0034] In one embodiment, at least one of the short video service 112 or the client device 104 includes an object tracking and detection model 110. The object tracking and detection model 110 may be used at least in part to detect (e.g., locate) one or more parts of an object (e.g., a body part of a user) in a feed captured by an image capture device. The object tracking and detection model 110 may be used, for example, to detect a body part that a user moves to the rhythm or beat of a pre-recorded audio overlay while creating a short video. For example, if a user moves his / her head / face to the rhythm or beat of a pre-recorded audio overlay while creating a short video, the object tracking and detection model 110 may be used to detect the user's head, face, and / or eyes in the feed captured by an image capture device.

[0035] The object tracking and detection model 110 may be configured to automatically extract information from an image, such as a photo or a poster, a scene, such as a real-world environment, or a video stream. In one embodiment, the object tracking and detection model 110 may use positive and negative training applied by machine learning algorithms or computer vision algorithms to extract information, such as labels or identifying attributes, from an image, scene, or video stream. Once extracted, the information may be used to detect at least one object, feature, or element of interest (e.g., one or more body parts) present in the image, scene, or video stream. The object tracking and detection model 110 may be configured to identify the object, feature, or element of interest (e.g., one or more body parts) detected in the image or scene. For example, the object tracking and detection model 110 may be configured to identify the detected body part as a specific body part, such as a head, a face, an eye, and / or any other specific body part.

[0036] In one embodiment, to identify an object, feature, or element of interest (e.g., one or more body parts) present in an image, scene, or video stream, the object tracking and detection model 110 compares the detected object, feature, or element of interest to information stored in a database, such as database 114. The information stored in the database may include information about the imaged and characterized object, feature, or element of interest (e.g., body part). If the detected object, feature, or element of interest matches an object, feature, or element of interest stored in the database, the object tracking and detection model 110 identifies the detected object, feature, or element of interest as the object, feature, or element of interest. For example, if the object tracking model detects a body part in a feed captured by an image capture device, the object tracking model may compare the body part to information stored in the database to determine what type of body part the detected body part is. For example, the body part may be compared to information stored in the database to determine that the body part is a head, a face, and / or any other particular body part.

[0037] In an embodiment, the information extracted by the object tracking and detection model 110 includes one or more measurements associated with an object, feature or element of interest (e.g., one or more body parts). For example, the one or more measurements may include information regarding the size of the body part, the position of the body part relative to other objects and / or body parts in the scene, or geometric or topological information. In one example, the object tracking and detection model 110 may match pixels in a feed obtained from a user's image capture device (e.g., a camera) with a set of points associated with a stored model of the body part. The object tracking and detection model 110 may be configured to determine a number of corresponding lines associated with the body part. For example, the set of points may be a series of connected line segments or links, e.g., a polyline, generated by the object tracking and detection model 110 and fitted to the set of pixels. The object tracking and detection model 110 may also determine the inclination angle of each link in the polyline. In short, to identify an object, it is known to identify lines in one or more graphic images or camera feeds with lines in a model associated with the object. Other techniques may be used to identify objects, such as body parts, in an image or camera feed.

[0038] The content application 106 may enhance graphics, video, sound, and / or other inputs on detected objects, such as detected body parts. For example, the content application 106 may place digital elements, such as one or more image overlays, on detected body parts in the camera feed. For example, if a head / face and eyes are detected in the camera feed via the object tracking and detection model 110, the content application 106 may place an image overlay on the user's eyes. The image overlay may include, for example, two circles covering at least one portion of each of the user's eyes. The image overlay may be any other shape and / or any color or size. As another example, if a different body part (e.g., a body part other than the head / face and eyes) is detected in the camera feed, the content application 106 may place an image overlay on the different body part such that at least one portion of the different body part is covered by the image.

[0039] An image overlay may move with the body part on which it is placed. For example, an image overlay over a user's eyes may move with the user's head / face or eyes. For example, when a user moves their head / face to the right (thus moving their eyes to the right), the image overlay over the user's eyes may also move to the right, so that the overlay remains over the user's eyes. Similarly, when a user moves their head / face to the left (thus moving their eyes to the left), the image overlay over the user's eyes may also move to the left, so that the overlay remains over the user's eyes. The image overlay over the user's eyes may move in any other direction such that the overlay remains over the user's eyes regardless of how or where the user moves their head / face.

[0040] In one embodiment, the content application 106 may place different or additional image overlays on the camera feed. For example, the content application 106 may place one or more target images on the camera feed. The target images may be overlaid at various locations on the camera feed. The various locations may be selected randomly and / or the locations may be selected according to an algorithm. The target images may be of any size, shape, or color. For example, the target images may be a smiley face, one or more circles, and / or any other image. The content application 106 may place one target image at a time on the camera feed and / or the content application 106 may place multiple targets at a time on the camera feed.

[0041] The image overlay may remain on the detected body part for at least one portion of the short video created by the user. During the creation of the short video, the user may attempt to align the image overlay with the one or more target images. The user may attempt to align the image overlay with one or more targets at a predefined time and / or at a time corresponding to the rhythm or beat of a pre-recorded audio overlay (e.g., a song clip). For example, the user may attempt to align the image overlay over his / her eyes with the target image in time with the beat of the song. The target image may be a smile with two "eyes" and the image overlay over his / her eyes may include a circle (or any other shape) covering at least one portion of each eye. The user may attempt to align the circle covering each of his / her eyes with the "eyes" of the smile in time with the beat of the song. Alternatively, any other body part overlay and / or target image may be used.

[0042] In one embodiment, the user may attempt to align the body part with the one or more target images (rather than the image overlay). The user may attempt to align the body part with the one or more targets at a predefined time and / or at a time corresponding to the rhythm or beat of a pre-recorded audio overlay (e.g., a song clip). For example, the user may attempt to align his / her eyes with the target image in time with the beat of the song. The target image may be a smiling face with two "eyes". The user may attempt to align his / her eyes with the "eyes" of the smiling face in time with the beat of the song. Alternatively, any other target image may be used.

[0043] In one embodiment, the content application 106 may utilize the object tracking and detection model 110 to determine whether a body part, such as the user's eyes, is aligned and / or approximately aligned with a target image during creation of the short video (e.g., during recording of the short video). For example, the object tracking and detection model 110 may be used to determine whether an image overlay over a body part (e.g., an overlay over the eyes) is aligned and / or approximately aligned with a target image (e.g., the "eyes" of a smile target). An image overlay over a body part (e.g., an overlay over the eyes) may be aligned with a target image when the image overlay over the body part and the target image completely overlap (e.g., are in the same position on the camera feed). An image overlay over a body part (e.g., an overlay over the user's eyes) may be approximately aligned with a target image when the image overlay over the body part and the target image partially or nearly completely overlap (e.g., within a predetermined distance threshold).

[0044] In one embodiment, the short video service 112 or at least one of the client devices 104a-d includes a beat tracking model 111. The content application 106 may utilize the beat tracking model 111 to determine whether one or more body parts are approximately aligned in time with one or more target images near the beat of a selected pre-recorded audio overlay (e.g., selected music). For example, the music may be selected by a user and / or the music may be selected randomly. The music may have a known beat and / or rhythm.

[0045] In one embodiment, the beat tracking model 111 may keep track of timestamps indicating one or more times corresponding to the beat and / or beats of the selected music. The beat tracking model 111 may also keep track of and / or generate timestamps indicating one or more times when the body part is in sync or approximately in sync with the target image. The beat tracking model 111 may compare the timestamps indicating the times when the body part is in sync or approximately in sync with the target image to timestamps corresponding to the beat and / or rhythm of the selected music to determine whether the body part is in sync or approximately in sync with the target image at a time that is close to the beat of the selected music.

[0046] The content application 106 may place a feedback image overlay during creation of the short video (e.g., during recording of the short video). The feedback image overlay may indicate whether the body part (e.g., the user's eyes) is aligned and / or approximately aligned in time with the target image near the beat and / or rhythm of the selected music. For example, the feedback image overlay may indicate whether the beat tracking model 111 has determined that the body part is aligned or approximately aligned in time with the target image near the beat of the selected music. The feedback image overlay may indicate to the user whether the body part is aligned with the target image on the beat of the selected music, approximately aligned with the target image on the beat of the selected music (e.g., on the beat or within a predetermined time of the beat), or not aligned with the target image on the beat of the selected music (e.g., not on the beat or not within a predetermined time of the beat).

[0047] The feedback image overlay may be different depending on whether the body part is in sync with the target image, approximately in sync with the beat, or not in sync with the beat. For example, if the body part is in sync with or approximately in sync with the target image to the beat of the selected music, a first type of feedback image overlay may be used. The first type of feedback image may be, for example, one or more heart shapes. If the body part is in sync with or approximately in sync with the target image to the beat of the selected music, a second type of feedback image overlay may be used. The second type of feedback image may be, for example, a green or blue circle. If the body part is not in sync with or approximately in sync with the target image to the beat, a third type of feedback image overlay may be used. The third type of feedback image may be, for example, an x ​​mark or a cross. Any other type of feedback image (e.g., any size, color, and / or shape) may be used. One or more of the first, second, or third feedback images may be the same. For example, the second and third types of feedback images may be the same.

[0048] In one embodiment, each of the different types of feedback images may be overlaid on the target image. For example, if the pair of eyes and / or the image overlay on the pair of eyes are aligned or approximately aligned with the "eyes" of the smile target image on beat, a first type of feedback image (e.g., a pair of hearts) may appear over the "eyes" of the smile target image. As another example, if the pair of eyes or the image overlay on the pair of eyes are aligned or approximately aligned with the "eyes" of the smile target image on beat, a second type of feedback image (e.g., a blue or green circle) may appear over the "eyes" of the smile target image. As another example, if the pair of eyes or the image overlay on the pair of eyes are not aligned or approximately aligned with the "eyes" of the smile target image on beat or approximately on beat, a third type of feedback image (e.g., an x ​​mark or cross) may appear over the "eyes" of the smile target image. In another embodiment, each of the different types of feedback images may additionally or alternatively be overlaid on a body part. For example, the feedback image may be overlaid on an image overlay over a body part (eg, an overlay over the eyes).

[0049] In one embodiment, the beat tracking model 111 may keep track of how many times a body part or an image overlay on the body part has been aligned or approximately aligned with a target image to the beat of a song. For example, the beat tracking model 111 may keep track of how many times an image overlay on the body part has been aligned or approximately aligned with a target image to the beat of a song to determine whether the body part has been aligned or approximately aligned with the target image to the beat of a song a predetermined number of times. The predetermined number may be, for example, a goal that the user attempts to achieve while creating the short video (e.g., while recording the short video). For example, while creating the short video, the user may attempt to align or approximately align the body part with the target image to the beat of a song a predetermined number of times, such as, for example, 5, 6, or 7 times (or any number of times).

[0050] In one embodiment, the short video service 112 or at least one of the client devices 104a-d includes a video segmentation model 109. A short video to be created may consist of a sequence of frames. Each frame in the sequence of frames may be an image, such as a 2D image. The video segmentation model 109 may be configured to segment the sequence of frames (i.e., the sequence of images) into multiple regions during creation of a short video to determine information about an object, feature, or element of interest. For example, the video segmentation model 109 may be configured to segment the sequence of frames (i.e., the sequence of images) into multiple regions during creation of a short video to determine information about one or more body parts. The one or more body parts may include body parts detected by the object tracking and detection model 110.

[0051] Information about the one or more body parts may be used to determine a boundary and / or a contour of the one or more body parts. Information about the one or more body parts may be used to distinguish the body parts from other objects in the sequence of frames. For example, the video segmentation model 109 may use information about a head or face to distinguish the head or face from other body parts (e.g., a neck or chest) in the sequence of frames. Once a body part is distinguished, the video segmentation model 109 may separate the body part from the remainder of the frames.

[0052] In one embodiment, the video segmentation model 109 may be configured to capture an image of the body part once the body part is separated from the remainder of the frame. The video segmentation model 109 may be configured to capture an image of the body part at a particular time. For example, the video segmentation model 109 may be configured to capture an image of the body part when the body part is aligned with the target image in time with the beat of the selected music. As another example, the video segmentation model 109 may be configured to capture an image of the body part when the body part is aligned with the target image approximately in time with the beat of the selected music. As another example, the video segmentation model 109 may be configured to capture an image of the body part when the body part is not aligned with the target image on or approximately in time with the beat of the selected music. As another example, the video segmentation model 109 may be configured to capture an image of the body part regardless of whether the body part is aligned with the target image in time with the beat of the selected music (e.g., the video segmentation model 109 may be configured to capture an image of the body part at random times and / or other predefined times). This may continue until the creation of the short video is completed.

[0053] In one embodiment, the content application 106 outputs one or more captured images of the body part to a top bar on the camera feed during creation of the short video (e.g., during recording of the short video). For example, if the video segmentation model 109 is set to capture an image of the body part when the body part aligns with the target image to the beat of the selected music, and the user has previously aligned the body part with the target image five times, five different captured images of the body part may be output to the top bar on the camera feed. Then, if the user aligns the body part with the target image a sixth time, the new, sixth captured image of the body part may be added to the top bar on the camera feed. This may continue until creation of the short video is finished.

[0054] The content application 106 may output animated overlays during creation of the short video (e.g., while the short video is being recorded). The animated overlays may include images of body parts acquired by the video segmentation model 109. For example, the animated overlays may be set to move across the camera feed while the short video is still being recorded. The animated overlays may be set to move across the camera feed in any direction, for example, toward a target image. The animated overlays may move across the camera feed at any speed. One or more animated overlays may move across the camera feed at any time. Multiple animated overlays may move across the camera feed during creation of the short video.

[0055] The animated overlay may be set to move across the camera feed to reach the target image at a predetermined time, e.g., a time corresponding to the beat of the selected music. For example, the beat tracking model 111 may keep track of times (e.g., timestamps) during the selected music that correspond to the beat and / or rhythm of the music. The location of the target image may be known. The timestamps associated with the beats of the music samples in combination with the location of the target image may be used to determine when the animated overlay should begin moving across the camera feed so that the animated overlay reaches the location of the target image at a desired (e.g., in sync with the beat) time.

[0056] In one embodiment, the content application 106 may output an animated overlay towards the end of the creation of the short video. For example, the creation of the short video may end when a predetermined period of time has elapsed. As another example, the creation of the short video may end when the image overlay over the body part has aligned or approximately aligned with the target image to the beat of the song a predetermined number of times. As mentioned above, the predetermined number may be, for example, a goal that the user attempts to achieve while creating the short video (e.g., while recording the short video). For example, while creating the short video, the user may attempt to align or approximately align the body part with the target image to the beat of the song a predetermined number of times, such as, for example, 5, 6, or 7 times (or any number of times). Before ending the creation of the short video, the content application 106 may output an animated overlay. The animated overlay may include an image of the body part obtained by the video segmentation model 109. For example, the animated overlay may be set to move down from the top bar and rotate around the user's head or face.

[0057] The user may know that the video segmentation model 109 is configured to capture one or more images of the body part. As a result, the user may attempt to distort and / or control the body part in a particular way so that the captured image is entertaining or amusing. For example, if the body part is a head or face, the user may attempt to create a humorous face when an image of his or her face is captured by the video segmentation model 109. As a result, any animated overlays based on the captured images output during the creation of the short video may be entertaining and may increase the likelihood that other users who watch the short video will find it entertaining.

[0058] In one embodiment, the animated overlay may include images of body parts belonging to one or more users other than the user creating the short video. For example, the animated overlay may include the head / face of a different user. The different user may be a friend, family member, or a complete stranger. For example, the different user may be a user of the short video service 112 who previously created a short video as described above. If the different user previously created a short video as described above, the body parts of that user may be segmented and stored as an image. This image may be used by one or more subsequent creators of the short video.

[0059] Animated overlays of other users' body parts may be set to move across the camera feed when the user attempts to align the user's body parts with one or more target images (e.g., while the short video is still being recorded). The animated overlays of the other users' body parts may be set to move across the camera feed in any direction, such as toward a target image. The animated overlays of the other users' body parts may move across the camera feed at any speed. The animated overlays of the other users' body parts may move across the camera feed at any time.

[0060] 2 illustrates an example process 200 performed by a user device (e.g., client device 104). Client device 104 may perform process 200 to generate a short video for uploading to a short video service (e.g., short video service 112). Once uploaded to the short video service, the short video may be viewed by users of the short video service. Although illustrated in FIG. 2 as a series of operations, one skilled in the art will appreciate that in various embodiments, the illustrated operations may be added, removed, reordered, or modified.

[0061] As described above, the feed captured by the image capture device may be used to detect (e.g., align) one or more body parts. For example, the client device may utilize the feed captured by the image capture device to detect the user's head or face. The one or more detected body parts may include a body part that moves to the rhythm or beat of a pre-recorded audio overlay by the user while creating the short video.

[0062] At least one portion of an object may be detected at 202. For example, the at least one portion of an object (e.g., a user) may be detected by an object tracking and / or detection model, e.g., object tracking and detection model 110. The object tracking and detection model may be used at least in part to detect (e.g., locate) one or more body parts in the feed captured by the image capture device. The object tracking and detection model may be used, for example, to detect body parts that a user moves to the rhythm or beat of a pre-recorded audio overlay while creating a short video. For example, if a user moves their head / face to the rhythm or beat of a pre-recorded audio overlay while creating a short video, the object tracking and detection model may be used to detect the user's head, face, and / or eyes in the feed captured by the image capture device.

[0063] Also, as described above, the content application 106 may place an image overlay on the camera feed. For example, the content application 106 may place one or more target images on the camera feed. The target images may be overlaid at various locations on the camera feed. The various locations may be selected randomly and / or the locations may be selected according to an algorithm. The target images may be of any size, shape, or color. For example, the target images may be a smiley face, one or more circles, and / or any other image. The content application 106 may place one target image at a time on the camera feed and / or the content application 106 may place multiple targets at a time on the camera feed.

[0064] During creation of the short video, the user may attempt to align the at least one portion with the one or more target images. The user may attempt to align the at least one portion with the one or more target images at a predefined time and / or at a time corresponding to the rhythm or beat of a selected pre-recorded audio overlay (e.g., a song clip). For example, the user may attempt to align his / her eyes with the target image in time with the beat of the song. The target image may be a smiling face with two "eyes". The user may attempt to align his / her eyes with the "eyes" of the smiling face in time with the beat of the song. Alternatively, any other target image may be used.

[0065] The content application 106 may utilize an object tracking and detection model to determine whether the at least one portion is aligned and / or approximately aligned with a target image during creation of the short video (e.g., during recording of the short video). The at least one portion may be tracked at 204. For example, an object tracking and detection model may be used to determine whether a pair of eyes and / or a face is aligned or approximately aligned with a target image (e.g., the "eyes" of a smile target). The at least one portion may be aligned with a target image when the at least one portion and the target image completely overlap. The at least one portion may be approximately aligned with a target image when the at least one portion and the target image partially or nearly completely overlap (e.g., within a predetermined distance threshold). At 206, it may be determined whether the at least one portion or at least one portion of the at least one portion is aligned or approximately aligned with a target image.

[0066] The short video created may consist of a series of frames. Each frame in the series of frames may be an image, such as a 2D image. A video segmentation model, e.g., video segmentation model 109, may be configured during creation of the short video to segment the series of frames (i.e., a series of images) into a number of regions to determine information about the at least one portion. Information about the one or more body parts may be used to determine a boundary and / or a contour of the at least one portion. Information about the at least one portion may be used to distinguish the at least one portion from other objects in the series of frames. For example, the video segmentation model may use information about a head or face to distinguish the head or face from other body parts (e.g., a neck or a chest) in the series of frames. Once the at least one portion is distinguished, the video segmentation model may separate the at least one portion from the rest of the frames.

[0067] The video segmentation model may be configured to acquire an image of the at least one portion when the at least one portion is separated from the remainder of the frame. At 208, the video segmentation model may be used to generate an image of the at least one portion. The video segmentation model may be configured to acquire an image of the at least one portion at a particular time. For example, the video segmentation model may be configured to acquire an image of the at least one portion when the at least one portion is aligned with a target image (regardless of whether the alignment is on or off the beat of the selected music). As another example, the video segmentation model may be configured to acquire an image of the at least one portion only when the at least one portion is aligned with the target image on or approximately on the beat of the selected music. As another example, the video segmentation model may be configured to acquire an image of the at least one portion when the at least one portion is not aligned with the target image on the beat of the selected music. As another example, the video segmentation model may be configured to acquire an image of the at least one portion at a random time and / or other predefined time unrelated to the beat of the selected music.

[0068] The content application 106 may output an animated overlay during creation of the short video (e.g., during recording of the short video). The animated overlay may include an image of the at least one portion obtained by a video segmentation model. For example, the animated overlay may be configured to move across the camera feed while the short video is still being recorded. At 210, an animated image of the at least one portion of the object or at least one portion of the at least one portion of the object may be output. The animated overlay may be configured to move across the camera feed in any direction, such as toward at least one other target image. The animated overlay may move across the camera feed at any speed. One or more animated overlays may move through the camera feed at any time. Multiple animated overlays may move across the camera feed during creation of the short video.

[0069] The animated overlay may be configured to move across the camera feed to reach the at least one other target image at a predetermined time, e.g., a time corresponding to the beat of a selected piece of music. For example, a beat tracking model, e.g., beat tracking model 111, may keep track of a time (e.g., a timestamp) during the selected piece of music that corresponds to the beat and / or rhythm of the music. The location of the at least one other target image may be known. A combination of a timestamp associated with the beat of a music sample and the location of the at least one other target image may be used to determine when the animated overlay should begin moving across the camera feed such that the animated overlay reaches the location of the at least one other target image at a desired (e.g., in sync with the beat) time.

[0070] The above process of tracking the at least one portion, determining whether the at least one portion has been aligned with a target image, generating an image of the at least one portion, and outputting an animated overlay of the image of the at least one portion may continue until the generation of the short video is terminated. For example, the creation of the short video may be terminated when the at least one portion has been aligned with the target image a predetermined number of times (e.g., seven times) and / or a predetermined time period (e.g., one minute) has elapsed. After the short video is created, the short video may be uploaded to the short video service for viewing by other users of the short video service.

[0071] 3 illustrates an example process 300 performed by a user device (e.g., client device 104). The client device 104 may perform the process 300 to generate a short video for uploading to a short video service (e.g., short video service 112). Once uploaded to the short video service, the short video may be viewed by users of the short video service. Although illustrated in FIG. 3 as a series of operations, one skilled in the art will appreciate that the illustrated operations may be added, removed, reordered, or modified in various embodiments.

[0072] Creation of a short video (e.g., recording of a short video) may be initiated. For example, a user wishing to create a short video may initiate creation via the content application 106. By way of example and not limitation, a short video may be a short video featuring a user moving one or more body parts in accordance with the rhythm or beat of a pre-recorded audio overlay (e.g., a pre-recorded audio overlay as described above). The pre-recorded audio overlay may include music that may be selected by the user and / or randomly selected. For example, a short video created by a user may be a short video featuring a user moving one or more body parts (e.g., the user's head and / or eyes) in response to the rhythm or beat of a selected piece of music. At 302, an image including a movement of an object (e.g., a user) may be acquired and recorded by an image acquisition device such as a camera of a client device (e.g., client device 104). The movement may include movement of one or more predetermined parts such as the user's head and / or eyes. These movements may include movement of one or more predetermined body parts toward one or more target images, for example.

[0073] As described above, one or more target images may be overlaid on the camera feed. The one or more target images may be overlaid at various locations on the camera feed. The various locations may be selected randomly and / or according to an algorithm. The one or more target images may be of any size, shape, or color. For example, the one or more target images may be a smiley face, one or more circles, and / or any other image. Only one target image may be overlaid on the camera feed at a time, or multiple target images may be overlaid on the camera feed at a time.

[0074] During creation of the short video, the user may attempt to align the one or more predefined body parts with the one or more target images. The user may attempt to align the one or more body parts with the one or more target images at predefined times and / or times corresponding to the rhythm or beat of a selected pre-recorded audio overlay (e.g., a song clip). For example, the user may attempt to align his / her eyes with the target image in time with the beat of the song. The target image may be a smile with two "eyes". The user may attempt to align his / her eyes with the "eyes" of the smile in time with the beat of the song. Alternatively, any other target image may be used.

[0075] At 304, it may be determined whether the at least one portion of an object (e.g., a user) is aligned or approximately aligned with a target image overlaid on a first image of the multiple video images. For example, an object tracking model, e.g., object tracking and detection model 110, may be used to determine whether the one or more body parts are aligned or approximately aligned with a target image (e.g., the "eyes" of a smile target). The one or more body parts may be aligned with a target image when the one or more body parts and the target image are completely overlapped (e.g., in the same position on the camera feed). The one or more body parts may be approximately aligned with a target image when the one or more body parts and the target image are partially or nearly completely overlapped (e.g., within a predetermined distance threshold).

[0076] It may be determined whether at least one portion of a user is aligned or approximately aligned with a target image in time close to a rhythmic point (e.g., beat) of the playback of the selected music. For example, a beat tracking model such as beat tracking model 111 may be utilized to determine whether the at least one portion of a user is aligned or approximately aligned with one or more target images in time close to the beat of a selected pre-recorded audio overlay (e.g., selected music). The music may have a known beat and / or rhythm. The beat tracking model may keep track of timestamps indicating the beat and / or one or more times corresponding to the beat of the selected music. The beat tracking model may also keep track of and / or generate timestamps indicating one or more times at which the at least one portion of a user is aligned or approximately aligned with a target image. The beat tracking model may compare timestamps indicating the time at which the at least one portion of a user is aligned or approximately aligned with a target image to timestamps corresponding to the beat and / or rhythm of the selected music to determine whether the body part is aligned or approximately aligned with a target image in time close to the beat of the selected music.

[0077] As described above, a video segmentation model, such as video segmentation model 109, may be configured to capture one or more images of the at least one portion of a user at a particular time. For example, the video segmentation model may be configured to capture one or more images of the at least one portion of a user when the at least one portion of a user is aligned or approximately aligned with a target image on the beat of a selected piece of music. As another example, the video segmentation model may be configured to capture one or more images of the at least one portion of a user when the at least one portion of a user is aligned or approximately aligned with a target image on the beat of a selected piece of music. As another example, the video segmentation model may be configured to capture one or more images of the at least one portion of a user when the at least one portion of a user is not aligned with a target image on or approximately aligned with the beat of a selected piece of music. As another example, the video segmentation model may be configured to capture one or more images of at least one portion of the user regardless of whether the at least one portion of the user is timed with the target image to the beat of the selected music (e.g., the video segmentation model may be configured to capture one or more images of the at least one portion of the user at random times and / or other predetermined times).

[0078] In one embodiment, the video segmentation model may be configured to capture one or more images of a different user or parts of a different user (e.g., a user other than the user who created the video). The parts of the different users may include any body parts of the different users, for example, the head / face of the different users. The different users may be friends, family, or complete strangers. For example, the different users may be users of the short video service 112 who have previously created short videos as described above. If the different users have previously created short videos as described above, the body parts of the users may be segmented and stored as images. The images may be used by one or more subsequent creators of short videos.

[0079] At 306, the at least one portion of the object or at least one portion of the at least one portion of the object may be segmented from the first image in accordance with a determination that the at least one portion is aligned with the target image in time close to a first rhythmic point of the playback of the selected music. In some embodiments, the at least one portion of the object has movement in the plurality of images. At least one portion of the at least one portion of the object in a second image of the plurality of images may be determined to be aligned with the target image in time close to a second rhythmic point. The at least one portion of the object or at least one portion of the at least one portion of the object may then be segmented from the second image. In one example, the first image, the second image, and the other image may be consecutive images with different timestamps.

[0080] The animated overlay may be generated based on the divided parts or portions. The animated overlay may be generated based on an image of the at least one part of an object (e.g., a user), an image of another object, a different part of the object, and / or an image of a different user. The animated overlay may be output over the camera feed during the creation of the short video. The animated overlay may be set to move across the camera feed while the short video is still being recorded. The animated overlay may be set to move across the camera feed in any direction, for example, towards a target image. The animated overlay may move across the camera feed at any speed. One or more animated overlays may move across the camera feed at any time. Multiple animated overlays may move across the camera feed during the creation of the short video.

[0081] The animated overlay may be configured to move across the camera feed to reach the target image at a predetermined time, e.g., a time corresponding to the beat of the selected music. For example, a beat tracking model may keep track of times (e.g., timestamps) during the selected music that correspond to the beat and / or rhythm of the music. The location of the target image may be known. The timestamps associated with the beats of the music samples in combination with the location of the target image may be used to determine when the animated overlay should begin moving across the camera feed so that the animated overlay reaches the location of the target image at a desired (e.g., in sync with the beat) time.

[0082] At 308, a first overlay may be generated and combined into the short video. Generating the short video may include combining at least one portion of a selected music playback, at least one portion of a video image (e.g., one or more frames of the captured video), and at least one portion of an animation into the video. In one example, a target where a user aligns the at least one portion of an object (e.g., a user) with a target image may result in the user exaggerating the movement of the generated content. For example, a user may attempt to distort and / or control the at least one portion in a particular way so that the captured image is entertaining or amusing. For example, if the at least one body part is a head or face, the user may attempt to create a funny face when an image of his or her face is captured by the video segmentation model. For example, generation of the short video may be terminated when the at least one portion of the user aligns with the target image a predetermined number of times (e.g., seven times) and / or when a predetermined time (e.g., one minute) has elapsed. After the short video is generated, it may be uploaded to the short video service for viewing by other users of the short video service.

[0083] 4 illustrates an example process 400 performed by a user device (e.g., client device 104). The client device 104 may perform process 400 to generate a short video for uploading to a short video service (e.g., short video service 112). Once uploaded to the short video service, the short video may be viewed by users of the short video service. Although illustrated in FIG. 4 as a series of operations, one skilled in the art will appreciate that the illustrated operations may be added, removed, reordered, or modified in various embodiments.

[0084] As discussed above, a client device, e.g., client device 104, may use the feed captured by the image capture device to detect (e.g., locate) one or more body parts, e.g., one or more body parts of a user. The one or more body parts detected by the camera may include a body part that the user moves to the rhythm or beat of a pre-recorded audio overlay while creating a short video. For example, if the user moves their head / face to the rhythm or beat of a pre-recorded audio overlay while creating a short video, the client device may use the feed captured by the image capture device to detect the user's head or face.

[0085] At 402, a face may be detected. The face may be detected (e.g., located) at least in part using an object tracking and detection model, such as the object tracking and detection model 110 described above. FIG. 5 illustrates an example of a UI 500 depicting a detected face 502. The detected face 502 may be the face of a user who wishes to create a short video. For example, the user may be a user of a short video service, such as the short video service 112. The face 502 may be detected, for example, by the object tracking and detection model.

[0086] At 404, a target image and eye overlay may be output. Graphics, video, sound, and / or other input may be overlaid on top of the detected face 502 depicted in FIG. 5. For example, one or more images, such as image overlay 504, may be overlaid on at least one portion of the detected face 502, such as the user's eyes. The image overlay 504 may, for example, include two circles covering at least one portion of each of the user's eyes. The image overlay 504 may be any other shape and / or any color or size. The image overlay 504 may cover both entire eyes or only one portion of the eyes. The image overlay 504 may move with the detected face 502. For example, as the user moves their head / face 502 to the right (and thus their eyes to the right), the image overlay 504 may also move to the right so that the image overlay 504 remains over the user's eyes. Similarly, when the user moves their head / face 502 to the left (and thus moves their eyes to the left), the image overlay 504 over the user's eyes may also move to the left, so that the image overlay 504 remains over the user's eyes. The image overlay 504 may also move in any other direction such that the image overlay 504 remains over the user's eyes regardless of how or where the user moves their head / face 502.

[0087] One or more target images 506 may additionally or alternatively be placed on the camera feed. The one or more target images 506 may be overlaid at various locations on the camera feed. The various locations may be selected randomly and / or according to an algorithm. The one or more target images 506 may be of any size, shape, or color. For example, the one or more target images 506 may be a smiley face, one or more circles, and / or any other image. One target image 506 may be output on the camera feed at a time, or multiple target images 506 may be output on the camera feed at a time.

[0088] Music may be selected before the user starts creating the short video (e.g., before starting recording). At 406, a music selection may be received. The music selection may be received, for example, from a user. Alternatively, the music selection may be selected randomly. The music may be selected from a music library, for example a music library associated with the short video service. Alternatively, the music may be selected from an online music database or a music library on the client device. The selected music may have a known beat and / or rhythm. For example, a timestamp may be associated with the known beat and / or rhythm of the selected music. The timestamp may indicate a time during the selected music that corresponds to the beat of the music.

[0089] At 408, recording may be started. The user may start recording, for example, after selecting music. Alternatively, recording may start automatically, for example, after selecting music. The selected music may be played while the video is being created (e.g., during recording). At 410, playback of at least one portion of the music may be started. The image overlay 504 may remain over the detected face 502 during at least one portion of the music playback. While the music is playing, the user may attempt to align the detected face 502 with the target image 506. For example, the user may attempt to align the image overlay 504 with the target image 506. The user may attempt to align the detected face 502 and / or image overlay 504 with the target image 506, for example, at a predetermined time and / or at a time corresponding to a predetermined rhythm or beat of the music. For example, the user may attempt to align the circles covering each of his or her eyes with the "eyes" of the smile target image 506 in time with the beat of the song.

[0090] 6 illustrates an example UI 600 depicting a user attempting to align a detected face 502 and / or image overlay 504 with a target image 506. The user may attempt to align the detected face 502 and / or image overlay 504 with the target image at a predetermined time and / or at a time corresponding to a predetermined rhythm or beat of music. As described above, an object tracking model (e.g., object tracking and detection model 110) may be utilized to determine whether the detected face 502 and / or image overlay 504 has been aligned or approximately aligned with the target image 506. If the detected face 502 and / or image overlay 504 and the target image 506 are perfectly overlapping (e.g., in the same position on the camera feed), the detected face 502 and / or image overlay 504 may be aligned with the target image 506. The detected face 502 and / or image overlay 504 and the target image 506 may be approximately aligned when the detected face 502 and / or image overlay 504 and the target image 506 overlap partially or nearly completely (e.g., within a predetermined distance threshold).

[0091] A video segmentation model, such as the video segmentation model 109 described above, may be configured to capture one or more images of the detected face 502 when the detected face 502 is aligned or approximately aligned with the target image 506. For example, the video segmentation model may be configured to capture one or more images of the detected face 502 when the detected face 502 is aligned or approximately aligned with the target image 506, regardless of whether the alignment is made to the beat of the selected music.

[0092] At 412, an image of the face may be generated and / or obtained. For example, the video segmentation model may segment the series of video frames (i.e., the series of images) into regions to determine information about the detected face 502. The information about the detected face 502 may be used to determine a boundary and / or a contour of the detected face 502. The information about the detected face 502 may be used to distinguish the detected face 502 from other objects in the series of frames. For example, the video segmentation model may use information about the detected face 502 to distinguish the detected face 502 from other body parts (e.g., neck or chest) in the series of frames. Once the detected face 502 is distinguished, the video segmentation model may separate the detected face 502 from the rest of the frames. Once the detected face 502 is separated from the rest of the frames, it may be saved as an image.

[0093] An animated overlay of the image of the detected face 502 may be set to move across the camera feed while the short video is still being recorded. For example, FIG. 6 depicts an animated overlay 604 of the image of the detected face 502 moving across the camera feed while the short video is still being recorded. The animated overlay 604 may be set to move across the camera feed in any direction, such as toward the target image 506. The animated overlay 604 may move across the camera feed at any speed. One or more animated overlays 604 may move across the camera feed at any time. Multiple animated overlays 604 may move across the camera feed during the creation of the short video.

[0094] The animated overlay 604 may be configured to move across the camera feed to reach the target image 506 at a predetermined time, e.g., a time corresponding to the beat of the selected music. For example, a beat tracking model, e.g., beat tracking model 111 described above, may keep track of times (e.g., timestamps) during the selected music that correspond to the beat and / or rhythm of the music. The location of the target image 506 may be known. The timestamps associated with the beats of the music samples in combination with the location of the target image 506 may be used to determine when the animated overlay 604 should begin moving across the camera feed such that the animated overlay 604 reaches the location of the target image 506 at a desired (e.g., in sync with the beat) time.

[0095] In one embodiment, a beat tracking model may be used to determine whether the detected face 502 is aligned or approximately aligned with the target image 506 at a time that is close to the beat of the selected music. The beat tracking model may keep track of timestamps indicating the beat of the selected music and / or one or more times corresponding to the beat. The beat tracking model may also keep track of and / or generate timestamps indicating one or more times when the detected face 502 is aligned or approximately aligned with the target image 506 (as determined by the object detection and tracking model). The beat tracking model may compare the timestamps indicating the time when the detected face 502 is aligned or approximately aligned with the target image 506 to timestamps corresponding to the beat and / or rhythm of the selected music to determine whether the alignment occurred at a time that is close to the beat of the selected music.

[0096] At 414, it may be determined whether the detected face 502 and / or image overlay 504 is aligned with the target image 506 in time with the beat. For example, a beat tracking model may be used to determine whether the detected face 502 and / or image overlay 504 is aligned with the target image 506 in time with, or approximately in time with, the beat of the selected music.

[0097] As described above, the feedback image overlay may be output during creation of the short video. The feedback image overlay may indicate whether the detected face 502 and / or image overlay 504 is aligned or approximately aligned in time with the target image 506 close to the beat and / or rhythm of the selected music. For example, the feedback image overlay may indicate whether the beat tracking model has determined that the detected face 502 and / or image overlay 504 is aligned or approximately aligned in time with the target image close to the beat of the selected music. The feedback image overlay may indicate whether the detected face 502 and / or image overlay 504 is aligned with the target image 506 in time with the beat of the selected music, is approximately aligned with the target image in time with the beat of the selected music (e.g., on the beat or within a predetermined time of the beat), or is not aligned with the target image in time with the beat of the selected music (e.g., not on the beat or not within a predetermined time of the beat). The feedback image overlay can differ depending on whether the detected face 502 and / or image overlay 504 is on-beat, barely on-beat, or not on-beat with the target image 506 .

[0098] If it is determined that the detected face 502 and / or image overlay 504 is approximately on beat with the target image 506, a feedback image approximately on beat may be output. For example, the UI 600 shows an exemplary type of feedback image 602. The feedback image 602 may indicate that the detected face 502 and / or image overlay 504 is approximately on beat with the target image 506. The feedback image 602 may be overlaid on the target image 506. For example, the feedback image 602 (e.g., a blue or green circle) may be overlaid on the "eyes" of the smile target image 506. In another embodiment, the feedback image 602 may additionally or alternatively be overlaid on the detected face 502 and / or image overlay 504. Although depicted as a circle in FIG. 6, the feedback image 602 may be of any size, color, and / or shape.

[0099] Alternatively, a different type of feedback image overlay may be output if the detected face 502 and / or image overlay 504 is aligned or approximately aligned with the target image 506 in time with the beat of the selected music. If it is determined that the detected face 502 and / or image overlay 504 and the target image 506 are aligned in time with the beat, the method 400 may proceed to 418. At 418, feedback may be output in time with the beat. For example, FIG. 7 depicts a feedback image 702. The feedback image 702 may indicate that the detected face 502 and / or image overlay 504 is aligned or approximately aligned with the target image 506 in time with the beat. The feedback image 702 may be overlaid on the target image 506. For example, the feedback image 702 (e.g., a heart shape) may be overlaid on the "eyes" of the smile target image 506. In another embodiment, the feedback image 702 may additionally or alternatively be overlaid on the detected face 502 and / or image overlay 504. Although depicted as a heart shape in FIG. 7, the feedback image 702 may be any size, color, and / or shape.

[0100] When the detected face 502 and / or image overlay 504 is aligned or approximately aligned with the target image 506 in time with the beat of the selected music, the image of the detected face 502 may be displayed in a top bar 708 of the video production feed. At 420, the image of the face may be displayed in a top bar of the camera feed. The image of the detected face 502 displayed in the top bar 708 may be generated / obtained by a video segmentation model at or near the time (e.g., on beat) when the detected face 502 and / or image overlay 504 is aligned or approximately aligned with the target image 506.

[0101] The user may know that the video segmentation model is configured to capture the detected face 502 for display in the top bar 708 if the detected face 502 and / or image overlay 504 are aligned or approximately aligned with the target image 506 in time with the beat of the selected music. As a result, the user may attempt to distort and / or control his / her face in a particular way so that the captured image of the detected face 502 is entertaining or amusing. For example, the user may attempt to create a comical face. As a result, the captured image output in the top bar 708 may be entertaining, which may increase the likelihood that other users who watch the short video will find it entertaining.

[0102] A new target image may appear after the user aligns or approximately aligns the detected face 502 and / or image overlay 504 with the target image 506. For example, FIG. 8 illustrates a UI 800 representing a user attempting to align the detected face 502 and / or image overlay 504 with a new target image 802. Similarly, a particular type of feedback image may be output depending on whether the detected face 502 and / or image overlay 504 and the new target image 802 are aligned or approximately aligned on beat. An additional image of the detected face 502 may be displayed in the top bar 708 of the video creation feed each time the detected face 502 and / or image overlay 504 is aligned or approximately aligned with a target image, e.g., the target image 802, on the beat of the selected music. For example, the additional image 804 of the detected face 502 may be generated / obtained by a video segmentation model at or near the time (e.g., on beat) when the detected face 502 and / or image overlay 504 is aligned or approximately aligned with the target image 802. The additional images 804 may be displayed in the top bar 708 along with the previously captured images 704 .

[0103] Referring again to FIG. 4, if it is determined that the detected face 502 and / or image overlay 504 and the target image 506 are not aligned on beat, the method 400 may proceed to 416. At 416, non-on beat feedback may be output. For example, FIG. 9 depicts a UI 900 showing a feedback image 902. The feedback image 902 may indicate that the detected face 502 and / or image overlay 504 are not aligned or approximately aligned on beat with the target image 506. The feedback image 902 may be overlaid on the target image 506. For example, the feedback image 902 (e.g., an x ​​or cross) may be overlaid on the "eyes" of the smile target image 506. In another embodiment, the feedback image 902 may additionally or alternatively be overlaid on the detected face 502 and / or image overlay 504. Although depicted as an x ​​or cross in FIG. 9, the feedback image 902 may be of any size, color, and / or shape. A new target image may appear after a user attempts and fails to align or approximately align the detected face 502 and / or image overlay 504 on beat with the target image 506. The user may again attempt to align the detected face 502 and / or image overlay 504 on beat with the new target image.

[0104] The above process of outputting a target image, determining whether the detected face 502 and / or image overlay 504 has aligned with the target image in time with the beat, outputting a feedback image, acquiring an image of the detected face 502, and outputting an animated overlay of the new target and / or acquired image may continue for the duration of the short video. The short video may continue, for example, until the user has aligned the detected face 502 and / or image overlay 504 with the target image a predetermined number of times. For example, the goal may be seven times. Recording of the short video may end when the user has aligned the detected face 502 and / or image overlay 504 with the target image in time with the beat seven times. Additionally or alternatively, creation of the short video may end after a predetermined period of time. For example, creation of the short video may end after a predetermined time regardless of how many times the user has aligned the detected face 502 and / or image overlay 504 with the target image in time with the beat. The predetermined period of time may be any period of time, such as 30 seconds or one minute.

[0105] At 422, it may be determined whether an image threshold and / or a time threshold are met. The image threshold may indicate a predetermined number of times a user attempts to align the detected face 502 and / or image overlay 504 with the target image in time with the beat. For example, the image threshold may be met if the user aligns the detected face 502 and / or image overlay 504 with the target image in time with the beat a predetermined number of times, for example, seven times. The time threshold may be any predetermined period of time, such as 30 seconds or one minute. If the predetermined period of time has elapsed, the time threshold may be met regardless of how many times the user aligns the detected face 502 and / or image overlay 504 with the target image in time with the beat. If it is determined that neither the image threshold nor the time threshold are met, the method 400 may return to 412. Conversely, if it is determined that either the image threshold nor the time threshold are met, the method 400 may proceed to 424. At 424, all captured images displayed in the top bar 708 may be output and rotated. For example, the images of detected face 502 displayed in top bar 708, including images 704, 804, and 904, may be set to move down from top bar 708 and rotate around detected face 502. Recording may be ended at 426. After the short video has been created (e.g., recorded), it may be uploaded to the short video service for viewing by other users of the short video service.

[0106] As described above, the animated overlay moving across the camera feed during the creation of the short video may include images of body parts belonging to one or more users other than the user creating the short video. FIG. 10 shows a UI 1000 depicting an animated overlay 1002 including an image of a part of a user different from the user creating the short video. For example, the animated overlay 1002 may include an image of the head or face of the different user. The different user may be a friend, family member, or a complete stranger. For example, the different user may be a user of the short video service 112 who previously created a short video as described above. If the different user previously created a short video as described above, the body part of that user (e.g., his / her face) may have been segmented and stored as an image. This image may be used by one or more subsequent creators of the short video.

[0107] An animated overlay 1002 of another user's body part may be configured to move across the camera feed as the user attempts to align the user's body part with one or more target images, e.g., target image 1004. FIG. 11 shows an example of a UI 1111 indicating that a user attempts to align the user's body part with a target image 1004 as the animated overlay 1002 of the other user's body part moves across the camera feed. The animated overlay 1002 of the other user's body part may be configured to move in any direction across the camera feed, e.g., toward the target image 1004. The animated overlay 1002 of the other user's body part may move across the camera feed at any speed. The animated overlay 1002 of the other user's body part may move across the camera feed at any time.

[0108] When a user aligns or approximately aligns their own body part with the target image 1004 in time with the beat of the selected music, one or more of the images used for the animated overlay 1002 of the other user's body part may be displayed in the top bar 1106 of the video creation feed. For example, if the animated overlay 1002 of the other user's body part is set to move across the camera feed toward the target image 1004, the target image 1004, the user's body part, and the animated overlay 1002 may all align or approximately align to the beat. Whenever this situation occurs, the image used for the particular animated overlay of the other user's body part may be moved to the top bar 1106. Multiple images 1104 of other users' body parts may be moved to the top bar 1106 during the creation of a short video.

[0109] Figure 12 illustrates a computing device that can be used in various aspects, such as the services, networks, modules, and / or devices illustrated in Figure 1. With respect to the example architecture of Figure 1, the message service, interface service, processing service, content service, cloud network, and clients may each be implemented by one or more instances of computing device 1200 of Figure 12. The computer architecture illustrated in Figure 12 illustrates a conventional server computer, workstation, desktop computer, laptop computer, tablet, network appliance, PDA, electronic reader, digital mobile phone, or other computing node that may be used to perform any aspect of the computer described herein, such as implementing the methods described herein.

[0110] Computing device 1200 may include a board or "motherboard," which is a printed circuit board that can be connected to multiple components or devices via a system bus or other electrical communication paths. One or more central processing units (CPUs) 1204 may be operatively coupled to a chipset 1206. CPU 1204 may be a standard programmable processor that performs arithmetic and logical operations necessary for the operation of computing device 1200.

[0111] The CPU(s) 1204 can perform the required operations by manipulating switching elements that transition from one discrete physical state to the next and change these states differentially. Switching elements typically include electronic circuits, such as flip-flops, that maintain one of two binary states, and electronic circuits, such as logic gates, that provide an output state based on a logical combination of the states of one or more other switching elements. These basic switching elements can be combined to create more complex logic circuits, including registers, adders / subtractors, arithmetic logic units, floating point units, etc.

[0112] CPU(s) 1204 may be augmented or replaced by other processing units, such as GPU(s), which may include processing units specialized, but not necessarily limited to, highly parallel computations, such as graphics and other visualization-related processing.

[0113] The chipset 1206 may provide an interface between the CPU 1204 and the remaining components and devices on the board. The chipset 1206 may provide an interface to a random access memory (RAM) 1208 used as a main memory in the computing device 1200. The chipset 1206 may also provide an interface to a computer-readable storage medium, such as a read-only memory (ROM) 1220 or a non-volatile RAM (NVRAM) (not shown), for storing basic routines that may facilitate starting the computing device 1200 and transmitting information between various components and devices. According to aspects described herein, the ROM 1220 or the NVRAM may also store other software components necessary for the operation of the computing device 1200.

[0114] The computing device 1200 can operate in a networked environment using logical connections to remote computing nodes and computer systems through a local area network (LAN). The chipset 1206 can include functionality for providing network connectivity through a network interface controller (NIC) 1222, such as a Gigabit Ethernet adapter. The NIC 1222 may be capable of connecting the computing device 1200 to other computing nodes over the network 1216. It should be understood that multiple NICs 1222 can be present in the computing device 1200 to connect the computing device to other types of networks and remote computer systems.

[0115] The computing device 1200 may be connected to a mass storage device 1228 that provides non-volatile storage for the computer. The mass storage device 1228 may store system programs, application programs, other program modules, and data, as described in more detail herein. The mass storage device 1228 may be connected to the computing device 1200 through a storage controller 1224 that is connected to the chipset 1206. The mass storage device 1228 may consist of one or more physical storage units. The mass storage device 1228 may include a management component 1210. The storage controller 1224 may interface with the physical storage units through a Serial Attached SCSI (SAS) interface, a Serial Advanced Technology Attachment (SATA) interface, a Fibre Channel (FC) interface, or any other type of interface for physically connecting and transmitting data between the computer and the physical storage units.

[0116] Computing device 1200 may store data on mass storage device 1228 by transforming the physical state of the physical storage units to reflect the stored information. The particular transformation of the physical state may depend on a variety of factors and different implementations of the present specification. Examples of such factors include, but are not limited to, the technology for implementing the physical storage devices and whether mass storage device 1228 is characterized as a primary storage device, a secondary storage device, etc.

[0117] For example, computing device 1200 can store information in mass storage device 1228 by issuing instructions via storage controller 1224 to change the magnetic properties of a particular location in a magnetic disk drive unit, the reflective or refractive properties of a particular location in an optical storage unit, or the electrical properties of a particular capacitor, transistor, or other discrete component in a solid-state storage unit. Other transformations of physical media are possible without departing from the scope and spirit of this specification, and the foregoing examples are provided merely for ease of explanation thereof. Computing device 1200 can also read information from mass storage device 1228 by detecting the physical state or characteristics of one or more particular locations in the physical storage unit.

[0118] In addition to the mass storage device 1228 discussed above, computing device 1200 may also have access to other computer-readable storage media for storing and retrieving information, such as program modules, data structures, or other data. Those skilled in the art will appreciate that computer-readable storage media can be any available media that provides non-transitory storage of data and that can be accessed by computing device 1200.

[0119] By way of example and not limitation, computer readable storage media can include volatile and non-volatile, transient and non-transient computer readable storage media, as well as removable and non-removable media implemented in any manner or technology. Computer readable storage media includes, but is not limited to, RAM, ROM, erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory or other solid state memory technology, compact disc ROM (CD-ROM), digital versatile disc (DVD), high definition DVD ("HD-DVD"), BLU-RAY or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage, other magnetic storage devices, or any other medium that can be used to store the desired information in a non-transitory manner.

[0120] A mass storage device, such as mass storage device 1228 shown in FIG. 12, may store an operating system for controlling the operation of computing device 1200. The operating system may include a version of the LINUX operating system. The operating system may include a version of Microsoft's WINDOWS server operating system. According to another embodiment, the operating system may include a version of the UNIX operating system. Also, various mobile phone operating systems, such as IOS and ANDROID, may be utilized. It should be understood that other operating systems may be utilized. Mass storage device 1228 may store other system or applications and data used by computing device 1200.

[0121] The mass storage device 1228 or other computer-readable storage medium may also be encoded with computer-executable instructions that, when loaded into computing device 1200, transform the computing device from a general-purpose computing system into a special-purpose computer capable of implementing aspects described herein. As described above, these computer-executable instructions transform computing device 1200 by defining how CPU 1204 transitions between states. Computing device 1200 may have access to a computer-readable storage medium that stores computer-executable instructions that, when executed by computing device 1200, can perform the methods described herein.

[0122] A computing device, such as the computing device 1200 shown in Figure 12, may further include an input / output controller 1232 for receiving and processing input from multiple input devices, such as a keyboard, a mouse, a touchpad, a touch screen, an electronic stylus pen, or other type of input device. Similarly, the input / output controller 1232 may provide output to a display, such as a computer monitor, a flat panel display, a digital projector, a printer, a plotter, or other type of output device. It should be understood that the computing device 1200 may not include all of the components shown in Figure 12, may include other components not explicitly shown in Figure 12, or may utilize an entirely different architecture than the architecture shown in Figure 12.

[0123] As described herein, a computing device may be a physical computing device, such as computing device 1200 of Figure 12. A computing node may also include a virtual machine host process and one or more virtual machine instances. Computer-executable instructions may be executed indirectly by the physical hardware of the computing device by interpreting and / or executing instructions that are stored and executed within the context of a virtual machine.

[0124] It should be understood that the methods and systems are not limited to particular methods, components, or implementations, and it should also be understood that the terminology used herein is used for the purpose of describing particular embodiments only, and is not intended to be limiting.

[0125] As used in the specification and the appended claims, the singular forms "a," "one," and "the" include plural referents unless the context clearly dictates otherwise. Ranges can be expressed herein as from "about" one particular value, and / or to "about" another particular value. When such a range is expressed, another embodiment includes the range from the one particular value and / or to the other particular value. Similarly, when values ​​are expressed as approximations, by use of the antecedent "about," it is to be understood that the particular value forms another embodiment. It is further to be understood that the endpoints of each of the ranges are significant both relative to the other endpoint, and independently of the other endpoint.

[0126] "Optional" or "optionally" means that the subsequently described event or circumstance may or may not occur, and the specification includes cases where said event or circumstance occurs and cases where said event or circumstance does not occur.

[0127] Throughout the description and claims of this specification, the word "comprise" and variations of that word, such as "including" and "including," mean "including, but not limited to," and are not intended to exclude, for example, other elements, integers, or steps. "Exemplary" denotes "an example of," and is not intended to convey an indication of a preferred or desirable embodiment. "Such as" is used for purposes of interpretation rather than a limiting sense.

[0128] Components are described that can be used to perform the described methods and systems. In describing combinations, subsets, interactions, groups, etc. of these components, specific references to each of the various individual and collective combinations and permutations of these components may not be explicitly described, and each of which is to be understood as specifically contemplated and described herein for all methods and systems. This applies to all aspects of this application, including but not limited to the operations in the described methods. Thus, where there are various additional operations that can be performed, it should be understood that each of these additional operations can be performed in any particular embodiment or combination of embodiments of the described methods.

[0129] The present method and system may be more readily understood by reference to the following detailed description of the preferred embodiments and examples contained therein, and the accompanying drawings and their descriptions.

[0130] As will be appreciated by those skilled in the art, the method and system may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the method and system may take the form of a computer program product on a computer-readable storage medium having computer-readable program instructions (e.g., computer software) embodied in the storage medium. More specifically, the method and system may take the form of web-implemented computer software. Any suitable computer-readable storage medium may be utilized, including hard disks, CD-ROMs, optical storage devices, or magnetic storage devices.

[0131] Embodiments of the methods and systems are described below with reference to block diagrams and flowcharts of methods, systems, devices, and computer program products. It should be understood that each block of the block diagrams and flowcharts, and combinations of blocks in the block diagrams and flowcharts, respectively, may be realized by computer program instructions. These computer program instructions can be loaded into a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute on the computer or other programmable data processing apparatus, produce means for implementing the functions specified in one or more blocks of the flowcharts.

[0132] These computer program instructions may be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to operate in a particular manner such that the instructions stored in the computer readable memory produce an article of manufacture including computer readable instructions for implementing the functions defined in one or more blocks of the flowcharts. The computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause the computer or other programmable data processing apparatus to perform a series of operational steps to produce a computer implemented process such that the instructions that execute on the computer or other programmable data processing apparatus provide steps for implementing the functions defined in one or more blocks of the flowcharts.

[0133] The various features and processes described above may be used independently of each other or may be combined in various ways. All possible combinations and subcombinations are intended to fall within the scope of the present disclosure. Furthermore, in some implementations, some method or process blocks may be omitted. The methods and processes described herein are not limited to any particular order, and the blocks or states associated therewith may be performed in other orders as appropriate. For example, the described blocks or states may be performed in an order other than the order specifically described, or multiple blocks or states may be combined within a single block or state. The example blocks or states may be performed sequentially, in parallel, or in some other manner. Blocks or states may be added to or deleted from the described example embodiments. The example systems and components described herein may be configured differently than described. For example, elements may be added, deleted, or rearranged as compared to the described example embodiments.

[0134] It will also be understood that while various items are shown as being stored in memory or on a storage device during use, these items, or portions thereof, may be transferred between memory and other storage devices for purposes of memory management and data integrity. Alternatively, in other embodiments, some or all of the software modules and / or systems may be executed in memory on another device and communicate with the illustrated computing system via computer-to-computer communications. Furthermore, in some embodiments, some or all of the systems and / or modules may be implemented or provided in other ways, such as at least partially in firmware and / or hardware, including, but not limited to, one or more application specific integrated circuits (ASICs), standard integrated circuits, controllers (e.g., by executing appropriate instructions, and including microcontrollers and / or embedded controllers), field programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and the like. Some or all of the modules, systems, and data structures may be stored (e.g., as software instructions or structured data) on a computer-readable medium, such as a hard disk, memory, network, or portable media product, for reading by an appropriate device or via an appropriate connection. The systems, modules, and data structures may be transmitted as a generated data signal (e.g., as part of a carrier wave or other analog or digital propagated signal) over a variety of computer-readable transmission media, including wireless-based media and wired / cable-based media, which may take a variety of forms (e.g., as part of a single or multiple analog signal, or as a number of discrete digital packets or frames). In other embodiments, such computer program products may take other forms. Thus, the invention may be practiced with other computer system configurations.

[0135] While the methods and systems have been described in connection with preferred embodiments and specific examples, it is not intended that the scope be limited to the specific embodiments, as the embodiments herein are intended in all respects to be illustrative and not restrictive.

[0136] Unless expressly stated otherwise, the methods described herein do not require that their operations be performed in a particular order. Thus, where a method claim does not actually recite the order in which its operations are to be followed, or where the claims or specification are not specifically recited to limit the operations to a particular order, no order is intended to be inferred in any manner. This applies to any possible non-expressive basis for interpretation, including logical issues regarding the arrangement of steps or operational flow, the simple meaning derived from grammatical constructions and punctuation, or the number or type of embodiments described in the specification.

[0137] As will be apparent to those skilled in the art, various modifications and variations can be made without departing from the scope or spirit of the present disclosure. Other embodiments will be apparent to those skilled in the art from consideration of the specification and practice described herein. It is intended that the specification and illustrative drawings be considered as exemplary only, with its true scope and spirit being indicated by the following claims.

Claims

1. 1. A computer-implemented method for creating content, comprising: acquiring a plurality of images having different timestamps including an object, the object including at least one portion having motion within the plurality of images; determining whether at least one portion of the at least one portion of the object in a first image is aligned with a target image overlaid on the first image and whether the at least one portion is aligned with the target image in a time proximate to a first rhythmic point of playback of the selected music; Segmenting the at least one portion of the object or the at least one portion of the at least one portion of the object from the first image in accordance with a determination that the at least one portion is aligned with the target image at the time proximate to the first rhythmic point of the playback of the selected music; generating a first overlay based on the at least one portion of the object or the at least one portion of the object segmented from the first image; the object is a body part of a user captured by an image capture device; Computer-implemented method.

2. determining whether the at least one portion of the at least one part of the object in a second image is aligned with the target image overlaid on the second image and whether the at least one portion is aligned with the target image in a time proximate to a second rhythmic point of the playback of the selected music; Segmenting the at least one portion of the object or the at least one portion of the at least one portion of the object from the second image in accordance with a determination that the at least one portion is aligned with the target image at the time proximate a second rhythmic point of the playback of the selected music; generating a second overlay based on the at least one portion of the object or the at least one portion of the object segmented from the second image.

10. The computer-implemented method of claim 1.

3. The computer-implemented method of claim 2 , wherein the first overlay and the second overlay are disposed in different locations on at least one portion of the plurality of images.

4. The computer-implemented method of claim 1 , wherein the first overlay includes an animation that moves over at least one portion of the plurality of images before the first overlay is placed in place.

5. The computer-implemented method of claim 1 , further comprising detecting the at least one portion of the object in the multiple images.

6. The computer-implemented method of claim 5 , further comprising comparing coordinates of the target image to coordinates of the at least one portion of the at least one part of the object in the multiple images.

7. overlaying an image over the at least one portion of the at least one part of the object; determining whether the image overlaid on the at least one portion is aligned with the target image.

6. The computer-implemented method of claim 5.

8. 1. A system comprising at least one computing device in communication with a computer memory, the computer memory including computer readable instructions that, when executed by the at least one computing device, provide the system with: acquiring a plurality of images having different timestamps including an object, the object including at least one portion having motion within the plurality of images; determining whether at least one portion of the at least one portion of the object in a first image is aligned with a target image overlaid on the first image and whether the at least one portion is aligned with the target image in a time proximate to a first rhythmic point of playback of the selected music; Segmenting the at least one portion of the object or the at least one portion of the at least one portion of the object from the first image in accordance with a determination that the at least one portion is aligned with the target image at the time proximate to the first rhythmic point of the playback of the selected music; generating a first overlay based on the at least one portion of the object or the at least one portion of the object segmented from the first image; Set it to perform operations including the object is a body part of a user captured by an image capture device; system.

9. determining whether the at least one portion of the at least one part of the object in a second image is aligned with the target image overlaid on the second image and whether the at least one portion is aligned with the target image in a time proximate to a second rhythmic point of the playback of the selected music; Segmenting the at least one portion of the object or the at least one portion of the at least one portion of the object from the second image in accordance with a determination that the at least one portion is aligned with the target image at the time proximate a second rhythmic point of the playback of the selected music; generating a second overlay based on the at least one portion of the object or the at least one portion of the object segmented from the second image. The system of claim 8.

10. The system of claim 8 , wherein the first overlay includes an animation that moves over at least one portion of the plurality of images before the first overlay is placed in place.

11. The operation is The system of claim 8 , further comprising detecting the at least one portion of the object in the multiple images.

12. The operation is The system of claim 11 , further comprising: comparing coordinates of the target image to coordinates of the at least one portion of the at least one part of the object in the multiple images.

13. The operation is generating and displaying a feedback image indicating whether the at least one portion of the at least one part of the object was aligned with the target image at the time proximate the first rhythmic point of the playback of the music. The system of claim 8.

14. The operation is and overlaying an image over the at least one portion of the at least one part of the object. The system of claim 8.

15. The operation is determining an overlay over the at least one portion of the at least one part of the object aligned with the target image. The system of claim 8.

16. A non-transitory computer-readable storage medium having computer-readable instructions stored thereon, the computer-readable instructions, when executed by a processor, causing the processor to: acquiring a plurality of images having different timestamps including an object, the object including at least one portion having motion within the plurality of images; determining whether at least one portion of the at least one portion of the object in a first image is aligned with a target image overlaid on the first image and whether the at least one portion is aligned with the target image in a time proximate to a first rhythmic point of playback of the selected music; Segmenting the at least one portion of the object or the at least one portion of the at least one portion of the object from the first image in accordance with a determination that the at least one portion is aligned with the target image at the time proximate to the first rhythmic point of the playback of the selected music; generating a first overlay based on the at least one portion of the object or the at least one portion of the object segmented from the first image; This allows you to perform operations including the object is a body part of a user captured by an image capture device; A non-transitory computer-readable storage medium.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the first overlay includes an animation before the first overlay is placed in place.

18. The operation is and comparing coordinates of the target image to coordinates of the at least one portion of the object in the plurality of images.

20. The non-transitory computer-readable storage medium of claim 16.

19. The operation is overlaying an image over the at least one portion of the at least one part of the object; determining whether the image overlaid on the at least one portion is aligned with the target image.

20. The non-transitory computer-readable storage medium of claim 16.

20. The operation is generating and displaying a feedback image indicating whether the at least one portion of the at least one part of the object was aligned with the target image at the time proximate the first rhythmic point of the playback of the music.

20. The non-transitory computer-readable storage medium of claim 16.

Images