Server, user device and computer program for providing video contents with avatar

Abstract

A server providing video content in which avatars participate may include an avatar registration unit that registers multiple avatars to participate in a stage video, a reference avatar selection unit that selects a reference avatar to adjust the synchronization between multiple avatars among the multiple avatars, a stage video editing unit that edits the stage video in which multiple avatars appear based on the pose data of the reference avatar, and an avatar video content generation unit that generates avatar video content including the edited stage video.

Description

Technology Field

[0001] The present invention relates to a server, a user terminal, and a method for providing video content in which an avatar participates. Background Technology

[0002] Recently, the market for virtual YouTubers, who provide one-person media broadcasts using 2D / 3D avatars, is expanding. In addition, there is a growing trend of participatory content where creators and viewers, or viewers themselves, participate in the production of video content.

[0003] Meanwhile, technologies for generating 3D avatars include kinematics technology that estimates the user's body shape and posture movements, and blend shape technology that designs the avatar's face.

[0004] Although this 3D avatar generation technology automatically generates an avatar by recognizing the user's appearance, the avatar is created based on a pre-set template, so there is little room for the user to freely create or change the avatar.

[0005] In addition, existing video production services using avatars have technical limitations in expressing hand movements or facial expressions. Prior art literature

[0006] Korean Published Patent Application No. 2012-0092960 (Published Aug. 22, 2012) The problem to be solved

[0007] The present invention aims to solve the problems of the aforementioned prior art by editing a stage video based on pose data of a selected reference avatar to adjust the synchronization between a plurality of avatars participating in the stage video, and by generating avatar video content including the edited stage video.

[0008] However, the technical problems that this embodiment aims to solve are not limited to the technical problems described above, and other technical problems may exist. means of solving the problem

[0009] As a technical means for achieving the technical problem described above, a server providing video content in which an avatar participates according to the first aspect of the present invention may include: an avatar registration unit that registers a plurality of avatars to participate in a stage video; a reference avatar selection unit that selects a reference avatar to adjust the synchronization between a plurality of avatars among the plurality of avatars; a stage video editing unit that edits the stage video in which the plurality of avatars appear based on pose data of the reference avatar; and an avatar video content generation unit that generates avatar video content including the edited stage video.

[0010] A user terminal providing video content in which an avatar participates according to a second aspect of the present invention may include a generating unit that generates an avatar to participate in a stage video; an editing unit that edits the stage video by adjusting the synchronization between a plurality of avatars participating in the stage video based on pose data of the avatar; and an output unit that outputs avatar video content including the edited stage video.

[0011] A method for providing video content in which avatars participate, performed by a content providing server according to a third aspect of the present invention, may include: registering a plurality of avatars to participate in a stage video; selecting a reference avatar to adjust the synchronization between the plurality of avatars among the plurality of avatars; editing the stage video in which the plurality of avatars appear based on pose data of the reference avatar; and generating avatar video content including the edited stage video.

[0012] The above-described means for solving the problem are merely exemplary and should not be interpreted as intended to limit the invention. In addition to the exemplary embodiments described above, additional embodiments described in the drawings and the detailed description of the invention may exist. Effects of the invention

[0013] According to any one of the means for solving the problem of the present invention described above, the present invention can edit a stage video based on pose data of a selected reference avatar to adjust the synchronization between a plurality of avatars participating in the stage video, and generate avatar video content including the edited stage video.

[0014] Through this, when multiple user terminals access a virtual stage space for producing stage video, each user can receive inter-user communication services through avatars corresponding to each user terminal appearing in the stage video.

[0015] In addition, multiple user terminals can access the virtual stage space to produce avatar video content together.

[0016] In addition, multiple user terminals can freely edit stage videos and their own avatars through an editing frame that allows editing stage videos (background sound sources, stage backgrounds, etc.) and avatars (avatar costumes, accessories, movements by body part, etc.). Brief explanation of the drawing

[0017] FIG. 1 is a configuration diagram of a video content provision system according to one embodiment of the present invention. FIG. 2 is a block diagram of a content providing server illustrated in FIG. 1 according to an embodiment of the present invention. FIGS. 3a to 3d are drawings for explaining a method for generating video content in which an avatar participates, according to an embodiment of the present invention. FIGS. 4a and 4b are drawings for explaining a method of adjusting the synchronization between a plurality of avatars according to an embodiment of the present invention. FIGS. 5a to 5c are drawings for explaining a method of adjusting the synchronization between a plurality of avatars using a guide character according to an embodiment of the present invention. FIG. 6 is a flowchart illustrating a method for providing video content in which an avatar participates on a content providing server, according to an embodiment of the present invention. FIG. 7 is a block diagram of a first user terminal shown in FIG. 1 according to an embodiment of the present invention. FIG. 8 is a drawing for explaining a method of editing stage images according to an embodiment of the present invention. FIG. 9 is a flowchart illustrating a method for providing video content in which an avatar participates at a first user terminal according to an embodiment of the present invention. Specific details for implementing the invention

[0018] Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0019] Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" with other components interposed between them. Furthermore, when a part is described as "including" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0020] In this specification, the term "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Additionally, one unit may be realized using two or more hardware, and two or more units may be realized by one hardware.

[0021] Some of the operations or functions described in this specification as being performed by a terminal or device may instead be performed by a server connected to said terminal or device. Likewise, some of the operations or functions described as being performed by a server may also be performed by a terminal or device connected to said server.

[0022] Hereinafter, specific details for implementing the present invention will be described with reference to the attached configuration diagram or process flowchart.

[0023] FIG. 1 is a configuration diagram of a video content provision system according to one embodiment of the present invention.

[0024] Referring to FIG. 1, a video content providing system may include a content providing server (100) and a plurality of user terminals (110). However, since the video content providing system of FIG. 1 is merely one embodiment of the present invention, the present invention is not limited to FIG. 1 and may be configured differently from FIG. 1 according to various embodiments of the present invention.

[0025] Generally, each component of the video content provision system of FIG. 1 is connected through a network (not shown). A network refers to a connection structure capable of exchanging information between each node, such as terminals and servers, and includes a Local Area Network (LAN), a Wide Area Network (WAN), the Internet (WWW: World Wide Web), wired and wireless data communication networks, telephone networks, wired and wireless television communication networks, etc. Examples of wireless data communication networks include, but are not limited to, 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, Bluetooth communication, infrared communication, ultrasonic communication, Visible Light Communication (VLC), and LiFi.

[0026] Multiple user terminals (110) can generate an avatar corresponding to each user terminal through an avatar service application.

[0027] A content providing server (100) may receive information on the opening of a virtual stage space from a first user terminal (110-1) that has applied for the opening of a virtual stage space through an avatar service application among a plurality of user terminals (110). Here, the information on the opening of a virtual stage space may include at least one of the number of user terminals that can access the virtual stage space, a background sound source of the stage video, and a stage background.

[0028] The content providing server (100) can open a virtual stage space for producing stage video based on the received virtual stage space opening information. At this time, the number of user terminals accessing the virtual stage space may be limited based on the number of user terminals that can access the virtual stage space included in the virtual stage space opening information.

[0029] When each of the multiple user terminals (110) connects to the virtual stage space opened by the first user terminal (110-1), multiple avatars corresponding to each user terminal (110) can appear in the stage image.

[0030] The content providing server (100) can select a reference avatar for adjusting the synchronization between multiple avatars among multiple avatars participating in the stage video as the first avatar corresponding to the first user terminal (110-1).

[0031] The content providing server (100) can edit a stage video in which multiple avatars appear based on the pose data of the first avatar corresponding to the first user terminal (110-1).

[0032] The content providing server (100) can generate avatar video content including edited stage video and transmit the generated avatar video content in real-time or non-real-time through a broadcast channel.

[0033] Multiple user terminals (110) or viewing terminals (not shown) can watch avatar video content transmitted through a broadcast channel in real-time or non-real-time.

[0034] A plurality of user terminals (110) may include mobile terminals capable of wireless communication, and according to various embodiments of the present invention, the plurality of user terminals (110) may be devices of various forms. For example, the plurality of user terminals (110) may be portable terminals capable of connecting to a remote server via a network. Here, an example of a portable terminal may include all types of handheld-based wireless communication devices that ensure portability and mobility, such as PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminals, smartphones, tablet PCs, etc. However, the user terminals (110) are not limited to the examples given above.

[0035] The operation of each component of the video content provision system of FIG. 1 will be explained in more detail below.

[0036] FIG. 2 is a block diagram of a content providing server (100) illustrated in FIG. 1 according to an embodiment of the present invention.

[0037] Referring to FIG. 2, the content providing server (100) may include an avatar registration unit (200), a space creation unit (210), a standard avatar selection unit (220), a stage video editing unit (230), and an avatar video content creation unit (240). However, the content providing server (100) illustrated in FIG. 2 is merely one embodiment of the present invention, and various modifications are possible based on the components illustrated in FIG. 2.

[0038] In the following, Figure 2 will be explained with reference to Figures 3a to 5c.

[0039] The avatar registration book (200) can register multiple avatars to participate in the stage video.

[0040] Specifically, the avatar registration unit (200) can generate an avatar corresponding to each of the multiple user terminals (110) based on the captured video and motion data of the user received from each of the multiple user terminals (110), and receive a registration decision regarding the use of each generated avatar from the multiple user terminals (110).

[0041] For example, the avatar registration unit (200) can generate a first avatar corresponding to the first user terminal (110-1) based on the first user's captured video and motion data received from the first user terminal (110-1), and receive the generated first avatar from the first user terminal (110-1).

[0042] For example, referring to FIG. 3a, the first user terminal (110-1) can activate an avatar service application and select one of a plurality of avatar types provided by the avatar service application.

[0043] If the camera captured by the first user terminal (110-1) is a 3D depth camera, the first user terminal (110-1) can capture a video of the first user through the 3D depth camera while the avatar service application is activated. Additionally, the first user terminal (110-1) can extract 3D motion data of the first user from the video of the first user captured through the 3D depth camera.

[0044] If the camera captured by the first user terminal (110-1) is a 2D camera, the first user terminal (110-1) can extract 2D motion data of the first user from the captured video of the first user captured by the 2D camera, and apply 3D conversion technology (e.g., 3D pose estimation technology, etc.) to the extracted 2D motion data to convert the 2D motion data into 3D motion data. At this time, the first user terminal (110-1) can predict a Z coordinate value from the 2D motion data through the 3D conversion technology and convert the 2D motion data into 3D motion data based on the predicted Z coordinate value.

[0045] Next, the first user terminal (110-1) can transmit an avatar creation request message to the content providing server (100) that includes information about the selected avatar and the first user's captured video and motion data.

[0046] Referring to FIG. 3a and FIG. 3b together, the avatar register (200) can generate a first avatar corresponding to the first user terminal (110-1) using information about a selected avatar included in an avatar creation request message received from the first user terminal (110-1) and motion data of the first user. For example, the avatar register (200) can generate a first avatar corresponding to the first user terminal (110-1) by mapping the motion data of the first user to an avatar previously selected by the first user terminal (110-1).

[0047] The space creation unit (210) receives information on the creation of a virtual stage space from the first user terminal (110-1) and can create a virtual stage space for producing a stage video based on the received information on the creation of a virtual stage space.

[0048] For example, referring to FIG. 3c, when a first avatar corresponding to a first user terminal (110-1) is registered, the first user terminal (110-1) can input information for opening a virtual stage space to open a virtual stage space that multiple other user terminals can access through an avatar service application. Here, the virtual stage space may be a space where the first user terminal (110-1) and multiple other user terminals can communicate with each other and produce video content together. Here, the information for opening the virtual stage space may include, for example, at least one of the number of user terminals that can access the virtual stage space, a background sound source for the stage video, and a stage background.

[0049] For example, referring to FIG. 3c and FIG. 3d together, when the space creation unit (210) receives information on the creation of a virtual stage space input through an avatar service application from the first user terminal (110-1), it can create a virtual stage space by setting the stage background included in the information on the creation of the virtual stage space as the background of the stage video (30) and setting the background sound source included in the information on the creation of the virtual stage space as the background music of the stage video (30). In addition, the space creation unit (210) can limit the number of user terminals accessing the virtual stage space based on the number of user terminals that can access the virtual stage space included in the information on the creation of the virtual stage space.

[0050] When a virtual stage space is created, the space creation unit (210) can transmit account information for the created virtual stage space to the first user terminal (110-1). At this time, the first user terminal (110-1) can transmit account information for the virtual stage space to other user terminals that wish to communicate in the virtual stage space. Additionally, other user terminals can access the virtual stage space created by the first user terminal (110-1) using the account information for the virtual stage space.

[0051] At this time, the number of user terminals accessing the virtual stage space is limited based on the number of users who can access the virtual stage space, and each avatar corresponding to the first user terminal (110-1) and at least one other user terminal that has accessed the virtual stage space appears in the stage image. For example, referring to FIG. 3d, when the second user terminal (110-2) accesses the virtual stage space opened by the first user terminal (110-1), the first avatar (301) corresponding to the first user terminal (110-1) and the second avatar (303) corresponding to the second user terminal (110-2) can appear in the stage image (30).

[0052] The stage video editing department (230) can display multiple avatars corresponding to each of the multiple user terminals (110) in the stage video based on the number of user terminals that can access the virtual stage space.

[0053] The reference avatar selection unit (220) can select a reference avatar to adjust the synchronization between multiple avatars among multiple avatars participating in the stage video. For example, the reference avatar selection unit (220) can select a first avatar corresponding to the first user terminal (110-1) that requested the opening of the first virtual stage space as the reference avatar.

[0054] The stage video editing department (230) can edit a stage video in which multiple avatars appear based on the pose data of a reference avatar.

[0055] For example, referring to FIG. 4a, the stage video editing unit (230) can extract first motion data of the first avatar (reference avatar) (401) and second motion data of the second avatar (403) for each video frame constituting the stage video. Additionally, the stage video editing unit (230) can compare the pose data included in the first motion data extracted for each video frame with the pose data included in the extracted second motion data. At this time, the stage video editing unit (230) can synchronize the coordinate information of the pose data included in the first motion data with the coordinate information of the pose data included in the second motion data by comparing them.

[0056] The stage video editing unit (230) can synchronize the pose data of other avatars based on the pose data of a reference avatar for each video frame constituting the stage video. For example, referring to FIG. 4b, the stage video editing unit (230) can edit the stage video by synchronizing the coordinate information for the pose data of the second avatar (403) based on the coordinate information for the pose data of the first avatar (reference avatar) (401).

[0057] Meanwhile, referring to FIG. 5a, the avatar registration unit (200) can collect external motion data and train a deep learning model (50) based on the collected external motion data. Here, the external motion data may include pose data contained in at least one external video (e.g., a dance video, etc.).

[0058] Additionally, the avatar register (200) can generate a guide character (501) using a learned deep learning model (50). Here, the guide character (501) may be a character used to adjust the synchronization between multiple avatars appearing in the stage video.

[0059] Referring to FIG. 5b, the stage video editing unit (230) can correct the pose data of the first avatar corresponding to the first user terminal (110-1) selected as the reference avatar (503) based on the pose data of the guide character (501).

[0060] Specifically, the stage video editing department (230) can calculate correction coordinate information for correcting the pose data of the reference avatar (503) by comparing the coordinate information for the pose data of the guide character (501) with the coordinate information for the pose data of the reference avatar (503), and correct the pose data of the reference avatar (503) based on the calculated correction coordinate information.

[0061] For example, the stage video editing department (230) may express the difference between the pose data of the guide character (501) and the pose data of the reference avatar (503) as a numerical value and provide guidance to the first user terminal (110-1).

[0062] Referring to FIG. 5b and FIG. 5c together, the stage video editing unit (230) can edit the stage video by synchronizing the pose data of the reference avatar (503) corrected according to the pose data of the guide character (501) with the pose data of another avatar (505).

[0063] The avatar video content generation unit (240) can generate avatar video content including edited stage video.

[0064] Meanwhile, when the stage video editing unit (230) receives a conversation message from a plurality of user terminals (110) corresponding to a plurality of avatars, it may include the conversation message in the stage video. For example, when the stage video editing unit (230) receives a voice message from a first user terminal (110-1) corresponding to a first avatar, it may cause the voice message of the first user terminal (110-1) to be output through the first avatar of the stage video. Alternatively, the stage video editing unit (230) may cause a voice message in which the voice of the first user terminal (110-1) is modulated to be output through the first avatar of the stage video. For example, when the stage video editing unit (230) receives a text message from a second user terminal corresponding to a second avatar, it may map a speech bubble containing the text message of the second user terminal to the second avatar of the stage video so that the speech bubble is output in the stage video.

[0065] The avatar video content generation unit (240) can generate avatar video content including a stage video containing a conversation message.

[0066] When the stage video editing unit (230) receives editing information for editing the avatar or the stage video from a user terminal corresponding to at least one avatar appearing in the stage video, it can edit the stage video or the avatar based on the received editing information.

[0067] For example, when the first user terminal (110-1) receives a request to modify the first avatar through an editing tool provided in the avatar service application (e.g., a request to apply special effects to each body part of the avatar, a request to change or add clothing or accessories, a request to apply animation effects to the avatar, etc.), the stage video editing unit (230) can modify the first avatar based on the request to modify the first avatar.

[0068] For example, when the first user terminal (110-1) receives an editing request for the stage background of the stage video through an editing tool provided in the avatar service application (e.g., a request to modify the stage background with a different template, a request to adjust the camera work included in the stage background, a request to add special effects to the stage video, etc.), the stage video editing unit (230) can edit the stage background of the stage video based on the received editing request for the stage background.

[0069] For example, when the first user terminal (110-1) receives an editing request for the background sound source of the stage video through an editing tool provided in the avatar service application (e.g., a request to adjust the sync or length of the background sound source, a request to change to another background sound source, etc.), the background sound source of the stage video can be edited based on the received editing request for the background sound source.

[0070] Meanwhile, those skilled in the art will fully understand that the avatar registration unit (200), the space creation unit (210), the standard avatar selection unit (220), the stage video editing unit (230), and the avatar video content creation unit (240) may each be implemented separately, or that one or more of these may be implemented in an integrated manner.

[0071] FIG. 6 is a flowchart illustrating a method of providing video content in which an avatar participates at a content providing server (100) according to an embodiment of the present invention.

[0072] Referring to FIG. 6, in step S601, the content providing server (100) may register a plurality of avatars to participate in the stage video. For example, the content providing server (100) may create an avatar corresponding to the first user terminal (110-1) based on the first user's captured video and motion data received from the first user terminal (110-1), and register the created avatar.

[0073] In step S601, the content providing server (100) can select a reference avatar to adjust the synchronization between multiple avatars among multiple avatars.

[0074] In step S601, the content providing server (100) can edit a stage video in which multiple avatars appear based on the pose data of a reference avatar. For example, the content providing server (100) can edit the stage video by synchronizing the pose data of each different avatar based on the pose data of the reference avatar for each video frame constituting the stage video.

[0075] In step S601, the content providing server (100) can generate avatar video content including edited stage video.

[0076] In the description above, steps S601 to S607 may be further divided into additional steps or combined into fewer steps, depending on an embodiment of the present invention. Additionally, some steps may be omitted as necessary, and the order of the steps may be changed.

[0077] FIG. 7 is a block diagram of the first user terminal (110-1) shown in FIG. 1 according to one embodiment of the present invention.

[0078] Referring to FIG. 7, the first user terminal (110-1) may include a generation unit (700), a transmission unit (710), an editing unit (720), and an output unit (730). However, the first user terminal (110-1) illustrated in FIG. 7 is merely one embodiment of the present invention, and various modifications are possible based on the components illustrated in FIG. 7.

[0079] The generating unit (700) can generate a first avatar corresponding to the first user terminal (110-1) that will participate in the stage video through the avatar service application when the first user terminal (110-1) activates the avatar service application.

[0080] The generation unit (700) can generate an avatar based on the first user's captured video and motion data captured through a camera installed on the first user terminal (110-1).

[0081] Specifically, the generating unit (700) can select one of the multiple types of avatars provided by the avatar service application and extract motion data and sound data of the first user for each video frame from the video of the first user captured through the camera installed on the first user terminal (110-1).

[0082] The generation unit (700) can generate a first avatar corresponding to a first user terminal (110-1) participating in a stage video by retargeting motion data of the first user to an avatar selected through an avatar service application and adding sound data of the first user to the voice of the selected avatar.

[0083] As another example, the transmission unit (710) transmits an avatar creation request message to the content providing server (100) that includes information about the avatar selected through the avatar service application, as well as the first user's captured video, motion data, and sound data, and the creation unit (700) can receive the avatar created based on the avatar creation request message from the content providing server (100).

[0084] The transmission unit (710) can transmit information on the opening of a virtual stage space for producing a stage video to the content providing server (100). Here, the information on the opening of the virtual stage space may include, for example, at least one of the number of user terminals that can access the virtual stage space, a background sound source for the stage video, and a stage background.

[0085] For example, referring to FIG. 3c, when the first user of the first user terminal (110-1) inputs information for opening a virtual stage space to open a virtual stage space that multiple other user terminals can access through an avatar service application, the transmission unit (710) can transmit the input information for opening a virtual stage space to the content providing server (100). Here, the virtual stage space may be a space where the first user terminal (110-1) and multiple other user terminals can communicate with each other and produce video content together.

[0086] For example, referring to FIG. 3c and FIG. 3d together, when the content providing server (100) receives information on opening a virtual stage space from the first user terminal (110-1), it can open a virtual stage space by setting the stage background included in the information on opening a virtual stage space as the background of the stage video (30) and setting the background sound source included in the information on opening a virtual stage space as the background music of the stage video (30). Here, the number of terminals that can access the virtual stage space may be limited based on the number of user terminals that can access the virtual stage space included in the information on opening a virtual stage space.

[0087] When the opening of the virtual stage space is completed, the receiving unit (not shown) can receive account information for the opened virtual stage space from the content providing server (100) through the avatar service application.

[0088] The transmission unit (710) can transmit account information for the virtual stage space to other user terminals that wish to communicate in the virtual stage space. Afterwards, other user terminals can access the virtual stage space opened by the first user terminal (110-1) using the account information for the virtual stage space received from the first user terminal (110-1).

[0089] At this time, the number of user terminals accessing the virtual stage space is limited based on the number of users who can access the virtual stage space, and each avatar corresponding to the first user terminal (110-1) and at least one other user terminal accessing the virtual stage space appears in the stage image.

[0090] The editorial department (720) can edit the first avatar and stage video through the editing tool of the avatar service application.

[0091] The editing department (720) can select one of the multiple avatar editing categories provided in the avatar service application (e.g., facial expression category, accessory category, clothing category, hair category, effect setting category, motion setting category for each body part, etc.) and, when one item from the list included in the selected avatar editing category (e.g., hair category) is selected, the selected item can be applied to the first avatar to edit the first avatar. For example, referring to FIG. 8, the editing department (720) can edit the facial expressions, hand shapes, etc. of the first avatar using the first editing tool (801) provided through the avatar service application.

[0092] The editing department (720) can select one of the multiple stage video editing categories (e.g., stage background category, stage effect setting category, sound setting category, etc.) provided through the avatar service application, and when one item from the list included in the selected stage video editing category (e.g., stage background category) is selected, the selected item can be applied to the stage video to edit the stage video. For example, referring to FIG. 8, the editing department (720) can change the stage background of the stage video and set stage background effects on the stage video through the second editing tool (803) provided through the avatar service application. Additionally, the editing department (720) can adjust the camera work set on the stage video, adjust the sync or length of the sound of the stage video, or change the background sound source of the stage video through the third editing tool (805).

[0093] The editing department (720) can edit the stage video by adjusting the synchronization between multiple avatars participating in the stage video based on the pose data of the first avatar corresponding to the first user terminal (110-1).

[0094] The editing department (720) may request the content providing server (100) to edit the stage video based on the pose data of the first avatar. Upon receiving the request to edit the stage video, the content providing server (100) may edit the stage video by synchronizing the pose data included in the second motion data of another avatar based on the pose data included in the first motion data of the first avatar for each video frame constituting the stage video.

[0095] The editorial department (720) may request the content provider server (100) to edit the stage video based on the pose data of the guide character. Here, the guide character may be a character created through a deep learning model learned based on external motion data (e.g., dance video, etc.) and may be a character used to adjust the synchronization between multiple avatars appearing in the stage video.

[0096] A content provider server (100) that receives a request to edit a stage video using a guide character can correct the pose data of the first avatar based on the pose data of the guide character, and edit the stage video by synchronizing the corrected pose data of the first avatar with the pose data of other avatars.

[0097] The output unit (730) can output avatar video content including edited stage video.

[0098] Meanwhile, those skilled in the art will fully understand that the generation unit (700), transmission unit (710), editing unit (720), and output unit (730) may each be implemented separately, or that one or more of them may be implemented in an integrated manner.

[0099] FIG. 9 is a flowchart illustrating a method for providing video content in which an avatar participates at a first user terminal (110-1) according to an embodiment of the present invention.

[0100] Referring to FIG. 9, in step S901, the first user terminal (110-1) can generate an avatar corresponding to the first user terminal (110-1) that will participate in the stage video.

[0101] In step S903, the first user terminal (110-1) can edit the stage video by adjusting the synchronization between multiple avatars participating in the stage video based on the pose data of the avatar corresponding to the first user terminal (110-1).

[0102] In step S905, the first user terminal (110-1) can output avatar video content including an edited stage video.

[0103] In the description above, steps S901 to S907 may be further divided into additional steps or combined into fewer steps, depending on an embodiment of the present invention. Additionally, some steps may be omitted as necessary, and the order between steps may be changed.

[0104] One embodiment of the present invention may also be implemented in the form of a recording medium comprising computer-executable instructions, such as program modules executed by a computer. A computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, and both removable and non-removable media. Additionally, a computer-readable medium may include all computer storage media. A computer storage medium includes both volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information, such as computer-readable instructions, data structures, program modules, or other data.

[0105] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0106] The scope of the present invention is defined by the claims set forth below rather than by the detailed description, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention. Explanation of the symbols

[0107] 100: Content delivery server 110: Multiple user terminals 200: Avatar Registry 210: Space Opening Section 220: Standard Avatar Selection Department 230: Stage Video Editing Department 240: Avatar Video Content Creation Section 700: Generation section 710: Transmission section 720: Editorial Department 730: Output section

Claims

Claim 1 A server providing video content in which avatars participate, comprising: an avatar registration unit for registering a plurality of avatars to participate in a stage video; a reference avatar selection unit for selecting a reference avatar among the plurality of avatars to adjust the synchronization between the plurality of avatars; and a stage video editing unit for editing the stage video in which the plurality of avatars appear based on pose data of the reference avatar. A content providing server comprising: an avatar video content generating unit that generates avatar video content including the above-mentioned edited stage video; the stage video editing unit compares the pose data of each of the plurality of avatars with the pose data of the reference avatar based on the pose data of the reference avatar, and synchronizes the pose data of each of the plurality of avatars with the pose data of the reference avatar according to the comparison result; the avatar registration unit collects external motion data corresponding to the pose data included in the external video, trains a deep learning model based on the collected external motion data, and generates a guide character using the trained deep learning model; and the stage video editing unit derives correction coordinate information for correcting the pose data of the reference avatar based on the difference value between the pose data of the guide character and the pose data of the reference avatar, and corrects the pose data of the reference avatar by guiding the correction coordinate information to a user terminal corresponding to the reference avatar. Claim 2 A content providing server according to claim 1, wherein, when the stage video editing unit receives a conversation message from a plurality of user terminals corresponding to the plurality of avatars, the conversation message is included in the stage video, and the avatar video content generating unit generates the avatar video content including the stage video containing the conversation message. Claim 3 A content providing server according to claim 1, wherein the avatar registration unit generates an avatar corresponding to the first user terminal based on the first user's captured video and motion data received from the first user terminal, and receives the generated avatar for registration. Claim 4 A content providing server according to claim 3, further comprising a space opening unit that receives information on opening a virtual stage space from the first user terminal and opens the virtual stage space for producing the stage video based on the received information on opening a virtual stage space, wherein the information on opening a virtual stage space includes at least one of the number of user terminals capable of accessing the virtual stage space, a background sound source of the stage video, and a stage background. Claim 5 A content providing server according to claim 4, wherein a plurality of avatars corresponding to each of the plurality of user terminals can appear in the stage image based on the number of user terminals accessible in the virtual stage space. Claim 6 delete Claim 7 delete Claim 8 delete Claim 9 A content providing server according to claim 1, wherein the stage video editing unit synchronizes the pose data of the corrected reference avatar with the pose data of another avatar based on the pose data of the corrected reference avatar. Claim 10 A user terminal providing video content in which an avatar participates, comprising: a generating unit for generating an avatar to participate in a stage video; an editing unit for editing the stage video by adjusting the synchronization between a plurality of avatars participating in the stage video based on pose data of the avatar; and an output unit for outputting avatar video content including the edited stage video, wherein the pose data of each of the plurality of avatars is compared with the pose data of the reference avatar based on the pose data of a reference avatar corresponding to one of the plurality of avatars, and the pose data of each of the plurality of avatars and the pose data of the reference avatar are synchronized according to the comparison result, external motion data corresponding to the pose data included in the external video is collected, a deep learning model is trained based on the collected external motion data, a guide character is generated using the trained deep learning model, correction coordinate information for correcting the pose data of the reference avatar is derived based on the difference value between the pose data of the guide character and the pose data of the reference avatar, and the correction coordinate information is guided to a user terminal corresponding to the reference avatar so that the pose data of the reference avatar is corrected. Claim 11 In claim 10, the generating unit generates the avatar based on the captured video and motion data of the user terminal, the user terminal. Claim 12 A user terminal according to claim 11, further comprising a transmission unit that transmits information on the opening of a virtual stage space for producing the stage video to a content provider server, wherein the information on the opening of the virtual stage space includes at least one of the number of user terminals accessible to the virtual stage space, a background sound source of the stage video, and a stage background. Claim 13 A user terminal according to claim 12, wherein a plurality of avatars corresponding to each of the plurality of user terminals can appear in the stage image based on the number of user terminals accessible in the virtual stage space. Claim 14 In claim 12, the editing unit requests the content providing server to edit the stage video based on the pose data of the avatar, and the edited stage video is edited by synchronizing the pose data of each different avatar based on the pose data of the avatar for each video frame constituting the stage video, the user terminal. Claim 15 A method for providing video content in which avatars participate, performed by a content providing server, comprising: a step of registering a plurality of avatars to participate in a stage video; a step of selecting a reference avatar among the plurality of avatars to adjust the synchronization between the plurality of avatars; a step of editing the stage video in which the plurality of avatars appear based on the pose data of the reference avatar; and a step of generating avatar video content including the edited stage video; further comprising a step of comparing the pose data of each of the plurality of avatars with the pose data of the reference avatar based on the pose data of the reference avatar; and a step of synchronizing the pose data of each of the plurality of avatars with the pose data of the reference avatar according to the comparison result; a step of collecting external motion data corresponding to the pose data included in the external video, training a deep learning model based on the collected external motion data, and generating a guide character using the trained deep learning model; and further comprising a step of deriving correction coordinate information for correcting the pose data of the reference avatar based on the difference value between the pose data of the guide character and the pose data of the reference avatar, and correcting the pose data of the reference avatar by guiding the correction coordinate information to a user terminal corresponding to the reference avatar. Claim 16 delete Claim 17 delete Claim 18 delete Claim 19 A content providing method according to claim 15, wherein the step of editing the stage video includes the step of synchronizing the pose data of the corrected reference avatar with the pose data of another avatar based on the pose data of the corrected reference avatar.

Images