Systems and methods for collaborative content creation in computing environment

The system allows multiple users to collaboratively create content in virtual environments using AI, addressing the limitations of current methods by analyzing user inputs for duplicates and redundancies to enhance creative capabilities and effects.

JP2024157533APending Publication Date: 2024-11-07TMRW FOUNDATION IP SARL
View PDF 0 Cites 5 Cited by

Patent Information

Application Number
JP2024067326
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-04-25
Filing Date
2024-04-18
Publication Date
2024-11-07

AI Technical Summary

Technical Problem

Current systems and methods do not allow multiple users to collaboratively create content using artificial intelligence, limiting the social and creative aspects of content creation.

Method used

A method and system for collaborative content creation using artificial intelligence that receives inputs from multiple users, analyzes for duplicate and redundant data, and generates actions based on machine learning models to enhance creative capabilities in virtual environments.

Benefits of technology

Enables multiple users to co-create content with enhanced creative effects in augmented, virtual, and simulated environments, facilitating intuitive and efficient collaborative interaction through persistent AI-assisted processing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024157533000001_ABST
    Figure 2024157533000001_ABST
Patent Text Reader

Abstract

To provide a method for executing collaborative content creation in a computing environment using artificial intelligence.SOLUTION: The method includes: receiving, by a content creation system, a first input from a first user and a second input from a second user; analyzing, by an input fusion system, the first input and the second input to determine the presence of duplicate data, redundancy data and / or prompt data by using a machine learning model; upon determining the presence of the prompt data from at least one of the first and second inputs, transmitting, by the input fusion system, the prompt data to an action generation system; generating, by the action generation system, first action data based on the prompt data and the machine learning model; and executing, by the action generation system, a first action based on the first action data.SELECTED DRAWING: Figure 6
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to creating content in a computing environment, and more particularly to systems and methods for performing collaborative content creation using artificial intelligence to enable multiple users to co-create content in a virtual environment with enhanced creative capabilities and / or effects. [Background technology]

[0002] Current technology allows human users to provide specific commands (e.g., through voice or typing) to computer programs that contain artificial intelligence (AI) to enable the AI ​​to perform certain activities. Some of the activities performed by AI are creative activities that require a certain level of training of the AI ​​algorithms. For example, existing AI platforms (e.g., Siri, ChatGPT, DALL-E, Mid-Journey, Stable Diffusion, etc.) all train computer models that can be used to create content that is responsive to user requests.

[0003] Of note, humans are inherently social beings, and often human creative capabilities are enhanced when they interact together to create content. However, current systems and methods do not enable humans to co-create with the help of AI computer models. Instead, current systems and methods focus on creation occurring at an individual level. Thus, the social and creative aspects of creation that can be accomplished collaboratively among multiple users with the help of evolving AI technologies are limited. For example, technologies utilizing current AI platforms (e.g., Siri, ChatGPT, DALL-E, Mid-Journey, Stable Diffusion, etc.) can convert speech to text and submit the text to a server, where the speech-to-text conversion can occur at the edge or in the cloud. The server can then send the text to the AI. Alternatively, a user can manually type and input one or more prompts into a computer to be sent to the AI, and some systems can create images or various types of content based on the user's input. However, all existing systems only take input commands from a single user and convert those input commands into corresponding actions. That is, none of the existing systems allow for the collaborative and creative aspects of creating content by multiple users. Summary of the Invention

[0004] The present disclosure is directed to overcoming one or more of these aforementioned problems and deficiencies. The background discussion provided herein is for the purpose of generally presenting the context of the present disclosure. Unless otherwise stated herein, the material described in this section is not prior art to the claims in this application, and is not admitted to be prior art or to suggest prior art by inclusion in this section.

[0005] In accordance with certain aspects of the present disclosure, systems and methods are provided for performing collaborative content creation using artificial intelligence to enable multiple users to co-create content in a virtual environment with enhanced creative capabilities and / or effects.

[0006] According to one aspect, a method is provided for performing collaborative content creation in a computing environment using artificial intelligence, which may include receiving, by a content creation system, a first input from a first user and a second input from a second user, analyzing, by an input fusion system, the first input and the second input by using a machine learning model to identify the presence of duplicate data, redundant data, and / or prompt data, transmitting, by the input fusion system, the prompt data to an action generation system upon identifying the presence of prompt data from at least one of the first input or the second input, generating, by the action generation system, first action data based on the prompt data and the machine learning model, and performing, by the action generation system, a first action in the computing environment based on the first action data.

[0007] In other aspects, any of the methods described herein may include any of the following steps or features: Upon identifying the presence of duplicate data in at least one of the first input or the second input, the input fusion system may remove the duplicate data from at least one of the first input or the second input. Upon identifying the presence of duplicate and redundant data in at least one of the first input or the second input, the input fusion system may remove the duplicate and redundant data from at least one of the first input or the second input. At least one of the first input or the second input may include text data. The input fusion system may convert at least one of the first input or the second input into transformed data. At least one of the first input or the second input may include at least one of image data, audio data, or tactile data. The transformed data may include transformed text data. Upon identifying the presence of duplicate or redundant data, the input fusion system may remove text data corresponding to at least one of the duplicate or redundant data from the transformed text data. The machine learning model may facilitate identifying the presence of duplicate data, redundant data, and / or prompt data based on direct or indirect user input. The computing environment may be a virtual environment. The computing environment may be an augmented environment. The first action may be creating or modifying an element within a space of the virtual environment. The element may be at least one of a visual element or an audio element. The first input and the second input may be received synchronously. The first input and the second input may be received asynchronously. The content creation system may generate a signal to display a graphical interface to the first user or the second user. The content creation system may receive a selection command from the first user or the second user. The content creation system may provide the user-created element in the computing environment based on the selection command. The graphical interface may include an adjustable timeline. The selection command may select a time period on the adjustable timeline.The user-created elements may be selected by the content creation system based on a time period.

[0008] According to one aspect, a computer system for performing collaborative content creation in a computing environment using artificial intelligence may be provided. The computer system may include a memory storing instructions and one or more processors configured to execute the instructions to perform operations. The operations may include receiving, by the content creation system, a first input from a first user and a second input from a second user, analyzing, by an input fusion system, the first input and the second input by using a machine learning model to identify the presence of duplicate data, redundant data, and / or prompt data, transmitting, by the input fusion system, the prompt data to an action generation system upon identifying the presence of prompt data from at least one of the first input or the second input, generating, by the action generation system, first action data based on the prompt data and the machine learning model, and performing, by the action generation system, a first action in the computing environment based on the first action data.

[0009] In other aspects, any of the systems described herein may include any of the following steps or features: Upon identifying the presence of duplicate data, the input fusion system may remove the duplicate data from at least one of the first input or the second input. Upon identifying the presence of duplicate and redundant data, the input fusion system may remove the duplicate and redundant data from at least one of the first input or the second input. At least one of the first input or the second input may include text data. The input fusion system may convert at least one of the first input or the second input into converted data. At least one of the first input or the second input may include at least one of image data, audio data, or tactile data. The converted data may include converted text data. Upon identifying the presence of duplicate or redundant data, the input fusion system may remove text data corresponding to at least one of the duplicate or redundant data from the converted text data. The machine learning model may facilitate identifying the presence of duplicate data, redundant data, and / or prompt data based on direct user input or indirect user input. The computing environment may be a virtual environment. The computing environment may be an augmented environment. The first action may be creating or modifying an element within a space of the virtual environment. The element may be at least one of a visual element or an audio element. The first input and the second input may be received synchronously. The first input and the second input may be received asynchronously.

[0010] According to one aspect, a non-transitory computer-readable medium can store instructions that, when executed by one or more processors of the computer system, cause the computer system to perform a method for performing collaborative content creation in a computing environment using artificial intelligence. The method can include receiving, by the content creation system, a first input from a first user and a second input from a second user, analyzing, by an input fusion system, the first input and the second input by using a machine learning model to detect at least one of duplicate data, redundancy data, or prompt data, transmitting, by the input fusion system, the prompt data to an action generation system upon detecting prompt data from at least one of the first input or the second input, generating, by the action generation system, first action data based on the prompt data and the machine learning model, and performing, by the action generation system, a first action in the computing environment based on the first action data.

[0011] According to one aspect, a method may be provided for performing content creation in a computing environment using artificial intelligence, which may include receiving, by a content creation system, a first input and a second input, analyzing, by an input fusion system, the first input and the second input by using a machine learning model to identify the presence of duplicate data, redundant data, and / or prompt data, transmitting, by the input fusion system, the prompt data to an action generation system upon identifying the presence of prompt data from at least one of the first input or the second input, generating, by the action generation system, first action data based on the prompt data and the machine learning model, and performing, by the action generation system, a first action in the computing environment based on the first action data.

[0012] In other aspects, any of the methods described herein may include any of the following steps or features: The first input and the second input may be received asynchronously. The first input and the second input may be received synchronously.

[0013] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.

[0014] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary aspects of the present disclosure and, together with the description, serve to explain the principles of the disclosure. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 illustrates an overview of an exemplary environment in which the systems, methods, and other aspects of the present disclosure may be implemented. [Diagram 2] FIG. 1 is a block diagram of an exemplary system for generating content creation data in a computing environment, according to an aspect of the present disclosure. [Diagram 3] 1 is a flowchart of an exemplary method for performing collaborative content creation using artificial intelligence in a computing environment, according to an aspect of the present disclosure. [Figure 4] FIG. 1 illustrates an exemplary computing environment for performing collaborative content creation using artificial intelligence, in accordance with aspects of the present disclosure. [Diagram 5] FIG. 1 illustrates an exemplary user interface for performing collaborative content creation using artificial intelligence in a computing environment, according to an aspect of the present disclosure. [Figure 6] 1 is a flowchart of another exemplary method for performing collaborative content creation in a computing environment, according to an aspect of the present disclosure. [Figure 7] FIG. 1 illustrates a computer system that may implement the techniques described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] In the following embodiments, systems and methods for performing collaborative content creation using artificial intelligence to enable multiple users to co-create content in a computing environment (e.g., an augmented reality environment, a virtual reality environment, and a two-dimensional (2D) or three-dimensional (3D) simulated environment) with enhanced creative capabilities and / or effects are described. As described above, existing systems utilize AI capabilities at an individual level. That is, a user can provide one or more input commands to an AI system, which provides a solution or response with a limited and one-way approach. Thus, there is currently no technical solution to facilitate collaborative content creation by multiple users in a computing environment that utilizes an artificial intelligence platform. Thus, there is a need for systems and methods that can facilitate a multi-directional, synchronous approach to achieving creative capabilities at a collaborative level that can enable co-creation among multiple users in a natural way. Such collaborative interactions among multiple users can create a sense that the users are simply having a conversation among each other. That is, the systems and methods of the present disclosure can automatically and intuitively detect key keywords during conversion and convert those keywords into prompts that can be translated into creative actions in the computing environment desired by the user (e.g., augmented reality environments, virtual reality environments, and two-dimensional (2D) or three-dimensional (3D) simulated environments). Additionally, these systems and methods can detect multiple forms of input, synchronously or asynchronously, from single or multiple users during one or more creative sessions to facilitate persistent AI-assisted processing of user inputs to provide iterative creative suggestions for personalized or collaborative content creation.Thus, the systems and methods disclosed herein significantly improve content creation and collaborative interaction of one or more users in a computing environment by providing intuitive and efficient processes and interfaces for enabling creative sessions driven by trained machine learning systems or modules in a non-traditional manner.

[0017] In the following embodiments, systems and methods are described for enabling multiple users to synchronously or asynchronously collaborate on content in a computing environment (e.g., an augmented reality environment, a virtual reality environment, and a two-dimensional (2D) or three-dimensional (3D) simulated environment) using artificial intelligence-based procedural technology. According to aspects of the present disclosure, these systems and methods may facilitate the incorporation of input data from one or more users. The input data may include, but is not limited to, audio data (e.g., voice, audio files, etc.), haptic data, text data, image data, video data, or any other type of data, synchronously or asynchronously. The systems and methods of the present disclosure may facilitate converting data to text data, analyzing the data, cleaning the data, extracting prompts from the data, and converting the prompts into creative actions having creative effects in the computing environment to generate creative works or content. According to aspects of the present disclosure, the process for cleaning the data may include, for example, removing unnecessary data from the text (e.g., removing duplication and redundancy). The creative effect may be graphical, which may include transforming one or more graphical images to make changes in shape, form, color, lighting (e.g., adding or removing light or light intensity), dimensions, elements (e.g., furniture), etc. in a computing environment. Additionally or alternatively, the creative effect may be based on musical compositions, artwork composition, textbooks, video game creation, architectural design, book creation, manufacturing of articles, and / or graphic design (e.g., creating virtual environments for businesses).

[0018] A computing environment or a particular area of ​​a computing environment may function similarly to a blank canvas on which creative effects may be applied. Such creative effects may be provided in real time or near real time on one or more of the surfaces or spaces. Additionally, the disclosed systems and methods may provide one or more creative suggestions to a user of a computing environment, where each suggestion may be targeted to a particular user based on the particular input of that user. Thus, the disclosed systems and methods improve the manner in which one or more users can create content and interact with other users in a computing environment by utilizing one or more machine learning models.

[0019] The subject matter of the present description will now be described more fully hereinafter with reference to the accompanying drawings, which form a part of this description and show, by way of illustration, certain exemplary embodiments. An embodiment or implementation described herein as "exemplary" should not be construed as preferred or advantageous over other embodiments or implementations, for example, but rather is intended to reflect or show that the embodiment(s) are "exemplary" embodiment(s). The subject matter can be embodied in a variety of different forms, and as such, the subject matter as is claimed or claimed is not intended to be construed as being limited to any exemplary embodiment described herein, which exemplary embodiments are provided for illustrative purposes only. Likewise, a reasonably broad scope of the subject matter as is claimed or claimed is intended. For example, the subject matter can be embodied as a method, device, component, or system, among other things. Thus, the embodiments can take the form of, for example, hardware, software, firmware, or any combination thereof (excluding software itself). As such, the following detailed description is not intended to be taken in a limiting sense.

[0020] Throughout this specification and the claims, terms may have a nuanced meaning beyond that explicitly stated, as suggested or implied by the context. Similarly, the phrase "in one embodiment" as used herein does not necessarily refer to the same embodiment, and the phrase "in another embodiment" as used herein does not necessarily refer to a different embodiment. For example, it is intended that the claimed subject matter includes, in whole or in part, a combination of the exemplary embodiments.

[0021] The terminology used hereinafter may be interpreted in its broadest reasonable manner, even if the term is used in conjunction with the detailed description of a particular specific example of the present disclosure. Indeed, even if a particular term is emphasized hereinafter, any term intended to be interpreted in any limited manner will be expressly and specifically defined as such in this detailed description section. Both the foregoing general description and the following detailed description are exemplary and explanatory only and are not limiting of the features claimed.

[0022] In this disclosure, the term "based on" means "based at least in part on." The singular forms "a," "an," and "the" include plural referents unless the context indicates otherwise. The term "exemplary" is used in the sense of "example" and not "ideal." The term "or" is intended to be inclusive and means either, any, some, or all of the listed items. The terms "comprises," "comprising," "includes," "including," or other variations thereof are intended to describe a non-exclusive inclusion, whereby a process, method, or product comprising a list of elements does not necessarily include only those elements, but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Relative terms such as "substantially" and "generally" are used to indicate possible variations of ±10% of the stated or understood value.

[0023] Referring now to the accompanying drawings, FIG. 1 illustrates an overview of an exemplary environment (or system(s)) 100 in accordance with one or more embodiments of the present disclosure. Environment 100 may include, for example, first user device(s) 110 and second user device(s) 120, which are configured to communicate with network system(s) 130 and collaborative content creation system(s) 140. Although two user device(s) 110 and 120 are shown in environment 100, additional user devices may be provided in environment 100 for communicating, synchronously or asynchronously, with network system(s) 130 and / or collaborative content creation system(s) 140 and for participating in multi-user collaborative content creation, in accordance with the present disclosure.

[0024] According to aspects of the present disclosure, the network system(s) 130 may define one or more networks, including wired or wireless networks. The network system(s) 130 may include, for example, the Internet and / or one or more cloud networks. Furthermore, the network system(s) 130 may include a public network, such as the Internet, a private network, such as an intranet, or a combination thereof, and may utilize various currently available or later developed networking protocols, including but not limited to TCP / IP-based networking protocols. The network system(s) 130 may be configured to communicatively couple the user device(s) 110 and 120 to the collaborative content creation system(s) 140 to enable communication of data between the user device(s) 110 and 120 and the collaborative content creation system(s) 140. The network system(s) 130 may generally be enabled to employ any form of computer-readable or machine-readable media for communicating information from one device to another. The network system(s) 130 may include communication methods by which information may travel between computing devices. The network system(s) 130 may be considered to be public or private network connections and may include, for example, a virtual private network, or encryption or other security mechanisms employed on the public Internet, etc.

[0025] In one embodiment, user device(s) 110 and 120 may communicate with collaborative content creation system(s) 140 directly or indirectly through network system(s) 130 or other available communication channels. In cases where user device(s) 110 and 120 communicate directly with collaborative content creation system(s) 140, collaborative content creation system(s) 140 may be implemented and configured to facilitate communication via one or more communication methods described in connection with network system(s) 130 above, for example.

[0026] According to aspects of the present disclosure, collaborative content creation system(s) 140 may include server system(s) 142, input fusion system(s) 144, generative AI system(s) 146, and computing environment generation system(s) 148. In some embodiments, collaborative content creation system(s) 140 may be one or more servers that may be configured to perform some or all of the functions of server system(s) 142, input fusion system(s) 144, generative AI system(s) 146, and computing environment generation system(s) 148 according to aspects of the present disclosure. The system(s), in this disclosure, may include various implementations that may include various electronic and computer systems. One or more of the implementations described herein may implement functions using two or more specific interconnected hardware modules or devices, with associated control and data signals that can be communicated between and through the modules, or as part of an application specific integrated circuit. Thus, the system(s) encompass software, firmware, and hardware implementations.

[0027] According to aspects of the present disclosure, the server system(s) 142 may include one or more data servers or databases that may be located on a cloud server / network, an edge server / network, within the network system(s) 130, and / or where the server system(s) 142 may be directly or indirectly integrated with the collaborative content creation system(s) 140. The server system(s) 142 may store and process data received from the user device(s) 110 and / or the network system(s) 130 to facilitate the performance of collaborative content creation in accordance with embodiments of the present disclosure. In addition, the server system(s) 142 may receive and transmit data or command signals to and from the input fusion system(s) 144, the generative AI system(s) 146, and the computing environment generation system(s) 148 to facilitate the performance of collaborative content creation.

[0028] According to aspects of the present disclosure, the input fusion system(s) 144 may receive one or more inputs or commands from the first user 112 and the second user 122. As described above, the environment 100 may include more than two users or more than two user devices. Thus, the input fusion system(s) 144 may receive inputs and commands from more than two users or more than two user devices. Alternatively, the input fusion system(s) 144 may receive one or more inputs or commands only from the first user 112 or the second user 112 depending on the type of collaborative session. That is, the collaborative content creation system(s) 140 may facilitate a single-user creation session or a multi-user creation session. In the present disclosure, the terms input and command may be used interchangeably. The user input or command may include, for example, but is not limited to, an audio input, a tactile input, a text input, an image input, a gesture input, or a video input. That is, any type of data that is receptive, for example, via sensors or other suitable means connected to or integrated with user devices in environment 100, may be considered to be a user input or command according to the present disclosure. A user command may be a direct or indirect command. For example, a direct command may refer to a command that may be configured to instruct collaborative content creation system(s) 140 to execute a command (e.g., changing a particular color to a selected area) that has been directly input by one or more users. Conversely, an indirect command may refer to a command that may be configured to instruct collaborative content creation system(s) 140 to infer, interpret, and extract information from a command (e.g., what is being said by a user). For example, a direct command may require less learning, analysis, and interpretation by collaborative content creation system(s) 140 because a user can simply select a particular area of ​​the creation and send the command directly. However, an indirect command may require more training (e.g., training a machine learning model).Because the collaborative content creation system(s) 140 take in the data and try to extract clean commands that may be as close as possible to what is being interpreted by the AI ​​or machine learning model. For example, indirect commands may be primarily used when one or more users are interacting among themselves in a creative session where the users may be brainstorming on what they think would be their creative work. Or, a single user may be speaking alone in a creative session to brainstorm on what they might think would be their creative work.

[0029] In one embodiment, the first user 112 and / or the second user 122 may input commands synchronously or asynchronously to the first user device(s) 110 and the second user device(s) 120, respectively. The input fusion system(s) 144 may receive the input commands from the user device(s) 110 and 120 directly or indirectly via the network system(s) 130 and / or the server system(s) 142. The input fusion system(s) 144 may then transcribe or convert the input commands into text data. For input commands that may be received in the form of text data, conversion to text data by the input fusion system(s) 144 may not be necessary. The input fusion system(s) 144 may then analyze the converted or received text data depending on the type or form of the input command by capturing the semantics of the text data (e.g., understanding the conversational context), detecting duplications and / or redundancies (e.g., unnecessary information), labeling the text data, and / or detecting prompts. The input fusion system(s) 144 may then clean or modify the analyzed text data, if necessary, for example, by removing detected duplications and / or redundancies. The input fusion system(s) 144 may then proceed to extract prompts, if any, from the analyzed and cleaned text data. In some embodiments, the input fusion system(s) 144 may utilize the trained machine learning model. The input fusion system(s) 144 may then send the extracted clean prompts to the generation AI system(s) 148. In some embodiments, the conversion or transcription of data may be performed on the user device(s) 110, 120.Thus, the user device(s) 110, 120 may implement computer code that enables conversion of audio or other types of data into text and transmit the transcribed text to the collaborative content creation system(s) 140. Alternatively, as described above, the transcription or conversion of data can be performed within the collaborative content creation system(s) 140 by the input fusion system(s) 142, such that the input device(s) 110, 120 may not need to perform any data transcription or conversion. Alternatively, all of the functions of the input fusion system(s) 142 may be performed by the generative AI system(s) 146, including the transcription or conversion of text, analysis of input or command data, and / or cleaning of input or command data.

[0030] In one embodiment, the labeling process may be performed by the input fusion system(s) 144 for any type of input or command data, such as image data, haptic data, video data, audio data, and / or other types of data. For data that does not include text data, the labeling process may be considered a type of transcription of commands into text. For example, if an image provided by a user (e.g., user 112, 122) including a moon in space is received by the input fusion system(s) 144, the input fusion system(s) 144 may generate labels including, e.g., moon, glowing, white, dark space, etc., which may be converted into actual prompts and creative works. As another example, in a music creation session where a classical music performance is uploaded, the input fusion system(s) 144 may generate labels including, e.g., harmonic music, classical music, soft music, etc., which may be converted into actual prompts and creative works. For data received in text, instead of performing the labeling process described above, a classification process may be performed as well. For example, if text data including a moon in space is received by the input fusion system(s) 144, the input fusion system(s) 144 may classify the text data as, for example, moon, shiny, white, dark space, etc. In one embodiment, the input fusion system(s) 144 may receive both text data and non-text data synchronously from one or more users. In this embodiment, the input fusion system(s) 144 may perform both transcription and classification simultaneously or sequentially. In one example, the labeling or classification process includes adding text that identifies the author of the received input (e.g., user 112 or 122). Such identification may be performed by one or more of recognizing the user device of the corresponding user, recognizing the user's voice or image (e.g., through facial recognition), etc.Additionally, the input fusion system(s) 144 can compare the information analyzed from the text data with the information analyzed from the non-text data to generate labels or categories that can be converted into actual prompts and creative works. For example, one or more labels identified from the non-text data and one or more classifications identified from the text data can be processed together by the input fusion system(s) 144 to effect conversion into prompts and creative works. The conversion and classification processes described above can be performed together simultaneously or sequentially.

[0031] According to aspects of the disclosure, the generative AI system(s) 148 may receive one or more prompts from the input fusion system(s) 144. In one embodiment, the generative AI system(s) 148 may generate one or more of the creative suggestions based on the one or more prompts received from the input fusion system(s) 144. Each suggestion may be relevant to a particular user based on the particular input of the particular user. For example, if the first user 112 sends a command to modify the color of the wall to red and the second user 122 sends a command to modify the color of the wall to blue, the generative AI system(s) 148 may recognize the difference between the commands based on the prompts received from the input fusion system(s) 144. The generative AI system(s) 146 may then label the first command as that of the first user 112 and the second command as that of the second user 122. The generative AI system(s) 146 may then proceed to prepare a first suggestion corresponding to the first user 112 and a second suggestion corresponding to the second user 122. Of course, the same or similar processes or methods for preparing suggestions may be applied to various other types of user commands (e.g., pictorial, textual, tactile, etc.). The generative AI system(s) 146 may then transmit the suggestions to the computing environment generation system(s) 148.

[0032] In one embodiment, the generative AI system(s) 146 may include one or more generative AI models that utilize input data received from the input fusion system(s) 144 and generate output data that may be used to facilitate the functioning of the generative AI system(s) 146. The generative AI models may include, for example, a combination of various AI algorithms used to represent and process content. Additionally, to generate one or more creative suggestions, the generative AI models may utilize natural language processing techniques to generate text to convert raw characters, such as letters, punctuation, and words, into sentences, parts of speech, entities, and actions, which may then be represented as vectors using multiple encoding techniques. Additionally, images may be converted into various visual elements, also represented as vectors, to be used by the generative AI system(s) 146 as one or more creative suggestions. In one embodiment, a specific neural network may be used to generate new content in response to a query or prompt. Techniques such as generative inverse networks (GAN) and variational autoencoders (VAE) may be utilized to generate one or more of the creative suggestions by the generative AI system(s) 146. In one embodiment, the AI ​​algorithms of the present disclosure may be trained with a pre-specified number of datasets prior to application of the AI ​​algorithms in the AI ​​system(s) 146. Additionally, the AI ​​algorithms may continue to learn during creative sessions performed in the environment 100. For example, the AI ​​system(s) 146 may learn from each user's creative style after each creative session. The AI ​​system(s) 146 may then improve the AI ​​algorithms through machine learning, such that the AI ​​system(s) 146 may generate and provide creative suggestions faster and / or with better quality.

[0033] According to aspects of the present disclosure, the computing environment generation system(s) 148 may facilitate the generation of one or more computing environments that users 112, 122 in the environment 100 may utilize to collaborate to create content. The computing environments may include, but are not limited to, for example, augmented reality environments, virtual reality environments, and two-dimensional (2D) or three-dimensional (3D) simulated environments. Examples of computing environments and user collaboration in computing embodiments are also disclosed in co-pending U.S. patent application Ser. No. 17 / 006,327, the entire disclosure of which is incorporated herein by reference, except for any disclaimers, disclaimers, and contradictions. Additionally, any of the machine learning algorithms disclosed in that co-pending application may be incorporated for use individually or in combination with the machine learning models used by the input fusion system(s) 144 and / or the generative AI system(s) 148.

[0034] In one embodiment, the computing environment generation system(s) 148 may facilitate the generation of one or more computing environments in an area of ​​an already existing virtual environment.

[0035] In one embodiment, the computing environment generating system(s) 148 may receive, for example, user commands, prompts, and / or suggestions from the user device(s) 110 and 120, the input fusion system(s) 144, and / or the generative AI system(s) 146. The computing environment generating system(s) 148 may then generate graphical and / or audio interfaces that users of the environment 100 may utilize to interact with each other to create content. For example, the computing environment generating system(s) 148 may generate one or more elements in the virtual environment through one or more user graphical representations (UGRs) or audio representations. These UGRs or audio representations may be created from real-time camera or microphone feeds captured by one or more cameras or microphones, for example, cameras or microphones integrated with the user devices (e.g., the user device(s) 110, 120) or cameras or microphones separate from but electrically connected to the user devices. UGRs can help users collaborate with each other to create content in a virtual environment. For example, if a first user 122 makes a movement that can be detected by the user device(s) 110, a UGR associated with the first user 112 in the virtual world can move or create one or more effects based on the movement made by the first user 112. Similarly, if a second user 122 makes a movement that can be detected by the user device(s) 120, another UGR associated with the second user 122 can move or create one or more effects based on the movement made by the second user 122.The input fusion system(s) 144 and / or the generative AI system(s) 146 may utilize one or more machine learning models to clean, analyze, and / or extract prompts to provide suggestions to the first user 112 and / or the second user 122 based on movements made by the first user 112 and the second user 122. Thus, the collaborative content creation system(s) 140 may facilitate the execution of collaborative content creation using artificial intelligence to enable multiple users to co-create content in a computing environment.

[0036] 2 illustrates a block diagram of an exemplary system 200 for generating content creation data 220 according to an embodiment of the present disclosure. The collaborative content creation system(s) 140 of the environment 100 illustrated in FIG. 1 may receive user data 210 from one or more users. In this embodiment, the collaborative content creation system(s) 140 are illustrated with only the input fusion system(s) 144 and the generative AI system(s) 146 for clarity of illustration and explanation. However, the collaborative content creation system(s) 140 may include all or some of the system(s) illustrated in FIG. 1.

[0037] Still referring to FIG. 2, the user data 210 may be generated by one or more users when they provide input or commands to one or more user devices (e.g., user device(s) 110, 120). The user devices may generate the first through n-th user input data 210a-210n synchronously or asynchronously. That is, if more than two users provide input or commands substantially simultaneously, the user input data 210a-210n may be said to be generated synchronously. Conversely, if two or more users do not provide input or commands substantially simultaneously, the user input data 210a-210n may be said to be generated asynchronously. Thus, if two or more of the user input data 210a-210n are sent to the collaborative content creation system(s) 140 synchronously, the two or more input data 210a-210n may overlap with each other, and vice versa. As described above in connection with FIG. 1, the user data 210 may be transmitted directly to the collaborative content creation system(s) 140 or indirectly via the network system(s) 130.

[0038] In one embodiment, the user data 210 may include any of audio data (e.g., voice, audio files, etc.), tactile data, text data, image data, video data, or any other type of data, individually or in combination, that may be captured or sensed by one or more sensors communicatively connected to the user device (e.g., user device(s) 110, 120). Alternatively, or in addition, the user data 210 may be input directly by one or more users (e.g., by manually typing commands or uploading a text file). For example, in the case of inputting audio data, the audio data may be captured or sensed by one or more microphones of the client device. Alternatively, the audio data may be uploaded directly by the user, for example, as an audio file including pre-recorded music and / or voice. In another example, the input data may be an image file uploaded through the user device or captured by the user's camera for transmission to the collaborative content creation system(s) 140. The audio and / or image files may be analyzed by the input fusion system(s) 144 to extract any prompts within the audio and / or image data. With respect to haptic data, a user may touch one or more surfaces on an input device, augmented environment, or virtual environment with their hands. For example, a user may make an input selection by physically or virtually touching or clicking one or more surfaces to be modified or steered. In one embodiment, one or more users may paint specific areas within the computing environment to create content (e.g., paint brush strokes as 2D lines or 3D curves that may define specific areas for creation or refinement).

[0039] Still referring to FIG. 2 , the collaborative content creation system(s) 140 may include a conversion system(s) 242, a redundancy / duplication detection system(s) 244, a prompt extraction system(s) 246, and a model generation system(s) 248. In one embodiment, upon receiving user data 210 from multiple users or input devices, the input fusion system(s) 144 may perform data cleaning on the user data 210 using one or more machine learning models. As described in the previous embodiment, the user data 210 may be converted into text data by the conversion system(s) 242 depending on the form or type of information provided by the user. In one embodiment, the conversion system(s) 242 may detect whether the user data 210 contains commands sent by two or more input devices substantially simultaneously. Upon detecting receipt of substantially simultaneous user commands, the redundancy / duplication detection system(s) 244 may clean relevant information (e.g., text or non-text information) in the input data 210 in a manner that is easy and efficient for an AI model (e.g., a machine learning model) to understand or process. For example, cleaning of the input information may be performed based on the uploaded modality corresponding to the machine learning model. The machine learning model may be generated and / or trained by the model generation system(s) 248. If there are confusing or unrecognizable (or unintelligible) commands (e.g., because overlaps between commands may result in unclear data or due to noise that may disrupt the flow of commands), the redundancy / duplication detection system(s) 242 may flag those data bits as unclear and send one or more clarification messages to the user to gain clarity regarding that particular command or message. The AI ​​models utilized by the input fusion system(s) 144 may be generated by one or more model generation system(s) 248.

[0040] In one embodiment, the model generation system(s) 248 may generate and / or train one or more of the AI ​​models (e.g., machine learning models). The model generation system(s) 248 may be integrated as part of and reside within the input fusion system(s) 144. Alternatively, the model generation system(s) 248 may be located within the network system(s) 130, on a cloud server / network, or on an edge server / network, but is not limited thereto. The model generation system(s) 248 may be a machine learning platform that may generate a library of components for generating machine learning models and machine learning applications. The model generation system(s) 248 may enable a user to generate machine learning applications without having detailed knowledge of a cloud-based network infrastructure or knowledge of how to generate code to build a model. The model generation system(s) 248 may analyze the identified data and the desired prediction and performance characteristics provided by the user to select one or more library components and associated application programming interfaces (APIs) for generating a machine learning application or model. The machine learning techniques may monitor and evaluate the output of the machine learning model to allow feedback and adjustments to the model. The machine learning application or model may be trained, tested, and compiled for export as standalone executable code. For example, the model generation system(s) 248 may utilize one or more large language models (LLMs) and / or machine learning models that are utilized to perform outpainting. In one embodiment, the model generation system(s) 248 may generate and store one or more library components that may be used for other machine learning applications, models, and / or systems (e.g., the generative AI system(s) 146).The model generation system(s) 248 may generate profiles that enable the input fusion system(s) 144 to make recommendations based on, for example, a user's past preferences. The generation system(s) 248 may discover the number and type of infrastructure resources required to achieve the desired results within the desired performance criteria.

[0041] In one embodiment, a single machine learning model generated by the model generation system(s) 248 may be utilized simultaneously or sequentially by the transformation system(s) 242, the redundancy / duplication detection system(s) 244, and the prompt extraction system(s) 246. Alternatively, multiple machine learning models generated by the model generation system(s) 248 may be utilized simultaneously or sequentially by the transformation system(s) 242, the redundancy / duplication detection system(s) 244, and the prompt extraction system(s) 246. Thus, the input fusion system(s) 144 may facilitate synchronous or asynchronous creative sessions in a computing environment by one or more users. Unlike traditional AI platforms that generally require a single command or request to be processed in a unilateral manner by an AI model, the systems and methods of the present disclosure facilitate the processing of multiple synchronous or asynchronous inputs provided by one or more users. For example, one or more users participating in content creation in a computing environment may provide inputs synchronously or asynchronously. Collaborative content creation system(s) 140 may process all of the inputs synchronously or asynchronously in a persistent manner, such that collaborative content creation system(s) 140 provided creative effects and / or suggestions based on one or more machine learning models before, during, and / or after inputs are provided by one or more users in one or more creative sessions.

[0042] Still referring to FIG. 2 , the redundancy / duplication detection system(s) 144 may detect one or more redundancies (e.g., unnecessary information) and / or duplications in the user data 210. For example, the redundancy / duplication detection system(s) 144 may detect repeated words and prompts in the user data 210, thereby leaving only information associated with content that adds value to a creative work of a single user or among multiple users in a computing environment. For example, if there are three users having a creative dialogue or conversation and two of the users are saying the same thing, the redundancy / duplication detection system(s) 244 may detect the redundancy and / or duplications, for example, by utilizing machine learning model(s) generated by the model generation system(s) 248, and enable the prompt extraction system(s) 246 to extract only the prompts that add value to the creative work. In one embodiment, the detection of redundancy and / or duplications may be performed before or after the cleaning process described in the previous embodiment. Additionally, if some of the users use filler words, repeat sentences, or make sounds that do not add value to the creative work among multiple users in the computing environment, the redundancy / duplication detection system(s) 144 may proceed to discard the redundancies and / or duplications in the user data 210 before extracting any prompts that may be present in the user data 210 by the prompt extraction system(s) 246. In this manner, the generative AI system(s) 146 may receive only those commands that are useful to the creative session, thereby increasing the efficiency and accuracy of identifying creative effects and suggestions based on user input.

[0043] 2, the generative AI system(s) 146 may include suggestion / effects generation system(s) 252 and model generation system(s) 254. The suggestion / effects generation system(s) 252 may generate content generation data 220 for generating creative effects in a computing environment (e.g., augmented reality environments, virtual reality environments, two-dimensional (2D) or three-dimensional (3D) simulated environments, etc.). 4 1 and 2. To aid in understanding the features of collaborative content creation according to embodiments of the present disclosure, the features of FIG. 4 Virtual environment in 400 Of course, any computing environment (e.g., augmented reality environments, virtual reality environments, two-dimensional (2D) or three-dimensional (3D) simulated environments, etc.) may be applicable to facilitate collaborative content creation according to embodiments of the present disclosure.

[0044] In one embodiment, the virtual environment 400 may be generated by a computing environment generation system (e.g., computing environment generation system(s) 148) according to content creation data 220 generated by generative AI system(s) 146. For simplicity, the virtual environment 400 Details of generating a virtual environment are omitted. A detailed description of generating computing environments (e.g., augmented reality environments, virtual reality environments, and two-dimensional (2D) or three-dimensional (3D) simulated environments, etc.) is provided in co-pending U.S. patent application Ser. No. 17 / 006,327, which is incorporated herein by reference in its entirety. Virtual Environments 400 is the first User Graphical Display (UGR) 402 , the second U.G.R. 404 , and the third UGR 406 UGR 402 ~ 406may be generated based on data or signals received from one or more cameras capturing images of the first, second, and third users. 402 ~ 406 The image may be generated based on an image removal process that removes background images, such that only images representing the user are left in the virtual environment. 400 This may allow the device to be inserted into the

[0045] In one embodiment, the virtual environment 400 is a space (or surface) 410 Space 410 may be provided to facilitate collaborative content creation by one or more users. That is, one or more users may create, for example, a UGR 402 ~ 406 Through space 410 For example, UGR 402 ~ 406 By element 408 In one embodiment, the space may be a creative space in the virtual environment. However, the creative space may also be a selected area, space, or object of the virtual environment. For example, the creative space may be a wall, an actual art canvas (for art paintings), a city model, a manufactured item, a book, etc., in which case all commands may be sent and processed with respect to that selected area, space, or object. 4 Although a table is shown in, any space or surface may be utilized to facilitate collaborative content creation according to embodiments of the present disclosure. For example, multiple users may collaborate to create elements 408 A virtual environment that allows you to create (e.g., drawings, paintings, writings, etc.) 400 Thus, the collaborative content creation system(s) 140 may be provided with a "blank canvas" to create a virtual environment. 400 of spaces and surfaces (e.g., 410) to facilitate or enable creative sessions to take place in a natural way, through conversations and / or through other means, in a way that creative changes can be seen in real or near real time.

[0046] 2, the content creation data 220 generated by the generative AI system(s) 146 may include, but is not limited to, graphical data 220a, audio data 220b, textual data 220c, and haptic data 220n. 4 Virtual environment in 400 In one embodiment, the content creation data 220 may include information or signals for providing creative effects in a virtual environment. 400 Elements in 408 This makes it easier to make changes to the element 408 Changes to the element 408 This may include changing the shape, form, color, brightness (e.g., adding or removing light or light intensity), dimensions, etc. of the element. 408 is a virtual environment 400 In addition, the element 408 may be expressed as, but is not limited to, a musical piece, a composition of art, a textbook, or an article of manufacture.

[0047] Still referring to FIG. 2, the suggestion / effect generation system(s) 252 may generate one or more creative suggestions for users, where each suggestion may be attributed to a particular user based on the particular input of that user. For example, the suggestion / effect generation system(s) 252 may label input commands received from multiple users based on each user providing or transmitting a particular input command. For example, if user A (e.g., first user 112) transmits a command to modify the color of a wall in a computing environment (e.g., an augmented reality environment, a virtual reality environment, and a two-dimensional (2D) or three-dimensional (3D) simulated environment) to red, and user B (e.g., second user 122) transmits a command to modify the color of the wall to blue, the suggestion / effect generation system(s) 252 may recognize those two separate commands. The suggestion / effect generation system(s) 252 can then label the first command as that of user A and the second command as that of user B, and can generate one or more suggestions corresponding to user A and different suggestions corresponding to user B. Of course, this process can be applied to other types of commands (e.g., image, text, haptic, etc.) as well.

[0048] In one embodiment, the suggestion / effect generation system(s) 252 may generate a representation of one or more surfaces or spaces (e.g., spatial) of a computing environment (e.g., augmented reality environments, virtual reality environments, and two-dimensional (2D) or three-dimensional (3D) simulated environments). 410) to generate content creation data 220 for facilitating one or more creative effects on the virtual environment. In this embodiment, the suggestion / effect generation system(s) 252 may continually refine the creative work as additional data is received from the user of the computing environment. That is, the suggestion / effect generation system(s) 252 may continually analyze user input data to generate creative effects and suggestions based on contextual iterations. For example, after the suggestion / effect generation system(s) 252 generates data or signals (e.g., content creation data 220) for providing a creative effect in the computing environment including all or most of the features based on the command submitted by the user, the generative AI system(s) 146 may review and continually further refine the current creative content as the user continues the creative session. Additionally, the user may also select a particular area for creating and refining creative content. For example, the user may select a particular area for creating and refining creative content on the virtual environment. 400 A user may make a selection by clicking, touching, painting brush strokes as 2D lines or 3D curves (e.g., to define a specific area for creation or refinement), typing text, or speaking to a particular area, object, volume, or space of the image, and then send his / her command. In some embodiments, the generative AI system(s) 146 may apply enhancement techniques independently after a creative session is over, based on sustained iterations. Enhancement techniques may include, for example, improving color, contrast, sharpness, brightness, smoothing surfaces, applying occlusion or other refinement techniques to a finalized or semi-finalized creative work. Similarly, for audio creation, enhancement techniques may include noise reduction, volume increase, tone enhancement, etc.

[0049] In one embodiment, the generative AI system(s) 146 may be configured to facilitate a real-time creative session in which a creative work is performed as one or more users speak. However, in some embodiments, if the desired creative work requires further processing, the generative AI system(s) 146 may notify the user of an estimated time of completion and may proceed to take the necessary time, either online or offline. For example, if the user finishes the creative session, the user may later receive a notification indicating that the collaborative content creation system(s) 140 have completed the creative work, and the user may access the completed creation for review and / or revision. Thus, the systems and methods of the present disclosure significantly improve the content creation and collaborative interaction of one or more users in a computing environment by utilizing the input fusion system(s) 144 and the generative AI system(s) 146 to perform synchronous or asynchronous processing of user inputs through sustained contextual iteration to generate creative suggestions and effects before, during, and after the creative collaborative session.

[0050] figure 3 teeth, Figure 1, Figure 2, Figure 4 An exemplary method for performing collaborative content creation using artificial intelligence in a computing environment (e.g., an augmented reality environment, a virtual reality environment, and a two-dimensional (2D) or three-dimensional (3D) simulated environment) according to the above-described embodiments described in relation to 300 1 shows a flowchart of the steps. 302 In this embodiment, a collaborative content creation system (e.g., collaborative content creation system(s) 140) may receive multi-user input commands from multiple users (e.g., users 112, 122). 3 The method shown in 300 To facilitate Figure 1, Figure 2, Figure 4 The method may utilize one or more of the system(s) and / or modules described in connection with the steps of:304 In step S200, the collaborative content creation system may transcribe the multi-user input commands into text. For example, the collaborative content creation system may convert the input commands into text for commands that are not provided in text form. These commands may be in the form of, for example, but not limited to, audio input, tactile input, image input, gesture input, or video input. 306 So, the collaborative content creation system is Figure 1, Figure 2, Figure 4 According to the above-described embodiment related to step 1, the converted text may be analyzed to detect redundancies and overlaps, to detect prompts, and / or to perform labeling of the text data. 308 Then, the collaborative content creation system may clean the analyzed and converted text, i.e., if any redundancies and duplications are detected in the converted text, the collaborative content creation system may remove the detected redundancies and duplications from the converted text. Furthermore, if necessary, labeling may be performed by the collaborative content creation system based on the form of the input commands converted into text. 310 The collaborative content creation system may then extract one or more prompts from the cleaned and converted text. The prompts may include information or data to facilitate application of creative effects in a computing environment. The computing environment may be any computing environment disclosed in connection with the preceding embodiments of the present disclosure (e.g., an augmented reality environment, a virtual reality environment, and a two-dimensional (2D) or three-dimensional (3D) simulated environment). 312 In one embodiment, the collaborative content creation system may execute prompts in the form of creative actions or effects in a computing environment. In accordance with an embodiment of the present disclosure, steps are provided to facilitate collaborative content creation by multiple users in a computing world. 304 ~ 312 One or more machine learning models may be used.

[0051] FIG. 5 illustrates an exemplary user interface 500 for performing collaborative content creation using artificial intelligence in a computing environment according to aspects of the disclosure. The user interface 500 may be provided in any of the computing environments described in connection with FIGS. 1-4 (e.g., an augmented reality environment, a virtual reality environment, and a two-dimensional (2D) or three-dimensional (3D) simulated environment). In an embodiment, the user interface 500 may be generated by the computing environment generation system(s) 148 based on data received from the input fusion system(s) 144 and / or the generative AI system(s) 146. The user interface 500 may be configured to be operated or navigated by multiple users 502, 504, 506. One or more of the users 502, 504, 506 may operate a slider 512 on a timeline in the user interface 510 as shown in FIG. 5. In some embodiments, a button, key, or other suitable graphical or audio element may be utilized in place of the slider 512. The timeline may include one or more time positions 514, 516, 518, where a user may position the slider 512 forward or backward on the timeline to review or modify creative effects made at a particular time period. For example, the time position 514 may define a prompt (e.g., prompt 1) that is associated with a position (e.g., X, Y, Z coordinates) in a time period (e.g., 3:00 minutes). In one embodiment, the computing environment generation system(s) 148 may generate one or more prompts associated with the selected time position (e.g., 514, 516, or 518). The user (502, 504, and / or 506) may then select a prompt, for example, by clicking on the prompt, which may be provided as a graphical element within the computing environment.Thus, the collaborative content creation system(s) 140 may enable users of the computing environment to slide back and forth through a timeline and click prompts or commands to perform persistent, repetitive functions to facilitate additional content creation or modification.

[0052] 6 illustrates a flowchart of another exemplary method 600 for performing collaborative content creation in a computing environment using artificial intelligence, according to aspects of the present disclosure. In particular, the method 600 may be performed in accordance with the embodiments described in connection with FIGS. 1-5. For example, the method 600 may be performed by one or more systems in the environment 100 and the system 200. Additionally, the method 600 may be performed in a virtual environment. 400 5 and may utilize a user interface 500. In step 602, the content creation system may receive a first input from a first user and a second input from a second user. The content creation system may be the collaborative content creation system(s) 140 shown in FIG. 1 and FIG. 2. In one embodiment, the first user may be the first user 112 and the second user may be the second user 122 shown in FIG. 1. Alternatively, the first and second users may be any of the users 502, 504, 506 shown in FIG. 5 or the second user 122 shown in FIG. 6. 4 The UGR shown in 402 , 404 , 406 The first input may be any user, represented by . The first input or the second input may include user data 210 shown in FIG. 2. For example, the first input may be first user input data 210a and the second input may be second user input data 210b shown in FIG. 2. Each of the first and second inputs may include, for example, audio data (e.g., voice, audio files, etc.), haptic data, text data, image data, video data, or any other type of data. In one embodiment, the first input and the second input may be received synchronously or asynchronously.

[0053] At step 604, the input fusion system analyzes the first input and the second input by using a machine learning model to identify the presence of duplicate data, redundant data, and / or prompt data. The input fusion system may be the input fusion system(s) 144 shown in FIG. 1 and FIG. 2. The machine learning model may be generated from the model generation system(s) 248 or the model generation system(s) 258. In one embodiment, the machine learning model may facilitate identifying the presence of duplicate data, redundant data, and / or prompt data based on a direct user input or an indirect user input. In one embodiment, the input fusion system may convert at least one of the first input or the second input into transformed data. In this embodiment, at least one of the first input or the second input may include at least one of image data, audio data, or tactile data. Furthermore, the transformed data may include transformed text data. In one embodiment, once the presence of duplicate data or redundant data is identified, the input fusion system may remove at least one of the duplicate data or redundant data from at least one of the first input or the second input. In one embodiment, upon identifying the presence of duplicate or redundant data, the input fusion system may remove from the converted text data the text data that corresponds to at least one of the duplicate or redundant data.

[0054] Upon determining the presence of prompt data from at least one of the first input or the second input in step 606, the input fusion system may transmit the prompt data to an action generation system, which may be the generative AI system(s) 146 in FIGS.

[0055] In step 608, the action generation system may generate the first action data based on the prompt data and the machine learning model. Alternatively, the first action data may be generated based on a second machine learning model. In one embodiment, the second machine learning model may be generated by the model generation system(s) 254 or the model generation system(s) 248.

[0056] At step 610, the action generation system may perform a first action in the computing environment based on the first action data. In one embodiment, the computing environment may be at least one of a virtual environment or an augmented environment. Alternatively, the computing environment may be, but is not limited to, an augmented reality environment, a virtual reality environment, and a two-dimensional (2D) or three-dimensional (3D) simulated environment. In one embodiment, the first action may create or modify an element in a space of at least one of the virtual environment or the augmented environment. Furthermore, the element may be at least one of a visual element or an audio element. In one embodiment, the content creation system may generate a signal to display a graphical interface to the first user or the second user. The graphical interface may be the user interface 500. Furthermore, the content creation system may receive a selection command from the first user or the second user. In one embodiment, the selection command may be sliding a slider 512 or clicking a graphical element associated with a prompt. Furthermore, the content creation system may provide a user-created element in the computing environment based on the selection command. In one embodiment, the graphical controller is an adjustable timeline. In one embodiment, the selection command may select a time period on an adjustable timeline, hi one embodiment, the user-created elements may be selected by the content creation system based on the time period.

[0057] In general, Fig. 3 and the process shown in FIG. Figure 1, Figure 2, Figure 4 Any processes discussed in this disclosure that are understood to be computer-implemented, such as the systems and / or interfaces described in connection with FIG. 5, may be executed or otherwise performed by one or more processors of a computer system, such as the user device(s) 110, 120, the collaborative content creation system(s) 140, the server system(s) 142, the input fusion system(s) 144, the generative AI system(s) 146, and the computing environment generation system(s) 148, as described above. The processes or process steps performed by one or more processors may also be referred to as operations. The one or more processors may be configured to execute such processes by having access to instructions (e.g., software or computer readable code) that, when executed by the one or more processors, cause the one or more processors to execute the processes. The instructions may be stored in a memory of the computer system. The processor may be a central processing unit (CPU), a graphics processing unit (GPU), or another type of processing unit.

[0058] A computer system, such as user device(s) 110, 120, collaborative content creation system(s) 140, server system(s) 142, input fusion system(s) 144, generative AI system(s) 146, and computing environment generation system(s) 148, or any other system that performs operations to facilitate collaborative content creation in a computing environment, may include one or more computing devices. When one or more processors of a computer system are implemented as multiple processors, the multiple processors can be included in a single computing device or distributed among multiple computing devices. When a computer system includes multiple computing devices, the memory of the computer system may include respective memory of each computing device of the multiple computing devices.

[0059] 7 illustrates an example of a computing device 700 of a computer system. The computing device 700 may include a processor(s) 710 (e.g., a CPU, GPU, or other processing unit), a memory 720, and a communication interface(s) 740 (e.g., a network interface) for communicating with other devices. The memory 720 may include volatile memory, such as RAM, and / or non-volatile memory, such as ROM and storage media. Examples of storage media include solid-state storage media (e.g., solid-state drives and / or removable flash memory), optical storage media (e.g., optical disks), and / or magnetic storage media (e.g., hard disk drives). The instructions (e.g., software or computer readable code) described above may be stored in any volatile and / or non-volatile memory components of the memory 720. The computing device 700 may further include input device(s) 750 (e.g., a keyboard, a mouse, a joystick, a controller, or a touch screen) and output device(s) 760 (e.g., a display, a head-up display, an AR display, a VR display, a printer) in some embodiments. For example, if the user device(s) 110, 120 may be embodied as a tablet computer, the user device(s) 110, 120 may have a touch screen and a display. The aforementioned elements of the computing device 700 may be connected to each other through a bus 730, which corresponds to one or more buses. In some embodiments, the processor(s) 710 of the computing device 700 includes both a CPU and a GPU.

[0060] Instructions executable by one or more processors may be stored on a non-transitory computer-readable medium. Thus, whenever a computer-implemented method is described in this disclosure, the disclosure shall also be understood to describe a non-transitory computer-readable medium storing instructions that, when executed by one or more processors, configure and / or cause the one or more processors to perform the computer-implemented method. Examples of non-transitory computer-readable media include RAM, ROM, solid-state storage media (e.g., solid-state drives), optical storage media (e.g., optical disks), and magnetic storage media (e.g., hard disk drives). A non-transitory computer-readable medium can be part of the memory of a computer system or can be separate from any computer system.

[0061] In the above description of exemplary embodiments, it should be understood that various features may be grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects reside in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of the present disclosure.

[0062] Moreover, although some embodiments described herein include some features included in other embodiments and not others, as will be understood by one of ordinary skill in the art, combinations of features of the various embodiments are intended to be within the scope of the disclosure and to form various embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0063] Thus, while particular embodiments have been described, those skilled in the art will recognize that other and further modifications may be made to those embodiments without departing from the spirit of the disclosure, and that all such changes and modifications are intended to be claimed as falling within the scope of the disclosure. For example, functionality may be added or removed from the block diagrams, operations may be swapped between functional blocks, and steps may be added or removed to methods described within the scope of the disclosure.

[0064] The subject matter disclosed above should be considered as illustrative and not restrictive, and the appended claims are intended to describe all such modifications, enhancements, and other embodiments that fall within the true spirit and scope of the present disclosure. Therefore, to the maximum extent permitted by law, the scope of the present disclosure should be determined by the broadest permissible interpretation of the following claims and their equivalents, and should not be limited or restricted by the foregoing detailed description. Although various embodiments of the present disclosure have been described, it will be apparent to those skilled in the art that many more embodiments and implementations are possible within the scope of the present disclosure. Thus, the present disclosure is not intended to be limited.

Claims

1. 1. A method for performing collaborative content creation in a computing environment using artificial intelligence, comprising: receiving, by a content creation system, a first input from a first user and a second input from a second user; analyzing, by an input fusion system, the first input and the second input by using a machine learning model to identify the presence of duplicate data, redundant data, and / or prompt data; transmitting, by the input fusion system, the prompt data to an action generation system upon identifying the presence of the prompt data from at least one of the first input or the second input; generating, by the action generation system, first action data based on the prompt data and the machine learning model; performing, by the action generation system, a first action in the computing environment based on the first action data; A method comprising:

2. removing, by the input merging system, the duplicate data from at least one of the first input or the second input upon identifying the presence of the duplicate data in at least one of the first input or the second input; The method of claim 1 further comprising:

3. removing, by the input merging system, the duplicate and redundant data from at least one of the first input or the second input upon identifying the presence of the duplicate and redundant data in at least one of the first input or the second input; The method of claim 1 further comprising:

4. The method of claim 1 , wherein at least one of the first input or the second input comprises text data.

5. The method comprises: converting, by the input fusion system, at least one of the first input or the second input into transformed data; Further comprising: at least one of the first input or the second input includes at least one of image data, audio data, or haptic data; the converted data includes converted text data; The method of claim 1.

6. removing, by the input fusion system, from the converted text data, text data corresponding to at least one of the duplicate data or the redundant data upon identifying the presence of the duplicate data or the redundant data; The method of claim 5 further comprising:

7. 10. The method of claim 1, wherein the machine learning model facilitates identifying the presence of the duplicate data, the redundant data, and / or the prompt data based on direct user input or indirect user input.

8. The method of claim 1 , wherein the computing environment is a virtual environment.

9. The method of claim 1 , wherein the computing environment is an augmented environment.

10. The method of claim 8 , wherein the first action is creating or modifying an element within a space of the virtual environment.

11. The method of claim 10 , wherein the element is at least one of a visual element or an audio element.

12. The method of claim 1 , wherein the first input and the second input are received synchronously.

13. The method of claim 1 , wherein the first input and the second input are received asynchronously.

14. The method comprises: generating, by the content creation system, a signal for displaying a graphical interface to the first user or the second user; receiving, by the content creation system, a selection command from the first user or the second user; providing, by the content creation system, a user-created element in the computing environment based on the selection command; Further comprising: the graphical interface includes an adjustable timeline; the selection command selects a time period on the adjustable timeline; the user-created elements are selected by the content creation system based on the time period; The method of claim 1.

15. 1. A computer system for performing collaborative content creation in a computing environment using artificial intelligence, comprising: A memory storing instructions; one or more processors configured to execute the instructions to perform operations; Equipped with The operation receiving, by a content creation system, a first input from a first user and a second input from a second user; analyzing, by an input fusion system, the first input and the second input by using a machine learning model to identify the presence of duplicate data, redundant data, and / or prompt data; transmitting, by the input fusion system, the prompt data to an action generation system upon identifying the presence of the prompt data from at least one of the first input or the second input; generating, by the action generation system, first action data based on the prompt data and the machine learning model; performing, by the action generation system, a first action in the computing environment based on the first action data; 2. A computer system comprising:

16. The operation removing, by the input merging system, the duplicate data from at least one of the first input or the second input upon identifying the existence of the duplicate data; The system of claim 15 further comprising:

17. The operation removing, by the input merging system, the duplicate data and the redundant data from at least one of the first input or the second input upon identifying the existence of the duplicate data and the redundant data; The system of claim 15 further comprising:

18. The system of claim 15 , wherein at least one of the first input or the second input comprises text data.

19. The operation converting, by the input fusion system, at least one of the first input or the second input into transformed data; Further comprising: at least one of the first input or the second input includes at least one of image data, audio data, or haptic data; the converted data includes converted text data; The system of claim 15.

20. The operation removing, by the input fusion system, from the converted text data, text data corresponding to at least one of the duplicate data or the redundant data upon identifying the existence of the duplicate data or the redundant data; 20. The system of claim 19, further comprising:

21. 20. The system of claim 15, wherein the machine learning model facilitates identifying the presence of the duplicate data, the redundant data, and / or the prompt data based on direct user input or indirect user input.

22. The system of claim 15 , wherein the computing environment is a virtual environment.

23. The system of claim 15 , wherein the computing environment is an augmented environment.

24. 23. The system of claim 22, wherein the first action is creating or modifying an element within a space of the virtual environment.

25. 25. The system of claim 24, wherein the element is at least one of a visual element or an audio element.

26. The system of claim 15 , wherein the first input and the second input are received synchronously.

27. The system of claim 15 , wherein the first input and the second input are received asynchronously.

28. A computer-readable storage medium storing instructions, comprising: The instructions, when executed by one or more processors of a computer system, cause the computer system to perform a method for performing collaborative content creation in a computing environment using artificial intelligence; The method comprises: receiving, by a content creation system, a first input from a first user and a second input from a second user; analyzing, by an input fusion system, the first input and the second input by using a machine learning model to detect at least one of duplicate data, redundant data, or prompt data; transmitting, by the input fusion system, the prompt data to an action generation system upon detection of the prompt data from at least one of the first input or the second input; generating, by the action generation system, first action data based on the prompt data and the machine learning model; performing, by the action generation system, a first action in the computing environment based on the first action data; A computer-readable storage medium comprising:

29. 1. A method for performing content creation in a computing environment using artificial intelligence, comprising: receiving, by a content creation system, a first input and a second input; analyzing, by an input fusion system, the first input and the second input by using a machine learning model to identify the presence of duplicate data, redundant data, and / or prompt data; transmitting, by the input fusion system, the prompt data to an action generation system upon identifying the presence of the prompt data from at least one of the first input or the second input; generating, by the action generation system, first action data based on the prompt data and the machine learning model; performing, by the action generation system, a first action in the computing environment based on the first action data; A method comprising:

30. 30. The method of claim 29, wherein the first input and the second input are received asynchronously.

31. 30. The method of claim 29, wherein the first input and the second input are received synchronously.

Citation Information

Cited By

  • Mapping a real-world room for a shared artificial reality environment

    US12586326B2

  • Mapping a real-world room for a shared artificial reality environment

    US12614355B2

  • Generating a summary of a conversation between users for an additional user in response to determining the additional user is joining the conversation

    US12712840B2

  • Artificial reality collaborative working environments

    US12719882B2

  • Collaborative Workspace for an Artificial Reality Environment

    US20240371059A1