Method for acquiring video related to scenario by utilizing neural network model

Abstract

Disclosed is a method by which one or more processors of a computing device acquire a video by utilizing a neural network model, according to one embodiment of the present disclosure. The method may comprise the steps of: acquiring a natural language input related to the video; acquiring at least one keyword on the basis of the natural language input; acquiring, on the basis of the at least one keyword, a prompt to be input into a neural network model; determining a weight for the prompt on the basis of the at least one keyword; and utilizing the neural network model to acquire the video on the basis of the prompt for which the weight has been determined.

Description

Method to acquire video regarding a scenario using a neural network model

[0001] The present disclosure relates to a method for acquiring video regarding a scenario using a neural network model. More specifically, the method involves acquiring a prompt and a corresponding weight based on at least one keyword obtained from a natural language input, and acquiring video based on the prompt with the determined weight using a neural network model, thereby optimizing various scene conditions such as text, emotion, and location to obtain natural conditions. Through this, the method enables the acquisition of an image in which emotional changes are reflected and background consistency is maintained, as well as a video in which continuity following scene transitions is visually naturally expressed.

[0002] Existing image generation models generally rely on a single text input to generate images, presenting challenges in maintaining consistency in emotional changes between scenes or backgrounds. Furthermore, even when relying on a single text input, keywords are not weighted to focus on important objects depending on the scene; consequently, in fields where visual storytelling is crucial—such as movies, animations, and games—it is difficult to accurately depict character emotional changes or prominent facial expressions in accordance with the scenario flow.

[0003] Additionally, depending on the situation, there was also the problem of being unable to reduce resources and costs associated with manual work during the process of adjusting various conditions according to the scenario development prior to filming.

[0004] Therefore, there is a growing need for a method to obtain natural conditions by optimizing various scene conditions such as text, emotions, and locations, thereby acquiring images that reflect emotional changes and maintain background consistency, as well as videos in which continuity following scene transitions is visually expressed naturally.

[0005] Meanwhile, although the present disclosure is derived at least based on the technical background examined above, the technical problem or objective of the present disclosure is not limited to solving the problems or disadvantages examined above. That is, in addition to the technical issues examined above, the present disclosure can cover various technical issues related to the contents described below.

[0006] The present disclosure relates to a method for acquiring video regarding a scenario using a neural network model. More specifically, the problem is to acquire a prompt and a corresponding weight based on at least one keyword obtained from natural language input, and to acquire a video based on the prompt with the determined weight using a neural network model, thereby optimizing various scene conditions such as text, emotion, and location to acquire natural conditions, and thereby to acquire a natural image in which emotional changes or consistency of the background are maintained, and a video in which continuity following scene transitions is visually naturally expressed.

[0007] Meanwhile, the technical problem that the present disclosure aims to solve is not limited to the technical problem mentioned above, and various technical problems may be included within the scope obvious to a person skilled in the art from the contents described below.

[0008] A method performed by a computing device according to an embodiment of the present disclosure for realizing the aforementioned task is disclosed. The method may include: a step of acquiring a natural language input related to an image; a step of acquiring at least one keyword based on the natural language input; a step of acquiring a prompt for inputting to a neural network model based on the at least one keyword; a step of determining a weight for the prompt based on the at least one keyword; and a step of acquiring an image based on the prompt with the determined weight using the neural network model.

[0009] Alternatively, the step of obtaining at least one keyword based on the natural language input may include the step of inputting the natural language input into a pre-trained language model; and the step of obtaining the at least one keyword based on one or more output data output based on the language model.

[0010] Alternatively, the step of obtaining at least one keyword based on the natural language input may include at least one of the following steps: obtaining a first keyword for distance based on the natural language input; obtaining a second keyword for an object based on the natural language input; obtaining a third keyword containing details about the object based on the natural language input; or obtaining a fourth keyword for background based on the natural language input.

[0011] Alternatively, the step of obtaining a second keyword for an object based on the natural language input may include at least one of the step of obtaining a second-1 keyword for a higher-level concept object based on the natural language input; or the step of obtaining a second-2 keyword for a lower-level concept object of the second-1 keyword based on the natural language input.

[0012] Alternatively, the step of obtaining a third keyword containing details about an object based on the natural language input may include at least one of the steps of obtaining a third-1 keyword related to one or more actions based on the natural language input; or obtaining a third-2 keyword related to one or more states based on the natural language input.

[0013] Alternatively, the neural network model may include a neural network model capable of performing at least one of encoding or decoding of input data.

[0014] Alternatively, the step of acquiring an image based on a weighted prompt using the neural network model may include at least one of the step of acquiring an image based on a weighted prompt using the neural network model; or the step of acquiring a video based on a weighted prompt using the neural network model.

[0015] Alternatively, the step of determining a weight for the prompt based on at least one keyword may include the step of determining a weight for the prompt based on the weight of at least one keyword included in the natural language input.

[0016] Alternatively, the step of determining a weight for the prompt based on at least one keyword may include, if the natural language input includes a 2-2 keyword for a sub-concept object of the 2-1 keyword, determining the weight for the 2-2 keyword to be greater than the weight of the 2-1 keyword.

[0017] Alternatively, the step of determining a weight for the prompt based on at least one keyword may include, when the natural language input contains a third-1 keyword associated with one or more actions, setting the weight of the keyword for an object associated with the third-1 keyword greater than the weight of the keyword for an object not associated with the third-1 keyword.

[0018] Alternatively, the step of determining a weight for the prompt based on at least one keyword may include, when the natural language input contains one or more third-2 keywords associated with a state, setting the weight of the keyword for an object associated with the third-2 keyword greater than the weight of the keyword for an object not associated with the third-2 keyword.

[0019] Alternatively, the method may further include the steps of: obtaining additional natural language input related to an additional image; obtaining at least one additional keyword based on the additional natural language input; obtaining an additional prompt for input to a neural network model based on the at least one additional keyword; determining a weight for the additional prompt based on the at least one additional keyword; and obtaining an additional image based on the image and the additional prompt with the determined weight using the neural network model.

[0020] A computer program stored in a computer-readable storage medium is disclosed in accordance with one embodiment of the present disclosure for realizing the aforementioned objectives. When the computer program is executed on one or more processors, the one or more processors perform operations to acquire an image using a neural network model, wherein the operations may include: an operation to acquire a natural language input related to an image; an operation to acquire at least one keyword based on the natural language input; an operation to acquire a prompt to be input to a neural network model based on the at least one keyword; an operation to determine a weight for the prompt based on the at least one keyword; and an operation to acquire an image based on the prompt with the determined weight using the neural network model.

[0021] Alternatively, the operation of obtaining at least one keyword based on the natural language input may include: the operation of inputting the natural language input into a pre-trained language model; and the operation of obtaining the at least one keyword based on one or more output data output based on the language model.

[0022] Alternatively, the operation of obtaining at least one keyword based on the natural language input may include at least one of the following: an operation of obtaining a first keyword for distance based on the natural language input; an operation of obtaining a second keyword for an object based on the natural language input; an operation of obtaining a third keyword including details about the object based on the natural language input; or an operation of obtaining a fourth keyword for background based on the natural language input.

[0023] Alternatively, the operation of obtaining a second keyword for an object based on the natural language input may include at least one of the operation of obtaining a second-1 keyword for a higher-level concept object based on the natural language input; or the operation of obtaining a second-2 keyword for a lower-level concept object of the second-1 keyword based on the natural language input.

[0024] Alternatively, the operation of obtaining a third keyword containing details about an object based on the natural language input may include at least one of the operation of obtaining a third-1 keyword related to one or more actions based on the natural language input; or the operation of obtaining a third-2 keyword related to one or more states based on the natural language input.

[0025] Alternatively, the operation of acquiring an image based on a weighted prompt using the neural network model may include at least one of the operation of acquiring an image based on a weighted prompt using the neural network model; or the operation of acquiring a video based on a weighted prompt using the neural network model.

[0026] Alternatively, the operation of determining a weight for the prompt based on at least one keyword may include an operation of determining a weight for the prompt based on the weight of at least one keyword included in the natural language input.

[0027] Alternatively, the operation of determining a weight for the prompt based on at least one keyword may include, when the natural language input includes a 2-2 keyword for a sub-concept object of the 2-1 keyword, determining the weight for the 2-2 keyword to be greater than the weight of the 2-1 keyword.

[0028] Alternatively, the operation of determining a weight for the prompt based on at least one keyword may include, when the natural language input contains a 3-1 keyword associated with one or more actions, setting the weight of the keyword for an object associated with the 3-1 keyword greater than the weight of the keyword for an object not associated with the 3-1 keyword.

[0029] Alternatively, the operation of determining a weight for the prompt based on at least one keyword may include, when the natural language input contains one or more 3-2 keywords associated with a state, setting the weight of the keyword for an object associated with the 3-2 keyword greater than the weight of the keyword for an object not associated with the 3-2 keyword.

[0030] A computing device according to one embodiment of the present disclosure for realizing the aforementioned task is disclosed. The device comprises at least one processor; and memory, wherein the processor may be configured to acquire a natural language input related to an image; acquire at least one keyword based on the natural language input; acquire a prompt for inputting to a neural network model based on the at least one keyword; determine a weight for the prompt based on the at least one keyword; and acquire an image based on the prompt with the determined weight using the neural network model.

[0031] The present disclosure relates to a method for acquiring video regarding a scenario using a neural network model. More specifically, a prompt and a corresponding weight are obtained based on at least one keyword obtained from a natural language input, and by using a neural network model to acquire video based on the prompt with the determined weight, various scene conditions such as text, emotion, and location are optimized to obtain natural conditions. Through this, natural images in which emotional changes or consistency of the background are maintained, and video in which continuity following scene transitions is visually naturally expressed can be acquired.

[0032] Meanwhile, the effects of the present disclosure are not limited to those mentioned above, and various effects may be included within the scope obvious to a person skilled in the art from the contents described below.

[0033] FIG. 1 is a block diagram of a computing device for acquiring an image using a neural network model according to one embodiment of the present disclosure.

[0034] FIG. 2 is a schematic diagram showing a network function according to one embodiment of the present disclosure.

[0035] FIG. 3 is a flowchart illustrating a method for acquiring an image using a neural network model according to one embodiment of the present disclosure.

[0036] FIG. 4 is a schematic diagram illustrating a process of acquiring an image based on at least one keyword using a neural network model according to one embodiment of the present disclosure.

[0037] FIG. 5 is a schematic diagram illustrating a process for determining a weight for a prompt based on at least one keyword according to one embodiment of the present disclosure, and acquiring an image based on the prompt with the determined weight using a neural network model.

[0038] FIG. 6 is a schematic diagram illustrating the process of acquiring additional images based on additional prompts with determined weights using a neural network model according to one embodiment of the present disclosure.

[0039] FIG. 7 is a brief and general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

[0040] Various embodiments are now described with reference to the drawings. In this specification, various descriptions are provided to provide an understanding of the present disclosure. However, it is evident that these embodiments can be practiced without such specific descriptions.

[0041] As used herein, terms such as “component,” “module,” “system,” etc. refer to computer-related entities, hardware, firmware, software, combinations of software and hardware, or executions of software. For example, a component may be, but is not limited to, a procedure executed on a processor, a processor, an object, an execution thread, a program, and / or a computer. For example, both an application executed on a computing device and the computing device itself may be a component. One or more components may reside within a processor and / or an execution thread. A component may be localized within a single computer. A component may be distributed among two or more computers. Additionally, these components may be executed from various computer-readable media having various data structures stored therein. Components may communicate through local and / or remote processes, for example, according to signals having one or more data packets (e.g., data from a component interacting with another component in a local system or distributed system, and / or data transmitted through signals to other systems and networks such as the Internet).

[0042] Furthermore, the term "or" is intended to mean an implicit "or" rather than an exclusive "or." That is, unless otherwise specified or evident from the context, "X uses A or B" is intended to mean one of the natural implicit substitutions. In other words, if X uses A; if X uses B; or if X uses both A and B, "X uses A or B" may apply to any of these cases. Additionally, the term "and / or" as used herein should be understood to refer to and include all possible combinations of one or more of the enumerated related items.

[0043] Additionally, the terms “comprising” and / or “comprising” should be understood to mean that such features and / or components are present. However, the terms “comprising” and / or “comprising” should be understood not to exclude the presence or addition of one or more other features, components and / or groups thereof. Furthermore, unless otherwise specified or clearly evident from the context to indicate a singular form, the singular in this specification and claims should generally be interpreted to mean “one or more.”

[0044] And, the term "at least one of A or B" should be interpreted to mean "a case including only A," "a case including only B," or "a combination of A and B."

[0045] Those skilled in the art should recognize that the various exemplary logical blocks, configurations, modules, circuits, means, logics, and algorithmic steps described in connection with the embodiments disclosed herein may be implemented in electronic hardware, computer software, or a combination of both. To clearly exemplify the interchangeability of hardware and software, various exemplary components, blocks, configurations, means, logics, modules, circuits, and steps have been generally described above in terms of their functionality. Whether such functionality is implemented in hardware or software depends on the specific application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each specific application. However, such decisions regarding implementation should not be construed as going beyond the scope of this disclosure.

[0046] The description of the presented embodiments is provided to enable those skilled in the art to use or practice the present invention. Various modifications to these embodiments will be apparent to those skilled in the art. The general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present invention is not limited to the embodiments presented herein. The present invention should be interpreted in the broadest possible scope consistent with the principles and novel features presented herein.

[0047] In the present disclosure, network functions, artificial neural networks, and neural networks may be used interchangeably.

[0048]

[0049] FIG. 1 is a block diagram of a computing device for improving data labeling convenience according to one embodiment of the present disclosure.

[0050] The configuration of the computing device (100) illustrated in FIG. 1 is merely a simplified example. In one embodiment of the present disclosure, the computing device (100) may include other configurations for performing the computing environment of the computing device (100), and only some of the disclosed configurations may constitute the computing device (100).

[0051] The computing device (100) may include a processor (110), memory (130), and a network unit (150).

[0052] The processor (110) may be composed of one or more cores and may include processors for data analysis and deep learning, such as a central processing unit (CPU) of a computing device, a general purpose graphics processing unit (GPGPU), and a tensor processing unit (TPU). The processor (110) may read a computer program stored in memory (130) and perform data processing for machine learning according to one embodiment of the present disclosure. According to one embodiment of the present disclosure, the processor (110) may perform operations for training a neural network model. The processor (110) may perform calculations for training a neural network model, such as processing input data for training in deep learning (DL), extracting features from input data, calculating errors, and updating weights of the neural network model using backpropagation. At least one of the CPU, GPGPU, and TPU of the processor (110) may process the training of the neural network model. For example, a CPU and a GPGPU can work together to process the training of a neural network model and the classification of data using the neural network model. Additionally, in one embodiment of the present disclosure, processors of a plurality of computing devices can be used together to process the training of a neural network model and the classification of data using the neural network model. Furthermore, a computer program executed on a computing device according to one embodiment of the present disclosure may be a CPU, GPGPU, or TPU executable program.

[0053] According to one embodiment of the present disclosure, the memory (130) can store any form of information generated or determined by the processor (110) and any form of information received by the network unit (150).

[0054] According to one embodiment of the present disclosure, the memory (130) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, a magnetic disk, and an optical disk. The computing device (100) may operate in conjunction with web storage that performs the storage function of the memory (130) on the internet. The description of the memory described above is merely an example and the present disclosure is not limited thereto.

[0055] A network unit (150) according to one embodiment of the present disclosure can use various wired communication systems such as a public switched telephone network (PSTN), xDSL (x Digital Subscriber Line), RADSL (Rate Adaptive DSL), MDSL (Multi Rate DSL), VDSL (Very High Speed ​​DSL), UADSL (Universal Asymmetric DSL), HDSL (High Bit Rate DSL), and a local area network (LAN).

[0056] Additionally, the network unit (150) presented in this disclosure may use various wireless communication systems such as CDMA (Code Division Multi Access), TDMA (Time Division Multi Access), FDMA (Frequency Division Multi Access), OFDMA (Orthogonal Frequency Division Multi Access), SC-FDMA (Single Carrier-FDMA), and other systems.

[0057] In the present disclosure, the network unit (150) can be configured regardless of the mode of communication, such as wired and wireless, and can be configured as various communication networks, such as a Personal Area Network (PAN) or a Wide Area Network (WAN). In addition, the network may be a known World Wide Web (WWW) and may utilize wireless transmission technology used for short-range communication, such as Infrared Data Association (IrDA) or Bluetooth. The technologies described in the present disclosure may also be used in other networks mentioned above.

[0058]

[0059] FIG. 2 illustrates an exemplary structure of an artificial intelligence-based model according to one embodiment of the present disclosure.

[0060] Throughout this specification, artificial intelligence model, artificial intelligence-based model, computational model, neural network, network function, and neural network may be used interchangeably.

[0061] A neural network can be composed of a set of interconnected computational units that may generally be referred to as nodes. These nodes may also be referred to as neurons. A neural network is composed of at least one node. The nodes (or neurons) constituting neural networks may be interconnected by one or more links.

[0062] In a neural network, one or more nodes connected via links can form relative input and output node relationships. The concepts of input and output nodes are relative; any node in an output node relationship with respect to one node may be in an input node relationship with respect to another node, and vice versa. As described above, the input node versus output node relationship can be generated based on links. One or more output nodes may be connected to a single input node via links, and vice versa.

[0063] In a relationship between an input node and an output node connected through a single link, the value of the output node's data can be determined based on the data input to the input node. Here, the link interconnecting the input node and the output node may have a weight. The weight can be variable and can be varied by the user or an algorithm to enable the neural network to perform the desired function. For example, if one or more input nodes are interconnected to a single output node by respective links, the output node's value can be determined based on the values ​​input to the input nodes connected to the output node and the weights set on the links corresponding to each input node.

[0064] As described above, a neural network consists of one or more nodes interconnected through one or more links, forming input-output node relationships within the network. The characteristics of a neural network can be determined by the number of nodes and links within the network, the relationships between the nodes and links, and the weight values ​​assigned to each link. For example, if two neural networks exist with the same number of nodes and links but different weight values ​​for the links, the two neural networks may be recognized as different from each other.

[0065] A neural network can be composed of a set of one or more nodes. A subset of nodes constituting a neural network can form a layer. Some of the nodes constituting a neural network can form a layer based on their distances from an initial input node. For example, a set of nodes with a distance of n from an initial input node can form n layers. The distance from the initial input node can be defined by the minimum number of links that must be traversed to reach that node from the initial input node. However, this definition of a layer is arbitrary for illustrative purposes, and the degree of a layer within a neural network can be defined in a way different from that described above. For example, a layer of nodes may be defined by its distance from a final output node.

[0066] In one embodiment of the present disclosure, a set of neurons or nodes may be defined by the expression a layer.

[0067] Initial input nodes may refer to one or more nodes within a neural network to which data is directly input without passing through links in their relationships with other nodes. Alternatively, in terms of link-based relationships between nodes within the neural network, they may refer to nodes that do not have other input nodes connected by links. Similarly, final output nodes may refer to one or more nodes within a neural network that do not have output nodes in their relationships with other nodes. Furthermore, hidden nodes may refer to nodes constituting the neural network that are neither initial input nodes nor final output nodes.

[0068] A neural network according to one embodiment of the present disclosure may have the number of nodes in the input layer equal to the number of nodes in the output layer, and may be a neural network in which the number of nodes decreases and then increases again as it progresses from the input layer to the hidden layer. Additionally, a neural network according to another embodiment of the present disclosure may have the number of nodes in the input layer less than the number of nodes in the output layer, and may be a neural network in which the number of nodes decreases as it progresses from the input layer to the hidden layer. Additionally, a neural network according to yet another embodiment of the present disclosure may have the number of nodes in the input layer greater than the number of nodes in the output layer, and may be a neural network in which the number of nodes increases as it progresses from the input layer to the hidden layer. A neural network according to yet another embodiment of the present disclosure may be a neural network in which the above-described neural networks are combined.

[0069] A deep neural network (DNN) can refer to a neural network that includes multiple hidden layers in addition to input and output layers. Deep neural networks allow for the identification of latent structures in data. That is, they can identify the latent structures of photos, text, videos, audio, protein sequences, gene sequences, peptide sequences, and music (e.g., what objects are in a photo, what the content and emotions of the text are, what the content and emotions of the audio are, etc.), and / or the binding affinity between peptides and MHCs. Deep neural networks may include convolutional neural networks (CNN), recurrent neural networks (RNN), autoencoders, restricted Boltzmann machines (RBM), deep belief networks (DBN), Q networks, U networks, Siamese networks, Generative Adversarial Networks (GAN), Transformers, etc. The description of the deep neural network described above is merely an example and the present disclosure is not limited thereto.

[0070] The artificial intelligence-based model of the present disclosure may be represented by a network structure of any structure described above, including an input layer, a hidden layer, and an output layer.

[0071] A neural network that can be used in an artificial intelligence-based model of the present disclosure may be trained in at least one of supervised learning, unsupervised learning, semi-supervised learning, transfer learning, active learning, or reinforcement learning. Training of a neural network may be a process of applying knowledge to the neural network to perform a specific operation.

[0072] Neural networks can be trained to minimize the error in their output. The training process involves repeatedly inputting training data into the network, calculating the error between the network's output and the target for the training data, and updating the weights of each node by backpropagating the error from the output layer to the input layer in a direction that reduces the error. In supervised learning, training data is used where the correct answer is labeled for each data point (i.e., labeled training data), whereas in unsupervised learning, the correct answer may not be labeled for each training data point. For instance, in the case of supervised learning for data classification, the training data may consist of data where each training point is labeled with a category. Labeled training data is input into the neural network, and the error can be calculated by comparing the network's output (category) with the labels of the training data. As another example, in the case of unsupervised learning for data classification, the error can be calculated by comparing the input training data with the neural network's output. The calculated error is backpropagated in the neural network (i.e., from the output layer to the input layer), and through backpropagation, the connection weights of each node in each layer of the neural network can be updated. The amount of change in the connection weights of each node being updated can be determined by the learning rate. The neural network's calculation of the input data and the backpropagation of the error can constitute a learning cycle (epoch). The learning rate can be applied differently depending on the number of iterations of the neural network's learning cycle. For example, a high learning rate can be used in the early stages of training to quickly achieve a certain level of performance and increase efficiency, while a low learning rate can be used in the later stages to improve accuracy.

[0073] In the training of neural networks, the training data is generally a subset of the real-world data (i.e., the data intended to be processed using the trained neural network). Consequently, a training cycle may exist where errors decrease on the training data but increase on the real-world data. Overfitting is a phenomenon where the network learns excessively on the training data, leading to increased errors on the real-world data. For example, a neural network trained on yellow cats might fail to recognize cats when seeing anything other than yellow, which can be considered a type of overfitting. Overfitting can act as a cause for increased errors in machine learning algorithms. Various optimization methods can be used to prevent this overfitting. To prevent overfitting, methods such as increasing the training data, regularization, dropout (which disables some nodes in the network during training), and the use of batch normalization layers can be applied.

[0074] A computer-readable medium storing a data structure according to one embodiment of the present disclosure is disclosed. The aforementioned data structure may be stored in a storage unit in the present disclosure, executed by a processor, and transmitted and received by a communication unit.

[0075] A data structure can refer to the organization, management, and storage of data that enables efficient access and modification of data. A data structure can refer to the organization of data for solving specific problems (e.g., data analysis, data retrieval, data storage, data modification). A data structure may also be defined by physical or logical relationships between data elements designed to support specific data processing functions. Logical relationships between data elements may include connections between user-defined data elements. Physical relationships between data elements may include actual relationships between data elements physically stored on a computer-readable storage medium (e.g., a permanent storage device). Specifically, a data structure may include sets of data, relationships between data, and functions or instructions applicable to the data. Through an effectively designed data structure, a computing device can perform operations while minimizing the use of the computing device's resources. Specifically, through an effectively designed data structure, a computing device can increase the efficiency of operations, reading, insertion, deletion, comparison, exchange, and retrieval.

[0076] Data structures can be classified into linear and non-linear data structures based on their form. A linear data structure is one where only one piece of data is connected to the next. Linear data structures can include lists, stacks, queues, and deques. A list can refer to a set of data that maintains an internal order. Lists can include linked lists. A linked list is a data structure where data is connected in a line, with each piece of data possessing a pointer. In a linked list, the pointer can contain information regarding the connection to the next or previous data. Depending on its form, a linked list can be represented as a singly linked list, a doubly linked list, or a circular linked list. A stack is a data arrangement structure that allows for restricted access to data. A stack can be a linear data structure where data can be processed (e.g., insertion or deletion) only at one end. Data stored in a stack can be a Last-In, First-Out (LIFO) data structure, meaning that the later an item is entered, the sooner it is retrieved. A queue is a data sequence structure that allows for limited access to data; unlike a stack, it can be a FIFO (First in First Out) data structure where data stored later is retrieved later. A deque is a data structure that can process data at both ends.

[0077] Non-linear data structures can be structures where multiple data are connected after a single piece of data. Non-linear data structures may include graph data structures. A graph data structure can be defined by vertices and edges, and an edge may include a line connecting two different vertices. Graph data structures may include tree data structures. A tree data structure may be a data structure where there is only one path connecting two different vertices among the multiple vertices included in the tree. In other words, it may be a data structure that does not form a loop in a graph data structure.

[0078] Throughout this specification, the terms artificial intelligence-based model, computational model, neural network, network function, and neural network may be used interchangeably. Hereinafter, they will be described uniformly as neural network. A data structure may include a neural network. Furthermore, a data structure including a neural network may be stored on a computer-readable medium. A data structure including a neural network may also include data preprocessed for processing by the neural network, data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, loss functions for learning the neural network, etc. A data structure including a neural network may include any of the components disclosed above. That is, a data structure including a neural network may be configured to include all or any combination thereof, such as data preprocessed for processing by the neural network, data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, and loss functions for learning the neural network. In addition to the configurations described above, a data structure including a neural network may include any other information that determines the characteristics of the neural network. Furthermore, the data structure may include any form of data used or generated during the computational process of the neural network, and is not limited to the foregoing. A computer-readable medium may include a computer-readable recording medium and / or a computer-readable transmission medium. A neural network may be composed of a set of interconnected computational units that may generally be referred to as nodes. These nodes may also be referred to as neurons. A neural network is composed of at least one node.

[0079] A data structure may include data input to a neural network. A data structure including data input to a neural network may be stored on a computer-readable medium. Data input to a neural network may include training data input during the neural network learning process and / or input data input to a neural network after training is complete. Data input to a neural network may include pre-processed data and / or data subject to pre-processing. Pre-processing may include a data processing process for inputting data into a neural network. Accordingly, a data structure may include data subject to pre-processing and data generated by pre-processing. The aforementioned data structure is merely an example, and the present disclosure is not limited thereto.

[0080] The data structure may include weights of the neural network. (In this specification, weights and parameters may be used interchangeably.) The data structure including the weights of the neural network may be stored on a computer-readable medium. The neural network may include multiple weights. The weights may be variable and may be varied by a user or an algorithm to enable the neural network to perform a desired function. For example, if one or more input nodes are interconnected to a single output node by respective links, the output node may determine the data value output from the output node based on values ​​input to the input nodes connected to the output node and weights set on the links corresponding to each input node. The aforementioned data structure is merely an example and the present disclosure is not limited thereto.

[0081] As an example rather than a limitation, weights may include weights that vary during the neural network learning process and / or weights for which neural network learning is completed. Weights that vary during the neural network learning process may include weights at the start of the learning cycle and / or weights that vary during the learning cycle. Weights for which neural network learning is completed may include weights for which the learning cycle is completed. Accordingly, a data structure containing the weights of a neural network may include a data structure containing weights that vary during the neural network learning process and / or weights for which neural network learning is completed. Therefore, the weights and / or combinations of each weight described above are included in the data structure containing the weights of a neural network. The aforementioned data structure is merely an example and the present disclosure is not limited thereto.

[0082] Data structures containing the weights of a neural network may be stored on a computer-readable storage medium (e.g., memory, hard disk) after undergoing a serialization process. Serialization may be a process of converting a data structure into a form that can be stored on the same or different computing devices and later reconstructed for use. A computing device may serialize the data structure to transmit and receive data over a network. A serialized data structure containing the weights of a neural network may be reconstructed on the same or different computing devices through deserialization. Data structures containing the weights of a neural network are not limited to serialization. Furthermore, data structures containing the weights of a neural network may include data structures designed to increase computational efficiency while minimizing the use of computing device resources (e.g., B-Tree, R-Tree, Trie, m-way search tree, AVL tree, Red-Black Tree in non-linear data structures). The foregoing is merely an example and the present disclosure is not limited thereto.

[0083] The data structure may include hyperparameters of the neural network. The data structure including the neural network hyperparameters may be stored on a computer-readable medium. The hyperparameters may be variables that are varied by the user. The hyperparameters may include, for example, a learning rate, a cost function, the number of learning cycle iterations, weight initialization (e.g., setting the range of weight values ​​subject to weight initialization), and the number of hidden units (e.g., the number of hidden layers, the number of nodes in the hidden layers). The aforementioned data structure is merely an example, and the present disclosure is not limited thereto.

[0084] Models according to one embodiment of the present disclosure can be trained to remove part or all of the predicted noise with respect to Gaussian distributed noise, thereby obtaining data in which part or all of the predicted noise has been removed.

[0085] A computing device (100) according to one embodiment of the present disclosure can train a neural network to obtain data in which part or all of the noise is removed, by removing part or all of the noise with respect to isotropic Gaussian distributed noise. At this time, the neural network may include a conditional noise prediction model. In addition, the conditional noise prediction model may include a U-Net structure in which the input and output have the same size, and may take data x(t) containing noise and a diffusion time step t as inputs, and predict and output the diffusion noise contained in the data x(t) containing noise.

[0086] A computing device (100) can perform a forward process of repeating the process of adding random Gaussian noise little by little over T time steps to original data x (0) that does not contain noise, and consequently obtaining isotropic Gaussian distributed noise x (T). The types of original data x (0) may include various examples such as audio data in addition to image data. Meanwhile, a forward process according to one embodiment of the present disclosure can be performed through the following formula.

[0087] [Mathematical Formula 1]

[0088]

[0089]

[0090] In the above mathematical formula 1 It can be used as a hyperparameter in the process of calculating the diffusion coefficient, and can be set to an arbitrary value, and 0< < < ... < It can be set to a value of <1. For example With this value of 0.0001 g can have a value of 0.02, and from until The value of can increase linearly and can increase along a cosine function, and T, representing the total number of diffusion steps, can be set to 1000. However, It is merely an example that it increases linearly or along a cosine function, and according to the embodiments of the present disclosure, depending on the type of the original data The amount of increase may be determined differently, and a specific explanation will be provided later through [Mathematical Formula 2]. In addition, can represent random Gaussian distribution noise. A general formula representing the data x(t) containing the noise at time step t in relation to the original data x(0) without noise and the included diffusion noise can be expressed as follows.

[0091] [Mathematical Formula 2]

[0092]

[0093] According to one embodiment of the present disclosure, in Equation 2 above, Equation (1) and Equation (2) are diffusion coefficients This is an expression that expresses the specific meaning of. In equation (1) of the above mathematical equation 2, the diffusion coefficient at a specific time step t. The above hyperparameters in 1 It can be calculated as the value after subtracting, and the diffusion coefficient in equation (2) of the above mathematical formula 2. can represent the diffusion coefficient accumulated sequentially from time step 1 to t. Therefore, in the above mathematical equation 2, equation (3) represents the data x(t) containing noise at time step t, the original data x(0) without noise, and the diffusion coefficient ( ) and random Gaussian distribution noise( It is expressed by a formula regarding ). In addition, in the above mathematical formula 2, formula (4) may represent the ratio of noise n(t) and the ratio of signals s(t) determined based on the type of original data according to one embodiment of the present disclosure. Accordingly, the noise-containing data x(t) at time step t can be expressed based on the ratio of noise n(t) and the ratio of signals s(t). Specifically, the noise-containing data x(t) at time step t approaches the form of the original data as the ratio of original data x(0) without noise (i.e., the ratio of signals s(t)) increases, and random Gaussian distribution noise ( As the ratio of ) (i.e., the ratio of noise n(t)) increases, it can approach the form of random Gaussian distribution noise.

[0094] For example, a computing device (100) may obtain first data by adding first noise to the original data based on the ratio of the determined noise in a forward process of adding random Gaussian noise little by little over T time steps to original data x (0) that does not contain noise, and obtain second data by adding second noise to the first data based on the ratio of the determined noise. In one embodiment, the type of original data x (0) may include various examples such as text data and audio data in addition to image data.

[0095] Through this, the computing device (100) can repeat the process of adding random Gaussian noise over T time steps based on the "ratio of noise determined by equation (4) in the above mathematical formula 2" for original data x (0) that does not contain noise, and consequently perform a forward process of obtaining isotropic Gaussian distributed noise x (T). However, the above forward process is not limited to the example of the above mathematical formula 2, and various processes of adding noise to data may be included in the forward process.

[0096] Additionally, the computing device (100) can train the neural network to perform a reverse process in which, in the opposite direction to the forward process, it repeats the process of removing random Gaussian noise over T time steps from isotropic Gaussian distributed noise x(T), and consequently obtains original data x(0) that does not contain noise. In this regard, the formula representing the reverse process can be expressed as follows.

[0097] [Mathematical Formula 3]

[0098]

[0099] In the above mathematical formula 3, formula (1) is the noise prediction result predicted by the neural network for "noise-containing data x(t)" ( This is a formula representing a reverse process of obtaining "previous step data x(t-1) with the noise partially removed" by removing ). For example, the computing device (100) can obtain "previous step first data x(1) with the second noise removed" by removing the noise prediction result predicted by the neural network for "second data x(2) containing the second noise." In the above mathematical formula 3, formula (2) is the diffusion coefficient at the current time step t. It means, and in equation (3) of the above mathematical formula 3 represents the dispersion parameter, and the diffusion coefficient It can be calculated based on [Equation 3]. However, the above reverse process is not limited to [Equation 3], and various processes for removing noise from data containing noise may be included in the reverse process.

[0100] Specifically, the computing device (100) inputs data x(t) containing noise and a time step t into a neural network, calculates a loss function by comparing the noise prediction result predicted by the neural network with the diffusion noise actually included, and trains the neural network by performing gradient descent according to the loss function. For example, the loss function calculated by comparing the noise prediction result predicted by the neural network with the diffusion noise actually included can be expressed by the following formula.

[0101] [Mathematical Formula 4]

[0102]

[0103] In the above [Equation 4], the loss function is actually the included diffusion noise ( ) and the noise prediction result predicted by the above neural network ( It can be calculated by comparing ). For example, the computing device (100) can predict "first noise included in the first data x (1)" and calculate a first loss function by comparing the predicted first noise and the first noise added to the original data x (0). The computing device (100) can predict "second noise included in the second data x (2)" and calculate a second loss function by comparing the predicted second noise and the second noise added to the first data x (1). However, the loss function is not limited to the example of Equation 4, and may include various loss functions calculated by comparing the noise prediction result with the actually included diffuse noise.

[0104] Additionally, the neural network may be trained to obtain "data x (1) with the second noise removed" by predicting the diffuse noise contained in the data x (2) containing the second noise and removing the predicted second noise. For example, the computing device (100) may also train the neural network based on at least one of the first loss function or the second loss function. Furthermore, the neural network may be trained to obtain original data x (0) with all noise removed by repeating the process of predicting the diffuse noise contained in the "data x (t) containing the noise" and removing the predicted diffuse noise one or more times to remove all the diffuse noise contained in the "data x (t) containing the noise". Meanwhile, in the process of training the neural network to remove diffuse noise from the data containing the noise, the ratio of noise is determined based on the type of original data, thereby allowing the neural network to be trained to generate data of better quality.

[0105] The computing device (100) can determine the ratio of noise based on the size of the data that can be represented for the type of original data determined above. Specifically, the computing device (10) can determine the ratio of noise such that the size of the noise added to the original data is smaller than the minimum size of the data that can be represented for the type of original data determined above. For example, if the type of original data is determined to be image data, the original data may have a depth of 8 bits per channel. Accordingly, the computing device (10) can determine the first ratio of noise such that the size of the noise added to the original data is smaller than 1 / (2^8) (= approximately 0.00039), which is the minimum size of the data that can be represented for image data.

[0106] Meanwhile, according to another embodiment of the present disclosure, when the type of the original data is determined to be audio data, the original data may have a depth of 16 bits per sample. Accordingly, the computing device (10) may determine a second ratio of noise such that the magnitude of the noise added to the original data is smaller than 1 / (2^16) (= approximately 0.000015), which is the minimum size of data representable for audio data. Accordingly, the second ratio of noise when the original data is audio data may be determined to be smaller than the first ratio of noise when the original data is image data. For example, the ratio of noise may be determined through the following mathematical formula.

[0107] [Mathematical Formula 5]

[0108]

[0109] Specifically, according to one embodiment of the present disclosure, the noise ratio n(t) can be determined to increase exponentially by referring to equation (1) in [Equation 5], and the signal ratio s(t) can be determined to decrease exponentially by referring to equation (2) in [Equation 5]. In this case, r may represent a hyperparameter that determines the shape of the curves of the noise ratio n(t) and the signal ratios s(t). However, [Equation 5] is merely an example, and the noise ratio and the signal ratio may be determined in various other ways. Through this, the neural network model can be trained so that the output generated by the neural network model is almost free from the influence of noise, even in data domains that are relatively sensitive to noise compared to image data (e.g., audio data).

[0110]

[0111] FIG. 3 is a flowchart illustrating a method for acquiring an image using a neural network model according to one embodiment of the present disclosure.

[0112] A computing device (100) according to one embodiment of the present disclosure may directly acquire "information for acquiring an image using a neural network model" or receive it from an external system. The external system may be a server, a database, etc., that stores and manages information for acquiring an image using a neural network model. The computing device (100) may use the information directly acquired or received from an external system as "input data for acquiring an image using a neural network model."

[0113] According to one embodiment of the present disclosure, a computing device (100) can acquire natural language input related to an image (S110). At this time, the natural language input related to the image may include natural language inputs such as text and audio related to a scenario used in the production of storyboards and storyboards for movies, animations, etc., and the natural language input related to the image may include, for example, text related to a specific scenario within a movie or a change in the emotion of a specific character, but is not limited to the above examples and various examples may be utilized. Meanwhile, the computing device (100) can acquire at least one keyword based on the natural language input, and an explanation related thereto is described below.

[0114] According to one embodiment of the present disclosure, a computing device (100) may obtain at least one keyword based on natural language input obtained through step S110 (S120). For example, the computing device (100) may input the natural language input into a pre-trained language model and obtain the at least one keyword based on one or more output data output based on the language model. At this time, the language model may use a pre-trained Transformer (GPT family) model, and examples include Google's Bard, Chat-GPT, etc., but are not limited thereto and various language models may be used. Additionally, the computing device (100) may obtain at least one of a first keyword for distance, a second keyword for object, a third keyword including details about the object, or a fourth keyword for background based on the natural language input. At this time, the first keyword regarding the distance may include a keyword regarding the sense of distance of a background or object, such as "seeing from afar," and the second keyword regarding the object may include a keyword regarding an object that may be included in the output image, such as a character, actor, person, animal, etc. Additionally, the third keyword containing detailed information about the object may include a keyword that expresses the second keyword in detail, and the fourth keyword regarding the background may include a keyword regarding a description or explanation of a background area other than the object expressed in the foreground. More specifically, the computing device (100) may obtain a second-1 keyword regarding a higher-level conceptual object, or a second-2 keyword regarding a lower-level conceptual object of the second-1 keyword, based on the natural language input.For example, the 2-1 keyword for the above-mentioned superordinate concept object may include concepts for comprehensive objects such as head and body, and the 2-2 keyword for the subordinate concept object of the above-mentioned 2-1 keyword may include, exemplarily, eyes, nose, mouth, eyebrows, etc., which are subordinate concepts of the head, and exemplarily include legs, hands, fingers, feet, etc., which are subordinate concepts of the body. As another example, the computing device (100) may obtain a 3-1 keyword related to one or more actions based on the above-mentioned natural language input, or obtain a 3-2 keyword related to one or more states based on the above-mentioned natural language input. At this time, the 3-1 keyword related to one or more actions may include keywords related to the behavior or activity of the object in the image, such as "moving eyes" or "unable to move," and the 3-2 keyword related to one or more states may include keywords related to the state of the object, such as "tensed expression" or "overwhelmed," but various examples other than the above examples may be utilized. Meanwhile, the computing device (100) may obtain a prompt for inputting into a neural network model based on at least one acquired keyword, and a specific explanation regarding this will be described later through FIGS. 4 to 6.

[0115] According to one embodiment of the present disclosure, a computing device (100) may obtain a prompt for inputting to a neural network model based on at least one keyword obtained through step S120 (S130). At this time, the neural network model includes a neural network model capable of performing at least one processing of encoding or decoding of the input data, and the computing device (100) may utilize a neural network model trained to remove part or all of the noise from the noise-containing data and to obtain restored data from which part or all of the noise has been removed as the neural network model. For example, a diffusion model, or a latent diffusion model such as stable diffusion or midjourney, may be utilized as the neural network model, but is not limited thereto and various examples may be utilized. Additionally, the above prompt may refer to a converted result that can be used as a condition in the process of acquiring an image when the keyword is input into the neural network model based on methods utilizing rule-based or language models such as GPT, but is not limited thereto and may include various commands corresponding to the input format in a computer program or system. Meanwhile, the computing device (100) may determine a weight for the prompt based on at least one keyword, and an explanation thereof will be described below.

[0116] According to one embodiment of the present disclosure, a computing device (100) can determine a weight for a prompt obtained through step S130 based on at least one keyword obtained through step S120 (S140). In this regard, the computing device (100) can determine a weight for each prompt input as a generation condition to the neural network model. For example, when the computing device (100) inputs "steak" and "red velvet cake" as prompts to the neural network model to obtain an image of a dinner, the computing device (100) can determine a weight of 0.8 for the keyword "steak" and a weight of 0.2 for the keyword "red velvet cake" and input them as generation conditions to the neural network model. Through this, in the image generated using the prompt reflecting the above weights, features of "steak" (e.g., light brown and bright red cross-section, dark brown upper and lower cross-sections, etc., features of well-grilled meat) can be expressed relatively more than features of "red velvet cake" (e.g., dark red cross-section, texture of a layered cake bread, etc., features of a red cake). On the other hand, if the computing device (100) determines the weight for the keyword "steak" to be 0.2 and the weight for the keyword "red velvet cake" to be 0.8 and inputs them as a generation condition to the neural network model (11), the features of "steak" in the generated result image can be expressed relatively less than the features of "red velvet cake." Accordingly, the computing device (100) can control the visual features expressed in the resulting image by adjusting the conditions for generating image information (e.g., the prompt and the weights thereof) during the process of acquiring an image using the neural network model. Meanwhile, the 0.8, 0.In addition to the example of 2, various other examples may be determined as weights, and various examples such as text embedding through a multi-modal encoder as well as prompts may be used as conditions for generation in the process of acquiring images or videos using the neural network model. Meanwhile, various examples may be used for the method of assigning weights to the conditions of the neural network model (11), such as the method of generating a feature map with weights reflected using a cross attention mechanism.

[0117] For example, the computing device (100) may determine a weight for the prompt based on the weight of at least one keyword in the natural language input. Alternatively, if the natural language input includes a 2-2 keyword for a sub-concept object of the 2-1 keyword, the computing device (100) may determine the weight for the 2-2 keyword to be greater than the weight of the 2-1 keyword. Additionally, if the natural language input includes a 3-1 keyword related to one or more actions, the computing device (100) may set the weight of the keyword for an object associated with the 3-1 keyword to be greater than the weight of the keyword for an object not associated with the 3-1 keyword. As another example, when the natural language input includes a third-2 keyword associated with one or more states, the computing device (100) may set the weight of the keyword for an object associated with the third-2 keyword to be greater than the weight of the keyword for an object not associated with the third-2 keyword. In this regard, the computing device (100) can optimize and reflect various scene conditions, such as text, emotion, and location, during the process of acquiring the video described later by determining a high weight for the prompt so that specific keywords regarding the details of the object are expressed in greater detail and occupy a larger screen ratio. Meanwhile, the computing device (100) may acquire the video based on the prompt with the determined weight using the neural network model, and an explanation thereof will be provided below through FIGS. 4 to 6.

[0118] According to one embodiment of the present disclosure, a computing device (100) can acquire an image based on a prompt with a determined weight through step S140 by utilizing the neural network model (S150). For example, the computing device (100) can acquire an image based on the prompt with the determined weight by utilizing the neural network model, or acquire a video based on the prompt with the determined weight. In this regard, the computing device (100) can acquire an image by determining a high weight for the prompt so that specific keywords regarding the details of an object are expressed in greater detail and occupy a large screen ratio, thereby optimizing and reflecting various scene conditions such as text, emotions, and locations, and can acquire a natural video in which visual features of scenes requiring detailed description, such as emotional changes, are well expressed.

[0119] Additionally, the computing device (100) may acquire additional natural language input related to the additional image for the image and acquire at least one additional keyword based on the additional natural language input. Furthermore, the computing device (100) may acquire an additional prompt for inputting into a neural network model based on the at least one additional keyword and determine a weight for the additional prompt based on the at least one additional keyword. Subsequently, the computing device (100) may acquire an additional image based on the image and the additional prompt with the determined weight by utilizing the neural network model. In this regard, during the process of acquiring the additional image, the computing device (100) may enhance the continuity between scenes and acquire an image in which each scene flows naturally by utilizing visual features such as lighting and changes in a person's emotions included in the previously acquired image in addition to the additional keyword with the determined weight. Meanwhile, a detailed explanation of the process by which the computing device (100) acquires an additional image based on the additional prompt with the determined weight will be described later through FIG. 6.

[0120]

[0121] FIG. 4 is a schematic diagram illustrating a process of acquiring an image based on at least one keyword using a neural network model according to one embodiment of the present disclosure.

[0122] Referring to FIG. 4, the computing device (100) can acquire natural language input related to the image, and the natural language input (20) related to the image may include text related to a scenario used in the production of storyboards and storyboards for movies, animations, etc., such as "Scene #1: Protagonist A was overwhelmed by the scenery of the magnificent mountains in the distance and could not move." The natural language input (20) related to the image may include, for example, text related to a specific scenario or emotional change of a specific character within the movie, but is not limited to the above examples.

[0123] Additionally, the computing device (100) may acquire at least one keyword (21 to 24) based on the acquired natural language input (20), and the computing device (100) may input the natural language input (20) into a pre-trained language model and acquire at least one keyword (21 to 24) based on one or more output data output based on the language model. At this time, the language model may use a pre-trained Transformer (GPT family) model, and examples include Google's Bard, Chat-GPT, etc., but are not limited thereto and various language models may be used.

[0124] Specifically, the computing device (100) can obtain, based on the natural language input (20), a first keyword (21) for distance, "distant," a second keyword (22) for object, "protagonist A," a third keyword (23) containing details about object, "overwhelmed" (23-1) and "unable to move" (23-2), or a fourth keyword (24) for background, "mountain scenery." Subsequently, the computing device (100) can obtain a prompt for inputting into a neural network model (11) based on at least one of the obtained keywords (21 to 24). At this time, the neural network model (11) includes a neural network model capable of performing at least one of encoding or decoding of input data, and the computing device (100) may utilize the neural network model (11) as a neural network model trained to remove part or all of the noise from data containing noise and to obtain restored data from which part or all of the noise has been removed. For example, a diffusion model, or a latent diffusion model such as stable diffusion or mid-journey, may be utilized as the neural network model (11), but is not limited thereto and various examples may be utilized. Additionally, the prompt may refer to a converted result that can be used as a condition in the process of acquiring an image when the keywords (21 to 24) are input into the neural network model (11) based on methods utilizing rule-based or language models such as GPT, but is not limited thereto and may include various commands corresponding to the input format in a computer program or system.

[0125] Subsequently, the computing device (100) can determine a weight (30) for the prompt based on the at least one keyword (21 to 24). Specifically, the computing device (100) can determine a weight (30) for each prompt that is input as a generation condition to the neural network model (11), and in this process, can determine a weight (30) for the prompt based on the proportion occupied by the at least one keyword (21 to 24). For example, since the computing device (100) has a larger proportion of the keyword "landscape of a magnificent mountain seen in the distance" compared to "protagonist A", it can determine the weight for the fourth keyword "landscape of a mountain" (24) to be 1.5, which is higher than the weight of the second keyword "protagonist A" (22), which is 0.5. Additionally, the computing device (100) can determine a weight for the "protagonist A" (22) to be 0.5, which is less than the reference value of 1, based on the first keyword regarding distance, "seeing from afar" (21), and input this as a generation condition to the neural network model (11). Through this, in the image (40) generated using the prompt (31) with the weight reflected therein, the feature of "a magnificent mountain landscape seen from afar" (e.g., a large, high mountain that appears above the head of the protagonist A and occupies a larger portion of the screen compared to the protagonist A) can be expressed relatively more than the feature of the specific person "protagonist A" (e.g., a back view of a person wearing a hooded hat and not moving).Accordingly, the computing device (100) can control the visual features expressed in the resulting image (40) by adjusting the conditions for generating image information (e.g., the prompt and the weight (31) for it) during the process of acquiring an image using the neural network model (11). Specifically, keywords that have a large proportion in the natural language input are expressed in greater detail, and by determining a high weight for the prompt to occupy a large screen ratio, a natural image reflecting visual importance and the context of the scenario can be acquired compared to the case where an image is acquired using only keywords. Meanwhile, in addition to the examples of 1.5 and 0.5, various other examples may be determined as weights, and various other examples may be used as conditions for generating images or videos during the process of acquiring an image or video using the neural network model (11). Meanwhile, various examples of methods for assigning weights to the generation conditions of the neural network model (11) may be utilized, such as a method of generating a feature map with weights reflected using a cross attention mechanism. Meanwhile, the computing device (100) can acquire an image by determining a weight for the prompt based on the details of the keyword, and an explanation thereof will be described later through FIG. 5.

[0126]

[0127] FIG. 5 is a schematic diagram illustrating a process for determining a weight for a prompt based on at least one keyword according to one embodiment of the present disclosure, and acquiring an image based on the prompt with the determined weight using a neural network model.

[0128] Referring to FIG. 5, the computing device (100) can acquire natural language input related to a video, and the natural language input (20') related to the video may include text related to a scenario used in the production of storyboards and storyboards for movies, animations, etc., such as "Scene #11: Protagonist B was moving his two eyes incessantly with a clearly tense expression." The natural language input (20') related to the video may include, for example, text related to a specific scenario or emotional change of a specific character within the movie, but is not limited to the above examples. Additionally, the computing device (100) can acquire at least one keyword (22-1' to 23-2') based on the acquired natural language input (20'), and the computing device (100) can input the natural language input (20') into a pre-trained language model and acquire the at least one keyword (22-1' to 23-2') based on one or more output data output based on the language model. At this time, the language model mentioned above may be a pre-trained Transformer (GPT family) model, and examples such as Google's Bard and Chat-GPT may be utilized, but are not limited to these and various language models may be used.

[0129] Specifically, the computing device (100) can obtain a 2-1 keyword (22-1') for a higher-level concept object, "Protagonist B," and a 2-2 keyword (22-2') for a lower-level concept object of "Protagonist B" (22-1), "Protagonist B's two eyes," based on the natural language input (20'). Additionally, the computing device (100) can obtain a 3-1 keyword, "moving eyes" (23-2'), related to the action of "Protagonist B's two eyes" (22-2'), and a 3-2 keyword, "tensed expression" (23-1'), related to the state of "Protagonist B" (22-1'). Subsequently, the computing device (100) can obtain a prompt for inputting into a neural network model (11) based on at least one of the obtained keywords (22-1' to 23-2'). At this time, the neural network model (11) includes a neural network model capable of performing at least one of encoding or decoding of input data, and the computing device (100) may utilize the neural network model (11) trained to remove part or all of the noise from data containing noise and to obtain restored data from which part or all of the noise has been removed. For example, a diffusion model, or a latent diffusion model such as stable diffusion or mid-journey, may be utilized as the neural network model (11), but is not limited thereto and various examples may be utilized.Additionally, the above prompt may refer to a converted result that can be used as a condition in the process of acquiring an image by inputting the keywords (22-1' to 23-2') into the neural network model (11) based on methods utilizing a rule-based or language model such as GPT, but is not limited thereto and may include various commands corresponding to the input format in a computer program or system.

[0130] Subsequently, the computing device (100) can determine a weight (30') for the prompt based on at least one keyword (22-1' to 23-2'). Specifically, the computing device (100) can determine a weight (30') for each prompt that is input as a generation condition to the neural network model (11), and in this process, if the natural language input (20') includes the 2-2 keyword "the two eyes of protagonist B" (22-2') for the sub-concept object of the 2-1 keyword "protagonist B" (22-1'), the computing device (100) can determine the weight for the 2-2 keyword "the two eyes of protagonist B" (22-2') to be greater than the weight of the "protagonist B" (22-1'). For example, the computing device (100) can determine the weight for "protagonist B's two eyes" (22-2') and "moving eyes" (23-2') as a weight of 2, which is higher than the weight of "protagonist B" (22-1') and "tensed expression" (23-1'), and input it as a generation condition to the neural network model (11). Through this, in the image (40') generated using the prompt (31') that reflects the weight, the feature of "protagonist B's eyes moving" (e.g., the feature of protagonist B's two eyes shaking while tense among his body parts) can be expressed relatively more than the feature of "protagonist B's tense expression" (e.g., the feature of protagonist B's overall face with dry lips and sweat) of a specific person.Accordingly, the computing device (100) can control the visual features expressed in the resulting image (40') by adjusting the conditions for generating image information (e.g., the prompt and the corresponding weight (31')) during the process of acquiring an image using the neural network model (11). By determining a high weight for the prompt so that specific keywords regarding the details of an object are expressed in greater detail and occupy a large screen ratio, the computing device can optimize and reflect various scene conditions such as text, emotion, and location, and acquire a natural image in which visual features for scenes requiring detailed description, such as emotional changes, are well expressed. Meanwhile, in addition to the examples of 2 and 1, various other examples may be determined as weights, and various examples may be utilized as conditions for generating images or videos during the process of acquiring an image or video using the neural network model (11). Meanwhile, the computing device (100) can acquire additional images based on additional prompts for which weights have been determined, and an explanation of this will be provided below through FIG. 6.

[0131]

[0132] FIG. 6 is a schematic diagram illustrating the process of acquiring additional images based on additional prompts with determined weights using a neural network model according to one embodiment of the present disclosure.

[0133] Referring to FIG. 6, the computing device (100) may include "Scene #2: The protagonist A, overwhelmed by the scenery, ended up shedding tears of emotion" as an additional natural language input (50) related to the image (40), but the additional natural language input (50) related to the image may not be limited to the above example and various examples may be utilized. Based on the additional natural language input (50), the computing device (100) may obtain at least one additional keyword, such as "protagonist A" (51), "overwhelmed" (52-1), "shed tears of emotion" (52-2), etc. Additionally, the computing device (100) may obtain an additional prompt for inputting to a neural network model (11) based on the at least one additional keyword (51 to 52-2), and determine a weight (60) for the additional prompt based on the at least one additional keyword (51 to 52-2). At this time, the examples described above in the present disclosure may be utilized in the process in which the computing device (100) determines the weight for the additional prompt. Subsequently, the computing device (100) may acquire an additional image (70) based on the image (40) and the additional prompt (61) for which the weight has been determined by utilizing the neural network model (11). Specifically, in the process of acquiring the additional image (70), the computing device (100) may acquire a video in which each scene requiring detailed description, such as emotional changes, naturally follows by utilizing visual features such as "visual features of lighting (sun) shining from the mountain towards protagonist A", "visual features of the bluish sky at dawn", and "emotional changes of protagonist A" in the image (40) of Scene 1 acquired through the example of FIG. 4, in addition to the additional prompt (61) for which the weight has been determined.

[0134]

[0135] FIG. 7 is a brief and general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

[0136] Although the present disclosure has been described as generally being implementable by a computing device, a person skilled in the art will be well aware that the present disclosure may be implemented in combination with computer-executable instructions and / or other program modules that can be executed on one or more computers and / or as a combination of hardware and software.

[0137] Generally, a program module includes routines, programs, components, data structures, etc., that perform a specific task or implement a specific abstract data type. Furthermore, a person skilled in the art will be well aware that the method of the present disclosure can be implemented in other computer system configurations, including single-processor or multi-processor computer systems, minicomputers, mainframe computers, as well as personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, etc. (each of which may be connected to and operated with one or more associated devices).

[0138] The embodiments described in this disclosure may also be implemented in a distributed computing environment in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0139] Computers typically include various computer-readable media. Any medium accessible by a computer may be a computer-readable medium, and such computer-readable media include volatile and non-volatile media, transitory and non-transitory media, and removable and non-removable media. By example, but not limiting, computer-readable media may include computer-readable storage media and computer-readable transmission media. Computer-readable storage media include volatile and non-volatile media, transitory and non-transitory media, and removable and non-removable media implemented by any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, DVD (digital video disk) or other optical disk storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or any other media that can be accessed by a computer and used to store desired information.

[0140] Computer-readable transmission media typically include all information transmission media that implement computer-readable instructions, data structures, program modules, or other data, etc., on a modulated data signal, such as a carrier wave or other transport mechanism. The term modulated data signal means a signal in which one or more of the characteristics of the signal are set or modified to encode information within the signal. By example, not limiting, computer-readable transmission media include wired media, such as wired networks or direct-wired connections, and wireless media, such as acoustic, RF, infrared, and other wireless media. Any combination of the media described above is also considered to be within the scope of computer-readable transmission media.

[0141] An exemplary environment (1100) for implementing various aspects of the present disclosure, including a computer (1102), is shown, wherein the computer (1102) includes a processing unit (1104), system memory (1106), and a system bus (1108). The system bus (1108) connects system components, including system memory (1106) (but not limited thereto), to the processing unit (1104). The processing unit (1104) may be any processor among various commercial processors. Dual processor and other multiprocessor architectures may also be used as the processing unit (1104).

[0142] The system bus (1108) may be any of several types of bus structures that can be additionally interconnected to a local bus using any of the memory bus, peripheral bus, and various commercial bus architectures. System memory (1106) includes read-only memory (ROM) (1110) and random access memory (RAM) (1112). The basic input / output system (BIOS) is stored in non-volatile memory (1110), such as ROM, EPROM, EEPROM, etc., and this BIOS includes basic routines that help transfer information between components within the computer (1102) at times such as during startup. The RAM (1112) may also include high-speed RAM, such as static RAM, for caching data.

[0143] The computer (1102) also includes an internal hard disk drive (HDD) (1114) (e.g., EIDE, SATA)—this internal hard disk drive (1114) may also be configured for external use within a suitable chassis (not shown)—a magnetic floppy disk drive (FDD) (1116) (e.g., for reading from or writing to a removable diskette (1118)), and an optical disk drive (1120) (e.g., for reading from a CD-ROM disk (1122) or reading from or writing to other high-capacity optical media such as a DVD). The hard disk drive (1114), the magnetic disk drive (1116), and the optical disk drive (1120) may each be connected to the system bus (1108) by a hard disk drive interface (1124), a magnetic disk drive interface (1126), and an optical drive interface (1128). The interface (1124) for implementing an external drive includes at least one or both of USB (Universal Serial Bus) and IEEE 1394 interface technologies.

[0144] These drives and associated computer-readable media provide non-volatile storage of data, data structures, computer-executable instructions, etc. In the case of a computer (1102), the drives and media correspond to storing any data in a suitable digital format. Although the description of computer-readable media above refers to HDDs, removable magnetic disks, and removable optical media such as CDs or DVDs, a person skilled in the art will know that other types of computer-readable media, such as zip drives, magnetic cassettes, flash memory cards, cartridges, etc., may also be used in exemplary operating environments and that any of these media may contain computer-executable instructions for performing the methods of the present disclosure.

[0145] A number of program modules, including an operating system (1130), one or more application programs (1132), other program modules (1134), and program data (1136), may be stored in the drive and RAM (1112). All or part of the operating system, application, module and / or data may also be cached in RAM (1112). It will be well known that the present disclosure may be implemented in various commercially available operating systems or combinations of operating systems.

[0146] The user can input commands and information into the computer (1102) through one or more wired / wireless input devices, such as a pointing device like a keyboard (1138) and a mouse (1140). Other input devices (not shown) may include a microphone, an IR remote control, a joystick, a game pad, a stylus pen, a touch screen, etc. These and other input devices are often connected to the processing unit (1104) via an input device interface (1142) connected to the system bus (1108), but may also be connected via other interfaces such as a parallel port, an IEEE 1394 serial port, a game port, a USB port, an IR interface, etc.

[0147] A monitor (1144) or other type of display device is also connected to the system bus (1108) via an interface such as a video adapter (1146). In addition to the monitor (1144), the computer generally includes other peripheral output devices (not shown), such as speakers, a printer, and so on.

[0148] The computer (1102) may operate in a networked environment using a logical connection to one or more remote computers, such as remote computer(s) (1148), via wired and / or wireless communication. The remote computer(s) (1148) may be a workstation, a computing device computer, a router, a personal computer, a portable computer, a microprocessor-based entertainment device, a peer device, or other conventional network node, and generally include many or all of the components described for the computer (1102), but for brevity, only the memory storage device (1150) is illustrated. The illustrated logical connection includes a wired / wireless connection to a local area network (LAN) (1152) and / or a larger network, e.g., a wide area network (WAN) (1154). Such LAN and WAN networking environments are common in offices and companies and facilitate enterprise-wide computer networks, such as intranets, all of which can be connected to a global computer network, e.g., the Internet.

[0149] When used in a LAN networking environment, the computer (1102) is connected to a local network (1152) via a wired and / or wireless communication network interface or adapter (1156). The adapter (1156) may facilitate wired or wireless communication to the LAN (1152), and the LAN (1152) may also include a wireless access point installed therein to communicate with the wireless adapter (1156). When used in a WAN networking environment, the computer (1102) may include a modem (1158), be connected to a communication computing device on the WAN (1154), or have other means to establish communication through the WAN (1154), such as through the Internet. The modem (1158), which may be an internal or external and a wired or wireless device, is connected to the system bus (1108) via a serial port interface (1142). In a networked environment, the program modules described for the computer (1102) or parts thereof may be stored in a remote memory / storage device (1150). It will be well known that the illustrated network connection is exemplary and that other means of establishing a communication link between computers may be used.

[0150] The computer (1102) operates to communicate with any wireless device or object that is deployed and operated via wireless communication, for example, a printer, scanner, desktop and / or portable computer, PDA (portable data assistant), communication satellite, any equipment or place associated with a wireless detectable tag, and a telephone. This includes at least Wi-Fi and Bluetooth wireless technologies. Accordingly, the communication may be a predefined structure as in a conventional network, or simply ad hoc communication between at least two devices.

[0151] Wi-Fi (Wireless Fidelity) enables connectivity to the Internet and other sources without wires. Wi-Fi is a wireless technology, similar to a cell phone, that allows devices, such as computers, to transmit and receive data indoors and outdoors—that is, anywhere within the coverage area of ​​a base station. Wi-Fi networks use a wireless technology called IEEE 802.11 (a, b, g, etc.) to provide secure, reliable, and high-speed wireless connections. Wi-Fi can be used to connect computers to each other, to the Internet, and to wired networks (using IEEE 802.3 or Ethernet). Wi-Fi networks can operate in unlicensed 2.4 and 5 GHz wireless bands, for example, at data rates of 11 Mbps (802.11a) or 54 Mbps (802.11b), or in products that include both bands (dual band).

[0152] Those skilled in the art of the present disclosure will understand that information and signals may be represented using any various different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced in the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0153] Those skilled in the art will understand that the various exemplary logic blocks, modules, processors, means, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented by electronic hardware, various forms of programs or design code (referred to herein as software for convenience), or a combination of all such. To clearly illustrate this interoperability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above in relation to their functions. Whether such functions are implemented as hardware or software depends on the design constraints imposed on the specific application and the overall system. Those skilled in the art may implement the functions described in various ways for each specific application, but such implementation decisions should not be interpreted as being outside the scope of this disclosure.

[0154] The various embodiments presented herein may be implemented as methods, devices, or articles manufactured using standard programming and / or engineering techniques. The term "article manufactured" includes a computer program, a carrier, or a medium accessible from any computer-readable storage device. For example, computer-readable storage media include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical discs (e.g., CDs, DVDs, etc.), smart cards, and flash memory devices (e.g., EEPROMs, cards, sticks, key drives, etc.). Additionally, the various storage media presented herein include one or more devices and / or other machine-readable media for storing information.

[0155] It should be understood that the specific order or hierarchy of steps in the presented processes is an example of exemplary approaches. It should be understood that the specific order or hierarchy of steps in the processes may be rearranged within the scope of this disclosure based on design priorities. The appended method claims provide elements of various steps in a sample order, but do not imply being limited to the specific order or hierarchy presented.

[0156] Description of the presented embodiments is provided so that a person skilled in the art may use or practice the present disclosure. Various modifications to these embodiments will be apparent to a person skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not limited to the embodiments presented herein, but should be interpreted in the broadest possible scope consistent with the principles and novel features presented herein.

[0157] As described above, the relevant details have been described in the best mode for carrying out the invention.

Claims

1. A method for acquiring an image using a neural network model executed by one or more processors of a computing device, wherein A step of acquiring natural language input related to an image; A step of obtaining at least one keyword based on the above natural language input; A step of obtaining a prompt for inputting into a neural network model based on at least one keyword; A step of determining a weight for the prompt based on at least one keyword; and A step of acquiring an image based on a prompt with determined weights using the above neural network model; including, method.

2. In Paragraph 1, The step of obtaining at least one keyword based on the above natural language input is, The step of inputting the above natural language input into a pre-trained language model; and A step of obtaining at least one keyword based on one or more output data output based on the above language model; including, method.

3. In Paragraph 2, The step of obtaining at least one keyword based on the above natural language input is, A step of obtaining a first keyword for distance based on the above natural language input; A step of obtaining a second keyword for an object based on the above natural language input; A step of obtaining a third keyword containing details about the object based on the above natural language input; or A step of obtaining a fourth keyword for background based on the above natural language input; including at least one of, method.

4. In Paragraph 3, The step of obtaining a second keyword for an object based on the above natural language input is, A step of obtaining a 2-1 keyword for a higher-level concept object based on the above natural language input; or A step of obtaining a 2-2 keyword for a sub-concept object of the 2-1 keyword based on the above natural language input; including at least one of, method.

5. In Paragraph 3, The step of obtaining a third keyword containing details about an object based on the above natural language input is, A step of obtaining a 3-1 keyword associated with one or more actions based on the above natural language input; or A step of obtaining a 3-2 keyword associated with one or more states based on the above natural language input; including at least one of, method.

6. In Paragraph 1, The above neural network model is, A neural network model capable of performing at least one of encoding or decoding of input data, method.

7. In Paragraph 1, The step of acquiring an image based on a weighted prompt utilizing the above neural network model is: A step of acquiring an image based on a weighted prompt using the above neural network model; or A step of acquiring a video based on a prompt with determined weights using the above neural network model; including at least one of, method.

8. In Paragraph 3, The step of determining a weight for the prompt based on at least one keyword is: A step of determining a weight for the prompt based on the weight of at least one keyword included in the natural language input; including, method.

9. In Paragraph 4, The step of determining a weight for the prompt based on at least one keyword is: If the natural language input includes a 2-2 keyword for a sub-concept object of the 2-1 keyword, the step of determining the weight for the 2-2 keyword to be greater than the weight of the 2-1 keyword; including, method.

10. In Paragraph 5, The step of determining a weight for the prompt based on at least one keyword is: If the above natural language input includes a 3-1 keyword associated with one or more actions, the step of setting the keyword weight for an object associated with the 3-1 keyword to be greater than the keyword weight for an object not associated with the 3-1 keyword; including, method.

11. In Paragraph 5, The step of determining a weight for the prompt based on at least one keyword is: If the above natural language input includes one or more 3-2 keywords related to a state, the step of setting the keyword weight for an object associated with the 3-2 keyword to be greater than the keyword weight for an object not associated with the 3-2 keyword; including, method.

12. In Paragraph 1, The above method is, A step of obtaining additional natural language input related to additional video; A step of obtaining at least one additional keyword based on the above additional natural language input; A step of obtaining an additional prompt for inputting into a neural network model based on at least one additional keyword; A step of determining a weight for the additional prompt based on at least one additional keyword; and A step of acquiring an additional image based on the image and an additional prompt with determined weights using the above neural network model; including, method.

13. A computer program stored on a computer-readable storage medium, wherein, when executed by one or more processors, the computer program causes the one or more processors to perform operations for acquiring an image using a neural network model, and said operations are: An action of acquiring natural language input related to an image; An operation to obtain at least one keyword based on the above natural language input; The operation of obtaining a prompt for inputting into a neural network model based on at least one keyword; An operation to determine a weight for the prompt based on at least one keyword; and An operation to acquire an image based on a prompt with determined weights using the above neural network model; including, A computer program stored on a computer-readable storage medium.

14. In Paragraph 13, The operation of obtaining at least one keyword based on the above natural language input is, The operation of inputting the above natural language input into a pre-trained language model; and An operation of obtaining at least one keyword based on one or more output data output based on the above language model; including, A computer program stored on a computer-readable storage medium.

15. In Paragraph 14, The operation of obtaining at least one keyword based on the above natural language input is, An operation to obtain a first keyword for distance based on the above natural language input; An operation to obtain a second keyword for an object based on the above natural language input; An operation to obtain a third keyword containing details about the object based on the above natural language input; or An operation to obtain a fourth keyword for background based on the above natural language input; including at least one of, A computer program stored on a computer-readable storage medium.

16. In Paragraph 15, The operation of obtaining a second keyword for an object based on the above natural language input is, An operation to obtain a 2-1 keyword for a higher-level concept object based on the above natural language input; or An operation to obtain a 2-2 keyword for a sub-concept object of the 2-1 keyword based on the above natural language input; including at least one of, A computer program stored on a computer-readable storage medium.

17. In Paragraph 15, The operation of obtaining a third keyword containing details about an object based on the above natural language input is, An action of obtaining a 3-1 keyword associated with one or more actions based on the above natural language input; or An operation to obtain a 3-2 keyword associated with one or more states based on the above natural language input; including at least one of, A computer program stored on a computer-readable storage medium.

18. In Paragraph 12, The operation of acquiring an image based on a weighted prompt utilizing the above neural network model is, An operation to acquire an image based on a prompt with determined weights using the above neural network model; or An operation to acquire a video based on a prompt with determined weights, utilizing the above neural network model; including at least one of, A computer program stored on a computer-readable storage medium.

19. In Paragraph 15, The operation of determining a weight for the prompt based on at least one keyword is, An operation to determine a weight for the prompt based on the weight of at least one keyword included in the natural language input; including, A computer program stored on a computer-readable storage medium.

20. In Paragraph 16, The operation of determining a weight for the prompt based on at least one keyword is, If the above natural language input includes a 2-2 keyword for a sub-concept object of the 2-1 keyword, the operation of determining the weight for the 2-2 keyword to be greater than the weight of the 2-1 keyword; including, A computer program stored on a computer-readable storage medium.

21. In Paragraph 17, The operation of determining a weight for the prompt based on at least one keyword is, If the above natural language input includes a 3-1 keyword associated with one or more actions, the operation of setting the keyword weight for an object associated with the 3-1 keyword to be greater than the keyword weight for an object not associated with the 3-1 keyword; including, A computer program stored on a computer-readable storage medium.

22. In Paragraph 17, The operation of determining a weight for the prompt based on at least one keyword is, If the above natural language input includes one or more 3-2 keywords related to a state, the operation of setting the keyword weight for an object associated with the 3-2 keyword to be greater than the keyword weight for an object not associated with the 3-2 keyword; including, method.

23. As a computing device, At least one processor; and memory Includes, The above at least one processor is, Acquire natural language input related to the image; Based on the above natural language input, at least one keyword is obtained; Obtaining a prompt for inputting into a neural network model based on at least one of the above keywords; Determining a weight for the prompt based on at least one keyword; and Configured to acquire an image based on a weighted prompt utilizing the above neural network model, Computing device.

Images