Method and system for generating contents using language model

Abstract

According to an aspect of the present invention, a method for generating content by using a language model is provided, the method comprising the steps of: extracting information associated with first content by referring to the first content, and causing a language model to be trained by referring to the information associated with the first content; and in response to acquisition of information on a request for generating second content, generating the second content by using the language model.

Description

Method and system for generating content using a language model

[0001] The present invention relates to a method and system for generating content using a language model.

[0002] As artificial intelligence technology rapidly develops, attempts are being made to apply it to various technological fields.

[0003] Generative AI (e.g., a language model like ChatGPT) may include AI that can generate information (e.g., content) based on specific requests (e.g., prompts) from a user.

[0004] Conventional generative AI has occasionally responded to simple questions (e.g., asking about the definition of a specific word) with responses completely outside the context of the question. Therefore, simply applying conventional generative AI to online content to generate new content has limited its ability to generate content appropriate to the context or intent.

[0005] Accordingly, the inventor(s) of the present invention propose a technology that enables the generation of appropriate second content that matches the context by extracting information (e.g., contextual information) associated with first content, training a language model by referring to the information associated with the first content, and generating second content using the language model.

[0006] <Prior Art Literature>

[0007] <Patent Document>

[0008] (Patent Document 1) Patent Publication No. 10-2022-0024579 (March 3, 2022)

[0009] The purpose of the present invention is to solve all of the problems of the above-mentioned prior art.

[0010] In addition, the present invention has another purpose of extracting information associated with the first content by referring to the first content, training a language model by referring to the information associated with the first content, and generating second content by using the language model in response to obtaining information regarding a request for generating second content.

[0011] In addition, another object of the present invention is to enable generation of suitable second content (e.g., second content that matches the context) by training a language model with reference to information associated with the first content.

[0012] In addition, another purpose of the present invention is to enable the generation of suitable content by training a language model with reference to the user's demographic information even when the user's content usage history is insufficient.

[0013] A representative configuration of the present invention to achieve the above purpose is as follows.

[0014] According to one aspect of the present invention, a method is provided, comprising: a step of extracting information associated with first content by referencing first content, training a language model by referencing the information associated with the first content, and a step of generating second content by using the language model in response to obtaining information regarding a request for generating second content.

[0015] According to another aspect of the present invention, a system is provided, including a learning unit that extracts information associated with the first content by referring to the first content and learns a language model by referring to the information associated with the first content, and a generation unit that generates the second content using the language model in response to obtaining information regarding a request for generating the second content.

[0016] In addition, a non-transitory computer-readable recording medium recording another method for implementing the present invention, another system, and a computer program for executing the method are further provided.

[0017] According to the present invention, information related to the first content is extracted by referring to the first content, a language model is trained by referring to the information related to the first content, and in response to information being obtained regarding a request for second content generation, second content can be generated using the language model.

[0018] In addition, according to the present invention, it is possible to generate suitable second content (e.g., second content that matches the context) by training a language model with reference to information associated with the first content.

[0019] In addition, according to the present invention, even if the user's content usage history is insufficient, suitable content can be generated by training a language model with reference to the user's demographic information.

[0020] FIG. 1 is a diagram schematically illustrating the configuration of an entire system for generating content according to one embodiment of the present invention.

[0021] FIG. 2 is a drawing detailing the internal configuration of a content creation system according to one embodiment of the present invention.

[0022] FIG. 3 is a diagram exemplarily showing a situation in which information associated with first content is extracted by referring to first content according to one embodiment of the present invention.

[0023] FIG. 4 is a diagram exemplarily showing a situation in which second content is generated according to one embodiment of the present invention.

[0024] <Explanation of symbols>

[0025] 100: Communications network

[0026] 200: Content Creation System

[0027] 210: Learning Department

[0028] 220: Generation Unit

[0029] 230: Communications Department

[0030] 240: Control Unit

[0031] 300: Device

[0032] 410: First Content

[0033] 420: Comment content

[0034] 510: User-generated content

[0035] 520: Second Content

[0036] The following detailed description of the present invention refers to the accompanying drawings, which illustrate specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention, while different from each other, are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be modified and implemented from one embodiment to another without departing from the spirit and scope of the present invention. Furthermore, it should be understood that the positions or arrangements of individual components within each embodiment may also be modified without departing from the spirit and scope of the present invention. Accordingly, the following detailed description is not to be taken in a limiting sense, and the scope of the present invention is to be construed to encompass the scope of the claims and all equivalents thereof. Like reference numerals in the drawings represent the same or similar elements throughout the several aspects.

[0037] Hereinafter, various preferred embodiments of the present invention will be described in detail with reference to the attached drawings so that a person having ordinary skill in the art to which the present invention pertains can easily practice the present invention.

[0038] Composition of the entire system

[0039] FIG. 1 is a diagram schematically illustrating the configuration of an entire system for generating content according to one embodiment of the present invention.

[0040] As illustrated in FIG. 1, the entire system according to one embodiment of the present invention may include a communication network (100), a content creation system (200), and a device (300).

[0041] First, the communication network (100) according to one embodiment of the present invention can be configured regardless of the communication mode such as wired communication or wireless communication, and can be configured with various communication networks such as a local area network (LAN), a metropolitan area network (MAN), and a wide area network (WAN). Preferably, the communication network (100) referred to herein may be the well-known Internet or the World Wide Web (WWW). However, the communication network (100) is not necessarily limited thereto, and may include at least a portion of a well-known wired or wireless data communication network, a well-known telephone network, or a well-known wired or wireless television communication network.

[0042] For example, the communication network (100) may be a wireless data communication network that implements conventional communication methods such as WiFi communication, WiFi-Direct communication, Long Term Evolution (LTE) communication, 5G communication, Bluetooth communication (including Bluetooth Low Energy (BLE) communication), infrared communication, ultrasonic communication, etc., at least in part. As another example, the communication network (100) may be an optical communication network that implements conventional communication methods such as LiFi (Light Fidelity), etc., at least in part.

[0043] Next, the content creation system (200) according to one embodiment of the present invention may perform a function of extracting information related to the first content by referencing the first content, training a language model by referencing the information related to the first content, and generating second content by using the language model in response to obtaining information regarding a second content creation request.

[0044] The configuration and function of the content creation system (200) according to the present invention will be described in detail below.

[0045] Next, a device (300) according to one embodiment of the present invention is a digital device that includes a function for communicating after connecting to a content creation system (200), and any digital device that has a memory means, a microprocessor, and a computing capability, such as a smart phone, a tablet, a smart watch, a smart band, smart glasses, a desktop computer, a notebook computer, a workstation, a PDA, a web pad, a mobile phone, etc., can be adopted as the device (300) according to the present invention.

[0046] In particular, the device (300) may include an application (not shown) that supports a user to receive functions according to the present invention from the content creation system (200). Such an application may be downloaded from the content creation system (200) or an external application distribution server (not shown). Meanwhile, the characteristics of such an application may be generally similar to the learning unit (210), generation unit (220), communication unit (230), and control unit (240) of the content creation system (200), which will be described later. Here, at least a portion of the application may be replaced with a hardware device or firmware device that can perform functions substantially identical to or equivalent thereto, as necessary.

[0047] Composition of the content creation system

[0048] Below, the internal configuration and functions of each component of the content creation system (200) that performs important functions for implementing the present invention will be examined.

[0049] FIG. 2 is a drawing showing in detail the internal configuration of a content creation system (200) according to one embodiment of the present invention.

[0050] As illustrated in FIG. 2, a content creation system (200) according to one embodiment of the present invention may be configured to include a learning unit (210), a generation unit (220), a communication unit (230), and a control unit (240). According to one embodiment of the present invention, at least some of the learning unit (210), the generation unit (220), the communication unit (230), and the control unit (240) may be program modules that communicate with an external system (not shown). These program modules may be included in the content creation system (200) in the form of an operating system, an application program module, or other program modules, and may be physically stored in various known memory devices. In addition, these program modules may be stored in a remote memory device that can communicate with the content creation system (200). Meanwhile, these program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform specific tasks or execute specific abstract data types, which will be described later, according to the present invention.

[0051] Meanwhile, although the content creation system (200) has been described as above, this description is exemplary, and it is obvious to those skilled in the art that at least some of the components or functions of the content creation system (200) may be realized within a device (300) or a server (not shown) or included within an external system (not shown) as needed.

[0052] First, the learning unit (210) according to one embodiment of the present invention can perform a function of extracting information related to the first content by referring to the first content, and learning a language model by referring to the information related to the first content.

[0053] Specifically, the learning unit (210) according to one embodiment of the present invention can extract information associated with the first content by referring to the first content.

[0054] For example, the first content according to one embodiment of the present invention may include content that can be obtained online (e.g., webtoon content, novel content, comic content, image content, video content, conversation content, voice content, etc.). However, it should be apparent to those skilled in the art that the first content according to one embodiment of the present invention may also include content that can be obtained offline as well as online (e.g., offline content that is converted electronically using OCR and that can be obtained as first content).

[0055] FIG. 3 is a diagram exemplarily showing a situation in which information associated with first content is extracted by referring to first content according to one embodiment of the present invention.

[0056] Information associated with the first content according to one embodiment of the present invention may include at least some of contextual information about the first content, contextual information about content associated with the first content, and contextual information about content derived from the first content.

[0057] For example, contextual information regarding first content according to one embodiment of the present invention may include at least one of information regarding words included in the first content, information regarding sentences included in the first content, information regarding the frequency of use of words included in the first content, and information regarding the subject matter of the first content. Furthermore, contextual information regarding first content according to one embodiment of the present invention may also include visual representations (e.g., pictures) that constitute the first content.

[0058] Referring to FIG. 3, context information regarding the first content (410) according to one embodiment of the present invention may include text information (410a, 410b, 410c, 410d, 410e, 410f, 410g, and 410h) included in the first content (410). Meanwhile, context information regarding the first content (410) according to one embodiment of the present invention may be such that at least some of the text information (410a, 410b, 410c, 410d, 410e, 410f, 410g, and 410h) is selected with reference to a preset condition, as described below.

[0059] For another example, context information about content associated with the first content according to one embodiment of the present invention may include at least one of context information of content that was referenced in the process of generating the first content (e.g., dictionary content), content with the same or similar subject matter as the first content (e.g., other content of the same genre as the first content), other content associated with the content creator associated with the first content (e.g., other webtoon content of the same author), and content that uses words included in the first content with a frequency of use exceeding a preset level a preset number of times or more (e.g., content in which a key word of the first content is used a large number of times).

[0060] As another example, contextual information regarding content derived from the first content according to one embodiment of the present invention may include user-generated content using the first content (e.g., user-generated comment content using webtoon content). Referring to FIG. 3, information regarding comment content (420) (i.e., content derived from the first content (410)) according to one embodiment of the present invention may be displayed on a user interface screen (i.e., indicating that there are 115 comment contents).

[0061] To improve the accuracy of a language model, a process of training the language model using refined information, excluding unnecessary information, may be necessary. Accordingly, according to one embodiment of the present invention, information associated with the first content may be selected by reference to at least some of the following: contextual information about the first content, contextual information about content associated with the first content, and contextual information about content derived from the first content, all of which are pre-defined conditions.

[0062] For example, the process of selecting with reference to a preset condition according to one embodiment of the present invention may be performed in a manner in which information associated with the first content is selected by excluding at least one of sentences consisting of slang (or buzzwords), incomplete words, and sentences consisting of conversational language.

[0063] For another example, the process of selecting with reference to preset conditions according to one embodiment of the present invention may be performed in a manner in which information associated with the first content is selected by selecting sentences composed of standard language.

[0064] As another example, the process of selecting with reference to a preset condition according to one embodiment of the present invention may be performed in a manner of excluding dialogue content included in the first content with reference to picture content (e.g., speech bubble content) included in the first content.

[0065] As another example, the learning unit (210) according to one embodiment of the present invention may determine whether information included in the first content is in a conversational format by referring to the picture content (e.g., speech bubble content) included in the first content, and may select first reference information by excluding text information associated with the content in a conversational format from the first content. Then, the learning unit (210) according to one embodiment of the present invention may select second reference information by excluding text information consisting of incomplete words from the first reference information, and may select the second reference information as information associated with the first content.

[0066] For example, referring to FIG. 3, according to one embodiment of the present invention, by excluding text information (410b, 410d, 410e, 410f, and 410g) associated with sentences in a conversational tone from among text information (410a, 410b, 410c, 410d, 410e, 410f, 410g, and 410h) of the first content (410), information associated with the first content can be selected (or extracted). Here, according to one embodiment of the present invention, a language model can be trained by referring to the text information (410a, 410c, and 410h) associated with the selected first content, and the efficiency of language model training and the accuracy of content generation can be increased.

[0067] For another example, continuing with reference to FIG. 3, the learning unit (210) according to one embodiment of the present invention may determine whether the content included in the first content is in a conversational form by referring to the picture content (e.g., speech bubble content) included in the first content, and may select text information (410a, 410c, and 410h) selected by excluding text information (410b, 410d, 410e, 410f, and 410g) associated with the content in a conversational form from the first content as first reference information. In addition, the learning unit (210) according to one embodiment of the present invention may select second reference information (410a and 410h) by excluding text information (410c) consisting of incomplete words from the first reference information, and may select (or extract) the second reference information as information associated with the first content.

[0068] However, the process of selecting the first reference information and the second reference information according to one embodiment of the present invention may be performed not only by selecting the first reference information and then the second reference information, but also by selecting the second reference information and then the first reference information. For example, the learning unit (210) according to one embodiment of the present invention may select text information selected by excluding text information consisting of incomplete words from among the information included in the first content as the second reference information. Then, the learning unit (210) according to one embodiment of the present invention may determine whether the second reference information is in a conversational tone by referring to the picture content included in the first content, and select the first reference information by excluding text information associated with the content consisting of a conversational tone from the second reference information, and may extract the first reference information as information associated with the first content.

[0069] According to the present invention, by extracting information suitable for learning a language model from among the information included in the first content and enabling the language model to be learned, the response accuracy of the language model can be improved.

[0070] In addition, the learning unit (210) according to one embodiment of the present invention may enable a language model to be trained by referring to information associated with the first content. Here, the language model according to one embodiment of the present invention may include a generative artificial intelligence (e.g., a generative artificial intelligence such as ChatGPT or Bard) that can generate content (e.g., text, images, videos, etc.) in response to a second content generation request (e.g., a prompt or input). In addition, the language model according to one embodiment of the present invention may include a language model associated with a deep learning-based generative artificial intelligence.

[0071] Next, the generation unit (220) according to one embodiment of the present invention can perform a function of generating the second content using a language model in response to information regarding a second content generation request being obtained.

[0072] Specifically, the second content according to one embodiment of the present invention may include content generated using a language model. Here, the second content according to one embodiment of the present invention may include at least one of: question content associated with the first content, comment content associated with the first content, question content associated with the comment content, and answer content for the question content.

[0073] Additionally, information regarding a second content creation request according to one embodiment of the present invention may include at least one of difficulty information regarding the second content, context information regarding the second content (e.g., topic of the second content), and type information regarding the second content (e.g., question or answer).

[0074] For example, a second content creation request according to one embodiment of the present invention may be made by providing user-generated content associated with the second content creation request.

[0075] FIG. 4 is a diagram exemplarily showing a situation in which second content is generated according to one embodiment of the present invention.

[0076] Referring to FIG. 4, user-generated content (510) associated with a second content creation request according to one embodiment of the present invention may be provided, and the creation unit (220) may create the second content (520) using a language model in response to information about the second content creation request being acquired (e.g., acquisition of user-generated content).

[0077] For example, referring to FIG. 4, when a user according to one embodiment of the present invention organizes the contents of the first content by himself / herself after using the first content to create user-generated content (510), the generation unit (220) according to one embodiment of the present invention can determine whether the contents of the first content are appropriately organized in the user-generated content (510), and if the contents of the first content are appropriately organized, can create content (i.e., second content (520)) that includes information that the contents of the first content are appropriately organized.

[0078] For another example, when a user according to one embodiment of the present invention uses first content and then creates question content (i.e., user-generated content) related to the first content, the generation unit (220) according to one embodiment of the present invention can obtain the question content as information regarding a second content creation request and create answer content corresponding to the question content.

[0079] As another example, when a user according to one embodiment of the present invention creates user-generated content (e.g., content including a request to create a question about webtoon A at a 4th grade elementary school level), the generation unit (220) according to one embodiment of the present invention can generate second content corresponding to the user-generated content.

[0080] Meanwhile, the learning unit (210) according to one embodiment of the present invention may perform a function to enable the language model to be learned by further referring to information associated with the user.

[0081] Specifically, information associated with a user according to one embodiment of the present invention may include at least one of demographic information of the user (e.g., information about age, gender, religion, residence, etc.) and information about the user's content usage history (e.g., information about the time the user used the content, information about the user's response to past question content (e.g., correct answer rate information), information about the time the user accessed the content creation system, etc.).

[0082] For example, a language model according to one embodiment of the present invention may be trained by further referencing information associated with a user (e.g., demographic information indicating that the user is a 4th grade male student and information regarding content usage history indicating that the user uses Webtoon A for an average of 15 minutes per day). Here, referring to FIG. 4, for example, when a 4th grade male student uses Webtoon A for an average of 15 minutes per day and creates user-generated content (510), the generation unit (220) may determine that the first content is appropriately organized in the user-generated content (510) and generate the second content (520).

[0083] Accordingly, referring to FIG. 4, when a user according to one embodiment of the present invention creates user-generated content (510) (e.g., content organized from first content), even if only the content organized from first content is created as the user-generated content (510) without including difficulty information of the second content (520) in the user-generated content (510), the language model can already be trained by referring to information associated with the user, and second content (520) customized to the user's learning ability can be created.

[0084] Meanwhile, according to one embodiment of the present invention, it may be assumed that a user has insufficient access history to the content creation system (200) (e.g., the user is using the first content for the first time or has used it for a short period of time). In this case, according to one embodiment of the present invention, it is difficult to train a language model by referencing information regarding the user's content usage history, making it difficult to generate customized second content for the user.

[0085] Accordingly, the learning unit (210) according to one embodiment of the present invention can perform a function of learning a language model by referring to the user's demographic information in response to the user's content usage history not meeting a preset condition.

[0086] For example, according to one embodiment of the present invention, when a third-year male high school student who has newly accessed the content creation system (200) (e.g., has no past usage history) provides a second content creation request for a question about a specific webtoon related to science, the creation unit (220) according to one embodiment of the present invention can create the second content (e.g., question content about a specific webtoon related to the science level of a third-year high school student) by referring to demographic information (e.g., information about being a third-year male high school student).

[0087] Meanwhile, the content creation system (200) according to one embodiment of the present invention may further include a management unit (not shown). Here, the management unit according to one embodiment of the present invention may update the language model by referring to the user's response regarding the second content being created.

[0088] Referring to FIG. 4, according to one embodiment of the present invention, a user may provide a response regarding the suitability of generated second content (e.g., a response regarding whether the generated second content is suitable), and the management unit according to one embodiment of the present invention may update the language model by referencing the user's response regarding the suitability of the second content. Accordingly, according to the present invention, the language model can be further improved by reflecting the user's feedback.

[0089] For example, a user using second content according to one embodiment of the present invention can provide information about the difficulty of the second content (e.g., information that the difficulty of the second content is somewhat high) to the content creation system (200), and the management unit can update the language model with reference to the information about the difficulty of the second content. Here, according to one embodiment of the present invention, when there is a new second content creation request from the same user, the creation unit (220) can use the updated language model to create second content suitable for the user (e.g., content with a lower difficulty than before).

[0090] Next, the communication unit (230) according to one embodiment of the present invention can perform a function that enables data transmission and reception from / to the learning unit (210) and the generation unit (220).

[0091] Finally, the control unit (240) according to one embodiment of the present invention can perform a function of controlling the flow of data between the learning unit (210), the generation unit (220), and the communication unit (230). That is, the control unit (240) according to one embodiment of the present invention can control the flow of data from / to the outside of the content creation system (200) or the flow of data between each component of the content creation system (200), thereby controlling the learning unit (210), the generation unit (220), and the communication unit (230) to perform their own functions.

[0092] The embodiments of the present invention described above may be implemented in the form of program commands that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, etc., either singly or in combination. The program commands recorded on the computer-readable recording medium may be specially designed and configured for the present invention or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. Hardware devices may be changed into one or more software modules to perform processing according to the present invention, and vice versa.

[0093] Although the present invention has been described above with specific details such as specific components and limited examples and drawings, these are provided only to help a more general understanding of the present invention, and the present invention is not limited to the above examples, and those with ordinary knowledge in the technical field to which the present invention pertains can make various modifications and changes based on this description.

[0094] Therefore, the idea of ​​the present invention should not be limited to the embodiments described above, and not only the scope of the patent claims described below but also all scopes equivalent to or equivalently modified from the scope of the patent claims are considered to fall within the scope of the idea of ​​the present invention.

Claims

1. A method for generating content using a language model, A step of extracting information related to the first content by referring to the first content, and training a language model by referring to the information related to the first content, and In response to obtaining information about a second content creation request, a step of creating the second content using the language model is included. method.

2. In paragraph 1, The information associated with the first content includes at least some of contextual information about the first content, contextual information about content associated with the first content, and contextual information about content derived from the first content. method.

3. In paragraph 1, The information associated with the first content is selected with reference to conditions in which at least some of the context information about the first content, the context information about content associated with the first content, and the context information about content derived from the first content are preset. method.

4. In paragraph 1, In the above learning step, the language model is trained by further referencing information associated with the user. method.

5. In paragraph 4, In response to the user's content usage history not meeting the preset conditions, the language model is trained by referring to the user's demographic information. method.

6. In paragraph 1, The information regarding the second content creation request includes at least one of difficulty information regarding the second content, context information regarding the second content, and type information regarding the second content. method.

7. In paragraph 1, The language model is updated with reference to the user's response regarding the second content generated above. method.

8. A non-transitory computer-readable recording medium recording a computer program for executing the method according to paragraph 1.

9. A system for generating content using a language model, A learning unit that extracts information related to the first content by referring to the first content and trains a language model by referring to the information related to the first content, and In response to obtaining information about a second content creation request, a generation unit is included that generates the second content using the language model. System.

10. In paragraph 9, The information associated with the first content includes at least some of contextual information about the first content, contextual information about content associated with the first content, and contextual information about content derived from the first content. System.

11. In paragraph 9, The information associated with the first content is selected with reference to conditions in which at least some of the context information about the first content, the context information about content associated with the first content, and the context information about content derived from the first content are preset. System.

12. In paragraph 9, The above learning unit allows the language model to be learned by further referencing information related to the user. System.

13. In paragraph 12, In response to the user's content usage history not meeting the preset conditions, the language model is trained by referring to the user's demographic information. System.

14. In paragraph 9, The information regarding the second content creation request includes at least one of difficulty information regarding the second content, context information regarding the second content, and type information regarding the second content. System.

15. In paragraph 9, The language model is updated with reference to the user's response regarding the second content generated above. System.

Images