Content provision system and interactive content provision method performed thereby
Patent Information
- Application Number
- PCT/KR2025/005177
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-04-16
- Publication Date
- 2026-10-22
Smart Images
Figure KR2025005177_22102026_PF_FP_ABST
Abstract
Description
Content provision system and method for providing interactive content by the same
[0001] The present invention relates to a content provision system that provides an environment in which a user can easily play content and enables the progression of content through interaction between a user and a device, and an interactive content provision method based thereon.
[0002]
[0003] The content described in this section merely provides background information regarding the present embodiment and does not constitute prior art.
[0004] Auditory stimulation can have a beneficial impact on the growth and development of infants and toddlers, who begin to perceive and learn about the world through auditory stimuli from the fetal stage. Various auditory stimuli significantly influence not only language development but also the cultivation of creativity and imagination. However, even when parents wish to deliver effective auditory stimulation to their children, there is a lack of means to stimulate the curiosity of the actual users—the infants—and provide an easy-to-use operating environment. Consequently, parents currently rely on digital video devices (TVs, smartphones, etc.) to indirectly deliver auditory stimulation by playing videos that prioritize visual stimulation. Therefore, there is a great demand for a means equipped with an operating environment that stimulates the curiosity of infants and toddlers and allows them to easily play various sounds without relying on their parents' digital video devices.
[0005] Furthermore, videos generally played on digital devices are merely one-way, passive content rather than interactive content designed to elicit user response; consequently, they are ineffective in fostering the development of social skills and agency, which are critically important during the early childhood stage. In particular, as the age of infants and toddlers increases, the importance of interaction grows even more; however, due to the lack of appropriate means to meet these demands, it is difficult to establish an environment at home that promotes the development of social skills and agency beyond communication with parents. Therefore, there is a need to develop a system that allows infant and toddler users to directly select the sounds they wish to play or interact with content through voice input.
[0006]
[0007] [Prior Art Literature]
[0008] [Patent Literature]
[0009] Registered Patent Publication No. 10-2547980
[0010]
[0011] The objective of the present invention is to provide a content provision system equipped with an operating environment that stimulates the auditory curiosity of users, including infants and toddlers, and enables easy playback of various sounds without passing through a digital video device, and a method for providing interactive content using the same.
[0012] The objective of the present invention is to provide a content provision system that allows a user, including infants and toddlers, to directly select a sound to play or interact with content through voice, and a method for providing interactive content by the same.
[0013] The objects of the present invention are not limited to those mentioned above, and other unmentioned objects and advantages of the present invention may be understood from the following description and will be more clearly understood by the embodiments of the present invention. Furthermore, it will be readily apparent that the objects and advantages of the present invention can be realized by the means and combinations thereof set forth in the claims.
[0014]
[0015] A content providing system according to an embodiment of the present invention comprises an identification device including tag information related to content, a content determination device that determines the content based on the tag information, and a content output device that provides the tag information to the content determination device when the identification device is recognized and outputs the content received from the content determination device. The content output device comprises at least one microphone for collecting user voice, a controller that provides the tag information to the content determination device when the identification device is recognized, and a content output station that outputs the content received from the content determination device.
[0016] Additionally, the above content includes at least one content element, and the content determination device determines the content element based on the tag information and the user voice, provided that if either of the case where a preset initial content element exists according to the tag information or the case where the user voice is not recognized during a reference waiting time is met, the content element is determined regardless of the user voice.
[0017] Additionally, the content determination device comprises: a communication module that receives the tag information and the user voice from the content output device; an agent module that requests a language model to generate a natural language response based on the user voice; a determination module that determines an agent module to interact with the user based on the tag information; and a generation module that generates the content based on the natural language response.
[0018] Additionally, the content determination device further includes at least one agent module selected by the determination module, characterized by dynamically generating an input sentence to be provided to the language model by combining information included in the user voice and preset role instruction information.
[0019] Additionally, the content determination device further includes at least one agent module selected by the determination module and characterized by identifying a language model in which role performance information is pre-embedded, and the agent module calls the identified language model and provides information included in the user voice to the identified language model to induce the generation of the natural language response.
[0020] Additionally, the above content includes a first content element in the form of a guide that induces a voice response from the user, the content output device collects the user's voice regarding the first content element, the content determination device analyzes the collected user voice to extract a first response corresponding to the first content element, and determines a second content element to be subsequently provided to the first content element based on the first response.
[0021] Additionally, if the content determination device is not recognized as a clear response to the first content element or if the voice recognition reliability is below a reference reliability, it generates an alternative content element based on the relationship between the first content element and the user voice and accumulated interaction flow data.
[0022] An interactive content provision method according to an embodiment of the present invention comprises, in the content provision method, the steps of: when an identification device including tag information related to content is recognized by a content output device, the content output device providing the tag information to a content determination device; the content output device collecting a user voice; the content determination device providing the content generated based on the tag information and the user voice to the content output device; and the content output device outputting the content received from the content determination device.
[0023] Additionally, the above content includes at least one content element, and the content element is determined based on the tag information and the user voice, provided that if either the case where a pre-set initial content element exists according to the tag information or the case where the user voice is not recognized during a reference waiting time is met, the content element is determined regardless of the user voice.
[0024] Additionally, the step of the content determination device providing the content to the content output device includes receiving the tag information and the user voice from the content output device, determining an agent module to interact with the user based on the tag information, requesting a language model to generate a natural language response based on the user voice, and generating the content based on the natural language response.
[0025] Additionally, the step of requesting a language model to generate a natural language response based on the user voice includes the step of dynamically generating an input sentence by combining information included in the user voice and pre-set role instruction information, and the step of providing the generated input sentence to the language model to request the generation of the natural language response.
[0026] Additionally, the step of requesting a language model to generate a natural language response based on the user voice includes the step of identifying a language model in which role performance information is embedded in advance, the step of calling the identified language model, and the step of providing information contained in the user voice to the identified language model to induce the generation of the natural language response.
[0027] Additionally, the content includes a first content element in the form of a guide that induces a user's voice response, and the step of the content output device collecting the user voice includes collecting the user voice for the first content element, and the step of the content determination device providing the content to the content output device includes analyzing the collected user voice to extract a first response corresponding to the first content element and determining a second content element to be subsequently provided to the first content element based on the first response.
[0028] In addition, if the collected user voice is not recognized as a clear response to the first content element or if the voice recognition reliability is below a reference reliability, the content determination device further includes the step of generating an alternative content element based on the relationship between the first content element and the user voice and accumulated interaction flow data.
[0029]
[0030] The content provision system and the interactive content provision method according to the present invention can stimulate the auditory curiosity of users, including infants and toddlers, and create an operating environment where various sounds can be easily played without going through a digital video device, thereby helping the developmental process of infants and toddlers, including language ability, creativity, and imagination.
[0031] In addition, the content provision system and the interactive content provision method according to the present invention can provide communication opportunities necessary for the development of social skills and activeness by allowing users, including infants and toddlers, to directly select a sound they wish to play or to interact with the content through voice.
[0032] In addition to the above, the specific effects of the present invention are described together with the specific details for implementing the invention below.
[0033]
[0034] FIG. 1 is a schematic diagram showing a content provision system according to an embodiment of the present invention.
[0035] FIG. 2 is a block diagram showing the configuration of a content output device according to an embodiment of the present invention.
[0036] Figure 3 illustrates the external appearance of a content output station as an example.
[0037] FIG. 4 illustrates a content output station and an identification device as examples.
[0038] FIG. 5 is a block diagram showing the configuration of a content determination device according to an embodiment of the present invention.
[0039] FIG. 6 is a flowchart illustrating a method for providing interactive content according to an embodiment of the present invention.
[0040] FIG. 7 is a flowchart illustrating a method for providing interactive content according to an embodiment of the present invention.
[0041] FIG. 8 is a flowchart illustrating a method for providing interactive content according to an embodiment of the present invention.
[0042] FIG. 9 is an exemplary diagram illustrating a method of operation between devices when a first model adaptation method is selected according to an embodiment of the present invention.
[0043] FIG. 10 is an exemplary diagram illustrating a method of operation between devices when a second model adaptation method is selected according to an embodiment of the present invention.
[0044] FIG. 11 is an illustrative diagram for explaining a method in which content is provided through interaction between a content providing device and a user according to an embodiment of the present invention.
[0045] FIG. 12 is an illustrative diagram for explaining a method in which content is provided through interaction between a content providing device and a user according to an embodiment of the present invention.
[0046]
[0047] Terms and words used in this specification and claims shall not be interpreted as being limited to their general or dictionary meanings. In accordance with the principle that an inventor may define the concept of a term or word to best describe their invention, they shall be interpreted in a meaning and concept consistent with the technical spirit of the invention. Furthermore, since the embodiments described in this specification and the configurations illustrated in the drawings are merely one embodiment of the invention and do not represent the entire technical spirit of the invention, it should be understood that various equivalents, modifications, and applicable examples capable of replacing them may exist at the time of filing this application.
[0048] The terms first, second, A, B, etc., as used in this specification and claims may be used to describe various components, but said components should not be limited by said terms. These terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.
[0049] The terms used in this specification and claims are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" should be understood as not precluding the existence or addition of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification.
[0050] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which this invention pertains.
[0051] Terms such as those defined in commonly used dictionaries should be interpreted as having meanings consistent with their meanings in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application. Furthermore, each component, process, procedure, or method included in each embodiment of the present invention may be shared within a scope that is not technically contradictory.
[0052]
[0053] A content providing system according to an embodiment of the present invention will be described below with reference to FIGS. 1 to 5.
[0054] FIG. 1 is a schematic diagram showing a content provision system according to an embodiment of the present invention. FIG. 2 is a block diagram showing the configuration of a content output device according to an embodiment of the present invention. FIG. 3 illustrates the external appearance of a content output station. FIG. 4 illustrates a content output station and an identification device. FIG. 5 is a block diagram showing the configuration of a content determination device according to an embodiment of the present invention.
[0055] Referring to FIG. 1, a content providing system (10) according to an embodiment of the present invention may include a content output device (100), a content determination device (200), an identification device (300), a user terminal (11), and a language model (12).
[0056] The content output device (100) is configured to output content. The content output device (100) can output content in the form of auditory stimuli including voice, music, and sound effects. Additionally, the content output device (100) can output content in the form of visual stimuli including color changes, images, and videos as needed. Additionally, the content output device (100) can output content in the form of tactile stimuli including vibration, positional movement, shape change, and temperature change as needed.
[0057] The content output device (100) can obtain tag information included in the identification device (300) when the identification device (300) is recognized in one area of the device. The content output device (100) can provide the tag information to the content determination device (200) so that content matching the identification device (300) is determined and generated.
[0058] The content determination device (200) is configured to provide content. The content determination device (200) can determine and generate content based on tag information provided by the content output device (100). The content determination device (200) can provide the generated content to the content output device (100) so that the content is output. For example, the content determination device (200) can generate content in the form of sound or provide pre-stored content in the form of sound to the content output device (100).
[0059] Meanwhile, in FIG. 1, the content determination device (200) is shown as an embodiment configured separately outside the content output device (100), but this is merely an example and the present invention is not limited thereto. As another embodiment of the present invention, the content determination device (200) may be mounted inside the content output device (100) to form a part of the content output device (100), or may be physically coupled with the content output device (100) and treated as an integral part.
[0060] The identification device (300) is configured to include tag information related to the content. For example, the identification device (300) may be recognized by the content output device (100) when it comes into contact with an area of the content output device (100) or approaches a location within a predetermined distance from the content output device (100). When the identification device (300) is recognized by the content output device (100), the tag information included in the identification device (300) may be provided to the content output device (100).
[0061] A user terminal (11) is connected to a content output device (100) and can control the content output device (100). The user terminal (11) refers to a communication terminal that provides a function to use web services in a wired / wireless communication environment or can operate a service provision app. For example, the user terminal (11) may be any one of a user's personal computer, smartphone, or tablet electronic device.
[0062] Here, the user of the content output device (100) and the user of the user terminal (11) may include an infant and the parents of the infant. The content output device (100) may have an infant as the primary user, and the user terminal (11) may have a parent as the primary user. However, this is exemplary and the embodiments of the present invention are not limited thereto.
[0063] A language model (12) refers to a model that generates a response based on natural language input. Here, the language model (12) may include a pre-trained artificial intelligence model that analyzes an input sentence in text form and generates a natural language-based response corresponding thereto.
[0064] As an example, the language model (12) can be implemented as a Large Language Model (LLM), in which case the language model (12) can infer semantic relationships between texts based on millions of parameters and generate a natural language response that fits the context of the input sentence.
[0065] The language model (12) may be implemented in a form embedded in an external server, but the embodiments are not limited thereto. For example, the language model (12) may be implemented in a form embedded in the content determination device (200) as needed. The language model (12) implemented outside or inside the content determination device (200) may be called by the instruction of the content determination device (200). The language model (12) may generate a natural language response based on at least one of user voice and prompt data provided by the content determination device (200). Here, the natural language response may include at least one of an initial utterance, a conversational response, a question response, a scenario development, and an answer feedback.
[0066] A communication network can connect a content output device (100), a content determination device (200), a user terminal (11), and a language model (12). For example, the communication network can provide a path necessary for data communication between the content output device (100) and the content determination device (200). Additionally, for example, the content determination device (200) can call the language model (12) through the communication network, and the communication network can provide a path necessary for data communication between the content determination device (200) and the language model (12).
[0067] Here, the communication network may include wired networks including LANs (Local Area Networks), WANs (Wide Area Networks), MANs (Metropolitan Area Networks), and ISDNs (Integrated Service Digital Networks), and wireless networks including wireless LANs, CDMA, Bluetooth, satellite communication, 3G, 4G, and 5G. However, this is exemplary and the embodiments of the present invention are not limited thereto.
[0068]
[0069] Referring to FIGS. 2 and FIGS. 3, the content output device (100) may include at least one microphone (110), a controller (120), and a content output station (130).
[0070] A content output device (100) according to an embodiment of the present invention may further include one or more other components as needed. For example, the content output device (100) may further include a communication device. Additionally, for example, the content output device (100) may further include a display.
[0071] The microphone (110) can collect user voice. Here, user voice can include not only spoken voice capable of conveying meaning, but also various forms of sound that the user can express. For example, sounds such as clapping, yawning, coughing, sniffling, and babbling that can convey the user's emotions or state can be recognized and collected as user voice. That is, all sounds of frequency, intensity, and pitch included within a predetermined range that can be recognized through the microphone (110) can be collected, and, if necessary, a step may be added in which sounds excluding user voice are filtered through a separate filtering means built into the microphone (110) and / or the content output device (100).
[0072] The controller (120) may be configured to perform comprehensive processing functions including data processing, computation, memory, information determination, communication with external devices, and signal control of the content output device (100). Specifically, the controller (120) may include at least one of a communication device, a memory, and a processor.
[0073] The controller (120) can provide tag information to the content determination device (200) when the identification device (300) is recognized. The controller (120) can control the content output device (100) so that content output is performed based on the data provided by the content determination device (200).
[0074] The controller (120) may store commands, information, or data associated with the operation of components included in the content output device (100). Additionally, the controller (120) may store and / or temporarily store various data acquired or used by at least one component of the content output device (100). In this case, the controller (120) may include memory, and may include volatile memory or non-volatile memory. The data may include input / output data for software and related commands.
[0075] The controller (120) may include a processor operatively connected to memory to perform overall processing functions of the content output device (100). In this case, the processor may control at least one other component included in the content output device (100) and perform various data processing or operations. For example, the processor may load commands or data received from other components into memory, process commands or data stored in memory, and store result data in memory.
[0076] Here, the processor may include one or more processors, including a main processor and an auxiliary processor, and as specific examples, may include an image signal processor (ISP), an application processor (AP), and a communication processor (CP).
[0077] The content output station (130) is configured to output content. For example, the content output station (130) may output content in the form of sound, including a speaker, and provide it to the user. The content output station (130) may output content provided by the content determination device (200), and may output sound including content, music, and sound effects as needed, upon the request of the controller (120).
[0078] The user can tag or dock the identification device (300) to the content output station (130), and as the identification device (300) is recognized by the content output station (130), content matching the identification device (300) can be output through the content output station (130). That is, the identification device (300) can serve as a key for content playback.
[0079] The content output station (130) may include at least one area capable of recognizing the identification device (300), and when the identification device (300) comes into contact with the area or approaches within a predetermined distance, a tag is formed and can be recognized by the content output station (130).
[0080] The content output station (130) can be implemented in various forms, but, for example, as shown in FIG. 3, it may have a form in which a docking space (D) of an identification device (300) is formed. Referring to the embodiment shown in FIG. 3 as an embodiment of the present invention, the content output station (130) may include a speaker (S) that outputs content in the form of sound, a volume control device (V) that controls the volume of the speaker, an operation status light (L) that indicates the operation status of the content output station (130) through a color change, and a playback track controller (C) for changing the output sound track.
[0081] According to the embodiment illustrated in FIGS. 3 and 4, the content output station (130) may have an exterior shape similar to a house with a docking space (D) formed therein, the volume control device (V) may be configured as the chimney of the house, and the playback track controller (C) may be configured as part of the roof of the house. An infant user may place the identification device (300) in the docking space (D) of the content output station (130) while purely recognizing that the identification device (300) resides in the content output station (130).
[0082] Tag information included in the identification device (300) may correspond to each piece of content. For example, the content may correspond to each sound form of content configured for auditory stimulation of infants and toddlers. As specific examples, the content may be one of the following: content for language learning such as counting numbers, bilingual repetition, onomatopoeia repetition, and mimetic repetition; content that enhances the user's imagination and creativity such as melodies, sound theaters, and folk tales; content that enables the user to perform physical activities through rhythm, such as animal songs and rhythmic songs; and content in which a parent has recorded their voice for the user's emotional development. The content may be composed of multiple sound tracks and may be changed, combined, or edited by the user as needed.
[0083] According to one embodiment of the present invention, the identification device (300) may include a plurality of identification devices (300) having different external shapes. When the identification device (300) is composed of a plurality, the plurality of identification devices (300) may each correspond to a plurality of different types of content. That is, the plurality of identification devices (300) having different external shapes may be configured to correspond to different types of auditory stimulation sound content for infants and toddlers. One of the plurality of identification devices (300) may be selected by a user and placed in a docking space (D) so as to be recognized by the content output device (100).
[0084] The identification device (300) can be implemented in various shapes, but, for example, as shown in FIG. 4, it can be formed in a size that can be held in one hand by an infant user. In this case, the docking space (D) can be formed as a space capable of sufficiently accommodating the identification device (300). The plurality of identification devices (300) can be composed of one of animal shapes, cute character shapes, and popularly known character shapes to stimulate the curiosity of the infant user. Additionally, the plurality of identification devices (300) can be formed with rounded corners without sharp edges for the safety of the infant user.
[0085] According to the embodiment illustrated in FIG. 4, a plurality of identification devices (300) are formed with an overall rounded shape and can be configured to have different shapes, such as a cute character (300A), a rabbit (300B), a bear (300C), a tiger (300D), and a chick (300E). For example, the cute character (300A) can be configured to correspond to science story content, the rabbit (300B) can be configured to correspond to world classic fairy tale content, the bear (300C) can be configured to correspond to children's music content, the tiger (300D) can be configured to correspond to adventure story content, and the chick (300E) can be configured to correspond to English education content.
[0086] For example, if an infant user selects a tiger (300D) from the identification device (300) and places it in the docking space (D), adventure story content can be played at the content output station (130) of the content output device (100). Additionally, if the infant user removes the tiger (300D) from the docking space (D) and places a bear (300C) there, the adventure story content is stopped and new children's music content can be played. Furthermore, if necessary, the system may be configured so that playback of the content is maintained even if the infant user removes the identification device (300) from the content output station (130) after tagging it.
[0087] Here, at least one of the contents corresponding to the plurality of identification devices (300) may include content in the form of sound that enables interaction with the user. That is, it may be implemented as content in the form of conversation or communication through the user's voice.
[0088] For example, a cute character (300A) can interact as science story content by telling science stories to the user, providing questions, and offering knowledge feedback based on the user's answers. Alternatively, for example, a rabbit (300B) can interact as world classic fairy tale content by telling a fairy tale to the user, checking user reactions at every branch of the story's progression, and changing the pace and direction of the story's development based on the user's response. Alternatively, for example, a bear (300C) can interact as children's music content by providing questions to check the user's emotional state and playing a song that matches the current mood based on the user's answers or reactions. Alternatively, for example, a tiger (300D) can interact as adventure story content by developing the story to make the user feel like they are the protagonist of an adventure scenario, and providing choices that the user can decide at every branch of the scenario to diversify the next scenario. Alternatively, for example, a chick (300E) can interact as English education content by having the user repeat English words and correcting the user's pronunciation.
[0089] The content output station (130) can recognize the tagged identification device (300) via near-field wireless communication (NFC). Each of the plurality of identification devices (300) may include an NFC tag, and the docking space (D) may include an NFC reader. Alternatively, if a tag area is included in a separate location among the content output devices (100) other than the docking space (D), the tag area may include an NFC reader.
[0090] Meanwhile, the aforementioned near-field communication (NFC) method is merely exemplary, and therefore the embodiments of the present invention are not limited thereto. That is, various wired or wireless communication methods capable of recognizing the identification device (300) may be selected. Additionally, depending on the selected communication method, additional components such as a reader may be further included in the content output device (100).
[0091]
[0092] Referring to FIG. 5, the content determination device (200) may include a communication module (210), a voice processing module (220), a determination module (230), an agent module (240), a generation module (250), and a DB module (260).
[0093] The communication module (210) can perform communication functions with an external device. For example, the communication module (210) can perform data communication with the content output device (100). The communication module (210) can receive tag information from the content output device (100) and, if there is a user's speech or response, can receive user voice from the content output device (100). In addition, the communication module (210) can transmit data regarding content to the content output device (100).
[0094] The voice processing module (220) can convert voice-type data into text-type data and text-type data into voice-type data. For example, the voice processing module (220) can convert user voice into text-type user input. Additionally, the voice processing module (220) can convert a text-type response written in the language model (12) into voice-type data so that it is output as sound at the content output station (130).
[0095] The decision module (230) can determine at least one of the agent modules (240). For example, the decision module (230) can determine an agent module (240) to interact with the user based on tag information. That is, the decision module (230) receives tag information of the identification device (300) received through the communication module (210) and can determine one agent module (240) corresponding to the received tag information.
[0096] The agent module (240) is configured to generate data necessary for interaction with the user in providing content corresponding to tag information, with the role being pre-set or dynamically adjusted according to the purpose of interaction with the user. The data generated by the agent module (240) may include input data to be provided to the language model (12). That is, in this case, the data generated by the agent module (240) may have the format of a prompt to be requested from the language model (12). Additionally, the data generated by the agent module (240) may include selection information and information regarding the response method for a specific language model (12) corresponding to the tag information.
[0097] Specifically, the agent module (240) may select various model adaptation methods to construct an input sentence for generating a natural language response, or specify a language model that corresponds to the method. As an example, the agent module (240) may select a first model adaptation method, and in this specification, the first model adaptation method may refer to a structure in which the agent module (240) combines an input sentence and transmits a request to the language model (12). That is, the first model adaptation method may be exemplified by a prompt-based method.
[0098] As another example, the agent module (240) may select a second model adaptation method, and in this specification, the second model adaptation method may refer to a structure in which the agent module (240) utilizes a language model (12) in which role performance information is pre-embedded, wherein the agent module (240) identifies the language model (12) and provides information included in the user's voice, and the identified language model (12) generates a response based thereon. That is, the second model adaptation method may be exemplified by a plurality of fine-tuned language model-based methods.
[0099] The agent module (240) selecting the first model adaptation method can generate input data for content generation based on interaction with the user, based on at least some of the tag information and user voice. For example, the agent module (240) receives user input converted into text form from the voice processing module (220) and can dynamically generate an input sentence to be provided to the language model (12) by combining information included in the user input with pre-set role instruction information. The agent module (240) can request the generation of a natural language response while providing the generated input sentence to the language model (12).
[0100] Meanwhile, according to another embodiment of the present invention, the agent module (240) may generate an input sentence to be provided to the language model (12) by using the user's voice in the form of voice directly. That is, in this case, the agent module (240) receives the user's voice directly from the communication module (210) without passing through the voice processing module (220), and can generate an input sentence to be provided to the language model (12) by interpreting the information contained in the user's voice in the form of voice.
[0101] As another embodiment of the present invention, the agent module (240) may generate response data necessary for content creation based on direct tag information and information included in the user voice. That is, in this case, the agent module (240) may receive the user voice directly from the communication module (210) without passing through a separate language model (12), interpret the information included in the user voice in the form of voice to generate response data, and then transmit it to the creation module (250).
[0102] Next, the agent module (240) selecting the second model adaptation method can identify a language model (12) in which role performance information is pre-embedded based on tag information. Then, for example, it can receive user input converted into text form from a voice processing module (220). In this case, the agent module (240) can call the identified language model (12) and provide the received user input to the identified language model (12) to induce the generation of a natural language response.
[0103] Meanwhile, according to another embodiment of the present invention, the agent module (240) can directly extract information contained in the user's voice in the form of voice and provide it to the language model (12). That is, in this case, the agent module (240) receives the user's voice directly from the communication module (210) without passing through the voice processing module (220), and can interpret the information contained in the user's voice in the form of voice and provide it to the language model (12).
[0104] The language model (12) may receive an input sentence in the form of a prompt from an agent module (240) corresponding to the tag information, and in this case, may generate a natural language response according to the input sentence. Additionally, when using a language model (12) in which role performance information is pre-embedded or sufficiently learned, such as in a multiple precision-learned language model method, the language model (12) may generate a natural language response according to the pre-embedded role performance information even if it receives only user input directly from an agent module (240) corresponding to the tag information.
[0105] The agent module (240) may be configured in multiple units, and each agent module (240) may generate data regarding different content. That is, agent modules (240) corresponding to different tag information may be configured separately, and accordingly, as shown in FIG. 5, the agent module (240) may include multiple detailed modules such as a first agent module (240A), a second agent module (240B), and a third agent module (240C).
[0106] Depending on the needs, at least some of the multiple agent modules (240A, 240B, 240C, …, 240K) may select the first model adaptation method, and the rest may select the second model adaptation method. Additionally, depending on the needs, the multiple agent modules (240A, 240B, 240C, …, 240K) may select a single unified model adaptation method.
[0107] The generation module (250) is configured to generate content. The generation module (250) may generate content by determining one of a plurality of pre-stored contents, or it may generate new content that reflects interaction with the user based on natural language responses. Since the content generated by the generation module (250) may have a text form, the content generated in text form may be transmitted to the voice processing module (220) and converted into a voice form.
[0108] Here, the content generated by the generation module (250) can be divided into data units (content elements) that are generated simultaneously. That is, in this specification, content is defined as a concept that includes each content element generated at each quarter during interaction (e.g., when it is the utterance sequence of one party in a conversation). In this case, the content includes at least one content element, and the generation module (250) of the content determination device (200) can determine the content element at each quarter based on tag information and user voice. For example, the content element may include utterance content that is initially provided to the user after the tag information is recognized. Additionally, the content element may include utterance content that is subsequently provided as a response to the user voice.
[0109] The DB module (260) can store and manage data generated within the content determination device (200), transmitted between internal configuration modules, or received through data communication with external devices. For example, the DB module (260) can store reference information and training data related to the agent module (240), user input, content elements, etc. Additionally, the data stored in the DB module (260) may further include content creation flow, user response history, and model adaptation information as needed.
[0110]
[0111] Hereinafter, FIGS. 6 to 8 describe a content provision method performed by the aforementioned content provision system (10) according to an embodiment of the present invention. In describing the content provision method, specific examples illustrated in FIGS. 9 to 12 are also referred to.
[0112]
[0113] First, FIG. 6 is a flowchart for explaining a method for providing interactive content according to an embodiment of the present invention. Each step shown in FIG. 6 relates to a method of operating a content output device (100).
[0114] When an identification device (300) containing tag information related to content is recognized by a content output device (100) (S101), the content output device (100) can provide the tag information to a content determination device (200) (S110). At this stage, the tag information provided to the content determination device (200) can be used importantly when the agent module (240) is determined by the determination module (230).
[0115] Subsequently, the content output device (100) can collect user voice and provide the collected user voice to the content determination device (200) (S120). Meanwhile, there are cases where content is output first and provided to the user even before user voice is collected. That is, each content element included in the content is determined based on tag information and user voice, but content elements can be provided even in the initial tag situation where there is no user voice.
[0116] Specifically, if an initial content element is pre-set according to tag information, the initial content element can be determined and provided regardless of the user's voice. Additionally, if the user's voice is not recognized during a reference waiting time, a subsequent content element can be determined regardless of the user's voice. That is, according to one embodiment of the present invention, if either the case where an initial content element is pre-set according to tag information or the case where the user's voice is not recognized during a reference waiting time is met, a content element can be determined regardless of the user's voice.
[0117] Even for interaction-based content, the initial content element (e.g., the opening utterance of an adventure story) must be provided to the user first during the initial tag situation so that conversational interaction can continue via subsequent user voice. In such cases, providing the initial content element independently of the user's voice enables a smooth and natural start to the content. Furthermore, young children with low language development levels may be unable to generate voice for extended periods due to reasons such as a lack of understanding of the question. In these situations, providing subsequent content elements even if the user's voice is not recognized for a preset waiting period allows the interactive content to proceed at an appropriate speed without indefinite interruption.
[0118] After the content output device (100) provides the user's voice (S120), the content determination device (200) can generate content based on tag information and the user's voice, and provide the generated content to the content output device. Then, the content output device (100) can receive the content from the content determination device (200) (S130) and can output the received content (S140). If the content output device (100) receives content in a sound format from the content determination device (200), the content output device (100) can play content in a sound format that helps stimulate the user's hearing.
[0119]
[0120] FIG. 7 is a flowchart for explaining an interactive content provision method according to an embodiment of the present invention, illustrating a specific method for determining content in a content determination device (200). Each step shown in FIG. 7 relates to a method of operation of the content determination device (200).
[0121] The content determination device (200) can receive tag information and user voice from the content output device (100) (S210), and can convert the user voice into user input in the form of text (S220). The user input becomes data necessary to request a subsequent response for interaction with the user from the language model (12). However, as described above, it is also possible to transmit the user voice directly to the agent module (240) without converting it into text. In this case, step S220 may be omitted.
[0122] The content determination device (200) can determine an agent module (240) to interact with the user based on tag information (S230). At this stage, one agent module (240) determined by the content determination device (200) can not only determine the output of content corresponding to a specific type, but also influence the determination of the method and content of the response used during the interaction process with the user. For example, depending on which agent module (240) is determined by the content determination device (200), the method of response, such as the tone of voice, speech speed, and vocabulary difficulty of the response applied during conversational interaction with the user, may vary.
[0123] Next, the content determination device (200) may request the language model (12) to generate a natural language response based on user input (S240). At this stage, the format of the request provided to the language model (12) may vary depending on the model adaptation method selected by the agent module (240).
[0124] Meanwhile, as previously discussed, in an embodiment where step S220 is omitted, the content determination device (200) may request the language model (12) to generate a natural language response based on the user's voice in the form of speech. However, for ease of understanding, the present specification describes an embodiment in which a natural language response is generated based on user input converted into text form as an example.
[0125] Specifically, when the agent model (240) selected for the first model adaptation method is determined, the content determination device (200) may generate a prompt and provide it to the language model (12). Such an embodiment is illustrated exemplarily in FIG. 9. Additionally, when the agent model (240) selected for the second model adaptation method is determined, the content determination device (200) may provide user input directly to the language model (12) containing specific role performance information instead of a prompt. Such an embodiment is illustrated exemplarily in FIG. 10.
[0126] The language model (12) can generate a natural language response based on a received request. Specifically, if it is determined to be an agent model (240) that has selected the first model adaptation method, the language model (12) can generate a natural language response based on a provided prompt. Additionally, if it is determined to be an agent model (240) that has selected the second model adaptation method, the called specific language model (12) can generate a natural language response based on user input and role performance information that it already possesses.
[0127] Then, the content determination device (200) can receive a natural language response generated from the language model (12) and generate content based thereon (S250). If the content is interactive content with the user, each content element can be generated at every quarter when the user's voice is provided.
[0128]
[0129] FIG. 8 is a flowchart for explaining an interactive content provision method according to an embodiment of the present invention, illustrating a specific method for determining subsequent content elements according to the characteristics of a user's voice in a content determination device (200). Each execution step illustrated in FIG. 8 relates to a method of operation of the content determination device (200).
[0130] First, the content according to an embodiment of the present invention may include a first content element in the form of a guidance that induces a voice response from the user. Then, the content determination device (200) may provide the first content element in the form of a guidance to the content output device (100) (S310). For example, the first content element may correspond to a question-type voice asking the user about their current mood. Alternatively, the first content element may correspond to a voice that informs the user of the first word in a word chain game.
[0131] When a first content element is provided to a content output device (100), the user may generate a user voice through subsequent speech, etc. The user voice for the first content element may be collected through the content output device (100), and the content determination device (200) may receive it from the content output device (100) (S320).
[0132] Here, the content determination device (200) can determine whether the user voice is recognized as a normal response to the first content element (S330). Specifically, the content determination device (200) can determine whether the collected user voice is recognized as a clear response to the first content element (S330). For example, regarding the first content element corresponding to the first word of a word chain game, there may be cases where the user gives a response completely unrelated to the game instead of a response to the next word chain word. An embodiment including a user voice that is not recognized as a clear response in this way is illustrated exemplarily in FIG. 11.
[0133] Next, the content determination device (200) can determine whether the voice recognition reliability of the user's voice is greater than or equal to the reference reliability (S340). Here, the reference reliability may be defined as a reference value for determining voice data at a level where the device can recognize at least one of the intention and content of the user's voice. The reference reliability may be set in advance in the content determination device (200) and may be changed by the administrator and / or user as needed. For example, regarding a first content element corresponding to a question-type voice asking about the current mood, the user response of a user with a low level of language development may be merely at the level of babbling, making it difficult to recognize the user's speech intention or content therefrom. An embodiment including a user voice with such low voice recognition reliability is illustrated exemplarily in FIG. 12.
[0134] As described above, when a user voice is recognized as a clear response to a first content element (S330) and the voice recognition reliability is determined to be greater than or equal to a reference reliability (S340), the content determination device (200) can analyze the collected user voice to extract a first response corresponding to the first content element (S350). That is, the content determination device (200) can recognize the user's interaction response to the first content element by extracting the content of the user voice recognized as a normal response as the first response.
[0135] Next, the content determination device (200) can determine a second content element to be subsequently provided to the first content element based on the first response (S360). That is, the content determination device (200) can determine a second content element to be subsequently provided based on the user's interaction response to the first content element, thereby allowing the development of the conversation or story to continue with a smooth and natural content flow.
[0136] Meanwhile, if the previously collected user voice is not recognized as a clear response to the first content element or if the voice recognition reliability is below the reference reliability, the content determination device (200) may generate an alternative content element based on the relationship between the first content element and the user voice and the accumulated interaction flow data (S370). Here, the alternative content element may refer to a content element generated by comprehensively considering the degree of relevance of the response to the first content element and the accumulated interaction flow data for the user, in cases where it is difficult to determine the subsequent second content element based solely on the user response to the first content element. For example, if the user voice to the first content element deviates from a general response, that is, the expected response range, a new story-developing content may be provided as an alternative content element to prevent the interaction flow from being interrupted.
[0137]
[0138] FIG. 9 is an exemplary diagram illustrating a method of operation between devices when a first model adaptation method is selected according to an embodiment of the present invention.
[0139] As illustrated in FIG. 9, when the content output device (100) recognizes the A identification device (S10), the corresponding tag information can be identified (S11). Then, the content output device (100) transmits the tag information to the content determination device (200) (S12), and the content determination device (200) can select a prompt-type A agent module based on this (S13). Then, the A agent module can generate a prompt to be provided to the language model (12) (S14), and the prompt may correspond to an input sentence dynamically generated by the 'story development content,' which is pre-set role instruction information for the A agent module.
[0140] When a prompt containing an input sentence generated by an agent module is transmitted to the language model (12) (S15), the language model (12) can generate a natural language response based on this (S16). In this case, the response generated by the language model (12) may correspond to a content element that starts the development of the story called 'Hansel and Gretel' as the first content element of the 'story development content' and grants the user the right to decide the story branching point.
[0141] The language model (12) can transmit the generated natural language response to the content determination device (200) (S17), and the natural language response in text form can be converted into voice form at the content determination device (200) to generate content in the form of voice data (S18). The content determination device (200) can transmit the generated content to the content output device (100) (S19), and the content output device (100) can output the content in voice form (S20).
[0142] The user can listen to a story played through the content output device (100) and make a speech that includes a decision regarding a story branching point. For example, the user can generate a voice with the content "sea" as a response to the question "Shall we go to the sea or to the mountains?", and the content output device (100) can collect such user voice (S21) and transmit it to the content decision device (200) (S22). Then, the content decision device (200) can convert the user voice into text (S23) and then generate a subsequent prompt to continue the interaction flow for the content (S24). Here, if the model adaptation method is selected as a prompt method, the content decision device (200) regenerates the entire prompt by combining the information included in the user input with pre-set role instruction information, as shown in FIG. 9.
[0143]
[0144] FIG. 10 is an exemplary diagram illustrating a method of operation between devices when a second model adaptation method is selected according to an embodiment of the present invention.
[0145] As illustrated in FIG. 10, when the content output device (100) recognizes the B identification device (S30), the corresponding tag information can be identified (S31). Then, the content output device (100) transmits the tag information to the content determination device (200) (S32), and the content determination device (200) can select a plurality of B agent modules of a truly learned language model method based on this (S33). Then, a specific language model (e.g., LLM for storytelling) containing the role performance information of the B agent module can be identified (S34). Then, the B agent module can immediately call the identified corresponding language model (12) without a prompt to provide to the language model (12) (S35) and provide user input as is.
[0146] A language model (12) that receives a model call from the B agent module can generate a natural language response based on this (S36). In this case, the response generated by the language model (12) may correspond to a content element that starts the development of the story of 'Hansel and Gretel' as the first content element of the 'story development content' and grants the user the right to decide the story branching point. That is, since a specific language model containing the role performance information of the B agent module has already been called, there is no need to input a specific request through a prompt.
[0147] The language model (12) can transmit the generated natural language response to the content determination device (200) (S37), and the natural language response in text form can be converted into voice form at the content determination device (200) to generate content in the form of voice data (S38). The content determination device (200) can transmit the generated content to the content output device (100) (S39), and the content output device (100) can output the content in voice form (S40).
[0148] The user can listen to a story played through the content output device (100) and make a utterance that includes a decision regarding a story branching point. For example, the user can generate a voice with the content "sea" as a response to the question "Shall we go to the sea or to the mountains?", and the content output device (100) can collect such user voice (S41) and transmit it to the content decision device (200) (S42). Then, the content decision device (200) can convert the user voice into text (S43) and provide it to a specific language model (e.g., a storytelling LLM) containing role performance information of the B agent module without the need to process it into a prompt. That is, if the model adaptation method is selected as a multiple truly learned language model method, the content decision device (200) can generate a request containing only data regarding the user's response (S44). Then, the request can be transmitted to the identified specific language model, which is the storytelling LLM (S45).
[0149]
[0150] FIG. 11 is an illustrative diagram for explaining a method in which content is provided through interaction between a content providing device and a user according to an embodiment of the present invention.
[0151] As illustrated in FIG. 11, when a content output device (100A) provides content about a space story to an infant user (UA), the response of the infant user (UA) may be recognized as being within a normal range, such as 'Earth!', or may be recognized as being within an abnormal range, such as 'I want to eat Earth jelly.', due to a lack of relevance to the preceding question content element.
[0152] According to an embodiment of the present invention, in the case of a user voice including a response recognized within a normal range, a content determination may be performed to directly influence the generation of a subsequent content element. However, in the case of a user voice including a response not recognized as a clear response to a preceding first content element, an alternative content element may be generated by comprehensively considering the relationship between the first content element and the user voice and the accumulated interaction flow data.
[0153]
[0154] FIG. 12 is an illustrative diagram for explaining a method in which content is provided through interaction between a content providing device and a user according to an embodiment of the present invention.
[0155] As illustrated in FIG. 12, when a content output device (100B) provides content that plays a song according to the current emotional state to an infant user (UB), the response of the infant user (UB) may not be recognized as normal speech due to a very low level of language development. According to an embodiment of the present invention, in the case of a user voice with high speech recognition reliability, content determination may be performed to directly influence the creation of subsequent content elements. However, in the case of a user voice with very low speech recognition reliability, which is below the standard reliability, alternative content elements may be created by comprehensively considering the relationship between the preceding first content element and the user voice and accumulated interaction flow data.
[0156]
[0157] As described above, the content provision system and the interactive content provision method according to the embodiment of the present invention can stimulate the auditory curiosity of users, including infants and toddlers, and create an operating environment where various sounds can be easily played without going through a digital video device, thereby helping the developmental process of infants and toddlers, including language ability, creativity, and imagination.
[0158] In addition, the content provision system and the interactive content provision method according to the embodiment of the present invention can provide communication opportunities necessary for the development of social skills and activeness by allowing a user, including infants and toddlers, to directly select a sound to play or interact with the content through voice.
[0159]
[0160] The above description is merely an illustrative explanation of the technical concept of the present embodiment, and a person skilled in the art to which the present embodiment belongs would be able to make various modifications and variations within the scope of the essential characteristics of the present embodiment. Accordingly, the present embodiments are intended to explain, not limit, the technical concept of the present embodiment, and the scope of the technical concept of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the present embodiment.
Claims
1. An identification device including tag information related to content; A content determination device that determines the content based on the above tag information; and It includes a content output device that provides the tag information to the content determination device when the identification device is recognized, and outputs the content received from the content determination device. The above content output device is, At least one microphone for collecting user voice; A controller that provides the tag information to the content determination device when the identification device is recognized; and A content output station that outputs the content received from the content determination device. Content provision system.
2. In Paragraph 1, The above content includes at least one content element, and The above content determination device is, The content element is determined based on the above tag information and the above user voice, If either of the following applies—where a preset initial content element exists according to the above tag information or where the user voice is not recognized during the reference waiting time—the content element is determined regardless of the user voice. Content provision system.
3. In Paragraph 1, The above content determination device is, A communication module that receives the tag information and the user voice from the content output device; An agent module that requests a language model to generate a natural language response based on the above user voice; A decision module for determining an agent module to interact with the user based on the above tag information; and A generation module that generates the content based on the above natural language response Content provision system.
4. In Paragraph 3, The above content determination device is, The system further comprises at least one agent module selected by the above-mentioned decision module, characterized by dynamically generating an input sentence to be provided to the language model by combining information included in the user voice and preset role instruction information. Content provision system.
5. In Paragraph 3, The above content determination device is, It further includes at least one agent module selected by the above-mentioned decision module and characterized by identifying a language model in which role performance information is pre-embedded, and The agent module calls the identified language model and provides information included in the user voice to the identified language model to induce the generation of the natural language response. Content provision system.
6. In Paragraph 1, The above content includes a first content element in the form of a guide that induces a voice response from the user, and The above content output device collects user voice for the first content element, and The content determination device analyzes the collected user voice to extract a first response corresponding to the first content element, and determines a second content element to be subsequently provided to the first content element based on the first response. Content provision system.
7. In Paragraph 6, The above content determination device is, If either of the cases applies where the collected user voice is not recognized as a clear response to the first content element or where the voice recognition reliability is below the reference reliability, an alternative content element is generated based on the relationship between the first content element and the user voice and the accumulated interaction flow data. Content provision system.
8. Regarding the method of providing content, When an identification device containing tag information related to content is recognized by a content output device, the content output device provides the tag information to a content determination device; The above content output device collects user voice; The step of the content determination device providing the content generated based on the tag information and the user voice to the content output device; and The above content output device includes the step of outputting the content received from the content determination device. Method of providing interactive content.
9. In Paragraph 8, The above content includes at least one content element, and The above content element is determined based on the tag information and the user voice, provided that if either the case where a pre-set initial content element exists according to the tag information or the case where the user voice is not recognized during a reference waiting time is met, the content element is determined regardless of the user voice. Method of providing interactive content.
10. In Paragraph 8, The step of the content determination device providing the content to the content output device is: A step of receiving the tag information and the user voice from the content output device; A step of determining an agent module to interact with the user based on the above tag information; A step of requesting a language model to generate a natural language response based on the above user voice; and A step comprising generating the content based on the natural language response Method of providing interactive content.
11. In Paragraph 10, The step of requesting a language model to generate a natural language response based on the above user voice is, A step of dynamically generating an input sentence by combining information included in the user voice and pre-set role instruction information; and A step comprising providing the above-generated input sentence to a language model and requesting the generation of the above-generated natural language response. Method of providing interactive content.
12. In Paragraph 10, The step of requesting a language model to generate a natural language response based on the above user voice is, A step of identifying a language model in which role performance information is embedded in advance; A step of calling the above-mentioned identified language model; and A step comprising providing information contained in the user voice to the identified language model to induce the generation of the natural language response. Method of providing interactive content.
13. In Paragraph 8, The above content includes a first content element in the form of a guide that induces a voice response from the user, and The step of the content output device collecting the user voice is, It includes the step of collecting user voice for the above-mentioned first content element, and The step of the content determination device providing the content to the content output device is: A step of analyzing the collected user voice to extract a first response corresponding to the first content element; and A step comprising determining a second content element to be subsequently provided to the first content element based on the first response. Method of providing interactive content.
14. In Paragraph 13, If either of the cases where the collected user voice is not recognized as a clear response to the first content element or where the voice recognition reliability is below a reference reliability, the content determination device further includes the step of generating an alternative content element based on the relationship between the first content element and the user voice and accumulated interaction flow data. Method of providing interactive content.