System and method for providing dynamic virtual environment reflecting context of conversation and computer program for the same
The dynamic virtual environment system addresses the issue of predetermined virtual environments by using AI to recognize user context and adjust the environment, enhancing educational efficiency through adaptive content selection.
Patent Information
- Application Number
- KR1020230041450
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2023-03-29
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2043-03-29
AI Technical Summary
Conventional virtual environments do not dynamically adapt to the user's context, leading to potential boredom and decreased learning efficiency, as they are predetermined and do not change based on the user's situation or context.
A dynamic virtual environment system that uses AI to recognize user conversation context and automatically adjusts the virtual environment by selecting and displaying relevant content based on context information, including changes to the virtual space and objects.
Enhances educational efficiency by providing a virtual environment that adapts to the user's context, optimizing content selection and improving communication and exchange within the virtual environment.
Smart Images

Figure 112023035607268-PAT00003_ABST
Abstract
Description
Technology Field
[0001] The embodiments relate to a system and method for providing a dynamic virtual environment and a computer program for the same. More specifically, the embodiments relate to a technology that enables efficient communication and exchange by recognizing a user's conversational context and automatically changing the virtual environment according to the recognized context when providing a virtual environment, such as a metaverse. Background Technology
[0002] With the recent advancement of technologies for constructing virtual environments such as the metaverse through information and communication technology, technologies enabling users to interact with Artificial Intelligence (AI) characters or interact with each other in virtual worlds are rapidly developing.
[0003] In particular, when virtual environments are utilized in education, there is an advantage in that learners can indirectly experience real-world situations while conducting educational courses within a virtual environment capable of various visual, auditory, and interactive elements. For example, students can actually observe the internal structure of a cell during biology class or experience the reenactment of historical events during history class. As an example of prior art utilizing virtual environments for education, Registered Patent Publication No. 10-2411058 discloses a rural farming education simulation server in which a user can cultivate crops, evaluate cultivation capabilities, and improve cultivation capabilities through learning within a metaverse space.
[0004] However, conventional technology merely provides a specific virtual environment that is predetermined and constructed, and the environment does not change according to the situation or context in which the user is situated. For example, in the case of the technology disclosed in Registered Patent Publication No. 10-2411058, rural farming education is conducted using a pre-set virtual plot of land. In this case, there are limitations in that learners may easily become bored with the virtual environment, and learning efficiency may decrease if the content necessary for education is not located in the appropriate place within the virtual environment. Prior art literature
[0005] Registered Patent Publication No. 10-2411058 The problem to be solved
[0006] According to one aspect of the present invention, a dynamic virtual environment providing system and method and a computer program for the same can be provided, which enable efficient communication and exchange by automatically recognizing the context of a user’s conversation through artificial intelligence (AI) or based on preset rules when providing a virtual environment such as a metaverse, and automatically changing the virtual environment according to the recognized context. means of solving the problem
[0007] A dynamic virtual environment providing system according to one aspect of the present invention comprises: a virtual environment providing unit configured to provide virtual environment data to be displayed on a user device; a receiving unit configured to receive input data from a user device; a database storing information related to a plurality of contents that can be displayed within the virtual environment; and a matching unit configured to extract context information associated with the user from the input data and to determine a content corresponding to the context information among the plurality of contents.
[0008] At this time, the virtual environment providing unit is further configured to change the virtual environment data provided to the user device so that the content determined by the matching unit is displayed on the user device.
[0009] In one embodiment, the virtual environment data is intended to display a virtual space and one or more objects located within the virtual space. In this case, the virtual environment providing unit is further configured to change one or more of the type of the virtual space, the display area within the virtual space, and the one or more objects based on content determined by the matching unit.
[0010] In one embodiment, the matching unit is further configured to extract context information by recognizing one or more of the user's conversation location, conversation partner, conversation purpose, and related objects represented by the input data based on natural language processing of the input data.
[0011] In one embodiment, the database is configured to further store one or more of attribute information and evaluation information related to each of the plurality of contents. In this case, the matching unit is further configured to determine the content corresponding to the context information among the plurality of contents by comparing one or more of the attribute information and the evaluation information with the context information.
[0012] In one embodiment, the receiving unit is further configured to receive the input data while the virtual environment data is being displayed on the user device. In this case, the virtual environment providing unit is further configured to update the virtual environment data based on the input data received while the virtual environment data is being displayed on the user device.
[0013] A method for providing a dynamic virtual environment according to one aspect of the present invention comprises: a dynamic virtual environment providing system providing virtual environment data to be displayed on a user device to a user device; a dynamic virtual environment providing system receiving input data from a user from the user device; a dynamic virtual environment providing system extracting context information associated with the user from the input data; a dynamic virtual environment providing system determining a content corresponding to the context information among a plurality of contents that can be displayed in a virtual environment; and a dynamic virtual environment providing system modifying the virtual environment data provided to the user device so that the content determined to correspond to the context information is displayed on the user device.
[0014] In one aspect of the present invention, a computer program may be stored on a computer-readable recording medium for implementing a method for providing a dynamic virtual environment according to the embodiments described above, in combination with hardware.
[0015] A computing device according to one aspect of the present invention includes: a memory in which executable instructions are stored; a communication module configured to communicate with an external system; and a processor connected to communicate with the memory and the communication module.
[0016] The processor is configured to provide virtual environment data to be displayed on the user device to the user device by executing the above instruction, receive user input data from the user device, extract context information associated with the user from the input data, determine content corresponding to the context information among a plurality of contents that can be displayed within the virtual environment, and modify the virtual environment data provided to the user device so that the content determined to correspond to the context information is displayed on the user device. Effects of the invention
[0017] A system and method for providing a dynamic virtual environment according to one aspect of the present invention can provide a virtual environment, such as a metaverse, by automatically recognizing the context of a user’s conversation through artificial intelligence (AI) or by recognizing the context based on preset rules, and by determining content suitable for the context from among pre-prepared content according to the recognized context, thereby dynamically changing the virtual environment so that the content is displayed.
[0018] By utilizing the dynamic virtual environment providing system and method according to one aspect of the present invention, optimal content is selected from the available content based on the perceived context, thereby providing the advantage of enabling more efficient communication and exchange in various environments where context must be explained during conversations within the virtual environment. The dynamic virtual environment providing system and method described above can be particularly advantageously applied to educational technology within a virtual environment, and thus, when conducting situational language education for learners, there is an advantage of enhancing educational efficiency through a virtual environment that changes according to the situation. Brief explanation of the drawing
[0019] FIG. 1 is a schematic block diagram of a dynamic virtual environment providing system according to one embodiment. FIG. 2 is a schematic block diagram showing the hardware configuration of a dynamic virtual environment providing system according to one embodiment. FIG. 3 is a flowchart showing each step of a method for providing a dynamic virtual environment according to one embodiment. FIG. 4 is a conceptual diagram showing an exemplary form of a virtual environment to which a dynamic virtual environment providing method according to one embodiment is applied. FIGS. 5 to 7 are conceptual diagrams showing exemplary forms in which the virtual environment shown in FIG. 4 is dynamically changed according to a method for providing a dynamic virtual environment according to one embodiment. Specific details for implementing the invention
[0020] Hereinafter, embodiments of the present invention will be examined in detail with reference to the drawings.
[0021] FIG. 1 is a schematic block diagram of a dynamic virtual environment providing system according to one embodiment.
[0022] Referring to FIG. 1, the dynamic virtual environment providing system (2) according to the present embodiment communicates with and operates with a user device (1), provides virtual environment data to be displayed on the user device (1), receives user input regarding the virtual environment from the user device (1), recognizes the context associated with the user based on this, and is configured to change the content to be displayed in the virtual environment according to the recognized context.
[0023] In one embodiment, the provision of a virtual environment by a dynamic virtual environment providing system (2) may be performed via a service server (3). The service server (3) is a server configured to allow a user to access it using a user device (1), and may be configured to provide virtual environment data that can be played on the user device (1), such as metaverse content or game content. For example, the service server (3) may be an application service server that enables the operation of an application (or app) running on the user device (1). Alternatively, the service server (3) may be a web server that provides a web page accessible by a web browser running on the user device (1).
[0024] The user is a user device, which is any computing device such as a mobile communication terminal like a smartphone, a personal computer, a notebook computer, a PDA (personal digital assistant), a tablet computer, a set-top box for IPTV (Internet Protocol Television), an Artificial Intelligence (AI) speaker, or a network server. You can access the service server (3) via a wired and / or wireless network in (1) and use the virtual environment content provided by the service server (3).
[0025] At this time, the dynamic virtual environment providing system (2) according to the embodiments may be configured to transmit and receive data with the service server (3) to determine or change the virtual environment content provided by the service server (3) to the user device (1). For example, the dynamic virtual environment providing system (2) may be a device that operates in conjunction with the service server (3) or may be implemented as a part of the service server (3). In addition, the dynamic virtual environment providing system (2) may receive user input regarding the virtual environment content directly from the user device (1) or receive it via the service server (3).
[0026] However, this is exemplary, and in other embodiments, the dynamic virtual environment providing system (2) may be configured to directly transmit virtual environment content to the user device without passing through the service server (3), in which case the service server (3) may be omitted. In addition, in other embodiments, the dynamic virtual environment providing system (2) itself may be implemented as a device linked to the user device (1) or as part of the user device (1).
[0027] In one embodiment, the dynamic virtual environment providing system (2) may include a receiving unit (21), a virtual environment providing unit (22), a database (DB) (23), and a matching unit (24). In addition, in one embodiment, the matching unit (24) may include a context recognition unit (241) and a content determination unit (242).
[0028] The devices described in this specification may be entirely hardware or have aspects that are partially hardware and partially software. For example, a user device (1), a dynamic virtual environment providing system (2), and a service server (3) may collectively refer to devices and related software for exchanging data of a specific format and content via electronic communication. In this specification, terms such as "unit," "module," "server," "system," "platform," "device," or "terminal" are intended to refer to a combination of hardware and software driven by said hardware. For example, the hardware may be a data processing device including a CPU or other processor. Additionally, software driven by the hardware may refer to a running process, object, executable, thread of execution, program, etc.
[0029] Additionally, each part constituting the dynamic virtual environment providing system (2) in this specification is not intended to refer to a physically separated separate component. That is, although the DB and each part (21-24) of the dynamic virtual environment providing system (2) in FIG. 1 are depicted as separate blocks distinct from one another, this is a functional distinction of the dynamic virtual environment providing system (2) based on the operations executed by them. Depending on the embodiment, some or all of the aforementioned DB and each part (21-24) may be integrated within the same device, or one or more parts may be implemented as separate devices physically separated from other parts. For example, each part of the dynamic virtual environment providing system (2) may be components connected to communicate with each other under a distributed computing environment.
[0030] The virtual environment providing unit (22) is configured to provide virtual environment data to the user device (1) to be displayed on the user device (1) communicating with the dynamic virtual environment providing system (2). For example, the virtual environment providing unit (22) may transmit virtual environment data via a communication method through a wired and / or wireless network, or provide virtual environment data by being directly connected to the user device (1).
[0031] In one embodiment, the user device (1) is connected to communicate with the service server (3), and the virtual environment providing unit (22) may transmit virtual environment data to the user device (1) in a manner that determines the data that the service server (3) transmits to the user device (1) in conjunction with the service server (3). For example, the service server (3) may be a server that provides virtual environment content, such as a game or a metaverse.
[0032] The receiving unit (21) is configured to receive user input regarding a virtual environment by communicating with a user device (1) of a user utilizing virtual environment content or a service server (3) providing such virtual environment content. At this time, the virtual environment may include a virtual space and one or more objects located within the virtual space. Additionally, user input regarding the virtual environment may include information such as the user's conversation location, conversation partner, purpose of conversation, and / or related objects, as data entered by the user for conversation with an AI character or another user.
[0033] In one embodiment, the receiving unit (21) can receive user input data through communication with a service server (3) that provides a virtual environment to the user device (1). For example, when an action performed by the user in a metaverse space or within a game is input to the corresponding service server (3), the receiving unit (21) can receive such input data from the service server (3).
[0034] Information related to multiple contents that can be displayed in a virtual environment is stored in the DB (23). Each content stored in the DB (23) is intended to provide a virtual environment implemented for a specific purpose to the user, and for each content, the type of virtual space defining the virtual environment, the display area within the virtual space, and one or more objects located within the virtual space may differ.
[0035] The content stored in the DB (23) may refer to the modeling data itself that defines the virtual environment, or it may refer to identification information for identifying the intended modeling data. For example, if the virtual environment provider (22) directly transmits virtual environment data to the user device (1), the modeling data constituting the virtual environment may be stored as content in the DB (23). However, if the virtual environment provider (22) determines the data to be provided by the service server (3) in a manner that interacts with the service server (3), the modeling data may not be directly stored in the DB (23), and only information that can identify the modeling data to be provided by the service server (3) to the user may be stored as content.
[0036] In one embodiment, the DB (23) may further store attribute information and / or evaluation information for each of the multiple contents. The attribute information may be assigned to each content in advance in the form of tags, etc. Additionally, the evaluation information may include scores (e.g., like, wrong, etc.) given by users, such as learners, to each content. This information may be used to determine content suitable for the user's context.
[0037] The matching unit (24) is configured to extract context information related to the user from user input data received by the receiving unit (21), and to determine content corresponding to the user's context information by referring to information on multiple contents stored in the DB (23). At this time, the virtual environment providing unit (22) can automatically replace the virtual environment exposed to the user with content suitable for the context by changing the virtual environment data provided to the user device (1) based on the content determined by the matching unit (24).
[0038] In one embodiment, the above process of receiving user input related to a virtual environment and automatically changing the virtual environment according to the context recognized therefrom may be performed in real time while the virtual environment data is displayed on the user device (1). That is, based on the input data received while the virtual environment data is displayed on the user device (1), the virtual environment to be displayed on the user device (1) at the next time point may be dynamically determined.
[0039] In one embodiment, the context recognition unit (241) of the matching unit (24) can extract context information by recognizing the user’s conversation location, conversation partner, conversation purpose and / or related objects, etc., represented by the input data, based on the user’s input data entered in the form of voice and / or text.
[0040] In one embodiment, the context recognition unit (241) may extract context information using any known or future-developed language model capable of natural language processing for recognizing and classifying dialogue and / or generating a response thereto, such as Natural Language Understanding (NLU), Dialogue State Tracking (DST), Dialogue Policy Learning (DPL), and Natural Language Generation (NLG). In this specification, a language model refers to a software means configured to classify dialogue content or / or generate a response corresponding to dialogue content in an automated manner through AI technologies such as machine learning and deep learning.
[0041] In another embodiment, the context recognition unit (241) may determine context information corresponding to an element by applying a pre-set rule to a specific element, such as a tag, during user input.
[0042] However, the aforementioned method for extracting context information is merely exemplary, and the method for extracting context information by the context recognition unit (241) is not necessarily limited to using a specific machine learning or deep learning model or network or a specific rule-based algorithm.
[0043] The content determination unit (242) of the matching unit (24) can determine the optimal content for the recognized context based on the context information recognized by the context recognition unit (241) and the content information stored in the DB (23). For example, it can determine which content is suitable by comparing the conversation location, conversation partner, purpose of conversation and / or related objects identified through the context information with the attribute information of each content.
[0044] In one embodiment, the content determination unit (242) assigns a score to each content according to the degree of consistency with context information, and determines the content with the highest score as the content suitable for the context when the difference between the score of a specific content and the score of another content is greater than or equal to a predetermined threshold value, but when the difference in scores is less than the threshold value, it may present a plurality of top-ranked contents determined in order of highest score to the user so that the user selects a specific content.
[0045] FIG. 2 is a schematic block diagram showing the hardware configuration of a dynamic virtual environment providing system according to one embodiment.
[0046] Referring to FIG. 2, a dynamic virtual environment providing system according to embodiments is implemented in the form of a computing device including hardware (200) and may include memory (201), a processor (202), a communication module (203) and an input / output unit (204).
[0047] Memory (201) is a non-transient computer-readable recording medium and may include permanent mass storage devices such as RAM (random access memory), ROM (read only memory), disk drive, SSD (solid state drive), and flash memory. Here, permanent mass storage devices such as ROM, SSD, flash memory, and disk drive may be included in the aforementioned device or server as separate permanent storage devices distinct from memory (201).
[0048] Additionally, the memory (201) may store an operating system and at least one program code (e.g., code for a security module or an application installed to provide a specific service). These software components may be loaded from a computer-readable recording medium separate from the memory (201). This separate computer-readable recording medium may include computer-readable recording media such as a floppy drive, disk, tape, DVD / CD-ROM drive, or memory card.
[0049] In another embodiment, software components may be loaded into memory (201) via a communication module (203) rather than a computer-readable recording medium. For example, at least one program may be loaded into memory (201) based on a computer program installed by files provided over a network by developers or a file distribution system (e.g., an application store service server) that distributes installation files for applications.
[0050] The processor (202) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor (202) by memory (201) or a communication module (203). For example, the processor (202) may be configured to execute instructions received according to program code stored in a recording device such as memory (201).
[0051] The communication module (203) can provide a function for the dynamic virtual environment providing system to communicate with a user device (1), a service server (3) and / or a learning server (4), etc., via a network. Additionally, the communication module (203) can provide a function for the dynamic virtual environment providing system to communicate with one or more other devices via a wired and / or wireless network. That is, the communication module (203) is a part that realizes each function module described above with reference to FIG. 1, by controlling its function by a processor (202) that references memory (201).
[0052] The input / output unit (204) may be a means for interfacing with an external input / output device (not shown). For example, the external input device may include devices such as a keyboard, mouse, microphone, camera, etc., and the external output device may include devices such as a display, speaker, haptic feedback device, etc. As another example, the input / output unit (204) may be a means for interfacing with a device in which the functions for input and output are integrated into one, such as a touchscreen.
[0053] In addition, in other embodiments, the dynamic virtual environment providing system may include more hardware components than those shown in FIG. 2 depending on the nature of the device to which it is applied. For example, when the dynamic virtual environment providing system is applied to a terminal device used by a user, it may be implemented to include at least some of the input / output devices described above, or it may include additional components such as a transceiver, a GPS (Global Positioning System) module, a camera, various sensors, a database, etc. As a more specific example, if the terminal device is a smartphone, it may be implemented to include various additional components such as an accelerometer or gyroscope sensor, a camera module, various physical buttons, buttons using a touch panel, input / output ports, and a vibrator for vibration, which are generally included in smartphones.
[0054] The dynamic virtual environment providing method described below with reference to FIGS. 3 to 7 can be performed by a dynamic virtual environment providing system implemented in the form of a computing device including the hardware (200) configuration described above with reference to FIG. 2. As an example, the dynamic virtual environment providing method may be provided to a user in the form of a service based on at least one of an application, software, and other programs that operate based on a user device and / or server.
[0055] In one embodiment, the processor (202) can provide virtual environment data to be displayed on the user device to the user device by executing instructions stored in memory (201), and can receive user input data from the user device.
[0056] Additionally, the processor (202) can extract context information related to the user from input data, determine a content corresponding to the context information among a plurality of contents that can be displayed in the virtual environment, and change the virtual environment data provided to the user device so that the content determined to be corresponding to the context information is displayed on the user device.
[0057] FIG. 3 is a flowchart showing each step of a method for providing a dynamic virtual environment according to one embodiment.
[0058] Referring to FIG. 3, first, a user can access a virtual environment content service provided by a dynamic virtual environment providing system (2) using a user device (1) (S1), and the dynamic virtual environment providing system (2) can transmit virtual environment data to the user device (1) to be displayed on the user device (1) (S2). In one embodiment, when the provision of virtual environment content is made through a service server (3), communication between the user device (1) and the dynamic virtual environment providing system (2) in the aforementioned steps (S1, S2) may be made via the service server (3). In another embodiment, the dynamic virtual environment providing system (2) itself may be implemented as part of the user device (1).
[0059] Next, the dynamic virtual environment providing system (2) can receive user input data for the virtual environment from the user device (1) of the user using the virtual environment (S3). The user input data may have a voice and / or text form, such as chat or voice conversation.
[0060] The dynamic virtual environment providing system (2) can extract context information defining the situation in which the user is placed or intends to use the virtual environment by recognizing the conversation location, conversation partner, conversation purpose and / or related objects, etc., through natural language processing of input data of the user using the virtual environment content (S4). Next, the dynamic virtual environment providing system (2) can determine corresponding content corresponding to the context information by referring to content information stored in the DB of the dynamic virtual environment providing system (2) (S6).
[0061] For example, assume that in a conversation between a user and an AI character or between a user and another user, the conversation content “I’m going to learn the conversation used in a restaurant from now on” is input. In this case, the dynamic virtual environment providing system (2) recognizes the conversation location “restaurant” from the input data, and the virtual space and / or objects within the virtual space may be determined as content corresponding to the context that corresponds to “restaurant”.
[0062] Whether a specific piece of content corresponds to a conversation location called a "restaurant" can be determined based on attribute information (e.g., tags, etc.) assigned in advance to each piece of content. In addition, in one embodiment, each piece of content may be further assigned evaluation information (e.g., scores, 'likes', etc.) given by users who use the content within the virtual environment, and the dynamic virtual environment providing system (2) may determine the content with relatively superior evaluation information among the content corresponding to context information as the corresponding content.
[0063] In one embodiment, the dynamic virtual environment providing system (2) may communicate with the user device (1) to allow the user to select content to be displayed (S5). For example, if there are multiple contents with attribute tags that match the context and the score or number of 'likes' of one of these contents is higher than a certain level compared to other contents, the one content is determined as the corresponding content; however, if there are multiple contents with similar evaluations, a list of contents may be presented to the user so that the user can select the corresponding content.
[0064] Although not shown in the drawing, if the provision of virtual environment content is done through the service server (3), the above content selection process may also be done via the service server (3).
[0065] When content that matches the user's context identified through conversation is determined (S6), the dynamic virtual environment providing system (2) can change the virtual environment data provided to the user based on this. For example, if the provision of virtual environment content is done through a service server (3), the dynamic virtual environment providing system (2) transmits virtual environment change information to the service server (3) (S7), and the service server (3) that receives the virtual environment change information can provide the changed virtual environment data to the user device (1) based on this (S8). However, this is exemplary, and in other embodiments, the dynamic virtual environment providing system (2) may directly provide the changed virtual environment data to the user device (1).
[0066] Hereinafter, several specific forms of change in virtual environment data based on context information will be explained with reference to FIGS. 4 to 7.
[0067] FIG. 4 is a conceptual diagram showing an exemplary form of a virtual environment to which a dynamic virtual environment providing method according to one embodiment is applied.
[0068] Referring to FIG. 4, in one embodiment of the present invention, the virtual environment may include a virtual space (300), an avatar (330) displayed corresponding to a user within the virtual space (300), and an avatar (340) displayed corresponding to a conversation partner. In this case, contextual information of the user related to the virtual environment (300) may be defined based on a conversation between the user and another user or AI character based on the avatar (330). Below, dynamic changes to the virtual environment are described based on an embodiment in which the users' avatars (330, 340) are displayed in the virtual environment. However, this is exemplary, and it is not necessary for the users' avatars (330, 340) to be displayed within the virtual space (300).
[0069] In one embodiment, changes to the virtual environment may include changes to the type of virtual space (300) of the virtual environment currently used by the user or changes to the type of specific area within the virtual space (300) where the user's interaction takes place. For example, the virtual space (300) may be classified into multiple types, such as a restaurant or a bank, depending on the situation in which a conversation occurs, and if a specific type of conversation space, such as a restaurant, is recognized as context information through the user's conversation, the virtual space (300) may be automatically replaced to display the recognized type of space.
[0070] Alternatively, the virtual space (300) may be divided into multiple areas (310, 320) having different characteristics within a single space, and the virtual environment may be changed so that the area (310, 320) where the user's avatar (330) is located changes according to the context of the user's conversation.
[0071] For example, in the embodiment illustrated in FIG. 4, the virtual space (300) is divided into a conference room space (310) and a pantry space (320), and the user's avatar (330) is located in the conference room space (310). At this time, if it is recognized through the user's conversation context that the user is having a conversation with another user regarding the pantry space (e.g., mentioning a coffee machine, sink, refrigerator, etc.), the space where the users' avatars (330, 340) are located can be automatically changed to the pantry space (320), as illustrated in FIG. 5.
[0072] In another embodiment, changes to the virtual environment may be displayed within the virtual space (300) and / or may include changes to one or more objects that the user can interact with. In the example illustrated in FIG. 4, objects such as a desk (331), a chair (332), a decoration (333), a laptop computer (334), and a tablet computer (335) are placed within the virtual space (300). The virtual environment may be implemented so that each object (331-335) can be interacted with when the user performs a specific action (e.g., clicking, dragging, etc.). At this time, if it is recognized through the context of the user’s conversation that the user is having a conversation involving an object called a ‘whiteboard,’ such as “Let’s mark on the whiteboard,” the virtual environment may be changed to add an object called a whiteboard (350) within the virtual space (300) adjacent to the location of the user’s avatars (330), as illustrated in FIG. 6.
[0073] Additionally, as illustrated in FIG. 7, the virtual space and / or objects displayed in the virtual space may be changed according to the user's conversation context. FIG. 7 illustrates a form in which the content within the virtual environment is automatically changed so that the virtual space (400) defining the virtual environment and the objects (401-403) within the virtual space (400) are changed to correspond to the 'classroom' as the user's conversation context is recognized as corresponding to the conversation location called 'classroom'.
[0074] The change in virtual environment content exemplified above with reference to FIGS. 4 to 7 can occur dynamically and in real time while the user is using the virtual environment. That is, while the virtual environment data is being displayed on the user device, context information of the conversation is recognized through the user's input data, and the virtual environment data to be provided to the user device can be changed immediately based on the context information.
[0075] In one embodiment, context information of a conversation may be determined by comprehensively considering utterances and responses between multiple users conversing with each other, rather than the input of a single user. For example, it is assumed that a conversation location called a "restaurant" is recognized when a first user inputs the utterance, "I will now learn the conversation used in a restaurant." In this case, in one embodiment, instead of immediately changing the virtual environment to content corresponding to a "restaurant," context information may be determined by further considering the responses input by one or more second users, who are conversation partners, to the first user's utterance.
[0076] For example, if the second user, like the first user, also mentions a conversation location called a 'restaurant,' the dynamic virtual environment providing system may recognize 'restaurant' as contextual information. Alternatively, even if the second user does not mention the word 'restaurant' in the same way as the first user, if the second user inputs a positive response to the first user's utterance (e.g., 'yes,' 'okay,' 'good,' 'okay,' etc.), the dynamic virtual environment providing system may recognize 'restaurant' as contextual information of the conversation. Whether the second user's conversation is a positive response may be determined based on a pre-configured word dictionary or natural language processing.
[0077] The operation of the method for providing a dynamic virtual environment according to the embodiments described above may be implemented at least partially as a computer program and recorded on a computer-readable recording medium. A computer-readable recording medium on which a program for implementing the operation according to the methods of the embodiments is recorded includes all types of recording devices in which data readable by a computer is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. Additionally, the computer-readable recording medium may be distributed across networked computer systems, and computer-readable code may be stored and executed in a distributed manner. Furthermore, the functional programs, codes, and code segments for implementing the present embodiment will be easily understood by a person skilled in the art to which the present embodiment belongs.
[0078] The present invention described above has been explained with reference to the embodiments illustrated in the drawings, but this is merely illustrative and those skilled in the art will understand that various modifications and variations of the embodiments are possible therefrom. However, such modifications should be considered to be within the technical scope of protection of the present invention.
Claims
Claim 1 A virtual environment providing unit configured to provide virtual environment data including metaverse content to be displayed on a user device; a receiving unit configured to receive user input data from the user device; and a database storing information related to a plurality of contents that can be displayed within the virtual environment; The system includes a matching unit configured to extract context information related to the user from the input data and to determine a content corresponding to the context information among the plurality of contents, wherein the virtual environment data includes a virtual space, a first avatar displayed corresponding to the user within the virtual space, and a second avatar displayed corresponding to the user's conversation partner, and the input data is data entered by the user for conversation with the conversation partner, and the receiving unit is further configured to receive the input data while the virtual environment data is displayed on the user device, and the matching unit is further configured to recognize one or more of the user's conversation location, conversation partner, conversation purpose, and related objects represented by the input data based on natural language processing of the input data, and, based on natural language processing of the conversation partner's input, to extract the context information based on one or more of the user's conversation location, conversation partner, conversation purpose, and related objects when the conversation partner inputs one or more of the user's conversation location, conversation partner, conversation purpose, and related objects identically or when the conversation partner inputs a positive response to the input data, and the virtual environment providing unit is further configured such that the content determined by the matching unit is displayed on the user device. A dynamic virtual environment providing system further configured to change the virtual environment data provided to the user device in real time based on the input data received while the virtual environment data is being displayed on the user device. Claim 2 A dynamic virtual environment providing system according to claim 1, wherein the virtual environment providing unit is further configured to change one or more of the type of virtual space, the display area within the virtual space, and one or more of the one or more objects based on the content determined by the matching unit. Claim 3 delete Claim 4 A dynamic virtual environment providing system according to claim 1, wherein the database is configured to further store one or more of attribute information and evaluation information related to each of the plurality of contents, and the matching unit is further configured to determine the content corresponding to the context information among the plurality of contents by comparing one or more of the attribute information and the evaluation information with the context information. Claim 5 delete Claim 6 A dynamic virtual environment providing system provides virtual environment data including metaverse content to be displayed on a user device to a user device; the dynamic virtual environment providing system receives user input data from the user device; the dynamic virtual environment providing system extracts context information associated with the user from the input data; and the dynamic virtual environment providing system determines a content corresponding to the context information among a plurality of contents that can be displayed within the virtual environment. The dynamic virtual environment providing system comprises the step of modifying the virtual environment data provided to the user device so that content determined to correspond to the context information is displayed on the user device, wherein the virtual environment data includes a virtual space, a first avatar displayed corresponding to the user within the virtual space, and a second avatar displayed corresponding to the user's conversation partner, and the input data is data entered by the user for conversation with the conversation partner, and the step of receiving the input data comprises the dynamic virtual environment providing system receiving the input data while the virtual environment data is displayed on the user device, and the step of extracting the context information comprises the dynamic virtual environment providing system recognizing one or more of the user's conversation location, conversation partner, conversation purpose, and related objects represented by the input data based on natural language processing of the input data;The dynamic virtual environment providing system comprises the step of extracting context information based on one or more of the user’s conversation location, conversation partner, conversation purpose, and related objects when, based on natural language processing of the conversation partner’s input, the conversation partner inputs one or more of the user’s conversation location, conversation partner, conversation purpose, and related objects identically or when the conversation partner inputs a positive response to the input data, and the step of changing the virtual environment data comprises the step of the dynamic virtual environment providing system updating the virtual environment data provided to the user device in real time based on the input data received while the virtual environment data is displayed on the user device. Claim 7 A computer program stored on a computer-readable recording medium to be combined with hardware to execute the method of providing a dynamic virtual environment according to claim 6. Claim 8 Memory storing executable commands; a communication module configured to communicate with an external system;A computing device comprising a processor connected to communicate with the memory and the communication module, wherein the processor is configured to provide virtual environment data including metaverse content to be displayed on the user device by executing the instruction, receive user input data from the user device, extract context information associated with the user from the input data, determine content corresponding to the context information among a plurality of contents that can be displayed within the virtual environment, and modify the virtual environment data provided to the user device so that the content determined to be corresponding to the context information is displayed on the user device, wherein the virtual environment data includes a virtual space, a first avatar displayed corresponding to the user within the virtual space, and a second avatar displayed corresponding to the user's conversation partner, wherein the input data is data entered by the user for conversation with the conversation partner, and wherein the processor receives the input data while the virtual environment data is displayed on the user device, recognizes one or more of the user's conversation location, conversation partner, conversation purpose, and related objects represented by the input data based on natural language processing of the input data, and, based on natural language processing of the conversation partner's input, the user's conversation location, conversation A computing device further configured to extract context information based on one or more of the user's conversation location, conversation partner, conversation purpose, and related objects when the conversation partner inputs one or more of the same person, conversation purpose, and related objects, or when the conversation partner inputs a positive response to the input data, and to update the virtual environment data provided to the user device in real time based on the input data received while the virtual environment data is displayed on the user device.
Citation Information
Patent Citations
Apparatus and method of providing virtual reality contents
KR1020220058071A
Generating method and device for generating virtual character, electronic device, storage medium and computer program
KR1020220167358A
Virtual reality presentation system, virtual reality presentation device, and virtual reality presentation method
JP2016045815A
User adaptive augmented reality mobile device and server and method thereof
KR1020130000160A
