Electronic device comprising artificial intelligence agent, and method for operating artificial intelligence agent

Abstract

The present disclosure relates to a device and a method for operating an artificial intelligence agent in a shared space. When a shared space event occurs, the artificial intelligence agent being communally operated in the shared space can be generated, a domain can be determined by the artificial intelligence agent, the determined domain can be loaded, information about users participating in the shared space event can be collected, and speech of the users can be processed on the basis of the determined domain and the information about the users.

Description

Electronic device including an artificial intelligence agent and method of operating the artificial intelligence agent

[0001] Certain embodiments relate to techniques providing electronic devices including artificial intelligence agents and methods of operating the electronic devices.

[0002] In extended reality (XR) environments, users are likely to perform necessary actions through voice interfaces, and voice assistants are expected to be utilized to handle this. Users can easily create spaces and invite each other to communicate. However, when inviting users to a shared space, an issue arises where the voice assistants supporting each user are activated simultaneously and enter the shared space. If voice assistants are used individually by each user, problems may arise where the voice assistants of other users recognize and process the voices of each user in the space, or personal information may be indiscriminately shared.

[0003] Certain embodiments seek to provide devices and methods for operating an artificial intelligence agent in a shared space.

[0004] An electronic device according to an exemplary embodiment comprises a memory; and a processor including a processing circuit, wherein at least one processor may be individually and / or collectively configured to, when a common space event occurs, create an artificial intelligence agent configured to operate in common in a common space, determine a domain in the artificial intelligence agent, load the determined domain, collect information about a user participating in the common space event, and process an utterance of the user based on the determined domain and the information about the user.

[0005] A method of operating an artificial intelligence agent according to an exemplary embodiment may include, when a common space event occurs, creating an artificial intelligence agent that operates in common in a common space; determining a domain in the artificial intelligence agent; loading the determined domain; collecting information about a user participating in the common space event; and processing an utterance of the user based on the determined domain and the information about the user.

[0006] The above and other aspects, features and advantages of specific embodiments will become more apparent from the following detailed description taken in conjunction with the accompanying drawings.

[0007] FIG. 1 is a flowchart illustrating a process of operating an artificial intelligence agent by considering a common space in an electronic device including an artificial intelligence agent according to an exemplary embodiment.

[0008] FIG. 2 is a flowchart illustrating a process of collecting information on a user participating in a shared space event in an electronic device including an artificial intelligence agent according to an exemplary embodiment.

[0009] FIG. 3 is a flowchart illustrating a process of providing history information when a shared space event ends in an electronic device including an artificial intelligence agent according to an exemplary embodiment.

[0010] FIG. 4 is a flowchart illustrating an example of processing when a user leaves a common space before a common space event ends in an electronic device including an artificial intelligence agent according to an exemplary embodiment.

[0011] FIG. 5 is a flowchart illustrating another example of handling a case where a user leaves a common space before a common space event ends in an electronic device including an artificial intelligence agent according to an exemplary embodiment.

[0012] FIG. 6 is a flowchart illustrating a process of adding a domain in an electronic device including an artificial intelligence agent according to an exemplary embodiment.

[0013] FIG. 7 is a diagram illustrating an example of an electronic device including an artificial intelligence agent operating in a common space according to an exemplary embodiment.

[0014] FIG. 8 is a diagram illustrating an example of a process in which an electronic device including an artificial intelligence agent operates in a common space for coordinating a vacation schedule according to an exemplary embodiment.

[0015] FIG. 9 is a diagram illustrating an example of providing history information in an electronic device including an artificial intelligence agent according to an exemplary embodiment.

[0016] FIG. 10 is a diagram illustrating a configuration of an electronic device including an artificial intelligence agent in a common space according to an exemplary embodiment.

[0017] FIG. 11 is a block diagram of an electronic device within a network environment according to an exemplary embodiment.

[0018] FIG. 12 is a diagram illustrating the structure of an electronic device implemented in the form of wearable augmented reality glasses according to an exemplary embodiment.

[0019] FIG. 13 is a block diagram illustrating an integrated intelligence system according to one embodiment of an exemplary embodiment.

[0020] FIG. 14 is a diagram showing a form in which relationship information between concepts and actions is stored in a database according to an exemplary embodiment.

[0021] Hereinafter, embodiments are described in detail with reference to the attached drawings. However, the embodiments may be modified in various ways, and the scope of the patent application is not limited or restricted by these embodiments. It should be understood that all modifications, equivalents, or alternatives to the embodiments are included within the scope of the patent application.

[0022] The terms used in the examples are for the purpose of description only and should not be construed as limiting. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, the terms "comprises" or "has" and the like are intended to specify the presence of a feature, number, operation, operation, component, part, or combination thereof described in the specification, but should be understood to not preclude the possibility of the presence or addition of one or more other features, numbers, operations, operations, components, parts, or combinations thereof.

[0023] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the embodiments pertain. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and shall not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0024] In addition, when describing with reference to the attached drawings, identical components will be assigned the same reference numerals regardless of the drawing numbers, and redundant descriptions thereof will be omitted. When describing embodiments, if a detailed description of a related known technology is judged to unnecessarily obscure the gist of the embodiment, the detailed description will be omitted.

[0025] Additionally, terms such as first, second, A, B, (a), (b), etc. may be used to describe components of the embodiments. These terms are only intended to distinguish the components from other components, and the nature, order, or sequence of the components are not limited by these terms. When a component is described as being "connected," "coupled," or "connected" to another component, it should be understood that the component may be directly or indirectly connected or connected to the other component, but another component may also be "connected," "coupled," or "connected" between each component. Therefore, the word "connected" includes both direct and indirect connections.

[0026] Components included in one embodiment and components with common functions will be described using the same names in other embodiments. Unless otherwise stated, the descriptions given in one embodiment may also apply to other embodiments, and detailed descriptions will be omitted to the extent of overlap.

[0027] Hereinafter, a device and method for operating an artificial intelligence agent in a common space according to an embodiment of the present invention will be described in detail with reference to the attached FIGS. 1 to 14.

[0028] FIG. 1 is a flowchart illustrating a process of operating an artificial intelligence agent by considering a common space in an electronic device including an artificial intelligence agent according to an exemplary embodiment.

[0029] Referring to FIG. 1, an electronic device including an artificial intelligence agent can determine whether a shared space event has occurred (110). If the result of operation 110 indicates that a shared space event has not occurred, the process returns to operation 110 as shown in FIG. 1.

[0030] When a common space event occurs as a result of the confirmation of operation 110, the electronic device including the artificial intelligence agent can create an artificial intelligence agent that operates in common in the common space (120).

[0031] And, the electronic device including the artificial intelligence agent can determine the domain (130).

[0032] In operation 130, an electronic device including an AI agent can analyze the title, topic, or description of a shared space to determine the corresponding domain. Alternatively, the electronic device including an AI agent can determine the domain by receiving information about the domain selected by the user who triggered the shared space event.

[0033] And, the electronic device including the artificial intelligence agent can load the determined domain (140). More specifically, the electronic device including the artificial intelligence agent can apply at least one model corresponding to the determined domain to at least one of an automatic speech recognition module (ASR module), a natural language understanding module (NLU module), a natural language generator module (NLG module), a text-to-speech module (TTS module), and an image processing module.

[0034] Additionally, an electronic device including an artificial intelligence agent can collect information about users participating in a shared space event (150). The operation of 150 will be described in more detail later with reference to FIG. 2.

[0035] Furthermore, an electronic device including an artificial intelligence agent can process the user's speech based on the determined domain and user information (160). The term "based" as used herein refers to at least the following:

[0036] FIG. 7 is a diagram illustrating an example of an artificial intelligence agent operating in a common space according to an exemplary embodiment.

[0037] Referring to FIG. 7, when a common space event for a meeting occurs, an electronic device including an artificial intelligence agent creates a common space (700) and users (721, 722, 723, 724) can display in the common space (700). At this time, the electronic device including an artificial intelligence agent can additionally display one artificial intelligence agent as a representative in the form of a virtual object (710).

[0038] FIG. 8 is a diagram illustrating an example of a process in which an artificial intelligence agent operates in a common space to coordinate a vacation schedule according to an exemplary embodiment.

[0039] Referring to Figure 8, we can see an example of a case where a company creates a common space to establish a summer vacation plan.

[0040] An electronic device including an artificial intelligence agent can create a common space when it detects the occurrence of a common space event (810) in which the title of the event is vacation schedule, the description of the common space is summer vacation schedule coordination, and the attendees are Kim OO, Hong OO, and Park OO, according to a user's request.

[0041] At this time, the electronic device including the artificial intelligence agent can load at least one domain (820) by referring to the title and description included in the shared space event (810).

[0042] At this time, at least one domain (820) may include a first domain (821) that may include a calendar and a second domain (822) that may include information on swimming pools, vacation spots, and beaches that are appropriate for a vacation location.

[0043] And, the electronic device including the artificial intelligence agent can send an invitation to a common space to attendees (830), Kim OO, Hong OO, and Park OO, request the schedules of each of Kim OO, Hong OO, and Park OO for vacation schedules, and receive information about the schedules from the attendees.

[0044]

[0045] FIG. 2 is a flowchart illustrating a process of collecting information on a user participating in a shared space event in an electronic device including an artificial intelligence agent according to an exemplary embodiment.

[0046] Referring to FIG. 2, an electronic device including an artificial intelligence agent can send an invitation message to at least one user participating in a shared space event (210).

[0047] And, the electronic device including the artificial intelligence agent can request information required for the common space event from at least one user participating in the common space event (220).

[0048] And, the electronic device including the artificial intelligence agent can receive information about each user participating in the common space event from each of at least one user participating in the common space event (230).

[0049] Users can distinguish between data to be provided and data to be unavailable, and data to be provided can be provided using an electronic device that includes an artificial intelligence agent.

[0050] User information, which is data provided by an electronic device including an artificial intelligence agent, can be divided into user public data, which is data that can be disclosed to other users of the public space, user private data, which is data that should not be disclosed to other users of the public space, public data, which is data related to other users of the public space, and personal data, which is data not related to other users of the public space.

[0051]

[0052] FIG. 3 is a flowchart illustrating a process of providing history information when a shared space event ends in an electronic device including an artificial intelligence agent according to an exemplary embodiment.

[0053] Referring to FIG. 3, when an electronic device including an artificial intelligence agent detects the end of a common space event (310), it can organize history information up to the end point and divide the history information into public data and private data (320).

[0054] Additionally, the electronic device including the artificial intelligence agent can transmit public data and personal data corresponding to each user to each user participating in the shared space event (330).

[0055] FIG. 9 is a diagram illustrating an example of providing history information in an electronic device including an artificial intelligence agent according to an exemplary embodiment.

[0056] Referring to FIG. 9, the common space artificial intelligence agent (900) can store and manage history information occurring within the common space by dividing it into public data (910) and personal data (920).

[0057] When the common space artificial intelligence agent (900) detects the end of a common space event, it organizes the history information up to the end point, divides the history information into public data (910) and personal data (920), and transmits the public data and personal data corresponding to each user to each user (940, 950).

[0058]

[0059] FIG. 4 is a flowchart illustrating an example of processing when a user leaves a common space before a common space event ends in an electronic device including an artificial intelligence agent according to an exemplary embodiment.

[0060] Referring to FIG. 4, when an electronic device including an artificial intelligence agent detects that a user has left the common space (410), it can organize history information up to the point at which the user left the common space (420).

[0061] Additionally, an electronic device including an artificial intelligence agent can transmit organized history information to a user who has left the shared space (430). In operation 430, the history information may be divided into public data and private data, as described in FIG. 3.

[0062]

[0063] FIG. 5 is a flowchart illustrating another example of handling a case where a user leaves a common space before a common space event ends in an electronic device including an artificial intelligence agent according to an exemplary embodiment.

[0064] Referring to FIG. 5, when an electronic device including an artificial intelligence agent detects that a user has left the common space (510), it can determine whether the common space event has ended (520).

[0065] When the joint space event is terminated as a result of the confirmation of operation 520, the electronic device including the artificial intelligence agent can organize the history information up to the termination point (530).

[0066] Additionally, an electronic device including an artificial intelligence agent can transmit organized history information to a user who has left the shared space (540). In operation 540, the history information may be divided into public data and private data, as described in FIG. 3.

[0067] In FIGS. 4 and 5, whether to provide history information at the time of leaving the common space to a user who has left the common space or history information at the time of the end of the common space event can be selectively applied based on the request of the user who has left the common space or the settings of the user who opened the common space event.

[0068]

[0069] FIG. 6 is a flowchart illustrating a process of adding a domain in an electronic device including an artificial intelligence agent according to an exemplary embodiment.

[0070] Referring to FIG. 6, an electronic device including an artificial intelligence agent can determine whether a domain addition event occurs (610).

[0071] In operation 610, when an electronic device including an artificial intelligence agent analyzes and processes a user's input, if the user's input cannot be processed by the currently applied domain, the electronic device may search for a domain corresponding to the user's input and generate a domain addition event that requests adding the searched domain or replacing it with the searched domain.

[0072] In operation 610, when an electronic device including an artificial intelligence agent receives a request from a user to add a new domain or replace a domain with a new domain, the electronic device may generate a domain addition event requesting the addition of a new domain or replacement with a new domain.

[0073] When a domain addition event occurs as a result of the confirmation of operation 610, an electronic device including an artificial intelligence agent can confirm at least one model corresponding to the domain for which an addition request has been made in the domain addition event (620).

[0074] And, the electronic device including the artificial intelligence agent can additionally apply or replace at least one model corresponding to the additionally requested domain to the corresponding module (630).

[0075] Specifically, an electronic device including an artificial intelligence agent can additionally or replaceably apply at least one model corresponding to an additional requested domain to at least one of an automatic speech recognition module (ASR module), a natural language understanding module (NLU module), a natural language generator module (NLG module), a text to speech module (TTS module), and an image processing module.

[0076]

[0077] FIG. 10 is a diagram illustrating a configuration of a device for operating an artificial intelligence agent in a common space according to an exemplary embodiment.

[0078] Referring to FIG. 10, an electronic device (1000) including an artificial intelligence agent may include a processor (1010), a communication unit (1030), a display unit (1040), a memory (1050), and a microphone (1060).

[0079] The communication unit (1030) is a communication interface device including a receiver and a transmitter, capable of processing spoken voice and communicating with an intelligent server that responds with the processing results. Furthermore, the communication unit (1030) can communicate with other electronic devices (1071-1073) participating in the shared space.

[0080] The display unit (1040) displays status information (or indicators), limited numbers and characters, moving pictures, and still pictures generated during the operation of an electronic device (1000) including an artificial intelligence agent. In addition, the display unit (1040) may display an artificial intelligence agent providing a voice assistant as a virtual object under the control of the processor (1010).

[0081] The memory (1050) stores an operating system, application programs, and storage data for controlling the overall operation of an electronic device (1000) including an artificial intelligence agent. Additionally, the memory (1050) can store user information collected through the information processing unit (1013) and history information generated during a shared space event.

[0082] The microphone (1060) can process external acoustic signals into electrical voice data.

[0083] The processor (1010) may include an agent management unit (1011), a model application unit (1012), an information processing unit (1013), a history management unit (1014), and an agent processing unit (1020).

[0084] Each "processor" in this disclosure may include a processing circuit or may include multiple processors. For example, as used in this disclosure, including in the claims, the term "processor" may include various processing circuits including at least one processor, wherein one or more of the at least one processor may be configured to perform various functions described herein, individually and / or collectively, in a distributed manner. When "processor," "at least one processor," and "one or more processors" are described herein as being configured to perform multiple functions, these terms include, but are not limited to, situations where one processor performs some of the recited functions and another processor performs other of the recited functions, and situations where a single processor may perform all of the recited functions. Furthermore, the at least one processor may include a combination of processors that perform various recited / disclosed functions, for example, in a distributed manner. The at least one processor may execute program instructions to achieve or perform various functions.

[0085] When a common space event occurs, the agent management unit (1011) can create an artificial intelligence agent that operates in common in the common space and determine a domain to be applied to the artificial intelligence agent.

[0086] At this time, the agent management unit (1011) can analyze the title, topic, or description of the shared space to determine the corresponding domain. Alternatively, the agent management unit (1011) can determine the domain by receiving information on the domain selected by the user who generated the shared space event.

[0087] When the agent management unit (1011) detects the occurrence of a domain addition event, it can transmit the domain for which addition has been requested to the model application unit (1012). At this time, when the agent management unit (1011) analyzes and processes the user's input, if the user's input cannot be processed with the currently applied domain, it can search for a domain corresponding to the user's input and generate a domain addition event requesting addition of the searched domain or replacement with the searched domain. In addition, when the agent management unit (1011) receives a request from the user to add a new domain or replace a domain with a new domain, it can generate a domain addition event requesting addition of a new domain or replacement with a new domain.

[0088] The model application unit (1012) can load the determined domain.

[0089] Specifically, the model application unit (1012) can apply at least one model corresponding to the requested domain to at least one of an automatic speech recognition module (ASR module), a natural language understanding module (NLU module), a natural language generator module (NLG module), a text to speech module (TTS module), and an image processing module.

[0090] The information processing unit (1013) may collect information about users participating in a public space event. More specifically, the information processing unit (1013) may send an invitation message to at least one user participating in the public space event, request information necessary for the public space event from at least one user participating in the public space event, and receive user information from each of at least one user participating in the public space event. At this time, the user information received is information provided by the user, and may include at least one of the following: user public data, which is data that may be disclosed to other users of the public space; user private data, which is data that should not be disclosed to other users of the public space; public data, which is data related to other users of the public space; and personal data, which is data not related to other users of the public space.

[0091] If a user participating in a shared space event leaves the shared space before the shared space event ends, the information processing unit (1013) can provide the user who left the shared space with organized history information up to the time of leaving the shared space.

[0092] If a user participating in a shared space event leaves the shared space before the shared space event ends, the information processing unit (1013) can provide the user who left the shared space with organized history information up to the end point after the shared space event ends.

[0093] Whether to provide users who leave a shared space with history information from the time they left or the time the shared space event ended can be selectively applied based on the request of the user who left or the settings of the user who initiated the shared space event. In this case, history information can be divided into public data and personal data for each user, and the personal data can only include data for that user.

[0094] When a shared space event ends, the information processing unit (1013) can provide organized history information up to the end point to each user participating in the shared space event. Specifically, when a shared space event ends, the information processing unit (1013) can divide the organized history information up to the end point into public data and personal data, and provide the public data and the personal data corresponding to each user to all electronic devices (1071, 1072, 1073) participating in the shared space event.

[0095] When the information processing unit (1013) provides history information to a user, if the electronic device (1000) including the artificial intelligence agent and the user receiving the history information use the same type of artificial intelligence agent, the history information is transmitted as is because the same data format is used.

[0096] When providing history information to a user, the information processing unit (1013) may convert and transmit the history information into a standardized data format or a data format used by the artificial intelligence agent used by the user receiving the history information, if the electronic device (1000) including the artificial intelligence agent and the user receiving the history information use different types of artificial intelligence agents.

[0097] The history management unit (1014) can store and manage history information including input data and processing result data processed by the artificial intelligence agent in the memory (1050).

[0098] The agent processing unit (1020) can process the user's speech based on the determined domain and user information.

[0099] The agent processing unit (1020) may include an automatic speech recognition module (ASR module) (1021), a natural language understanding module (NLU module) (1022), a natural language generator module (NLG module) (1023), a text-to-speech conversion module (TTS module) (1024), and an image processing module (1025).

[0100] The ASR module (1021) can convert received voice input into text data.

[0101] The NLU module (1022) can identify the user's intention using text data of voice input.

[0102] The NLG module (1023) can change the specified information into text format.

[0103] The TTS module (1024) can change information in text form into information in voice form.

[0104] The image processing module (1025) can analyze the received image.

[0105] The agent processing unit (1020) may be configured to be included in an electronic device (1000) including an artificial intelligence agent, but may also be implemented as an intelligent device located externally, and may respond with processing results through communication via a communication unit (1030).

[0106]

[0107] Meanwhile, the electronic device (1000) including the artificial intelligence agent of FIG. 10 may be configured in the form of an electronic device (1101) in a network environment as in FIG. 11 below, or may be configured in the form of wearable augmented reality glasses (1200) as in FIG. 12.

[0108] FIG. 11 is a block diagram of an electronic device (1101) within a network environment (1100) according to an exemplary embodiment.

[0109] Referring to FIG. 11, in a network environment (1100), an electronic device (1101) may communicate with an electronic device (1102) via a first network (1198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (1104) or a server (1108) via a second network (1199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (1101) may communicate with the electronic device (1104) via the server (1108). According to one embodiment, the electronic device (1101) may include a processor (1120), a memory (1130), an input module (1150), an audio output module (1155), a display module (1160), an audio module (1170), a sensor module (1176), an interface (1177), a connection terminal (1178), a haptic module (1179), a power management module (1188), a battery (1189), a communication module (1190), a subscriber identification module (1196), or an antenna module (1197). In some embodiments, the electronic device (1101) may omit at least one of these components (e.g., the connection terminal (1178)), or may have one or more other components added. In some embodiments, some of these components (e.g., sensor module (1176), camera module (1180), or antenna module (1197)) may be integrated into a single component (e.g., display module (1160)).

[0110] The processor (1120) may, for example, execute software (e.g., a program (1140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (1101) directly or indirectly connected to the processor (1120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (1120) may store commands or data received from other components (e.g., a sensor module (1176) or a communication module (1190)) in a volatile memory (1132), process the commands or data stored in the volatile memory (1132), and store result data in a non-volatile memory (1134) (which may include, for example, an internal memory (1136) and / or an external memory (1138). According to one embodiment, the processor (1120) may include a main processor (1121) (e.g., a central processing unit or an application processor) or an auxiliary processor (1123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (1121). For example, when the electronic device (1101) includes the main processor (1121) and the auxiliary processor (1123), the auxiliary processor (1123) may be configured to use less power than the main processor (1121) or to be specialized for a given function. The auxiliary processor (1123) may be implemented separately from the main processor (1121) or as a part thereof.

[0111] The auxiliary processor (1123) may control at least a portion of functions or states associated with at least one component (e.g., the display module (1160), the sensor module (1176), or the communication module (1190)) of the electronic device (1101), for example, on behalf of the main processor (1121) while the main processor (1121) is in an inactive (e.g., sleep) state, or together with the main processor (1121) while the main processor (1121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (1123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (1180) or a communication module (1190)). In one embodiment, the auxiliary processor (1123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (1101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (1108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0112] Meanwhile, the processor (1120) can perform the operation of the processor (1010) of FIG. 10.

[0113] The memory (1130) can store various data used by at least one component (e.g., the processor (1120) or the sensor module (1176)) of the electronic device (1101). The data can include, for example, software (e.g., the program (1140)) and input data or output data for commands related thereto. The memory (1130) can include a volatile memory (1132) or a non-volatile memory (1134).

[0114] The program (1140) may be stored as software in memory (1130) and may include, for example, an operating system (1142), middleware (1144), or an application (1146).

[0115] The input module (1150) can receive commands or data to be used in a component of the electronic device (1101) (e.g., a processor (1120)) from an external source (e.g., a user) of the electronic device (1101). The input module (1150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0116] The audio output module (1155) can output audio signals to the outside of the electronic device (1101). The audio output module (1155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0117] The display module (1160) can visually provide information to an external party (e.g., a user) of the electronic device (1101). The display module (1160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (1160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0118] The audio module (1170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (1170) can acquire sound through the input module (1150), output sound through the sound output module (1155), or an external electronic device (e.g., electronic device (1102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (1101).

[0119] The sensor module (1176) can detect the operating status (e.g., power or temperature) of the electronic device (1101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (1176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, a Hall sensor, or an illuminance sensor.

[0120] Additionally, the sensor module (1176) may further include a camera module capable of capturing still images and moving images. In this case, the camera module may include one or more lenses, image sensors, image signal processors, or flashes.

[0121] The interface (1177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (1101) with an external electronic device (e.g., the electronic device (1102)). In one embodiment, the interface (1177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0122] For example, the electronic device (1101) can transmit a video signal to an external electronic device through a connection terminal (1178). The electronic device (1101) can transmit a video signal for the external electronic device to output a video to a display module of the external electronic device.

[0123] The connection terminal (1178) may be for outputting a video signal or for outputting an audio signal. For example, the connection terminal (1178) may be for outputting a video signal and an audio signal simultaneously. For example, the electronic device (1101) may output a video signal and an audio signal to an external electronic device through an interface such as HDMI, DP, or Thunderbolt from the connection terminal (1178) that outputs a video signal and an audio signal simultaneously.

[0124] The connection terminal (1178) may include a connector through which the electronic device (1101) may be physically connected to an external electronic device (e.g., the electronic device (1102)). According to one embodiment, the connection terminal (1178) may include, for example, an HDMI connector, a DP connector, a Thunderbolt connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0125] The haptic module (1179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (1179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0126] The power management module (1188) can manage power supplied to the electronic device (1101). According to one embodiment, the power management module (1188) can be implemented, for example, as at least a part of a power management integrated circuit (PMIC).

[0127] A battery (1189) may power at least one component of the electronic device (1101). In one embodiment, the battery (1189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0128] A communication module (1190) including a communication circuit can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (1101) and an external electronic device (e.g., electronic device (1102), electronic device (1104), or server (1108)), and the performance of communication through the established communication channel. The communication module (1190) can operate independently from the processor (1120) (e.g., application processor) and can include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (1190) can include a wireless communication module (1192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (1194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, a corresponding communication module can communicate with an external electronic device (1104) via a first network (1198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (1199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (1192) can verify or authenticate the electronic device (1101) within a communication network such as the first network (1198) or the second network (1199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (1196).

[0129] A wireless communication module (1192) including a communication circuit can support a 5G network and next-generation communication technologies following a 4G network, such as NR access technology (new radio access technology). NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (1192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (1192) may support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (1192) may support various requirements specified in the electronic device (1101), an external electronic device (e.g., the electronic device (1104)), or a network system (e.g., the second network (1199)). According to one embodiment, the wireless communication module (1192) may support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0130] The antenna module (1197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (1197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (1197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (1198) or the second network (1199), may be selected from the plurality of antennas by, for example, the communication module (1190). A signal or power may be transmitted or received between the communication module (1190) and the external electronic device via the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (1197).

[0131] According to various embodiments, the antenna module (1197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0132] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0133] According to one embodiment, commands or data may be transmitted or received between the electronic device (1101) and an external electronic device (1104) via a server (1108) connected to a second network (1199). Each of the external electronic devices (1102 or 1104) may be the same or a different type of device as the electronic device (1101). According to one embodiment, all or part of the operations executed in the electronic device (1101) may be executed in one or more of the external electronic devices (1102, 1104, or 1108). For example, when the electronic device (1101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (1101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (1101). The electronic device (1101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (1101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (1104) may include an Internet of Things (IoT) device. The server (1108) may be an intelligent server utilizing machine learning and / or a neural network.In one embodiment, an external electronic device (1104) or server (1108) may be included in the second network (1199). The electronic device (1101) may be applied to intelligent services (e.g., smart homes, smart cities, smart cars, or healthcare) based on 5G communication technology and IoT-related technology.

[0134]

[0135] FIG. 12 is a diagram illustrating the structure of an electronic device implemented in the form of wearable augmented reality glasses according to an exemplary embodiment.

[0136] Referring to FIG. 12, an electronic device (1200) may be worn on a user's face and may provide the user with images related to an augmented reality service and / or a virtual reality service.

[0137] In one embodiment, the electronic device (1200) includes a first display (1205), a second display (1210), a first screen display unit (1215a), a second screen display unit (1215b), an input optical member (1220), a first transparent member (1225a), a second transparent member (1225b), a lighting unit (1230a, 1230b), a first PCB (1235a), a second PCB (1235b), a first hinge (1240a), a second hinge (1240b), a first camera (1245a, 1245b, 1245c, 1245d), a plurality of microphones (e.g., a first microphone (1250a), a second microphone (1250b), a third microphone (1250c)), a plurality of speakers (e.g., a first speaker (1255a), a second It may include a speaker (1255b)), a battery (1260), a second camera (1275a, 1275b), a third camera (1265), and a visor (1270a, 1270b).

[0138] In one embodiment, the displays (e.g., the first display (1205) and the second display (1210)) may include, for example, a liquid crystal display (LCD), a digital mirror device (DMD), a liquid crystal on silicon (LCoS), an organic light emitting diode (OLED), or a micro light emitting diode (micro LED). Although not shown, when the displays are made of one of the liquid crystal display, the digital mirror display, or the silicon liquid crystal display, the electronic device (1200) may include a light source that irradiates light onto a screen output area of ​​the display. In another embodiment, when the displays are capable of generating light on their own, for example, when they are made of one of the organic light emitting diodes or the micro LED, the electronic device (1200) may provide a good quality virtual image to the user even without including a separate light source. In one embodiment, if the display is implemented with an organic light-emitting diode or micro LED, a light source is unnecessary, so the electronic device (1200) can be made lighter. In the following, a display capable of generating light on its own is referred to as a self-luminous display, and the description is based on the assumption of a self-luminous display.

[0139] Displays according to various embodiments (e.g., the first display (1205) and the second display (1210)) may be composed of at least one micro LED (micro light emitting diode). For example, the micro LED can express red (R), green (G), and blue (B) by self-luminescence, and is small in size (e.g., 100 μm or less), so that one chip can implement one pixel (e.g., one of R, G, and B). Accordingly, when the display is composed of the micro LED, it can provide a high resolution without a backlight unit (BLU).

[0140] Not limited thereto, one pixel may include R, G, and B, and one chip may be implemented with multiple pixels including R, G, and B.

[0141] In one embodiment, the display (e.g., the first display (1205) and the second display (1210)) may be configured with a display area composed of pixels for displaying a virtual image and light-receiving pixels (e.g., photo sensor pixels) arranged between the pixels for receiving light reflected from the eye, converting it into electrical energy, and outputting it.

[0142] In one embodiment, the electronic device (1200) can detect the direction of the user's gaze (e.g., eye movement) through light-receiving pixels. For example, the electronic device (1200) can detect and track the direction of the user's gaze with respect to the right eye and the direction of the user's gaze with respect to the left eye through one or more light-receiving pixels constituting the first display (1205) and one or more light-receiving pixels constituting the second display (1210). The electronic device (1200) can determine the position of the center of the virtual image based on the direction of the user's gaze with respect to the right and left eyes (e.g., the direction in which the pupils of the user's right and left eyes are gazing) detected through one or more light-receiving pixels.

[0143] In one embodiment, light emitted from displays (e.g., a first display (1205) and a second display (1210)) may pass through a lens (not shown) and a waveguide to reach a first screen display unit (1215a) formed on a first transparent member (1225a) positioned to face the user's right eye and a second screen display unit (1215b) formed on a second transparent member (1225b) positioned to face the user's left eye. For example, light emitted from displays (e.g., a first display (1205) and a second display (1210)) may pass through a waveguide to be reflected by a grating area formed on an input optical member (1220) and screen display units (1215a, 1215b) and transmitted to the user's eyes. The first transparent member (1225a) and / or the second transparent member (1225b) may be formed of a glass plate, a plastic plate, or a polymer, and may be manufactured to be transparent or translucent.

[0144] In one embodiment, a lens (not shown) may be positioned in front of a display (e.g., a first display (1205) and a second display (1210)). The lens (not shown) may include a concave lens and / or a convex lens. For example, the lens (not shown) may include a projection lens or a collimation lens.

[0145] In one embodiment, the screen display unit (1215a, 1215b) or the transparent member (e.g., the first transparent member (1225a), the second transparent member (1225b)) may include a lens including a waveguide, a reflective lens.

[0146] In one embodiment, the waveguide may be made of glass, plastic, or polymer, and may include nano-patterns formed on one surface of the inner or outer surface, for example, a grating structure having a polygonal or curved shape. According to one embodiment, light incident on one end of the waveguide may be propagated within the display waveguide by the nano-patterns and provided to the user. In one embodiment, a waveguide composed of a free-form prism may provide the incident light to the user through a reflective mirror. The waveguide may include at least one diffractive element, for example, a diffractive optical element (DOE), a holographic optical element (HOE), or a reflective element, for example, a reflective mirror. In one embodiment, the waveguide may guide light emitted from the display (1205, 1210) to the user's eyes by using at least one diffractive element or reflective element included in the waveguide.

[0147] According to various embodiments, the diffractive element may include an input optical member (1220) / output optical member (not shown). For example, the input optical member (1220) may mean an input grating area, and the output optical member (not shown) may mean an output grating area. The input grating area may serve as an input terminal that diffracts (or reflects) light output from a display (e.g., a first display (1205) and a second display (1210)) (e.g., a micro LED) to transmit the light to a transparent member (e.g., a first transparent member (1225a), a second transparent member (1225b)) of a screen display unit (1215a, 1215b). The output grating region can act as an outlet to diffract (or reflect) light transmitted to the transparent member of the waveguide (e.g., the first transparent member (1225a), the second transparent member (1225b)) toward the user's eyes.

[0148] According to various embodiments, the reflective element may include a total internal reflection (TIR) ​​optical element or a total internal reflection waveguide. For example, total internal reflection may refer to a method of guiding light such that light (e.g., a virtual image) entering through an input grating region is 100% reflected from one surface (e.g., a specific surface) of the waveguide, thereby transmitting 100% of the light to the output grating region.

[0149] In one embodiment, light emitted from a display (1205, 1210) may be guided along an optical path through an input optical member (1220) into a waveguide. Light traveling within the waveguide may be guided toward a user's eyes through an output optical member. The screen display (1215a, 1215b) may be determined based on the light emitted toward the user's eyes.

[0150] In one embodiment, the first camera (1245a, 1245b, 1245c, 1245d) may include a camera used for 6 degrees of freedom (6DoF), 6DoF head tracking, hand detection and tracking, gesture and / or spatial recognition. For example, the first camera (1245a, 1245b, 1245c, 1245d) may include a global shutter (GS) camera to detect head and hand movements and track movements.

[0151] For example, the first camera (1245a, 1245b, 1245c, 1245d) may be a stereo camera for head tracking and spatial recognition, and cameras of the same specifications and performance may be applied. The first camera (1245a, 1245b, 1245c, 1245d) may be a GS camera with excellent performance (e.g., image drag) for detecting and tracking fine movements such as rapid hand movements and fingers.

[0152] According to various embodiments, the first camera (1245a, 1245b, 1245c, 1245d) may be a rolling shutter (RS) camera. The first camera (1245a, 1245b, 1245c, 1245d) may perform SLAM functions through spatial recognition and depth shooting for 6 DoF. The first camera (1245a, 1245b, 1245c, 1245d) may perform a user gesture recognition function.

[0153] In one embodiment, the second camera (1275a, 1275b) may be used to detect and track the pupil. The second camera (1275a, 1275b) may be referred to as an ET (eye tracking) camera. The second camera (1275a, 1275b) may track the direction of the user's gaze. The electronic device (1200) may consider the direction of the user's gaze and position the center of the virtual image projected on the screen display unit (1215a, 1215b) according to the direction in which the user's pupil is looking.

[0154] The second camera (1275a, 1275b) for tracking the gaze direction may be a GS camera that can detect the pupil and track rapid eye movements. The second cameras (1275a, 1275b) may be installed for the left and right eyes, respectively, and the second cameras (1275a, 1275b) for the left and right eyes may be cameras with identical performance and specifications.

[0155] In one embodiment, the third camera (1265) may be referred to as HR (high resolution) or PV (photo video) and may include a high-resolution camera. The third camera (1265) may include a color camera equipped with functions for obtaining high-quality images, such as an AF (auto focus) function and an OIS (optical image stabilizer). However, the third camera (1265) is not limited thereto, and may include a GS (global shutter) camera or an RS (rolling shutter) camera.

[0156] In one embodiment, at least one sensor (e.g., a gyroscope sensor, an acceleration sensor, a magnetometer sensor, a touch sensor, an ambient light sensor, and / or a gesture sensor), and a first camera (1245a, 1245b, 1245c, 1245d) can perform at least one of head tracking for 6DoF, pose estimation & prediction, gesture and / or spatial recognition, and slam function with depth shooting.

[0157] In another embodiment, the first camera (1245a, 1245b, 1245c, 1245d) may be used separately as a camera for head tracking and a camera for hand tracking.

[0158] In one embodiment, the lighting units (1230a, 1230b) may have different uses depending on the attachment location. For example, the lighting units (1230a, 1230b) may be attached together with the first camera (1245a, 1245b, 1245c, 1245d) mounted around a hinge (e.g., a first hinge (1240a), a second hinge (1240b)) connecting a frame and a temple or around a bridge connecting the frames. When shooting with a GS camera, the lighting units (1230a, 1230b) may be used as a means of supplementing the ambient brightness. For example, the lighting units (1230a, 1230b) may be used when it is difficult to detect a subject to be shot due to a dark environment or mixing or reflecting light from multiple light sources.

[0159] In one embodiment, a PCB (e.g., a first PCB (1235a), a second PCB (1235b)) may include a processor (not shown), a memory (not shown), and a communication module (not shown) that control components of the electronic device (1200).

[0160] Each embodiment of the present disclosure can be used in combination with other embodiments described in the present disclosure.

[0161] A communication module (not shown) may support the establishment of a direct (e.g., wired) or wireless communication channel between the electronic device (1200) and an external electronic device, and the performance of communication through the established communication channel. The PCB may transmit electrical signals to components constituting the electronic device (1200).

[0162] A communication module (not shown) may include one or more communication processors that operate independently from the processor and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (not shown) may include a wireless communication module (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (e.g., a local area network (LAN) communication module, or a power line communication module). Any of these communication modules (not shown) may communicate with an external electronic device via a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA), or a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or a WAN). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips).

[0163] The wireless communication module can support 5G networks and next-generation communication technologies beyond 4G networks, such as new radio access technology (NR). NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimizing terminal power and connecting multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency communications (URLLC (ultra-reliable and low-latency communications)). The wireless communication module can support high-frequency bands (e.g., mmWave bands) to achieve high data rates, for example. The wireless communication module may support various technologies to secure performance in high-frequency bands, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna.

[0164] The electronic device (1200) may further include an antenna module (not shown). The antenna module may transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a first PCB (1235a), a second PCB (1235b)). In one embodiment, the antenna module may include a plurality of antennas (e.g., an array antenna).

[0165] In one embodiment, multiple microphones (e.g., a first microphone (1250a), a second microphone (1250b), and a third microphone (1250c)) can process external acoustic signals into electrical voice data. The processed voice data can be utilized in various ways depending on the function being performed (or the application being executed) by the electronic device (1200).

[0166] In one embodiment, multiple speakers (e.g., first speaker (1255a), second speaker (1255b)) can output audio data received from a communication module or stored in a memory.

[0167] In one embodiment, one or more batteries (1260) may be included and may supply power to components that make up the electronic device (1200).

[0168] In one embodiment, the visor (1270a, 1270b) can adjust the amount of external light transmitted into the user's eyes according to the transmittance. The visor (1270a, 1270b) can be positioned in front or behind the screen display unit (1215a, 1215b). The front of the screen display unit (1215a, 1215b) can mean the direction opposite to the user wearing the electronic device (1200), and the back can mean the direction toward the user wearing the electronic device (1200). The visor (1270a, 1270b) can protect the screen display unit (1215a, 1215b) and adjust the amount of external light transmitted.

[0169] For example, the visor (1270a, 1270b) may include an electrochromic element that changes color according to the applied power to control the transmittance. Electrochromism is a phenomenon in which the color changes due to an oxidation-reduction reaction caused by the applied power. The visor (1270a, 1270b) can control the transmittance of external light by utilizing the color change of the electrochromic element.

[0170] For example, the visor (1270a, 1270b) may include a control module circuit and an electrochromic element. The control module may control the electrochromic element to adjust the transmittance of the electrochromic element.

[0171]

[0172] In addition, the electronic device (1000) including the artificial intelligence agent of FIG. 10 may be implemented as the integrated intelligence system of FIG. 13, and when the agent processing unit (1020) of the electronic device (1000) including the artificial intelligence agent is implemented as an external configuration, the agent processing unit (1020) may be implemented as the integrated intelligence system of .

[0173] FIG. 13 is a block diagram illustrating an integrated intelligence system according to one embodiment of an exemplary embodiment.

[0174] Referring to FIG. 13, an integrated intelligence system of one embodiment may include a user terminal (1300), an intelligent server (1400), and a service server (1500).

[0175] The user terminal (1300) of one embodiment may be a terminal device (or electronic device) that can connect to the Internet, and may be, for example, a mobile phone, a smart phone, a personal digital assistant (PDA), a laptop computer, a TV, white goods, a wearable device, an HMD, or a smart speaker.

[0176] According to the illustrated embodiment, the user terminal (1300) may include a communication interface (1310), a microphone (1320), a speaker (1330), a display (1340), a memory (1350), or a processor (1360). The above-listed components may be operatively or electrically connected to each other.

[0177] The communication interface (1310) of one embodiment may be configured to connect to an external device and transmit and receive data. The microphone (1320) of one embodiment may receive sound (e.g., user speech) and convert it into an electrical signal. The speaker (1330) of one embodiment may output the electrical signal as sound (e.g., voice). The display (1340) of one embodiment may be configured to display an image or video. The display (1340) of one embodiment may also display a graphical user interface (GUI) of a running app (or application program).

[0178] In one embodiment, the memory (1350) may store a client module (1351), a software development kit (SDK) (1353), and multiple apps (1355). The client module (1351) and the SDK (1353) may constitute a framework (or solution program) for performing general functions. In addition, the client module (1351) or the SDK (1353) may constitute a framework for processing voice input.

[0179] In one embodiment, the memory (1350) may be a program for performing a specified function for the plurality of apps (1355). According to one embodiment, the plurality of apps (1355) (e.g., 1355_1, 1355_2) may include a first app (1355_1) and a second app (1355_2). According to one embodiment, each of the plurality of apps (1355) may include a plurality of operations for performing a specified function. For example, the apps may include an alarm app, a message app, and / or a schedule app. According to one embodiment, the plurality of apps (1355) may be executed by the processor (1360) to sequentially perform at least some of the plurality of operations.

[0180] The processor (1360) of one embodiment can control the overall operation of the user terminal (1300). For example, the processor (1360) can be directly or indirectly electrically connected to a communication interface (1310), a microphone (1320), a speaker (1330), and a display (1340) to perform a designated operation.

[0181] The processor (1360) of one embodiment may also execute a program stored in the memory (1350) to perform a designated function. For example, the processor (1360) may execute at least one of the client module (1351) or the SDK (1353) to perform the following operations for processing voice input. The processor (1360) may control the operations of multiple apps (1355), for example, through the SDK (1353). The following operations described as operations of the client module (1351) or the SDK (1353) may be operations executed by the processor (1360). Here, each "module" may include a circuit such as a processing circuit.

[0182] The client module (1351) of one embodiment can receive a voice input. For example, the client module (1351) can receive a voice signal corresponding to a user utterance detected through a microphone (1320). The client module (1351) can transmit the received voice input to the intelligent server (1400). The client module (1351) can transmit status information of the user terminal (1300) to the intelligent server (1400) along with the received voice input. The status information can be, for example, execution status information of an app.

[0183] The client module (1351) of one embodiment can receive a result corresponding to the received voice input. For example, the client module (1351) can receive a result corresponding to the received voice input if the intelligent server (1400) can produce a result corresponding to the received voice input. The client module (1351) can display the received result on the display (1340).

[0184] In one embodiment, the client module (1351) may receive a plan corresponding to the received voice input. The client module (1351) may display the results of executing multiple operations of the app according to the plan on the display (1340). For example, the client module (1351) may sequentially display the results of executing multiple operations on the display. The user terminal (1300) may, for example, display only some of the results of executing multiple operations (e.g., the result of the last operation) on the display.

[0185] According to one embodiment, the client module (1351) may receive a request from the intelligent server (1400) to obtain information necessary to produce a result corresponding to a voice input. According to one embodiment, the client module (1351) may transmit the necessary information to the intelligent server (1400) in response to the request.

[0186] The client module (1351) of one embodiment can transmit result information of executing multiple operations according to a plan to the intelligent server (1400). The intelligent server (1400) can use the result information to confirm that the received voice input has been processed correctly.

[0187] The client module (1351) of one embodiment may include a voice recognition module. According to one embodiment, the client module (1351) may recognize voice inputs that perform limited functions through the voice recognition module. For example, the client module (1351) may execute an intelligent app to process voice inputs to perform organic actions through designated inputs (e.g., "Wake up!").

[0188] An intelligent server (1400) of one embodiment may receive information related to a user voice input from a user terminal (1300) via a communication network. According to one embodiment, the intelligent server (1400) may convert data related to the received voice input into text data. According to one embodiment, the intelligent server (1400) may generate a plan for performing a task corresponding to the user voice input based on the text data.

[0189] In one embodiment, the plan may be generated by an artificial intelligence (AI) system. The AI ​​system may be a rule-based system, a neural network-based system (e.g., a feedforward neural network (FNN), a recurrent neural network (RNN), or a combination of the above, or another AI system. In one embodiment, the plan may be selected from a set of predefined plans, or may be generated in real time in response to a user request. For example, the AI ​​system may select at least one plan from a plurality of predefined plans.

[0190] In one embodiment, the intelligent server (1400) may transmit results according to the generated plan to the user terminal (1300), or transmit the generated plan to the user terminal (1300). According to one embodiment, the user terminal (1300) may display results according to the plan on a display. According to one embodiment, the user terminal (1300) may display results of executing actions according to the plan on a display.

[0191] An intelligent server (1400) of one embodiment may include a front end (1410), a natural language platform (1420), a capsule database (1430), an execution engine (1440), an end user interface (1450), a management platform (1460), a big data platform (1470), or an analytic platform (1480).

[0192] The front end (1410) of one embodiment can receive a voice input from a user terminal (1300). The front end (1410) can transmit a response corresponding to the voice input.

[0193] According to one embodiment, the natural language platform (1420) may include an automatic speech recognition module (ASR module) (1421), a natural language understanding module (NLU module) (1423), a planner module (1425), a natural language generator module (NLG module) (1427), or a text to speech module (TTS module) (1429).

[0194] An automatic speech recognition module (1421) of one embodiment can convert a voice input received from a user terminal (1300) into text data. A natural language understanding module (1423) of one embodiment can use the text data of the voice input to determine the user's intention. For example, the natural language understanding module (1423) can perform syntactic analysis or semantic analysis to determine the user's intention. The natural language understanding module (1423) of one embodiment can use linguistic features (e.g., grammatical elements) of morphemes or phrases to determine the meaning of words extracted from the voice input, and can match the meaning of the determined words to the intention to determine the user's intention.

[0195] In one embodiment, the planner module (1425) can generate a plan using the intent and parameters determined by the natural language understanding module (1423). According to one embodiment, the planner module (1425) can determine a plurality of domains necessary to perform a task based on the determined intent. The planner module (1425) can determine a plurality of operations included in each of the plurality of domains determined based on the intent. According to one embodiment, the planner module (1425) can determine parameters necessary to execute the determined plurality of operations or result values ​​output by the execution of the plurality of operations. The parameters and the result values ​​can be defined as concepts of a specified format (or class). Accordingly, the plan can include a plurality of operations and a plurality of concepts determined by the user's intent. The planner module (1425) can operationally (or hierarchically) determine the relationship between the plurality of operations and the plurality of concepts. For example, the planner module (1425) can determine the execution order of a plurality of actions based on the user's intention based on a plurality of concepts. In other words, the planner module (1425) can determine the execution order of a plurality of actions based on parameters required for the execution of the plurality of actions and results output by the execution of the plurality of actions. Accordingly, the planner module (1425) can generate a plan including association information (e.g., ontology) between the plurality of actions and the plurality of concepts. The planner module (1425) can generate the plan using information stored in a capsule database (1430) in which a set of relationships between concepts and actions is stored.

[0196] The natural language generation module (1427) of one embodiment can convert specified information into text format. The information converted into text format may be in the form of natural language speech. The text-to-speech conversion module (1429) of one embodiment can convert information in text format into information in speech format.

[0197] According to one embodiment, some or all of the functions of the natural language platform (1420) may also be implemented in the user terminal (1300).

[0198] The capsule database (1430) can store information on the relationships between multiple concepts and actions corresponding to multiple domains. According to one embodiment, a capsule may include multiple action objects (or action information) and concept objects (or concept information) included in a plan. According to one embodiment, the capsule database (1430) may store multiple capsules in the form of a concept action network (CAN). According to one embodiment, the multiple capsules may be stored in a function registry included in the capsule database (1430).

[0199] The capsule database (1430) may include a strategy registry that stores strategy information required when determining a plan corresponding to a voice input. The strategy information may include reference information for determining one plan when there are multiple plans corresponding to a voice input. According to one embodiment, the capsule database (1430) may include a follow-up registry that stores information on follow-up actions for suggesting follow-up actions to a user in a given situation. The follow-up actions may include, for example, follow-up utterances. According to one embodiment, the capsule database (1430) may include a layout registry that stores layout information of information output through the user terminal (1300). According to one embodiment, the capsule database (1430) may include a vocabulary registry that stores vocabulary information included in capsule information. According to one embodiment, the capsule database (1430) may include a dialog registry that stores information on dialogue (or interaction) with the user. The capsule database (1430) may update stored objects through a developer tool. The developer tool may include, for example, a function editor for updating action objects or concept objects. The developer tool may include a vocabulary editor for updating vocabulary. The developer tool may include a strategy editor for creating and registering strategies that determine plans.The developer tool may include a dialog editor that creates a dialogue with the user. The developer tool may also include a follow-up editor that activates follow-up goals and allows editing of follow-up utterances that provide hints. The follow-up goals may be determined based on currently set goals, user preferences, or environmental conditions. In one embodiment, the capsule database (1430) may also be implemented within the user terminal (1300).

[0200] The execution engine (1440) of one embodiment can produce a result using the generated plan. The end user interface (1450) can transmit the produced result to the user terminal (1300). Accordingly, the user terminal (1300) can receive the result and provide the received result to the user. The management platform (1460) of one embodiment can manage information used in the intelligent server (1400). The big data platform (1470) of one embodiment can collect user data. The analysis platform (1480) of one embodiment can manage the quality of service (QoS) of the intelligent server (1400). For example, the analysis platform (1480) can manage the components and processing speed (or efficiency) of the intelligent server (1400).

[0201] In one embodiment, a service server (1500) may provide a designated service (e.g., food ordering or hotel reservation) to a user terminal (1300). According to one embodiment, the service server (1500) may be a server operated by a third party. In one embodiment, the service server (1500) may provide information for generating a plan corresponding to the received voice input to an intelligent server (1400). The provided information may be stored in a capsule database (1430). In addition, the service server (1500) may provide result information according to the plan to the intelligent server (1400).

[0202] In the integrated intelligence system described above, the user terminal (1300) can provide various intelligent services to the user in response to user input. The user input may include, for example, input via a physical button, touch input, or voice input.

[0203] In one embodiment, the user terminal (1300) may provide a voice recognition service through an intelligent app (or voice recognition app) stored internally. In this case, for example, the user terminal (1300) may recognize a user utterance or voice input received through the microphone and provide the user with a service corresponding to the recognized voice input.

[0204] In one embodiment, the user terminal (1300) may perform a designated action based on the received voice input, either alone or in conjunction with the intelligent server and / or service server. For example, the user terminal (1300) may execute an app corresponding to the received voice input and perform a designated action through the executed app.

[0205] In one embodiment, when a user terminal (1300) provides a service together with an intelligent server (1400) and / or a service server, the user terminal may detect user speech using the microphone (1320) and generate a signal (or voice data) corresponding to the detected user speech. The user terminal may transmit the voice data to the intelligent server (1400) using a communication interface (1310).

[0206] According to one embodiment, an intelligent server (1400) may generate a plan for performing a task corresponding to a voice input received from a user terminal (1300), or a result of performing an operation according to the plan, in response to the voice input. The plan may include, for example, a plurality of operations for performing a task corresponding to the user's voice input, and a plurality of concepts related to the plurality of operations. The concepts may define parameters input to the execution of the plurality of operations, or result values ​​output by the execution of the plurality of operations. The plan may include association information between the plurality of operations and the plurality of concepts.

[0207] In one embodiment, the user terminal (1300) can receive the response using the communication interface (1310). The user terminal (1300) can output a voice signal generated within the user terminal (1300) to the outside using the speaker (1330), or can output an image generated within the user terminal (1300) to the outside using the display (1340).

[0208]

[0209] FIG. 14 is a diagram showing a form in which relationship information between concepts and actions is stored in a database according to an exemplary embodiment.

[0210] Referring to FIG. 14, the capsule database (e.g., capsule database (1430)) of the intelligent server (1400) can store capsules in the form of a CAN (concept action network). The capsule database can store operations for processing tasks corresponding to a user's voice input and parameters necessary for the operations in the form of a CAN (concept action network).

[0211] The capsule database may store a plurality of capsules (e.g., capsule(A)(1601), capsule(B)(1604)) corresponding to each of a plurality of domains (e.g., applications). In one embodiment, one capsule (e.g., capsule(A)(1601)) may correspond to one domain (e.g., location (geo), application). In addition, one capsule may correspond to at least one service provider (e.g., CP 1 (1602) or CP 2 (1603)) for performing a function for a domain related to the capsule. In one embodiment, one capsule may include at least one operation (1610) and at least one concept (1620) for performing a specified function.

[0212] The above natural language platform (1420) can generate a plan for performing a task corresponding to a received voice input using capsules stored in a capsule database. For example, the planner module (1425) of the natural language platform can generate a plan using capsules stored in a capsule database. For example, a plan (1607) can be generated using actions (16011, 16013) and concepts (16012, 16014) of capsule A (1610) and actions (16041) and concepts (16042) of capsule B (1604).

[0213]

[0214] According to one embodiment, an electronic device includes a memory and a processor, wherein the processor is configured to, when a shared space event occurs, generate an artificial intelligence agent that operates in common in the shared space, determine a domain in the artificial intelligence agent, load the determined domain, collect information about a user participating in the shared space event, and process an utterance of the user based on the determined domain and the information about the user. The artificial intelligence agent can be trained and used as described in the present disclosure.

[0215] In one embodiment, the processor may determine the corresponding domain by analyzing a title, topic, or description of the common space when determining the domain.

[0216] According to one embodiment, when determining the domain, the processor may determine the domain by receiving selected domain information of a user who generated the common space event.

[0217] According to one embodiment, when loading the determined domain, the processor may apply at least one model corresponding to the determined domain to at least one of an automatic speech recognition module (ASR module), a natural language understanding module (NLU module), a natural language generator module (NLG module), a text to speech module (TTS module), and an image processing module.

[0218] According to one embodiment, the processor may collect information of users participating in the common space event by sending an invitation message to users participating in the common space event, requesting information necessary for the common space event from users participating in the common space event, and receiving information of the users from each user participating in the common space event.

[0219] According to one embodiment, the user's information may include at least one of the user's public data, which is data that may be disclosed to other users of the public space, the user's private data, which is data that should not be disclosed to other users of the public space, the public data, which is data related to other users of the public space, and the personal data, which is data not related to other users of the public space.

[0220] According to one embodiment, if a user participating in the shared space event leaves the shared space before the shared space event ends, the processor may provide the user who left the shared space with organized history information up to the time of leaving the shared space.

[0221] According to one embodiment, if a user participating in the shared space event leaves the shared space before the shared space event ends, the processor may provide the user who left the shared space with organized history information up to the end point after the shared space event ends.

[0222] According to one embodiment, when the shared space event ends, the processor may provide organized history information up to the end point after the shared space event ends to each user participating in the shared space event.

[0223] According to one embodiment, when the shared space event ends, the processor may divide the organized history information up to the end point into public data and personal data after the shared space event ends, and provide the public data and the personal data corresponding to each user to each user participating in the shared space event.

[0224] According to one embodiment, when the processor detects the occurrence of a domain addition event, the processor may identify at least one model corresponding to a domain requested to be added in the domain addition event, and additionally apply or replace the at least one model corresponding to the domain requested to be added to at least one of an automatic speech recognition module (ASR module), a natural language understanding module (NLU module), a natural language generator module (NLG module), a text to speech module (TTS module), and an image processing module.

[0225] According to one embodiment, when analyzing and processing the user's input, if the user's input cannot be processed by the currently applied domain, the processor may search for a domain corresponding to the user's input and generate a domain addition event requesting addition of the searched domain or replacement with the searched domain.

[0226] According to one embodiment, when the processor receives a request from the user to add a new domain or replace a domain with the new domain, the processor may generate an event to add a domain or request to replace the domain with the new domain.

[0227] According to one embodiment, a method for operating an artificial intelligence agent may include, when a common space event occurs, creating an artificial intelligence agent that operates in common in a common space; determining a domain in the artificial intelligence agent; loading the determined domain; collecting information about a user participating in the common space event; and processing an utterance of the user based on the determined domain and the information about the user.

[0228] In one embodiment, the operation of determining the domain may determine the corresponding domain by analyzing the title, topic or description of the common space.

[0229] According to one embodiment, the operation of determining the domain may determine the domain by receiving selected domain information of a user who generated the shared space event.

[0230] According to one embodiment, the operation of loading the determined domain may apply at least one model corresponding to the determined domain to at least one of an automatic speech recognition module (ASR module), a natural language understanding module (NLU module), a natural language generator module (NLG module), a text to speech module (TTS module), and an image processing module.

[0231] According to one embodiment, the operation of collecting information of at least one user participating in the common space event may include: sending an invitation message to the at least one user participating in the common space event; requesting information necessary for the common space event from the at least one user participating in the common space event; and receiving information of the user from each of the at least one user participating in the common space event.

[0232] According to one embodiment, the user's information may include at least one of the user's public data, which is data that may be disclosed to other users of the public space, the user's private data, which is data that should not be disclosed to other users of the public space, the public data, which is data related to other users of the public space, and the personal data, which is data not related to other users of the public space.

[0233] According to one embodiment, a method of operating an artificial intelligence agent may further include providing, to a user who has left the common space, organized history information up to the time of leaving the common space, if the user participating in the common space event leaves the common space before the common space event ends.

[0234] According to one embodiment, a method of operating an artificial intelligence agent may further include, when a user participating in the shared space event leaves the shared space before the shared space event ends, providing the user who leaves the shared space with organized history information up to the end point after the shared space event ends.

[0235] According to one embodiment, the method of operating an artificial intelligence agent may further include, when the shared space event ends, providing, to each user participating in the shared space event, organized history information up to the end point after the shared space event ends.

[0236] According to one embodiment, when the shared space event ends, the operation of providing the organized history information up to the end point after the shared space event ends to each of all users participating in the shared space event may include the operation of dividing the organized history information up to the end point after the shared space event ends into public data and private data; and the operation of providing the public data and the private data corresponding to each user to each of all users participating in the shared space event.

[0237] According to one embodiment, a method for operating an artificial intelligence agent may further include: detecting an occurrence of a domain addition event; identifying at least one model corresponding to a domain requested to be added in the domain addition event; and additionally applying at least one model corresponding to the domain requested to be added to at least one of an automatic speech recognition module (ASR module), a natural language understanding module (NLU module), a natural language generator module (NLG module), a text-to-speech module (TTS module), and an image processing module, or replacing a currently applied domain with the requested domain.

[0238] According to one embodiment, the operation of detecting the occurrence of the domain addition event may include: when analyzing and processing the user's input, if the user's input cannot be processed with the currently applied domain, an operation of searching for a domain corresponding to the user's input; and an operation of generating the domain addition event to request adding the searched domain or replacing it with the searched domain.

[0239] According to one embodiment, the operation of detecting the occurrence of the domain addition event may include the operation of receiving a request from the user to add a new domain or replace a domain with the new domain; and the operation of generating the domain addition event that requests the addition of the new domain or replacement with the new domain.

[0240]

[0241] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may store program commands, data files, data structures, etc., singly or in combination. The program commands recorded on the medium may be those specially designed and configured for the embodiment or may be those known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of the program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiment, and vice versa.

[0242] Software may include a computer program, code, instructions, or a combination of one or more of these, and may configure a processing device to perform a desired operation or, independently or collectively, command the processing device. The software and / or data may be stored on any type of machine, component, physical device, virtual equipment, computer storage medium, or device, for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over networked computer systems, and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

[0243] Although the embodiments have been described with limited drawings as described above, those skilled in the art will appreciate that various technical modifications and variations can be applied based on the above. For example, appropriate results can be achieved even if the described techniques are performed in a different order from the described method, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different form from the described method, or are replaced or substituted with other components or equivalents. While the present disclosure has been described and illustrated with reference to various embodiments, it is to be understood that these various embodiments are illustrative and not limiting. Furthermore, those skilled in the art will appreciate that various modifications in configuration and details are possible without departing from the full scope and spirit of the present disclosure, including the scope of the following claims and their equivalents. Furthermore, it will be appreciated that any of the embodiments described in the present disclosure can be used in conjunction with other embodiments described in the present disclosure.

[0244] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.

Claims

1. In electronic devices, Memory: and At least one processor comprising a processing circuit Including, At least one processor of the above, When a common space event occurs, an AI agent configured to operate in common in the common space is created, and the domain is determined by the AI ​​agent. Load the domain determined above, We collect information from users who participate in the above shared space events, Individually and / or collectively configured to process the user's utterance based on the determined domain and the user's information. Electronic devices.

2. In paragraph 1, At least one processor including the above processing circuit, When determining the above domain, individually and / or collectively configured to analyze the title, topic or description of the above common space to determine the corresponding domain. Electronic devices.

3. In any one of paragraphs 1 and 2, At least one processor including the above processing circuit, When determining the above domain, Individually and / or collectively configured to receive the selected domain information of the user who caused the above shared space event and determine the domain. Electronic devices.

4. In any one of paragraphs 1 to 3, At least one processor including the above processing circuit, When loading the above determined domain, Individually and / or collectively configured to apply at least one model corresponding to the determined domain to at least one of an automatic speech recognition module (ASR module), a natural language understanding module (NLU module), a natural language generator module (NLG module), a text to speech module (TTS module), and an image processing module. Electronic devices.

5. In any one of paragraphs 1 to 4, At least one processor including the above processing circuit, Individually and / or collectively configured to send an invitation message to at least one user participating in the shared space event, request information required for the shared space event from the at least one user participating in the shared space event, and receive information about the user from each of the at least one user participating in the shared space event, thereby collecting information about the user participating in the shared space event. Electronic devices.

6. In any one of paragraphs 1 to 5, The above user's information is: Public data of the user, which is data that can be disclosed to other users of the public space, Private data of the above user, which should not be disclosed to other users of the above public space; Public data, which is data related to other users of the above public space; and Personal data that is not related to other users of the above public space An electronic device comprising at least one of:

7. In any one of paragraphs 1 to 6, At least one processor including the above processing circuit, If a user participating in the above shared space event leaves the shared space before the shared space event ends, the history information organized up to the time of leaving the shared space is individually and / or collectively configured to be provided to the user who left the shared space. Electronic devices.

8. In any one of paragraphs 1 to 7, At least one processor including the above processing circuit, If a user participating in the above shared space event leaves the shared space before the shared space event ends, the shared space event is individually and / or collectively configured to provide the user who left the shared space with the organized history information up to the end point after the shared space event ends. Electronic devices.

9. In any one of paragraphs 1 to 8, At least one processor including the above processing circuit, When the above shared space event ends, it is individually and / or collectively configured to provide the organized history information up to the end point to each user participating in the shared space event after the above shared space event ends. Electronic devices.

10. In any one of paragraphs 1 to 9, At least one processor including the above processing circuit, When the above shared space event ends, the history information organized up to the end point after the above shared space event ends is divided into public data and personal data. Individually and / or collectively configured to provide each user participating in the above shared space event with the above public data and the above personal data corresponding to each user. Electronic devices.

11. In any one of paragraphs 1 to 10, At least one processor including the above processing circuit, When detecting the occurrence of a domain addition event, at least one model corresponding to the domain requested for addition in the domain addition event is checked, Individually and / or collectively configured to additionally apply or replace at least one model corresponding to the additionally requested domain to at least one of an automatic speech recognition module (ASR module), a natural language understanding module (NLU module), a natural language generator module (NLG module), a text to speech module (TTS module), and an image processing module. Electronic devices.

12. In any one of paragraphs 1 to 11, At least one processor including the above processing circuit, When analyzing and processing the user's input, if the user's input cannot be processed with the currently applied domain, a domain corresponding to the user's input is searched for, Individually and / or collectively configured to cause the occurrence of the domain addition event requesting addition of the searched domain or replacement with the searched domain. Electronic devices.

13. In any one of paragraphs 1 to 12, At least one processor including the above processing circuit, When receiving a request from the above user to add a new domain or replace a domain with the new domain, the above domain addition event is individually and / or collectively configured to generate a request to add the new domain or replace with the new domain. Electronic devices.

14. A method for operating an artificial intelligence agent, When a common space event occurs, an action is taken to create an artificial intelligence agent that operates in common in the common space; The action of determining a domain in the above artificial intelligence agent; An action to load the above determined domain; Actions to collect information of users participating in the above shared space event; and An action to process the user's speech based on the determined domain and the user's information. A method including:

15. In paragraph 14, The action of determining the above domain is: At least one of the determinations of the corresponding domain is included by analyzing the title, topic or description of the above common space. method.

Images