Method and apparatus for tracking participants of collaboration activity using on-device AI(Artificial Intelligence)

Abstract

A method and device for tracking participants in cooperative activities using on-device artificial intelligence are disclosed. The method comprises: a step of identifying each participant by distinguishing at least one of the timbre and location of each participant by referring to at least one of input voice and video when a cooperative activity begins; a step of obtaining a unique identifier from each participant's terminal using an Ultra-wideband (UWB) module and receiving user detail information corresponding to each participant's unique identifier from an information providing server; a step of identifying each participant by generating matching information by matching each participant's user detail information with the identified participant; and a step of controlling an LLM-based on-device artificial intelligence to distinguish actions and utterances recognized during the progress of the cooperative activity by participant using the matching information and to generate participant-specific tracking information that reflects the context.

Description

Technology Field

[0001] The present invention relates to a technology for automatically tracking offline cooperative activities performed by a number of people together, and more specifically, to a method and apparatus for specifically identifying participants in cooperative activities and tracking the speech and actions of each participant using on-device artificial intelligence. Background Technology

[0002] As online gatherings become more active in many fields, collaborative activities such as meetings, seminars, and study groups are often conducted online. In such cases where participants connect online, it is easy to identify the participants using their connection information and to track their online activities and update their activity history using artificial intelligence.

[0003] However, there are several technical limitations in situations where multiple participants gather offline to conduct collaborative activities. First, it is difficult for the AI ​​module to identify each individual participant present at an offline gathering. Additionally, it is not easy for the AI ​​module to grasp the context of conversations or activities among the participants.

[0004] When multiple participants perform cooperative activities while a dedicated application on a user terminal using an artificial intelligence module is running, it is possible to convert the conversation content into text form through voice analysis or to differentiate the conversation content for each participant and convert it into text form by analyzing the tone.

[0005] However, according to these conventional technologies, the best that can be achieved is to generate meeting minutes as an outcome of collaborative activities, and there are technical limitations to artificial intelligence tracking the behavior of each participant in offline collaborative activities or actively intervening in the collaborative activities. This is because the information that artificial intelligence can acquire is in the form of voice or video, making it difficult to track participants as in cases where participants connected online perform collaborative activities online. The problem to be solved

[0006] The present invention aims to resolve the problems of the prior art as described above by providing a method and device that can identify users participating in cooperative activities at an offline meeting and track the speech or actions of the participants using on-device artificial intelligence equipped on a participant's terminal and Ultra-wideband (UWB) communication.

[0007] In addition, the present invention aims to provide a method and apparatus capable of analyzing conversations and cooperative behaviors among participants in offline cooperative activities using on-device artificial intelligence, and evaluating each participant's achievement, understanding, contribution, etc. means of solving the problem

[0008] To achieve the above objectives, a method for tracking participants in a cooperative activity using on-device artificial intelligence according to an embodiment of the present invention may include: a step of identifying each participant by distinguishing at least one of the tone and location of each participant by referring to at least one of the input voice and video when an offline cooperative activity begins; a step of obtaining a unique identifier from each participant's terminal using an Ultra-wideband (UWB) module and receiving user detailed information corresponding to each participant's unique identifier from an information providing server; a step of specifying each participant by generating matching information by matching each participant's user detailed information with each participant identified above; and a step of controlling an LLM-based on-device artificial intelligence to distinguish actions and utterances recognized during the progress of the cooperative activity by participant using the matching information and to generate tracking information for each participant that reflects the context.

[0009] The step of generating the matching information may include controlling the LLM-based on-device artificial intelligence to analyze an attribute including at least one of an attitude, understanding, contribution, and opinion regarding each participant's past cooperative activities using activity history information included in the user details.

[0010] The step of controlling an LLM-based on-device artificial intelligence to generate tracking information for each participant may include a step of analyzing the context of utterances in the cooperative activity by referring to the attributes of the analyzed participant.

[0011] The tracking method may further include the step of controlling the LLM-based on-device artificial intelligence to create a result of the cooperative activity using the tracking information for each participant, while creating an integrated result that reflects the result of a previous cooperative activity related to the cooperative activity by referring to the activity history information included in the user details of each participant received from the information providing server.

[0012] An electronic device according to one embodiment of the present invention comprises: an input module for receiving commands or data to be used in the electronic device from outside the electronic device; a display module for providing information to outside the electronic device; an Ultra-wideband (UWB) module for communicating with another electronic device in an ultra-wideband frequency band; a processor electrically connected to the input module, the display module, and the UWB module; and a memory electrically connected to the processor and including an LLM-based on-device artificial intelligence module.

[0013] The above processor can control the LLM-based on-device artificial intelligence to identify each participant by distinguishing at least one of the timbre and location of each participant by referring to at least one of the input voice and video when an offline cooperative activity begins, obtain a unique identifier from each participant's terminal using the UWB module, retrieve user detail information corresponding to each participant's unique identifier from an information providing server, and generate matching information by matching each participant's user detail information with the identified participant to identify each participant, and use the matching information to distinguish actions and utterances recognized during the cooperative activity by participant and generate context-reflecting tracking information for each participant. Effects of the invention

[0014] According to the present invention, by using an on-device artificial intelligence and an ultra-wideband sensor equipped on a participant's terminal, users participating in cooperative activities at an offline meeting can be identified and the participants' speech or actions can be tracked.

[0015] In addition, according to the present invention, by using on-device artificial intelligence to analyze conversations and cooperative behaviors among participants in offline cooperative activities and evaluating each participant's achievement, understanding, and contribution, and by actively intervening in offline cooperative activities based on this or providing customized content to the group of participants or each participant, the efficiency and satisfaction of offline cooperative activities can be maximized. Brief explanation of the drawing

[0016] FIG. 1 is a diagram illustrating the network configuration of a cooperative activity participant tracking system using on-device artificial intelligence according to an embodiment of the present invention. FIG. 2 is a diagram illustrating the configuration of a host terminal that performs a method for tracking cooperative activity participants by having an on-device artificial intelligence according to an embodiment of the present invention. FIG. 3 is a diagram illustrating the functional configuration of a dedicated application for performing a method of tracking cooperative activity participants using on-device artificial intelligence according to an embodiment of the present invention. FIG. 4 is a flowchart illustrating a method for tracking participants in cooperative activities using on-device artificial intelligence according to an embodiment of the present invention. Specific details for implementing the invention

[0017] The terms used in this specification will be briefly explained, and the invention will be described in detail.

[0018] The terms used in this invention have been selected based on currently widely used general terms, taking into account their functions within the invention; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this invention should be defined not merely by their names, but based on their meanings and the overall content of the invention.

[0019] When a part of the specification is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as "...means," "...part," and "module" as used in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware or software, or as a combination of hardware and software.

[0020] Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0021] FIG. 1 is a schematic diagram showing the network configuration of a cooperative activity participant tracking system using on-device artificial intelligence according to one embodiment of the present invention.

[0022] Participants in the cooperative activity will attend the offline meeting carrying a smart device with an ultra-wideband (UWB) module built in, and the cooperative activity participant tracking system may be composed of multiple user terminals (110) carried by the participants and an information providing server (100). The user terminals (110) are smart devices of participants attending the cooperative activity meeting offline, and may be divided into at least one host terminal (200) with an on-device artificial intelligence module and a dedicated application for tracking cooperative activity participants installed, and user terminals (101, 102) of other participants(s).

[0023] The host terminal (200) may be implemented as a computing device equipped with a cooperative activity participant tracking system that includes software and hardware for identifying participants in offline cooperative activities and tracking cooperative activities such as speech using LLM (Large Language Model)-based on-device artificial intelligence. A dedicated application implementing the cooperative activity participant tracking system may provide cooperative activity participant tracking services to a user by using an API provided by an LLM-based on-device artificial intelligence model.

[0024] LLM, or Large Language Model, is also called a giant language model; it is a language model composed of artificial neural networks with numerous parameters and is a key element enabling AI chatbot technology. It is trained using self-supervised or semi-self-supervised learning with a significant amount of unlabeled text.

[0025] A user terminal (110) can communicate with another user terminal or an information provider server (100) through a network. Depending on the installation environment, the network (100) may be configured as a wired network such as Ethernet, Power Line Communication, telephone line communication devices and RS-serial communication, a mobile communication network, a Wireless LAN (WLAN), Wi-Fi, Bluetooth and ZigBee, or a combination thereof.

[0026] The user terminal (110) may be any computing device capable of wired and / or wireless communication. For example, the user terminal may include a smartphone, mobile phone, navigation, desktop computer, laptop, digital broadcasting terminal, PDA (Personal Digital Assistants), PMP (Portable Multimedia Player), tablet PC, game console, wearable device, IoT (Internet of Things) device, VR (Virtual Reality) device, AR (Augmented Reality) device, etc.

[0027] The user terminal (110) is equipped with a UWB module and can determine each other's locations through the built-in UWB sensor. UWB technology is a short-range wireless communication protocol that operates via radio waves at high frequencies. It is characterized by very precise spatial recognition and directionality, and operates to enable mobile devices to recognize their surrounding environment well. Through UWB, various devices are intelligently connected, allowing them to perform various functions ranging from secure remote payments to finding the location of a remote control. In addition, because accurate searching is possible over a large area, it is possible to find a restaurant at an airport or locate a parked car using a smartphone. With the recent advancement of autonomous vehicles, UWB is being installed in smartphones, and this technology is being utilized to support vehicle digital key services.

[0028] Participants in the cooperative activity will have their unique identifiers registered with the information provision server (100) in advance through the user terminal (110) by performing membership registration processing with the information provision server (100). When the cooperative activity starts, the host terminal (200) can determine the terminal locations of other participants and obtain their unique identifiers through the built-in UWB sensor. The host terminal (200) can transmit each participant's unique identifier to the information provision server (100) and receive and use detailed information about the corresponding participant in response.

[0029] The user details of a participant are a set of information collected about the user, which may include user attribute information, history information regarding previous collaborative activities, learning history information, etc. The host terminal (200) can access the user details using each participant's unique identifier, and as the collaborative activity progresses, the LLM-based on-device artificial intelligence of the host terminal (200) can use the user details to specifically identify each speaker, analyze the participant's actions and utterances, and store them separately for each participant. In this way, the process of the collaborative activity of the participants can be tracked, and active intervention appropriate to the situation regarding the collaborative activity can be performed based on the tracking information.

[0030] FIG. 2 is a diagram illustrating the configuration of a host terminal that performs a method for tracking cooperative activity participants by having an on-device artificial intelligence according to an embodiment of the present invention.

[0031] The host terminal (200) may be implemented to include a dedicated application (210) for tracking participants in cooperative activities, an LLM-based on-device artificial intelligence module (220), and a UWB module (230).

[0032] In one embodiment, the dedicated application (210) performs a two-step process of specifically identifying the participants and then tracking the actions of each participant in order to intervene beyond simply receiving the participants' speech content and organizing it into text.

[0033] The specific identification of participants performing offline collaborative activities can again be composed of the following two stages.

[0034] First, when participants possessing user terminals start a cooperative activity in an offline environment, the cooperative activity tracking application (210) receives the participants' speech (201) and / or video (203) and can determine the location of each participant by analyzing the direction of the voice or the video. In addition, the direction of the input voice, tone, and appearance of the participants can be distinguished and used to identify each participant. Subsequently, even if participants speak freely, it becomes possible to distinguish whose speech it is and save it.

[0035] Next, the cooperative activity tracking application (210) can identify the terminal locations of other participants through the UWB module (230) and obtain information about each participant's unique identifier. The obtained unique identifier can be used to retrieve detailed information about the participant from an external information provider server (100) or local memory. By matching the retrieved detailed information with the participant identified in the previous step, each participant can be identified more specifically by the matching information. Based on this matching information, the LLM-based on-device artificial intelligence module (220) can track utterances and cooperative behaviors input as the cooperative activity proceeds, on a per-participant basis.

[0036] FIG. 3 is a diagram illustrating the functional configuration of a dedicated application for performing a method for tracking cooperative activity participants using on-device artificial intelligence according to an embodiment of the present invention, and FIG. 4 is a flowchart illustrating a method for tracking cooperative activity participants using on-device artificial intelligence according to an embodiment of the present invention.

[0037] When the host, who is the user of the host terminal (200), opens and runs a dedicated application on their terminal (200) and then starts an offline meeting, the cooperation activity begins.

[0038] When an offline cooperative activity begins, the participant identification unit (301) can determine how many people are present and identify each participant by distinguishing at least one of the voice tone and location of each participant by referring to at least one of the input voice and video (step S41). Accordingly, when participants speak later, it becomes possible to distinguish whose speech it is and store it.

[0039] The unique identifier acquisition unit (302) acquires a unique identifier from each participant's terminal using the built-in UWB (230) module, and the participant identification unit (303) transmits each participant's unique identifier to an external information providing server (100) (S42). In response to this, the information providing server (100) will search for a previously registered unique identifier and the corresponding user information and transmit detailed information about the requested participants.

[0040] In this way, according to the present invention, the efficiency of cooperative activities can be increased by automatically collecting location and identification information of participants through UWB communication in offline meetings.

[0041] User details may include basic information including personal details of the user, activity history information regarding past collaborative activities related to current collaborative activities, learning history information, achievement information, etc.

[0042] The participant identification unit (303) receives detailed information for each participant from the information providing server (100) (S43), and generates matching information by matching each participant identified in step S41 with the detailed information received from the information providing server (100), and can specifically identify each of the cooperative activity participants using this (S44).

[0043] According to one embodiment, an LLM-based on-device artificial intelligence (220) can be controlled to analyze an attribute including at least one of an attitude, understanding, contribution, and opinion regarding the past cooperation activities of the participant using activity history information included in user details.

[0044] The tracking information generation unit (304) can control an LLM-based on-device artificial intelligence (220) to generate tracking information for each participant that reflects context by distinguishing between actions and utterances perceived during the cooperative activity using the matching information of the participants specifically identified above. In order to generate tracking information, the attributes of the participants may also be referenced when analyzing the context of utterances in the cooperative activity.

[0045] According to one embodiment, the LLM-based on-device artificial intelligence (220) can be controlled to create a result of a collaborative activity using the tracking information for each participant generated in this way, and to create an integrated result that reflects the result of a previous collaborative activity related to the collaborative activity by referring to the activity history information included in the user details of each participant received from the information providing server (100).

[0046] On the other hand, since the conversation content between the LLM and a single user maintains consistency, it is possible for the LLM to understand the conversation in text form and infer the context. However, when multiple participants exchange conversations or engage in various interactions, it is difficult to judge the context between utterances, as well as their relationships, roles, and levels of participation.

[0047] For example, when Participant A asks a question and Participant B answers, a context of question and answer exists between Participant A's question and Participant B's answer amidst numerous utterances recorded in a sequence. As another example, when Participant B elaborates on a previously given explanation, a context of elaboration exists between Participant B's original explanation and the elaboration.

[0048] Furthermore, it is necessary not only to simply convert the content of the utterance into text and analyze it, but also to conduct contextual analysis regarding various situations, such as the speaker's voice volume or emotion, whether they interrupt others, whether they wait for the current speaker to finish before speaking, and what role specific participants primarily play in their speech.

[0049] In other words, new collaborative activity analysis technology is needed to identify the context and interactions between input utterances as the collaborative activity progresses, and to analyze the contributions and achievements of each participant.

[0050] According to one embodiment, after identifying and specifically identifying each participant as described above, a collaborative activity analysis method consisting of four steps can be performed as follows. These steps are necessary to track the utterances or actions of each participant during offline collaborative activities, and to track and analyze the context of individual utterances and actions. In order to efficiently use LLM-based on-device artificial intelligence, it is necessary to create separate LLM sessions to perform each step and to control these LLM sessions so that they operate independently of each other.

[0051] First, the first LLM session can control the conversion of the content of utterances recognized during the collaborative activity into text and the storage associated with the participants who are the utterers. For example, in the first LLM session, Participant A uttered Text #1 and Participant B uttered Text #2, and the order in which the utterances were made can be stored.

[0052] The first LLM session is to generate data for use in other LLM sessions, and in particular, to provide foundational data that the second LLM session will use to analyze the relationships between utterances.

[0053] According to one embodiment, utterances for each participant can be defined as a data structure in which the order can be known, such as in the form of a linked list, and all utterances of all participants can also be stored as a data structure in which the order can be determined.

[0054] According to one embodiment, each utterance content can be stored in a database and defined to have field values ​​such as the speaker, utterance content, and utterance time for each utterance.

[0055] Next, the second LLM session can be controlled to analyze and define the relationship between the currently recognized utterance and previously stored utterances. Since simply converting each utterance into text is insufficient for analyzing collaborative activities, it searches for utterances among previous ones that are related to the current utterance; if such utterances exist, the relationship between those utterances and the current utterance is defined and stored.

[0056] The second LLM session provides relationship information necessary for the third LLM session described later to structure the context of the conversation.

[0057] According to one embodiment, in a second LLM session, the relationship between existing utterances and a new utterance can be identified and defined, and the relationship between two or more can be connected using a specific data structure such as a pointer. Specifically, a specific utterance among one's own or another's past utterances can be connected to the current utterance using (1) a specific data structure such as a pointer, and (2) the type of the relationship can be determined and stored. Examples of types of relationships between utterances include elaboration, rebuttal, question, and answer.

[0058] For example, if Participant B makes a statement to the effect that "I think differently about what A said earlier," in the second LLM session, the AI ​​detects the utterance mentioned by Participant B among Participant A's previous utterances in context. The detected utterance by Participant A and the current utterance by Participant B can be linked, and the relationship between the two utterances can be defined and stored as a "rebuttal" type.

[0059] Next, the third LLM session can control the structuring and representation of the context between utterances saved during collaborative activities. Accordingly, it becomes possible to analyze the mood or overall intent of the conversation, going beyond simply converting each utterance into text.

[0060] According to one embodiment, in the third LLM session, the correlation of utterances in the current collaborative activity can be analyzed by reflecting each participant's previous activity history information regarding previous collaborative activities related to the current collaborative activity.

[0061] In this way, the third LLM session can separate utterances that are contextually connected within the entire conversation, even when multiple participants speak incoherently to one another during a collaborative activity, and can structure and represent the context of the utterances into a data structure that indicates the direction of pointers, the relationship between utterances, and the flow of the conversation.

[0062] For example, when Participant C presented Topic #1, it becomes possible to structure and express the flow of the conversation, such as what answer Participant E gave and what additional explanation Participant G provided.

[0063] The third LLM session provides structured data necessary for the fourth LLM session, described below, to analyze the conversation context for participant evaluation.

[0064] Finally, the 4th LLM session can control the generation of evaluation information regarding each participant's collaborative activities by comprehensively using the data processed by the previous sessions.

[0065] According to one embodiment, in the fourth LLM session, at least one of each participant's achievement, understanding, level of participation, and contribution can be evaluated using the context analyzed in the previous session, and personalized customized content or services can be provided based on this. At this time, control can be exercised to qualitatively evaluate how actively a participant participated in the collaborative activity, what role a participant played in the collaborative activity, etc., and express this in a sentence.

[0066] According to one embodiment, in the fourth LLM session, by analyzing the frequency of each participant's remarks, the importance of the content, etc., and evaluating their contribution, the fairness and transparency of the meeting can be enhanced and active participation of the participants can be encouraged.

[0067] As an example of designing a data structure to store the content of utterances, each utterance can be defined as a node, and a node may include speaker identification information, utterance content, utterance time, a pointer to the next node, a pointer to the related node, and a relationship type field. Relationships can be represented by connecting the nodes storing all utterance content into a single linked list, managing linked lists separately for each speaker, and using a separate list to define the relationships between specific utterances.

[0069] The following is an example defining the above node structure.

[0070] plaintext

[0071] struct Node {

[0072] int speaker_id; / / Speaker ID

[0073] string content; / / utterance content

[0074] timestamp speak_time; / / Speech time

[0075] Node* next; / / Pointer to the next utterance node

[0076] Node* related_node; / / Pointer to related utterance node

[0077] string relation_type; / / Relationship type (e.g., counter-argument, elaboration, etc.)

[0078] };

[0079] The following is an example of a linked structure connecting the above nodes.

[0080] · (1) Full list of utterances:

[0081] Head -> Node1 -> Node2 -> Node3 -> ... -> NodeN

[0082] · (2) List by speaker (e.g., Speakers A, B):

[0083] Speaker A: HeadA -> NodeA1 -> NodeA2 -> ...

[0084] Speaker B: HeadB -> NodeB1 -> NodeB2 -> ...

[0085] (3) Relationships between utterances:

[0086] NodeA1 (Speaker A) -> NodeB2 (Speaker B) with relation_type "Counterargument"

[0088] The above-mentioned 1st to 4th LLM sessions operate independently but can share data and processing results with one another. By having the artificial intelligence in each session operate independently while organically exchanging data to perform tasks, a series of processes—from text conversion of utterances and analysis of relationships between utterances to structuring conversational context and participant evaluation—can be automated, thereby improving the quality of the meeting.

[0089] Furthermore, during offline collaborative activities, the conversation content of each participant can be analyzed in real time, and the relationship between the speaker and the utterance can be defined and structured. Previously, recording or analyzing conversation content required significant human resources, but according to the present invention, accuracy and efficiency can be increased through an automated method.

[0090] Meanwhile, although the above disclosed an embodiment in which participants in a collaborative activity are tracked using an on-device artificial intelligence module equipped on a user's smart device and the collaborative activity is analyzed based thereon, in other embodiments, the collaborative activity participants can be tracked and the collaborative activity is analyzed based thereon by calling an API (Application Programming Interface) of an artificial intelligence module equipped on a separate server device.

[0091] A method according to one embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the present invention, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0092] Although embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention as defined in the following claims also fall within the scope of the present invention. Explanation of the symbols

[0093] 100: Information provision server 110: Participant terminal 200: Host terminal 101, 102: Other participant terminals

Claims

Claim 1 A method for tracking participants in a cooperative activity using on-device artificial intelligence, performed by at least one processor, comprising: (a) identifying each participant by distinguishing at least one of the timbre and location of each participant by referring to at least one of the input voice and video when an offline cooperative activity begins; (b) obtaining a unique identifier from each participant's terminal using an Ultra-wideband (UWB) module and receiving user detail information corresponding to each participant's unique identifier from an information providing server; (c) identifying each participant by generating matching information by matching each participant's user detail information with each participant identified above. and (d) a step of controlling an LLM-based on-device artificial intelligence to distinguish actions and utterances recognized during the progress of the cooperative activity by participant using the matching information and to generate participant-specific tracking information reflecting the context; wherein in step (d), a first LLM session to a fourth LLM session operating independently is created to efficiently use the LLM-based on-device artificial intelligence, and in each session, the LLM-based on-device artificial intelligence organically shares data with one another; and in order to generate participant-specific tracking information by organically sharing data, the utterance content by speaker in the first LLM session is converted into text; in the second LLM session, relationship information is generated by analyzing the relationship type between utterances regarding the current utterance and previous utterance converted into text; in the third LLM session, the context of the utterance is structured based on the relationship information and the user detail information (wherein the user detail information includes each participant's activity history information regarding previous cooperative activities related to the current cooperative activity); and in the fourth LLM session, the structured A method characterized by generating evaluation information by evaluating at least one of the achievement, understanding, degree of participation, and contribution of a participant's collaborative activity based on context. Claim 2 A method according to claim 1, wherein in step (c), the LLM-based on-device artificial intelligence generates user attribute information by analyzing an attribute including at least one of an attitude, understanding, contribution, and opinion regarding each participant's past cooperative activities using the activity history information. Claim 3 A method according to paragraph 2, wherein in step (d) above, the context of the utterance in the cooperative activity is analyzed by referring to the user attribute information. Claim 4 In an electronic device, an input module for receiving commands or data to be used in the electronic device from outside the electronic device; a display module for providing information to outside the electronic device; an Ultra-Wideband (UWB) module for communicating with another electronic device in an ultra-wideband frequency band; and a processor electrically connected to the input module, the display module, and the UWB module.The system includes a memory electrically connected to the processor and comprising an LLM-based on-device artificial intelligence module, wherein the processor: (a) identifies each participant by distinguishing at least one of the timbre and location of each participant by referring to at least one of the input voice and video when an offline cooperative activity begins; (b) obtains a unique identifier from each participant's terminal using the UWB module and retrieves user detail information corresponding to each participant's unique identifier from an information providing server; (c) identifies each participant by generating matching information by matching each participant's user detail information with the identified participant; and (d) controls the LLM-based on-device artificial intelligence to distinguish actions and utterances recognized during the cooperative activity by participant using the matching information and to generate participant-specific tracking information reflecting the context; wherein in (d), a first to fourth LLM session is created to enable efficient use of the LLM-based on-device artificial intelligence, and in each session, the LLM-based on-device artificial intelligence organically shares data with each other. An electronic device characterized by generating tracking information for each participant by organically sharing data, wherein in the first LLM session, the utterance content for each speaker is converted into text; in the second LLM session, relationship information is generated by analyzing the relationship type between utterances regarding the current utterance and previous utterance converted into text; in the third LLM session, the context of the utterance is structured based on the relationship information and the user detail information (wherein the user detail information includes each participant's activity history information regarding previous cooperative activities related to the current cooperative activity); and in the fourth LLM session, at least one of the achievement, understanding, degree of participation, and contribution to the participant's cooperative activity is evaluated based on the structured context to generate evaluation information.

Images