Ai-based bidirectional intercom system and control method thereof

Abstract

An AI-based bidirectional intercom system according to the present invention may include: a user terminal that collects voice input from a user, performs voice-to-text conversion and natural language understanding to generate intercom control commands; smart glasses that display communication status or control results within the user's field of vision through a transparent display and perform voice input and output; an intercom server that establishes voice communication sessions between multiple users and relays voice data; a management server that manages user authorization information and channel policy information, and controls the operation of the intercom server according to user authentication results or security events; and a user-customized AI learning device that collects user voice data, learns and generates a user-customized AI model, and provides the generated authentication model to the management server or the user terminal.

Description

Technology Field

[0001] The present invention relates to an AI-based bidirectional intercom system and a method for controlling the same. More specifically, it relates to an AI-based bidirectional intercom system and a method for controlling the same that apply a user-customized AI authentication model in a voice-based bidirectional intercom environment using smart glasses, thereby enabling only legitimate users to create and maintain communication sessions and ensuring communication security even in situations of malicious equipment theft or disguised use. Background Technology

[0003] In industrial sites, broadcast production facilities, construction sites, and security and safety management environments, it is necessary for multiple workers to collaborate simultaneously and perform real-time voice communication. To this end, voice communication systems utilizing walkie-talkies, intercom devices, or mobile terminals are widely used. Recently, with the advancement of network-based voice communication technology, intercom systems that relay voice communication between multiple users via a server are being provided in various forms.

[0004] However, most conventional intercom systems are designed based on physical button operation or simple touch input. For example, users must directly press buttons on the terminal or touch the screen to perform actions such as channel switching, activating speaking, or switching to receive-only mode. This method is inconvenient to use in environments where both hands cannot be free during work, and it can reduce safety and work efficiency, particularly in situations requiring communication while working at heights, moving, or operating equipment.

[0005] Furthermore, while some systems incorporate speech recognition capabilities, most remain at the level of simple keyword recognition. In other words, operating based solely on whether a specific word has been uttered limits their ability to accurately understand the meaning embedded in the user's natural language speech and extend it into various control commands. Consequently, even slight variations in speech expression can lead to misrecognition or the execution of unwanted control actions.

[0006] Meanwhile, existing intercom systems often process voice communication data and control signals using the same communication path or protocol. In such structures, problems may arise where the transmission of control signals is delayed due to increased voice data transmission volume, or conversely, voice communication quality is degraded due to control signal processing. In particular, for devices with limited battery capacity, such as wearable terminals, this integrated processing structure becomes a factor that increases power consumption.

[0007] With the recent proliferation of wearable display devices such as smart glasses, user interfaces that display information within the user's field of vision are gaining attention. However, conventional technology often utilizes smart glasses merely as simple display devices or limits them to mirroring only a portion of information from a terminal screen. Consequently, there are limitations in intuitively providing intercom-related information, such as communication status, channel selection status, or voice reception status.

[0008] Furthermore, in intercom environments where multiple users participate simultaneously, it is necessary to dynamically manage channel-specific communication policies, user-specific speaking rights, and communication priorities. However, in conventional technologies, these policies are often applied through fixed rules or manual administrator settings, making it difficult to respond flexibly to real-time situational changes. In particular, there is a problem of increased user fatigue because notifications or displays are provided in the same manner even when the same event occurs repeatedly.

[0009] Therefore, there is a need for a new type of intercom system that can interpret a user's natural language utterance based on meaning, convert it into accurate communication control commands, efficiently process control signals and voice communication through separate paths, and provide the results in an intuitive form on a wearable display. In addition, there is an increasing need for intercom technology that can dynamically reflect channel and user policies and adaptively control the output method according to the importance and frequency of events. The problem to be solved

[0011] The present invention aims to provide an AI-based two-way intercom system and a control method thereof, which apply a user-customized AI authentication model in a voice-based two-way intercom environment using smart glasses to ensure that only legitimate users can create and maintain communication sessions, and to secure communication security even in situations of malicious equipment theft or disguised use.

[0012] In addition, the present invention aims to provide an AI-based bidirectional intercom system and a control method thereof, which can improve the accuracy of voice command recognition and ease of use compared to conventional technology that relied on fixed keyword input by interpreting a user's natural language utterance based on speech recognition and natural language understanding and applying it to intercom control.

[0013] In addition, the present invention aims to provide an AI-based bidirectional intercom system and a control method thereof that can maintain stable voice communication quality even in a wearable terminal environment by processing control signals and voice communication data through separate communication paths, and can reduce battery consumption to enable long-term use.

[0014] In addition, the present invention aims to provide an AI-based bidirectional intercom system and a method for controlling the same, which enables safe and intuitive intercom control even in work environments where both hands are not free, by providing a user interface in the form of a virtual layer on a transparent display of smart glasses, thereby allowing the user to immediately perceive the communication status and control results within their field of vision.

[0015] In addition, the present invention aims to provide an AI-based bidirectional intercom system and a control method thereof, which enables orderly voice communication and reduces communication interference even in environments where multiple users participate simultaneously, by dynamically adjusting the transmission and reception status according to user authority, channel priority, and communication conditions through channel-level policy-based control.

[0016] In addition, the present invention aims to provide an AI-based bidirectional intercom system and a control method thereof that can improve the reliability of the intercom system and user satisfaction by reducing user fatigue caused by repetitive notifications through effect control that considers the importance and frequency of events, and by more clearly conveying urgent or important events. means of solving the problem

[0018] An AI-based bidirectional intercom system for achieving these purposes may include: a user terminal that collects user voice input, performs voice-to-text conversion and natural language understanding to generate intercom control commands; smart glasses that display communication status or control results within the user's field of vision via a transparent display and perform voice input and output; an intercom server that establishes voice communication sessions between multiple users and relays voice data; a management server that manages user authorization information and channel policy information, and controls the operation of the intercom server according to user authentication results or security events; and a user-customized AI learning device that collects user voice data, learns and generates a user-customized AI model, and provides the generated authentication model to the management server or the user terminal.

[0019] In one embodiment, the user-customized AI learning device can extract a feature vector from user voice data and generate an authentication model that distinguishes between registered users and non-registered users based on the feature vector.

[0020] In one embodiment, the management server can store the authentication model received from the user-customized AI learning device and distribute it to the user terminal.

[0021] In one embodiment, if the re-authentication result is determined to be less than a threshold value during a communication session, the management server may transmit a session termination or transmission blocking command to the intercom server.

[0022] In addition, an AI-based bidirectional intercom method for achieving this purpose may include the steps of: a user terminal collecting a user's voice input; the user terminal performing speech-to-text conversion and natural language understanding on the voice input to generate an intercom control command; the user terminal transmitting a communication request to an intercom server based on the intercom control command; the intercom server establishing a voice communication session between multiple users and relaying voice data; a management server managing user authorization information and channel policy information, and controlling the operation of the intercom server according to user authentication results or security events; a user-customized AI learning device collecting user voice data and learning and generating a user-customized authentication model based on the user voice data; and providing the generated authentication model to the management server or the user terminal. Effects of the invention

[0024] According to the present invention, by interpreting a user's natural language utterance based on speech recognition and natural language understanding and applying it to intercom control, there is an advantage in that the accuracy of voice command recognition and ease of use can be improved compared to conventional technology that relied on fixed keyword input.

[0025] In addition, according to the present invention, by processing control signals and voice communication data through separate communication paths, stable voice communication quality can be maintained even in a wearable terminal environment, and there is an advantage of being able to use it for a long time by reducing battery consumption.

[0026] Furthermore, according to the present invention, by providing a user interface in the form of a virtual layer on a transparent display of smart glasses, the user can immediately perceive the communication status and control results within their field of vision, and the invention aims to provide an AI-based bidirectional intercom system and a method for controlling the same, which enables safe and intuitive intercom control even in a work environment where both hands are not free.

[0027] In addition, according to the present invention, by dynamically adjusting the transmission and reception status according to user authority, channel priority, and communication conditions through channel-level policy-based control, there is an advantage that communication interference is reduced and orderly voice communication can be achieved even in an environment where multiple users participate simultaneously.

[0028] In addition, according to the present invention, user fatigue caused by repetitive notifications is reduced through effect control that considers the importance and frequency of events, and urgent or important events are conveyed more clearly, which has the advantage of improving the reliability and user satisfaction of the intercom system. Brief explanation of the drawing

[0030] FIG. 1 is a network configuration diagram for explaining an AI-based bidirectional intercom system according to one embodiment of the present invention. FIG. 2 is a network configuration diagram for explaining an AI-based bidirectional intercom system according to another embodiment of the present invention. FIG. 3 is a block diagram illustrating the internal structure of an intercom server according to an embodiment of the present invention. FIG. 4 is a block diagram illustrating the internal structure of smart glasses according to one embodiment of the present invention. FIG. 5 is a diagram illustrating voice-based intercom control and visual information display status using smart glasses according to an embodiment of the present invention. FIG. 6 is a block diagram illustrating an embodiment in which a management server and an AI learning device according to one embodiment of the present invention are configured as a single integrated server. Specific details for implementing the invention

[0031] The aforementioned objectives, signatures, and advantages are described in detail below with reference to the attached drawings, thereby enabling those skilled in the art to easily implement the technical concept of the present invention. In describing the present invention, detailed descriptions of known technologies related to the present invention are omitted if it is determined that such descriptions would unnecessarily obscure the essence of the invention. Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings. In the drawings, the same reference numerals are used to indicate the same or similar components.

[0033] FIG. 1 is a network configuration diagram for explaining an AI-based bidirectional intercom system according to one embodiment of the present invention.

[0034] Referring to FIG. 1, the AI-based two-way intercom system includes a user terminal (100), smart glasses (200), an intercom server (300), a management server (400), and a user-customized AI learning device (500).

[0035] The user terminal (100) functions as a core control terminal that performs voice-based intercom control, user authentication, communication session control, and interaction with smart glasses (200) in an AI-based bidirectional intercom system according to the present invention. The user terminal (100) can be implemented as a smartphone, tablet, or portable communication terminal, is connected to smart glasses (200) via short-range wireless communication, and can be connected to an intercom server (300) and a management server (400) via a wide-area communication network.

[0036] The user terminal (100) first collects the user's speech through smart glasses (200) or its own microphone. When the user speaks a voice command while wearing the smart glasses (200), the corresponding voice data is transmitted to the user terminal (100), and the user terminal (100) receives it and performs voice-to-text conversion. The result of the voice-to-text conversion is interpreted as an intercom control command through a natural language understanding process. At this time, the user terminal (100) does not perform simple keyword matching, but rather extracts the intent and target information of the speech and determines it as at least one of a predefined set of commands such as “display,” “select,” “close,” “channel switching,” “speaking activation,” and “listening-only switching.”

[0037] The user terminal (100) can convert voice text into an embedding vector and calculate the similarity with the embedding vector corresponding to the command set to determine the command with the highest similarity as the command to be executed. If the similarity value is below a set threshold, the execution of the command is suspended or re-uttering is induced to prevent malfunction. In addition, even for the same utterance, the interpretation result can be applied differently depending on the current system state, such as the active channel state or node information displayed on the HUD.

[0038] The user terminal (100) transmits a communication request to the intercom server (300) after confirming the command. The communication request may include user identification information, channel identification information, request type, and authentication status information. The intercom server (300) establishes or changes a voice communication session according to the request. The communication protocol is not limited to a specific method and, in one embodiment, may be implemented based on WebRTC.

[0039] As an important feature of the present invention, the user terminal (100) performs real-time user authentication using a user-customized authentication model provided by an AI learning device (500) to prevent unauthorized use of smart glasses (200). The user terminal (100) generates a user feature vector from input voice data and calculates a user suitability score by comparing it with the authentication model. Controls the transmission of a communication request to the intercom server (300) or the maintenance of an existing session only when the suitability score is greater than or equal to a threshold value.

[0040] Additionally, the user terminal (100) can periodically reanalyze voice characteristics not only at the time of login but also during the communication session to determine whether there has been an interception or impersonation. For example, if the voice characteristics of the speaker differ significantly from those of the registered user during communication, the user terminal (100) can temporarily suspend the session or send a warning signal to the intercom server (300) to restrict communication. This prevents information leakage resulting from equipment theft in special operations environments, secure communication environments, or security communication environments.

[0041] The user terminal (100) transmits the communication status or control result received from the intercom server (300) to the smart glasses (200). The smart glasses (200) display the information within the user's field of vision so that the current channel status, whether speaking is active, or warning information can be intuitively recognized. The user terminal (100) may also generate voice guidance to provide auditory feedback, such as "Channel switched" or "User authentication failed".

[0042] In another embodiment, the user terminal (100) can dynamically adjust the mapping relationship of the command set to improve command recognition accuracy by analyzing the user's repeated usage pattern. For example, if a specific user repeatedly uses a specific expression, the weight of the command corresponding to that expression can be increased to provide a personalized intercom control environment.

[0043] In this way, the user terminal (100) integrates voice collection, natural language-based command interpretation, embedding-based command confirmation, user authentication, communication session control, and display linkage with smart glasses (200), thereby enabling safe and reliable two-way intercom communication without button operation.

[0044] The smart glasses (200) operate as a wearable terminal that displays the communication status within the user's field of vision and performs voice input and output in the AI-based two-way intercom system according to the present invention, and supports intercom communication in conjunction with the user terminal (100). The smart glasses (200) go beyond being a simple display device and are controlled in conjunction with an operation flow that distinguishes between legitimate users and malicious users in an environment where security is required.

[0045] When the smart glasses (200) are worn by a user, they enter an initial activation state. After activation, the smart glasses (200) check the connection status with the user terminal (100), and when the connection is complete, they switch to standby mode. In this state, when the user starts speaking, the smart glasses (200) collect voice signals and transmit them to the user terminal (100). The user terminal (100) uses the voice data to perform user authentication and command interpretation.

[0046] For example, when an actual registered user wears smart glasses (200), if the user says “switch to channel 3,” the smart glasses (200) transmit the corresponding voice data to the user terminal (100).

[0047] The user terminal (100) analyzes the voice characteristics of the speaker using a learned authentication model and determines that the suitability with the registered user is greater than or equal to a threshold value. If authentication is successful, the user terminal (100) transmits a channel switching request to the intercom server (300) and transmits the result to the smart glasses (200).

[0048] The smart glasses (200) display “Channel 3” information and the current communication status within the user’s field of vision. Then, when the user says “speak,” the transmission mode is activated, a visual highlight appears, and the received voice of another user is output through the earphones. During this process, the smart glasses (200) maintain a normal communication state according to the authentication result of the user terminal (100).

[0049] On the other hand, assuming a case where a malicious user steals and wears the smart glasses (200), it operates as follows. Even if the malicious user wears the smart glasses (200) and utters commands such as “disable all listening” or “enable speaking,” the smart glasses (200) transmit the corresponding voice data to the user terminal (100).

[0050] The user terminal (100) analyzes the voice characteristics of the speaker and calculates the similarity with the registered user. At this time, if the suitability score is below a threshold value, it is determined that authentication has failed. When authentication failure is notified, the smart glasses (200) do not activate the speaking function and output a warning display within the user's field of vision or display a function restriction state.

[0051] Additionally, if authentication failures occur more than a certain number of times, the smart glasses (200) may disable the entire intercom function or block the session connection under the control of the user terminal (100).

[0052] As another example, a situation can be assumed where a registered user is communicating normally after logging in, but the equipment is stolen during communication. While communication is proceeding with the registered user wearing the smart glasses (200), if the smart glasses (200) are passed to another person and that person attempts to speak, the smart glasses (200) transmit the speech to the user terminal (100).

[0053] The user terminal (100) may be configured to periodically re-verify voice characteristics even during an existing communication session, and determines that the session is abnormal if the newly entered voice characteristics are significantly different from those of the registered user. In this case, the smart glasses (200) immediately block the transmission function under the control of the user terminal (100), display a warning message within the field of view, or indicate that the communication session has ended. This prevents information leakage caused by the theft of equipment or disguised use during operations.

[0054] In another embodiment, the smart glasses (200) may change the display mode according to the authentication status received from the user terminal (100). In the authentication success state, a general communication UI is displayed, and in the authentication failure or suspicion state, a restricted UI is displayed so that the transmit button or speech indicator is not activated. Even if a forced blocking command is issued with administrator privileges in an emergency situation, the smart glasses (200) may stop the communication function by reflecting the command.

[0055] In this way, the smart glasses (200) are not merely devices that collect voice data and perform display, but also distinguish between normal users and malicious users in conjunction with the AI-based authentication results of the user terminal (100), and dynamically control communication functions according to the authentication status, thereby providing safe and reliable communication in an intercom environment where security is required.

[0056] The intercom server (300) operates as a relay server that establishes voice communication sessions between multiple users in an AI-based bidirectional intercom system according to the present invention, relays voice data on a channel basis, and performs communication control according to user authority and authentication status. The intercom server (300) can be operated in a cloud environment, a dedicated server based on a closed network, or a security network dedicated to military or security, and can be designed to operate stably even in a mission-critical environment.

[0057] The intercom server (300) receives a communication request from a user terminal (100). The communication request may include user identification information, channel identification information, request type, and authentication status information. The intercom server (300) determines the legitimacy of the request based on user authorization information and channel policy information provided by the management server (400).

[0058] For example, in the case where an actual registered user wears smart glasses (200) and speaks normally, the user terminal (100) confirms through an AI-based authentication model that the user suitability score is greater than or equal to a threshold value and then sends a request to the intercom server (300) to participate in the channel or activate speaking.

[0059] The intercom server (300) receives the request, connects the user to a designated channel, and establishes a voice communication session. Subsequently, when the user speaks, the intercom server (300) relays the received voice stream in real time to other user terminals belonging to the same channel. Even when multiple users participate in a multi-party communication environment, the intercom server (300) manages the stream on a channel basis to deliver appropriate voice data to each user.

[0060] Additionally, the intercom server (300) can control the communication mode according to a channel-specific policy. For example, a specific channel may be configured to allow only receiving-only users, or to grant speaking rights only to specific users. When the actual owner attempts to speak, if the user is granted transmission rights on that channel, the intercom server (300) normally allows transmission and relays the voice.

[0061] On the other hand, if a malicious user attempts to steal and use the smart glasses (200), the following actions are performed. When the malicious user attempts to speak, the corresponding voice data is transmitted to the user terminal (100), and the user terminal (100) calculates a suitability score through an authentication model.

[0062] If the suitability score is below the threshold value, the user terminal (100) may transmit authentication failure status information to the intercom server (300) or block the communication request itself. When the intercom server (300) receives a request containing an authentication failure status, it refuses the terminal's participation in the channel or does not grant transmission rights. As a result, even if a malicious user connects to the channel, voice transmission is blocked.

[0063] As another example, a situation can be assumed where an actual registered user is logged in and communicating, but the equipment is stolen during the communication. If a malicious user attempts to speak after the theft, the user terminal (100) periodically re-verifies voice characteristics even during the communication session to determine an abnormal state.

[0064] When an authentication failure is detected, the user terminal (100) sends a request to the intercom server (300) to terminate the session or block transmission. The intercom server (300) receives the request and immediately blocks the transmission stream or terminates the session. At the same time, the intercom server (300) may report an abnormal event to the management server (400) to generate a security alert.

[0065] The intercom server (300) also performs a communication quality maintenance function. It adjusts the quality of the voice stream according to the network status and performs adaptive control when the latency or packet loss rate exceeds a standard. Even when multiple channels are operated simultaneously, resources can be prioritized for the emergency channel according to a priority policy.

[0066] Additionally, the intercom server (300) can record communication logs and event logs on a channel basis. For example, if repeated authentication failures occur at a specific terminal, the logs can be transmitted to the management server (400) to enable additional security measures. This allows tracking not only simple theft attempts but also repeated impersonation attempts.

[0067] In this way, the intercom server (300) performs stable two-way voice relay for communication requests from normal users, while blocking channel access or transmission rights for malicious users in conjunction with the authentication result of the user terminal (100), thereby providing reliable intercom communication in an environment where security is required.

[0068] The management server (400) operates as a central management server that comprehensively manages user authority, channel configuration, terminal status, and security policy in an AI-based bidirectional intercom system according to the present invention. The management server (400) provides channel-unit policies in conjunction with the intercom server (300), monitors user authentication status and abnormal events, and performs communication control authority in environments where security is required.

[0069] The management server (400) first manages user registration information. Each user is registered with unique identification information, along with authority information, affiliated group, list of channels available for participation, ability to speak, and emergency authority. Additionally, device identification information for smart glasses (200) and user terminals (100) can also be managed. This allows for restricting communication so that a specific user can only communicate on a specific device.

[0070] When an actual registered user wears smart glasses (200) and logs in through a user terminal (100), the user terminal (100) transmits user identification information along with the authentication result to an intercom server (300), and the intercom server (300) can query the management server (400) for this.

[0071] The management server (400) checks the user's authority and whether they can access the channel, and then provides the allowed channel information and policy information to the intercom server (300). For example, if a specific user is configured to access only channel 1 and channel 3, the management server (400) reflects this policy and causes the intercom server (300) to reject requests for access to other channels. In this case, the actual owner can communicate normally within the allowed range.

[0072] On the other hand, if a malicious user attempts to steal and use the smart glasses (200), the following actions are performed. When the malicious user attempts to log in or join a channel, the authentication result of the user terminal (100) can be transmitted to the management server (400) through the intercom server (300).

[0073] When the management server (400) receives authentication failure information or abnormal sign information, it may apply a restriction policy to the terminal or user account. For example, if authentication failures occur repeatedly, it may send a command to the intercom server (300) to temporarily lock the account or block channel access for the device.

[0074] As another example, a situation can be assumed where an actual registered user is logged in and communicating, but the equipment is stolen during the communication. If the user terminal (100) detects an abnormal state during the re-authentication process while communicating, the information is transmitted to the management server (400) via the intercom server (300).

[0075] The management server (400) may record this as a security event and instruct the intercom server (300) to immediately terminate the session or send a warning notification to the administrator terminal. At the same time, the policy may be changed to require an additional authentication procedure for the user account.

[0076] The management server (400) also dynamically manages the channel configuration. For example, in a specific operation mode or a situation where the security level is raised, it may set some channels to switch to private mode or grant transmission rights only to specific users. In emergency situations, it may grant mandatory control rights over all channels to specific users or administrators. These policies are transmitted to the intercom server (300) and reflected immediately.

[0077] Additionally, the management server (400) collects and analyzes communication logs and authentication logs. If repeated authentication failures occur or abnormal channel access attempts are detected by a specific user or device, the management server (400) determines this as abnormal behavior and generates a security alert. If necessary, access rights to the user or device can be permanently restricted or configured to be reactivated after administrator approval.

[0078] In this way, the management server (400) maintains a stable communication environment based on the authority of a normal user, while simultaneously performing authority restriction and session blocking quickly in the event of an access attempt by a malicious user or equipment theft, thereby strengthening the security of the entire intercom system.

[0079] The AI ​​learning device (500) performs the role of learning, updating, and distributing a user-customized authentication model to prevent unauthorized or malicious use of smart glasses (200) in an AI-based two-way intercom system according to the present invention.

[0080] The AI ​​learning device (500) can be implemented as a single server, a distributed server, a cloud computing environment, or a dedicated security server, and can be logically combined with a management server (400) or operated independently as needed.

[0081] The AI ​​learning device (500) first receives user identification information and user registration procedure information for the user to be learned. For example, when a new user registration request is received from the management server (400), the AI ​​learning device (500) may create a learning session for the user and provide a guidance signal or collection policy so that user learning information can be collected from the user terminal (100) or smart glasses (200).

[0082] Here, user learning information may include at least one of the user's unique vocal characteristics, intonation patterns, speech rate, speech rhythm, changes in vocal intensity, frequency distribution characteristics, and repetitive speech structures. Additionally, behavioral characteristic information such as the user's wearing pattern, head movement pattern, or usage habit may be additionally included.

[0083] The AI ​​learning device (500) receives voice data collected from a user terminal (100) or smart glasses (200) and performs preprocessing. In the preprocessing process, at least one of noise removal, volume normalization, silent interval removal, speech interval segmentation, and removal of substandard quality data may be performed.

[0084] For example, in a special operations environment or a noisy field environment, voice data mixed with background noise may be collected, so the AI ​​learning device (500) can maintain the learning quality by excluding voice samples below a reliability standard from learning or by lowering the weight according to the noise intensity.

[0085] The AI ​​learning device (500) extracts user-specific features from preprocessed voice data to generate a user feature vector. The feature vector may include frequency domain features, time domain features, and speech habit-based features, and may be configured to reflect, for example, the spectral characteristics of the voice signal, formant distribution, pitch change pattern, energy change curve, speech rate, pause interval pattern between words, or the repetition tendency of specific pronunciations. Additionally, when user speech text is obtained, the vocabulary selection tendency, sentence length distribution, repeated phrases, and sentence structure pattern of the speech text may be combined to enhance the user's unique speech characteristics.

[0086] The AI ​​learning device (500) learns a user-customized authentication model using the generated user feature vector. The authentication model may be implemented as at least one of a classification model for user identification, a metric learning model for calculating user similarity, or a representation learning model for forming a user-specific embedding space.

[0087] For example, a user-customized authentication model can be trained to output a score representing the similarity between an input voice feature vector and a registered user, and can be configured to determine a legitimate user if the similarity score is above a threshold.

[0088] In addition, considering cases where a person other than the registered user intentionally imitates the voice or repeats specific phrases to disguise it, the AI ​​learning device (500) may learn by including disguised speech patterns as negative samples, or reflect the temporal consistency and micro-vocalization characteristics of the speech as additional criteria for discrimination.

[0089] The AI ​​learning device (500) evaluates the performance of the learned authentication model. The performance evaluation may include the recognition rate for registered users, the misrecognition rate for other people, the detection rate for spoofed speech, and robustness in noisy environments. For example, if the misrecognition rate, which incorrectly identifies a person who is not a registered user as a registered user, exceeds a threshold value, the AI ​​learning device (500) may improve the reliability of the model by raising the quality standard of the training data or requesting the collection of additional samples.

[0090] The AI ​​learning device (500) distributes the finally confirmed authentication model to the user terminal (100). The user terminal (100) stores the distributed authentication model and performs authentication using real-time voice input whenever the smart glasses (200) are used. At this time, the AI ​​learning device (500) may provide model signature information or a verification hash value together to ensure the integrity of the model and prevent tampering, and the user terminal (100) can verify the safety of the model using the verification information when loading the model.

[0091] Additionally, the AI ​​learning device (500) can update the authentication model based on additional data collected during use. For example, if the user's voice condition changes over time or the wearing environment changes, the AI ​​learning device (500) can retrain the model by accumulating new samples that satisfy certain conditions and redistribute the updated model to the user terminal (100). At this time, if voice samples in the section where authentication failures are repeated are analyzed and determined to be an attempt at impersonation, the pattern is added as a negative sample and retrained, thereby improving the detection performance against malicious use.

[0092] In this way, the AI ​​learning device (500) learns a user-customized authentication model based on the user's unique voice and behavioral characteristic data, distributes the learned model to the user terminal (100), and strengthens security through continuous data accumulation and retraining, thereby providing the effect of preventing information leakage due to theft or malicious use of the smart glasses (200).

[0094] FIG. 2 is a network configuration diagram for explaining an AI-based bidirectional intercom system according to another embodiment of the present invention.

[0095] Referring to FIG. 2, the AI-based two-way intercom system includes a user terminal (100), smart glasses (200), an intercom server (300), and a management server (400).

[0096] The user terminal (100) is implemented as a smartphone and operates as a central control unit that performs voice-based control in the intercom system according to the present invention.

[0097] The user terminal (100) is linked with smart glasses (200), an intercom server (300), and a management server (400) to integrally perform voice command recognition, communication control judgment, and user feedback provision.

[0098] The user terminal (100) receives voice data extracted from the user's speech from the smart glasses (200). The received voice data is converted into text form through a voice-to-text conversion process executed within the user terminal (100).

[0099] At this time, the user terminal (100) can be configured to enable real-time voice command recognition without relying on an external server by analyzing voice text using a lightweight language model running inside the smartphone.

[0100] The voice-to-text conversion result is mapped to a predefined set of commands in the user terminal (100). The set of commands may include multiple commands for intercom control, such as “display,” “select,” “close,” “channel switching,” “speaking activation,” and “listening-only switching.”

[0101] The user terminal (100) converts the input voice text into an embedding vector form and calculates the similarity with the embedding vector corresponding to the command set, thereby determining the command with the highest similarity as the command to be executed.

[0102] At this time, the command determination process by the user terminal (100) is modeled by [Equation 1].

[0104] [Mathematical Formula 1]

[0105]

[0106] u: Voice data obtained from user's speech,

[0107] STT(): Speech-to-text conversion function,

[0108] NLU(): Natural Language Understanding-based Semantic Normalization Function,

[0109] e(): embedding vector transformation function,

[0110] v k : Embedding vectors corresponding to each instruction included in the instruction set,

[0111] τ: Similarity threshold for command execution,

[0113] Accordingly, the user terminal (100) determines the command to be executed by considering the semantic-based similarity of the user's natural language utterance rather than a simple keyword match, and converts the determined command into a corresponding control trigger and applies it to intercom control.

[0114] As described above, the confirmed command acts as a control trigger. For example, when a user makes a utterance designating a specific client node or channel, the user terminal (100) recognizes the utterance as one of the command sets and controls the connection status of a corresponding node among a plurality of client nodes displayed on the HUD of the smart glasses (200) to selectively activate or deactivate. Accordingly, the user can control the communication target and communication status using only voice without physical input operations.

[0115] When a command execution is determined, the user terminal (100) distinguishes the control signal transmission path according to the type of command. Lightweight control information, such as changes in the display state of the smart glasses (200), whether speaking is enabled, and selected client node information, is transmitted to the smart glasses (200) using a Bluetooth Low Energy communication method.

[0116] On the other hand, audio data used for real-time voice communication between users is transmitted and received with the intercom server (300) via a WebRTC-based audio channel. Due to this dual structure, stable voice communication is possible while minimizing battery consumption of the smart glasses (200).

[0117] Additionally, the user terminal (100) performs input / output functions to allow the user to intuitively recognize the result of command execution. Specifically, the user terminal (100) can provide auditory feedback using voice synthesis technology for the result of command execution or changes in communication status.

[0118] For example, when the channel switch is completed, a guidance voice such as “Channel switched” may be output. At the same time, changes in the channel selection status, speaking active status, or connection status are visually displayed on the HUD of the smart glasses (200).

[0119] Accordingly, by receiving auditory and visual feedback simultaneously, the user can intuitively recognize that a call session switch or a change in communication status has been successfully made. This configuration enables the user to clearly understand the communication status even in work environments where both hands must be used, thereby significantly improving the usability of a hands-free intercom environment.

[0120] In this way, the user terminal (100) interprets voice data extracted from the user's speech through a voice-to-text conversion and command mapping process, utilizes the interpretation result as a trigger for HUD control and intercom communication control of the smart glasses (200), and enables intuitive voice-based intercom control by simultaneously providing auditory and visual feedback.

[0121] Smart glasses (200) operate as a wearable output terminal that intuitively provides the user with the communication status and control results in an intercom system according to the present invention.

[0122] The smart glasses (200) include a transparent display for providing communication status and control results in real time while maintaining the user's field of vision, and are configured to display a user interface in the form of a virtual layer superimposed on the transparent display.

[0123] Here, the virtual layer is an overlay screen rendered so that graphic elements are superimposed on the user's field of vision regarding the real environment, intuitively providing the user with channel status, voice reception status, speaking activity status, and connection status of specific client nodes related to intercom communication.

[0124] The smart glasses (200) determine the type of UI element to be displayed, the display location, the display intensity, and the display duration based on a status value or control command received from a user terminal (100), and render it as a virtual layer on the display area of ​​a transparent display.

[0125] For example, when a user utters a “display” command and triggers a status check of a specific client node, the smart glasses (200) may superimpose an icon corresponding to the client node, an indication of the connection status (connected / waiting / disconnected), and a highlight border indicating the current selected focus on the lens.

[0126] Conversely, if the user issues the “Close” command, the virtual layer UI currently being displayed is removed or its transparency is increased to minimize visual obstruction while transitioning to a standby state.

[0127] The smart glasses (200) implement an interactive UI by not simply outputting UI elements in a fixed manner, but by changing them dynamically according to communication events.

[0128] For example, when a voice reception event occurs from an intercom server (300), the user terminal (100) transmits a status value corresponding to the event to the smart glasses (200), and the smart glasses (200) flash a specific area on the lens, highlight a channel icon, or output a background highlight so that the user immediately recognizes that voice reception has occurred.

[0129] At this time, when events occur simultaneously on multiple channels, the smart glasses (200) may prioritize displaying only specific channels according to priority rules or sequentially switch displays according to time order to reduce the cognitive burden caused by information overcrowding.

[0130] The smart glasses (200) are based on a structure that processes interactions related to selection and switching based on voice. That is, even if the smart glasses (200) do not receive direct touch input from the user, when the user terminal (100) interprets a voice command and transmits a “selection” or “switching” state value, the UI changes according to that state value, such as moving the focus on the HUD or confirming the selected item.

[0131] For example, when a user utters a “select” command, the smart glasses (200) fix the display of the currently highlighted client node or additionally render a selection confirmation display so that the user can immediately check the selection result.

[0132] Additionally, the smart glasses (200) can vary the display method to ensure visibility in the work environment. For example, in an environment where contrast is required due to high external illumination, the thickness of the outline of the UI element or the brightness level can be increased, and in a night or indoor environment, the display intensity can be lowered to reduce glare.

[0133] In another embodiment, when the user is set to be working at a height or moving, the smart glasses (200) may minimize text display and output only an icon-centered, simplified UI, or move the display area to the periphery of the field of view so as not to obstruct the working field of view.

[0134] The smart glasses (200) can immediately reflect the received state value in the rendering pipeline to minimize display delay, and when a display change occurs, UI transitions can be performed using fade-in / fade-out or gradual highlighting to prevent the screen from suddenly jumping. For example, when voice reception ends, the highlight display is controlled to be maintained for a certain period of time and then gradually disappear, so that the user naturally recognizes the end of the event.

[0135] In this way, the smart glasses (200) render an interactive UI superimposed as a virtual layer on a transparent display in real time, thereby enabling the user to immediately recognize the communication status and control results even in an environment where both hands are not free, and to enable hands-free intercom control combined with voice-based commands.

[0136] The smart glasses (200) are connected to the user terminal (100) via wireless communication and perform primary detection of voice input, visual output of status information, and transmission and reception of minimal control signals.

[0137] The smart glasses (200) collect ambient sound through a microphone placed in the frame or temple area, and activate a voice signal only when the user actually starts speaking using a voice activity detection function implemented inside.

[0138] Accordingly, the smart glasses (200) keep the microphone in a low-power standby state until the user's speech is detected, preventing unnecessary environmental noise from being continuously collected. Through this operation, the possibility of misrecognition is reduced and battery consumption is minimized even in noisy work environments.

[0139] When a user's speech is detected, the smart glasses (200) transmit voice data along with a detection signal to the user terminal (100). At this time, the smart glasses (200) does not perform complex analysis of the voice data itself, but instead determines only minimal state information, such as whether speech has started or ended, and transmits it to the user terminal (100), thereby minimizing the computational burden on the smart glasses (200). Accordingly, the interpretation and execution of voice commands are performed by the user terminal (100), and the smart glasses (200) focus on their role as an edge terminal.

[0140] The smart glasses (200) perform a visual output operation based on control signals and status information received from the user terminal (100). For example, when a specific channel is selected or a speaking state is activated at the user terminal (100), the smart glasses (200) overlay a graphic element representing the state in real time on the lens area. The graphic element displayed at this time may include at least one of an icon, a color change, a border highlight, text, or a brief symbol.

[0141] Additionally, the smart glasses (200) provide visual feedback in response to a voice reception event received from a user terminal (100). For example, when voice is received from another user, the smart glasses (200) output a visual effect in response to the event, such as flashing a specific area of ​​the lens or changing the background brightness, so that the user can immediately recognize the current communication status. At this time, the intensity or display pattern of the effect may vary depending on the intensity or duration of the voice reception.

[0142] The smart glasses (200) use a Bluetooth low-power communication method for communication with the user terminal (100). Accordingly, the smart glasses (200) do not directly process data with a large bandwidth, such as voice communication data, but only transmit and receive lightweight data such as status values, control commands, and display information. Due to this structure, the smart glasses (200) can operate stably even in a long-term wearing environment.

[0143] In one embodiment, the smart glasses (200) may selectively output display information according to the priority of status information received from the user terminal (100). For example, when voice is received simultaneously from multiple channels, the smart glasses (200) may display only the status of the most important channel according to a preset priority rule, or sequentially switch and display status information.

[0144] In another embodiment, the smart glasses (200) can change the display mode according to the work environment or user settings. For example, in situations such as working at height or driving, the size and contrast of the displayed information can be increased, and unnecessary text information can be omitted to minimize visual obstruction. Conversely, in a relatively stable environment, channel numbers, brief user identification information, or communication mode information can be displayed together.

[0145] Additionally, the smart glasses (200) perform a state return operation according to the control result transmitted from the user terminal (100). For example, when a communication session is terminated or a channel is released, the smart glasses (200) remove the graphic elements that were being displayed and switch to a default standby state screen. At this time, the switching process can be performed in a gradual fade-out manner so as not to burden the user's field of vision.

[0146] In this way, the smart glasses (200) provide a wearable intercom environment in which the user can immediately recognize the communication status without physical operation through primary detection of voice input, visual output of status information, and low-power interaction with the user terminal (100).

[0147] The intercom server (300) operates as a central server that relays real-time voice communication between a plurality of user terminals (100) in an intercom system according to the present invention and controls the creation, maintenance, and termination of communication sessions. The intercom server (300) manages the communication status in conjunction with the user terminals (100), smart glasses (200), and management server (400), and controls voice communication between users to be performed stably.

[0148] When the intercom server (300) receives a communication connection request from a user terminal (100), it initiates a communication session setup procedure based on user identification information and channel identification information included in the request.

[0149] At this time, the intercom server (300) determines whether to allow the requested communication session by referring to user authority information and channel access authority information provided by the management server (400). If the communication session is allowed, the intercom server (300) establishes a communication connection to enable real-time voice communication between user terminals (100).

[0150] When a communication session is established, the intercom server (300) relays bidirectional voice communication by transmitting voice data received from a user terminal (100) to another user terminal (100).

[0151] In this process, the intercom server (300) adjusts the transmission of voice data according to the network conditions to minimize the deterioration of call quality caused by increased latency or packet loss. Accordingly, the user terminal (100) can perform stable real-time voice communication even in a web-based environment.

[0152] The intercom server (300) manages group communication and individual communication separately. For example, in the case of group communication, voice data is transmitted simultaneously to multiple user terminals (100) belonging to the same channel, and in the case of individual communication, voice data is controlled to be transmitted only between specific user terminals (100). Through this, communication interference is prevented even in an environment where multiple channels are operated simultaneously.

[0153] Additionally, the intercom server (300) changes the call state according to a communication control request received from the user terminal (100). For example, if a specific user requests microphone deactivation, the intercom server (300) stops transmission so that the user's voice data is not transmitted to other users. Conversely, if a request to enable the microphone is received, the communication state is switched so that the user's voice data is transmitted again.

[0154] When a voice reception event occurs, the intercom server (300) transmits status information corresponding to the event to the user terminal (100). The user terminal (100) uses this status information to change the visual effect on the screen or the display state of the smart glasses (200), thereby enabling the user to intuitively recognize the current communication status. At this time, the intercom server (300) can select and transmit the event with the highest priority among multiple voice reception events occurring simultaneously.

[0155] Furthermore, the intercom server (300) sets up a new communication session while releasing or maintaining the existing communication session in response to a channel switching request received from the user terminal (100).

[0156] For example, if a user selects a different channel via a voice command, the intercom server (300) maintains communication continuity by clearing the communication status of the existing channel and establishing a communication connection for the selected channel.

[0157] Additionally, when a request to terminate a communication session or a network error occurs, the intercom server (300) terminates the session and transmits the relevant status information to the user terminal (100). Through this, the user terminal (100) can reflect the call termination or error status in the user interface. If necessary, the intercom server (300) can transmit the communication history or error information to the management server (400) to be used for operation management and quality analysis.

[0158] In this way, the intercom server (300) relays voice communication between user terminals (100) and controls the communication status in an integrated manner, thereby providing a stable and intuitive intercom service in various terminal environments.

[0159] The management server (400) operates as a server that manages operational information of users, channels, and organizational units in the intercom system according to the present invention, and supports the operation of the entire system by providing management information to the intercom server (300) and the user terminal (100). The management server (400) systematically configures the communication environment of organizational units and provides standard information to consistently maintain user access rights.

[0160] The management server (400) creates an organizational unit management structure based on configuration information input from an administrator terminal. For example, it creates group information representing a specific organization or work unit and registers user accounts and communication channel information belonging to the group. At this time, the management server (400) manages the active or inactive status of the group to collectively control whether users and channels belonging to a specific group can communicate.

[0161] When a user account creation request is received, the management server (400) registers account information including user identification information, user type, and affiliated group information. At this time, the management server (400) sets the communication channel range and communication authority accessible to the user account, and this information is subsequently used in the authentication and channel loading process of the user terminal (100). For example, it may be configured so that only users belonging to the same group can access a specific group channel.

[0162] Additionally, the management server (400) performs the function of creating and managing communication channels. The management server (400) creates a group communication channel or a fixed communication channel based on the administrator's input and defines the range of users who can belong to each channel. The generated channel information is provided to the user terminal (100) or the intercom server (300) and is used as reference information when setting up an actual communication session.

[0163] When the management server (400) receives an authentication request from the user terminal (100), it verifies the validity of the user account, group affiliation information, and channel access rights, and transmits the verification result to the user terminal (100). Based on this information, the user terminal (100) displays a list of accessible channels on the screen and restricts the communication range according to the information when the user requests communication.

[0164] Additionally, the management server (400) provides management information to the intercom server (300) to support proper communication control. For example, if a specific user belongs to a disabled group, the management server (400) can transmit the relevant information to the intercom server (300) to restrict the creation of communication sessions. Accordingly, the communication policy of the organization unit can be reflected in the communication operation in real time.

[0165] In one embodiment, the management server (400) can store the change history of users or channels and utilize it for operational management. For example, when management events such as the creation of user accounts, the addition of channels, or changes in permissions occur, the management server (400) can store the history and provide it to an administrator as needed. Through this, the change history of communication configurations can be systematically managed even in a large-scale organizational environment.

[0166] Additionally, the management server (400) can collect status information for the stable operation of the communication system. For example, communication session status or error occurrence information transmitted from the intercom server (300) may be stored, and this information may be used for service quality analysis or improvement of operation policies.

[0167] In this way, the management server (400) maintains management information of users, channels, and organizational units in an integrated manner and provides it to the entire intercom system, thereby enabling systematic and stable operation of the intercom service even in an environment where multiple users participate simultaneously.

[0169] FIG. 3 is a block diagram illustrating the internal structure of an intercom server according to an embodiment of the present invention.

[0170] Referring to FIG. 3, the intercom server (300) includes a communication unit (310), a channel management unit (320), an input / output control unit (330), an effect control unit (340), and a central control unit (350).

[0171] The communication unit (310) performs the function of setting, maintaining, and releasing a real-time audio stream between the intercom server (300) and the user terminal (100). When the communication unit (310) receives a request for communication connection or session setup from the user terminal (100), it initiates a procedure for setting up an audio communication session based on user identification information, channel identification information, and media setup information of the terminal included in the request.

[0172] At this time, the communication unit (310) negotiates an audio codec, sampling rate, number of channels, and encryption options through signaling with the user terminal (100), and forms a connection state capable of transmitting an audio stream by exchanging connection candidate information for network path setting.

[0173] The communication unit (310) receives an audio stream received from a user terminal (100) in real time after a session is formed, and transmits the stream to another user terminal (100) corresponding to the same channel or communication partner.

[0174] In the case of a group communication channel, the communication unit (310) simultaneously distributes and relays audio streams to multiple user terminals (100) belonging to the same channel, and in the case of private communication, controls so that the audio stream is transmitted only to a designated counterpart user terminal (100).

[0175] At this time, the communication unit (310) can perform transmission queue management, buffer control, and timestamp-based synchronization processing so that delays do not accumulate or voice quality deteriorates during the process of transmitting the sender's audio stream to multiple receivers.

[0176] The communication unit (310) supports switching between audio transmission and reception states. For example, when a request to enable speaking is received from a user terminal (100), the communication unit (310) activates the audio transmission path received from the user terminal (100), and when a request to switch to listening-only mode is received, the transmission path is restricted so that the audio stream transmitted from the user terminal (100) is not delivered to another user.

[0177] Accordingly, the communication unit (310) supports switching between a two-way audio communication state and a receive-only communication state, and can change only the transmission direction while maintaining the session so that the call session is not unnecessarily reset even if a communication state switch occurs.

[0178] In addition, the communication unit (310) can detect changes in bandwidth, latency, or packet loss rate to mitigate the degradation of communication quality due to changes in network status, and in response, perform quality correction processing such as transmission rate control, packet retransmission, or error concealment processing.

[0179] For example, if the network quality of a specific user terminal (100) deteriorates below a reference value, the communication unit (310) may switch the audio quality mode to a low-bandwidth mode or adjust buffer parameters so that voice interruption is minimized even if frame loss occurs.

[0180] When a request to terminate a session or a request to switch channels is received, the communication unit (310) stops transmitting audio streams with the corresponding user terminal (100) and releases session resources. At this time, the communication unit (310) performs resource release on a channel basis so as not to affect the communication of other user terminals (100) during the session termination or switching process, and saves status information regarding the terminated session as record data of the intercom server (300) or transmits it to the management server (400) so that it can be used for operation management or quality analysis.

[0181] In this way, the communication unit (310) establishes an audio communication session between user terminals (100), relays an audio stream according to the channel type, and performs transmission and reception state switching and network quality correction processing, thereby enabling stable real-time intercom audio communication in various terminal environments.

[0182] The channel management unit (320) performs the function of creating, maintaining, and managing a logical channel through which communication takes place between user terminals (100) in the intercom server (300). When the channel management unit (320) receives a request to create a channel from a user terminal (100) or a management server (400), it creates a new communication channel based on the channel identification information, channel type information, and user information for participation included in the request. At this time, the channel management unit (320) may set the channel as a group communication channel or a private communication channel.

[0183] When a channel management unit (320) receives a request to participate in a channel from a user terminal (100), it verifies whether the requested user account has the authority to participate in the channel. If participation is permitted as a result of the verification, the channel management unit (320) adds the user to the list of participating users of the channel and updates the current status information of the channel. Conversely, for users who do not have the authority to participate, the channel participation may be restricted. This verification process can be performed by referring to organizational unit information or channel access authority information provided by the management server (400).

[0184] The channel management unit (320) continuously manages changes in the status of user terminals (100) belonging to the channel. For example, when a specific user enters or leaves the channel, the channel management unit (320) detects the event and updates the list of participating users of the channel. In addition, the channel management unit (320) maintains channel status information, such as the number of users in the channel, the number of active users, or the number of users available to call, and can provide such information to other components of the intercom server (300).

[0185] When a channel management unit (320) receives a channel switching request from a user terminal (100), it releases the participation status on the existing channel and sets the participation status on the new channel. At this time, the channel management unit (320) interacts with the communication unit (310) to ensure that communication resources on the existing channel are properly released, and provides necessary channel information so that a communication session can be established on the new channel. Accordingly, the user can continuously switch channels even while on a call.

[0186] The channel management unit (320) can apply communication policies on a channel-by-channel basis. For example, it can be configured to allow only a receive-only mode for a specific channel or to grant speaking rights only to restricted users. These channel-specific policies are managed by the channel management unit (320), and when a communication request from a user terminal (100) is received, the policy is applied and transmitted to the communication unit (310).

[0187] At this time, the channel management unit (320) can determine the operation mode of the channel by comprehensively considering user identification information, channel identification information, user authority, channel priority and network status, and this determination process can be modeled by [Equation 2].

[0189] [Mathematical Formula 2]

[0190]

[0191] m*: Optimal channel operation mode determined by the channel management unit (320),

[0192] M: A set of channel operation modes including simultaneous transmit / receive mode, receive-only mode, transmit-only mode, and transmit / receive blocking mode,

[0193] P(m|u,c): Suitability of policy-allowed modes for user and channel combinations,

[0194] Q c (m): Score reflecting channel type and priority,

[0195] R u (m): Score reflecting user permissions,

[0196] C(m,n): Communication cost based on network status,

[0198] Accordingly, the channel management unit (320) can perform policy-based channel control that comprehensively considers user, channel, and network status, going beyond simple rule-based control.

[0199] Additionally, the channel management unit (320) manages the active or inactive state of the channel. For example, if a specific channel is inactive, the channel management unit (320) may restrict new participation or communication requests for that channel from being processed. Conversely, when the channel is reactivated, it switches the state so that communication requests for that channel are processed normally.

[0200] The channel management unit (320) can generate status information to transmit event information to the user terminal (100) when a channel-related event occurs. The user terminal (100) can use this status information to visually display changes in the channel status on the HUD of the smart glasses (200) or on the terminal screen. Through this, the user can intuitively recognize the current status of the channel and the participation status.

[0201] In this way, the channel management unit (320) integrates the creation of communication channels, participation management, switching processing, and application of channel policies, thereby enabling systematic and stable intercom channel operation even in an environment where multiple users participate simultaneously.

[0202] The input / output control unit (330) performs the function of controlling the input / output status related to voice transmission and reception between the intercom server (300) and the user terminal (100). The input / output control unit (330) controls the active or inactive status of voice input and voice output for each user based on a communication control request received from the user terminal (100) or channel policy information transmitted from the channel management unit (320).

[0203] When the input / output control unit (330) receives a communication control request from a user terminal (100), such as speaking activation, switching to listening-only mode, or speaking termination, it determines an input / output state corresponding to the request. For example, if a speaking activation request is received, the transmission state is activated so that voice data input from the user terminal (100) is transmitted to another user terminal through the communication unit (310). Conversely, if a request to switch to listening-only mode is received, voice transmission from the user terminal (100) is blocked, and the input / output state is switched so that only voice data transmitted from another user is received.

[0204] Additionally, the input / output control unit (330) can restrict the input / output state by reflecting the channel policy set by the channel management unit (320). For example, if a specific channel is set as a reception-only channel, the input / output control unit (330) controls the system so that voice transmission does not occur even if a request to enable speaking is received from the user terminal (100). In this way, the input / output control unit (330) determines the final voice input / output state by comprehensively considering the user request and the channel policy.

[0205] The input / output control unit (330) continuously monitors changes in state that occur during communication. For example, if the network connection status of the user terminal (100) becomes unstable or if the transmission of the voice stream is interrupted by the communication unit (310), the input / output control unit (330) may temporarily disable the input / output status of the user or switch it to a standby state. This prevents incomplete voice data from being transmitted to other users.

[0206] Additionally, the input / output control unit (330) can generate state information based on changes in the input / output state and transmit it to other components. For example, when a specific user's speaking state is activated or deactivated, the input / output control unit (330) transmits the corresponding state information to the effect control unit (340), and the effect control unit (340) can provide this to the user terminal (100) or smart glasses (200) so that visual or auditory feedback is output.

[0207] The input / output control unit (330) can adjust the input / output state to minimize voice conflicts between users even in an environment where multiple user terminals (100) participate in communication simultaneously. For example, when multiple users switch to a speaking state simultaneously within the same channel, the input / output control unit (330) can control only a specific user to maintain the speaking state according to priority information or channel policy, or temporarily restrict transmission by the remaining users.

[0208] In this way, the input / output control unit (330) dynamically controls the active state of voice transmission and reception by comprehensively considering user requests, channel policies, and communication status, thereby enabling stable and orderly voice communication even in an intercom environment where multiple users participate.

[0209] The effect control unit (340) performs the function of generating visual and auditory feedback information corresponding to a change in communication status, a user action event, or a channel status event in the intercom server (300).

[0210] The effect control unit (340) receives status information transmitted from the communication unit (310), the channel management unit (320), and the input / output control unit (330), generates effect output information corresponding to the status information, and provides it to the user terminal (100) or smart glasses (200).

[0211] The effect control unit (340) detects events occurring during voice communication between user terminals (100). For example, when a specific user switches to a speaking state, or when voice reception starts or ends, the effect control unit (340) generates a state value corresponding to the event. This state value may be information for distinguishing between a state indicating voice transmission, a state indicating voice reception, or a waiting state.

[0212] The effect control unit (340) also controls the effect output for channel-related events transmitted from the channel management unit (320).

[0213] For example, when a user enters or exits a channel, when a channel switch occurs, or when a specific channel is switched to an active or inactive state, the effect control unit (340) can generate visual emphasis information or guidance information corresponding to the event. Through this, the user can intuitively recognize the change in the current channel state.

[0214] Additionally, the effect control unit (340) determines the type of effect to be reflected in the user interface based on user-specific input / output status information transmitted from the input / output control unit (330).

[0215] For example, control can be exercised to highlight an icon representing a specific user when their speaking state is active, or to output a blinking indicator showing the reception status when voice is being received. Conversely, if transmission and reception are blocked, a visual indicator showing an inactive state can be displayed.

[0216] The effect control unit (340) can transmit the generated effect information to the user terminal (100) or to the smart glasses (200) via the user terminal (100).

[0217] Smart glasses (200) intuitively provide the user with a communication status or whether an event has occurred within their field of vision by displaying a virtual layer-shaped user interface superimposed on a transparent display based on the transmitted effect information.

[0218] At this time, the effect control unit (340) can control the display duration, emphasis intensity, or display priority of the effect so that the user does not get confused even if multiple events occur simultaneously.

[0219] Additionally, the effect control unit (340) can perform effect control linked with auditory feedback. For example, when voice guidance is output from the user terminal (100), the effect control unit (340) can provide status information so that a visual display linked to the voice guidance is simultaneously output. Accordingly, the user can receive both auditory guidance and visual guidance together, thereby more clearly perceiving changes in the communication status.

[0220] The effect control unit (340) can adjust the effect output method according to the frequency of event occurrence or user settings. For example, for events that occur repeatedly, the effect intensity can be lowered or the display time shortened to reduce user fatigue, and for urgent events, the priority can be increased to provide clearer visual emphasis.

[0221] At this time, the process of determining the effect output priority and output intensity by the effect control unit (340) can be modeled by [Equation 3].

[0223] [Mathematical Formula 3]

[0224]

[0225] : A set of events that the effect control unit (340) can process,

[0226] e*: Event selected as a result of priority calculation,

[0227] I*: Intensity of the effect output in response to the selected event,

[0228] S(e): Importance of the event,

[0229] U(e, u): Weights reflecting user settings or permissions,

[0230] P(c): Channel priority,

[0231] D(e): decay term over time since the event occurred,

[0232] O(e,t): Over-output penalty based on repetition frequency,

[0234] Accordingly, the effect control unit (340) can perform adaptive effect control that reflects event characteristics and user environment, rather than displaying according to simple fixed rules.

[0235] In this way, the effect control unit (340) converts various communication and channel events occurring throughout the intercom system into visual and auditory feedback and provides them to the user, thereby enabling intuitive recognition of complex communication states and improving the usability of the intercom system even in a hands-free environment.

[0236] The central control unit (350) is a core control component that integrates and controls multiple function modules in the intercom server (300), and adjusts the overall operation flow of the server by comprehensively analyzing communication control requests, channel status changes, user events, and system status information received from the user terminal (100).

[0237] The central control unit (350) organically links the operations of the communication unit (310), channel management unit (320), input / output control unit (330), and effect control unit (340), thereby enabling the intercom system to maintain consistent policies and state transitions.

[0238] When the central control unit (350) receives a communication control request based on a voice command from a user terminal (100), it analyzes the type of the request and determines a processing path.

[0239] For example, when a request related to channel selection or channel switching is received, the channel management unit (320) is instructed to update the channel status, and when a request such as speaking activation or listening-only switching is received, the input / output control unit (330) is instructed to change the transmission / reception status.

[0240] At this time, the central control unit (350) determines the control order so that the control is performed sequentially when a single request requires multiple sub-controls.

[0241] Additionally, the central control unit (350) determines whether a request from a user terminal (100) is allowed by policy by referring to the channel policy and user authority information set in the channel management unit (320). For requests that are not allowed by policy, the central control unit can control the execution of the request or perform an alternative action.

[0242] For example, when a request to enable speaking is received on a receiving-only channel, the central control unit (350) may not transmit the request to the input / output control unit (330), but instead control the effect control unit (340) to inform the user of the restriction status.

[0243] The central control unit (350) monitors changes in the session status of the communication unit (310) and performs control in response when changes in communication quality or network status are detected. For example, if the network quality of a specific user terminal (100) deteriorates, the central control unit (350) may instruct the communication unit (310) to restrict audio transmission to the user or temporarily adjust the channel policy through the channel management unit (320). This prevents the deterioration of communication quality from affecting the entire channel.

[0244] Additionally, the central control unit (350) manages the frequency and cumulative status of events occurring throughout the system. For example, if the same type of event occurs repeatedly within a short period of time, the central control unit (350) may instruct the effect control unit (340) to reduce the output intensity or shorten the display time so as not to excessively deliver the event to the user.

[0245] Conversely, when an urgent event or a high-priority event occurs, the control parameters of the effect control unit (340) can be adjusted so that it is processed with priority over existing events. This over-output penalty control based on the frequency of repeated occurrences reduces user fatigue and provides the effect of ensuring that important events are clearly recognized.

[0246] The central control unit (350) controls the system to maintain state consistency even when multiple events occur simultaneously. For example, when a channel switching request and a speaking state change request are received simultaneously, the central control unit (350) adjusts the control order to perform the channel switching first and then apply the speaking state change. This prevents state inconsistency between the user terminal (100), smart glasses (200), and intercom server (300).

[0247] In this way, the central control unit (350) integrates and determines the status information received from each component within the intercom server (300) and the control request from the user terminal (100) to adjust the control flow, thereby enabling policy-based and context-aware intercom control and providing stable and consistent communication services even in a complex multi-user environment.

[0249] FIG. 4 is a block diagram illustrating the internal structure of smart glasses according to one embodiment of the present invention.

[0250] Referring to FIG. 4, the smart glasses (200) include a transparent display unit (210), a voice input / output unit (220), a communication unit (230), a sensor unit (240), a glasses control unit (250), and a power supply unit (260).

[0251] The transparent display unit (210) is configured to visually provide intercom-related information while maintaining the user's forward field of vision in the smart glasses (200), and renders a user interface in the form of a virtual layer based on status information transmitted from the user terminal (100) and the intercom server (300).

[0252] When display target information is input from the glasses control unit (250), the transparent display unit (210) generates a graphic element corresponding to the information and outputs it superimposed on a display area on the lens. At this time, the graphic element displayed may include at least one of an icon indicating a channel selection status, a client node connection status, whether speaking is active, a voice reception status, or a channel switching result, a highlight, a border, or a color change.

[0253] The transparent display unit (210) outputs graphic elements in a semi-transparent or partially transparent manner so as not to excessively obstruct the user's field of vision. For example, it may be configured to display only minimal icons in the default standby state and temporarily output an highlighted UI only when a specific event occurs. After the event ends, the display intensity is gradually reduced or the display is removed after a certain period of time, thereby reducing the user's visual fatigue.

[0254] Additionally, the transparent display unit (210) can dynamically adjust the display position and display method. For example, depending on the user's wearing status, head movement, or work environment, the display area can be moved to the center or periphery of the field of view, or the display size and brightness can be adjusted. Such adjustments can be performed based on sensor information or user setting information received from the sensor unit (240).

[0255] The transparent display unit (210) can change the display method according to effect information transmitted from the effect control unit (340). For example, when a voice reception event occurs, a specific area is flashed or the color is changed to emphasize the reception status, and when a channel switching or speech activation event occurs, the selected channel or user node is highlighted. At this time, if multiple events occur simultaneously, the display order or emphasis intensity may be adjusted according to priority information.

[0256] Additionally, the transparent display unit (210) can operate in conjunction with auditory feedback provided by the user terminal (100). For example, while voice guidance is output from the user terminal (100), the transparent display unit (210) simultaneously provides a visual display corresponding to the guidance, thereby allowing the user to more clearly perceive the communication status or control results.

[0257] In this way, the transparent display unit (210) renders communication status and control information generated from the user terminal (100) and the intercom server (300) in real time as an interactive UI in the form of a virtual layer, thereby allowing the user to intuitively perceive the communication status without directly operating the terminal and improving intercom usability in a hands-free environment.

[0258] The voice input / output unit (220) is configured to collect voice input from a user in smart glasses (200) and provide voice output according to intercom communication, and operates by including one or more microphones and one or more speakers. The voice input / output unit (220) is linked with a user terminal (100) and an intercom server (300) to support stable performance of voice-based intercom functions in a hands-free environment.

[0259] The microphone of the voice input / output unit (220) detects the user's speech and collects a voice signal. At this time, the voice input / output unit (220) may be configured to determine the time when the user's speech actually begins using a voice activity detection function, and to activate and collect voice data only when speech is detected. Accordingly, unnecessary voice collection is restricted in a standby state where the user is not speaking, thereby reducing power consumption and reducing the possibility of misrecognition due to ambient noise.

[0260] The collected voice data is transmitted to the user terminal (100) through the communication unit (230). The user terminal (100) uses the transmitted voice data to perform voice-to-text conversion and natural language understanding-based command interpretation, or controls the transmission of voice data to another user terminal through the intercom server (300). In this process, the voice input / output unit (220) minimizes the computational load of the smart glasses (200) by focusing on the collection and transmission of voice signals without performing complex calculations.

[0261] The speaker of the voice input / output unit (220) outputs the voice of the other user transmitted through the intercom server (300) or voice guidance generated from the user terminal (100). For example, when the other user switches to a speaking state, the corresponding voice is output in real time, and when a channel switch or a change in communication status occurs, voice guidance generated from the user terminal (100) can be output through the speaker.

[0262] The voice input / output unit (220) can dynamically change its voice input / output state according to the control of the glasses control unit (250). For example, when the user switches to listening-only mode, the microphone input is disabled and only the speaker output is enabled, and when the speaking-enabled state is enabled, both the microphone input and the speaker output are enabled. Additionally, when a speaking end command is received, the microphone input is blocked and voice transmission is stopped.

[0263] Additionally, the voice input / output unit (220) may have its voice output characteristics adjusted according to the communication status or user settings. For example, in an environment with loud external noise, the speaker output volume may be automatically increased, or the voice may be controlled to be delivered in an output manner that is close to the user's ear. Conversely, in a quiet environment, the output intensity may be lowered to reduce user fatigue.

[0264] In this way, the voice input / output unit (220) efficiently collects the user's speech and stably outputs the communication voice, thereby supporting smooth voice input and voice output in a hands-free intercom environment using smart glasses (200).

[0265] The communication unit (230) is configured to perform wireless communication between the smart glasses (200) and the user terminal (100), and supports the smart glasses (200) to be stably connected to the intercom system without performing complex calculations on its own. The communication unit (230) transmits and receives control signals and status information using a low-power wireless communication method and operates in a structure that delegates voice communication processing to the user terminal (100).

[0266] The communication unit (230) establishes a wireless connection with the user terminal (100) when power is applied to the smart glasses (200). At this time, the communication unit (230) can be configured to minimize battery consumption even in a long-term wearing environment by maintaining the connection using a low-power short-range wireless communication protocol. Once the connection is established, the communication unit (230) is linked with the glasses control unit (250) to receive control commands and status information transmitted from the user terminal (100).

[0267] The communication unit (230) receives a control signal transmitted from the user terminal (100) and transmits it to the glasses control unit (250). The control signal may include speech activation, listening-only switching, channel selection result, HUD display information, or effect output information. The glasses control unit (250) interprets the transmitted control signal and controls the operation of the transparent display unit (210), voice input / output unit (220), and sensor unit (240).

[0268] Additionally, the communication unit (230) transmits status information collected from the smart glasses (200) to the user terminal (100). For example, information on whether speech has started detected by the voice input / output unit (220), or information on the wearing status or operation status detected by the sensor unit (240), can be transmitted to the user terminal (100) through the communication unit (230). Based on this information, the user terminal (100) performs voice command processing or communication control with the intercom server (300).

[0269] The communication unit (230) is designed so that communication delay is minimized during the transmission and reception of control signals and status information. To this end, the communication unit (230) performs communication focused on control packets consisting of a small amount of data, and large-capacity data such as voice data is processed through a separate communication path via the user terminal (100). Accordingly, the smart glasses (200) can focus on intercom control without the burden of voice streaming processing.

[0270] Additionally, the communication unit (230) can continuously monitor the connection status to detect a deterioration in communication quality or a disconnection situation. If a communication failure is detected, the communication unit (230) transmits the status to the glasses control unit (250) and the user terminal (100) so that an attempt to reconnect or alternative control is performed. During this process, the user may be visually notified of the change in the connection status through the transparent display unit (210).

[0271] In this way, the communication unit (230) is responsible for low-power, low-latency communication between the smart glasses (200) and the user terminal (100), thereby enabling the smart glasses (200) to function as an output and input auxiliary terminal of the intercom system and performing the role of improving the stability and power efficiency of the entire system.

[0272] The communication unit (230) is configured to perform wireless communication between the smart glasses (200) and the user terminal (100), and supports the smart glasses (200) to be stably connected to the intercom system without performing complex calculations on its own. The communication unit (230) transmits and receives control signals and status information using a low-power wireless communication method and operates in a structure that delegates voice communication processing to the user terminal (100).

[0273] The communication unit (230) establishes a wireless connection with the user terminal (100) when power is applied to the smart glasses (200). At this time, the communication unit (230) can be configured to minimize battery consumption even in a long-term wearing environment by maintaining the connection using a low-power short-range wireless communication protocol. Once the connection is established, the communication unit (230) is linked with the glasses control unit (250) to receive control commands and status information transmitted from the user terminal (100).

[0274] The communication unit (230) receives a control signal transmitted from the user terminal (100) and transmits it to the glasses control unit (250). The control signal may include speech activation, listening-only switching, channel selection result, HUD display information, or effect output information. The glasses control unit (250) interprets the transmitted control signal and controls the operation of the transparent display unit (210), voice input / output unit (220), and sensor unit (240).

[0275] Additionally, the communication unit (230) transmits status information collected from the smart glasses (200) to the user terminal (100). For example, information on whether speech has started detected by the voice input / output unit (220), or information on the wearing status or operation status detected by the sensor unit (240), can be transmitted to the user terminal (100) through the communication unit (230). Based on this information, the user terminal (100) performs voice command processing or communication control with the intercom server (300).

[0276] The communication unit (230) is designed so that communication delay is minimized during the transmission and reception of control signals and status information. To this end, the communication unit (230) performs communication focused on control packets consisting of a small amount of data, and large-capacity data such as voice data is processed through a separate communication path via the user terminal (100). Accordingly, the smart glasses (200) can focus on intercom control without the burden of voice streaming processing.

[0277] Additionally, the communication unit (230) can continuously monitor the connection status to detect a deterioration in communication quality or a disconnection situation. If a communication failure is detected, the communication unit (230) transmits the status to the glasses control unit (250) and the user terminal (100) so that an attempt to reconnect or alternative control is performed. During this process, the user may be visually notified of the change in the connection status through the transparent display unit (210).

[0278] In this way, the communication unit (230) is responsible for low-power, low-latency communication between the smart glasses (200) and the user terminal (100), thereby enabling the smart glasses (200) to function as an output and input auxiliary terminal of the intercom system and performing the role of improving the stability and power efficiency of the entire system.

[0279] The sensor unit (240) is configured to detect the wearing status of the smart glasses (200), user movements, and surrounding environment, and to control the operation of the smart glasses (200) according to the situation, and operates by including one or more sensors. The sensor unit (240) is linked with the glasses control unit (250) and dynamically adjusts the output method and operation status of the smart glasses (200) based on the detected sensor information.

[0280] The sensor unit (240) may include a wear detection sensor for detecting whether the user is wearing the smart glasses (200). The wear detection sensor determines whether the smart glasses (200) are worn on the user's face or head, and controls the main functions of the smart glasses (200) to be activated only when the wearing state is detected. For example, when the user is not wearing the smart glasses (200), the operation of the transparent display unit (210) and the voice input / output unit (220) is restricted, and when wearing is detected, the corresponding components can be automatically activated.

[0281] Additionally, the sensor unit (240) may include a posture sensor or an acceleration sensor for detecting the user's movement or change in posture. These sensors can detect the user's head movement, change in direction, or stationary state, and can be used to adjust the display position or display intensity of the transparent display unit (210). For example, if the user is moving quickly or moving their head significantly, the size of the display element may be reduced or the display frequency lowered to minimize obstruction of the user's view.

[0282] The sensor unit (240) may include an illuminance sensor for detecting the brightness of the surrounding environment. The ambient brightness information detected by the illuminance sensor is transmitted to the glasses control unit (250) so that the brightness or contrast of the transparent display unit (210) is automatically adjusted. Through this, the HUD information can maintain appropriate visibility regardless of changes in the indoor and outdoor environment.

[0283] Additionally, the sensor unit (240) can assistly determine the user's speech environment in conjunction with the voice input / output unit (220). For example, the sensor unit (240) can indirectly determine the user's operating state or environmental noise and utilize this to adjust the microphone activation conditions or voice output characteristics of the voice input / output unit (220).

[0284] Sensor information detected by the sensor unit (240) can be transmitted to the user terminal (100) through the communication unit (230). The user terminal (100) can use the transmitted sensor information as auxiliary information for voice command processing or transmit it to the intercom server (300) to be reflected in channel control or user status determination.

[0285] In this way, the sensor unit (240) detects the wearing status of the smart glasses (200), user movements, and environmental information, and adjusts the operation of the smart glasses (200) according to the situation, thereby improving user convenience and safety and reducing unnecessary power consumption.

[0286] The glasses control unit (250) is a core control component that integrally controls the operation of each component of the smart glasses (200), and interprets control commands received from the user terminal (100) and state information generated inside the smart glasses (200) to adjust the operation of the transparent display unit (210), voice input / output unit (220), communication unit (230), and sensor unit (240).

[0287] When the glasses control unit (250) receives a control command from the user terminal (100) through the communication unit (230), it analyzes the type of the control command. For example, if a command related to speech activation, listening-only switching, channel switching results, or intercom status change is received, the glasses control unit (250) determines an internal control sequence corresponding to the command. According to the determined control sequence, a corresponding HUD display is output to the transparent display unit (210), and the input or output state of the voice input / output unit (220) is changed.

[0288] Additionally, the glasses control unit (250) continuously monitors sensor information received from the sensor unit (240) and adjusts the operating state of the smart glasses (200) according to the situation. For example, if the wearing state is detected by the wearing detection sensor, the glasses control unit (250) can control the operation of the transparent display unit (210) and the voice input / output unit (220) to be restricted or switched to a low-power mode. Conversely, if the wearing state is detected, the corresponding components are controlled to be automatically activated.

[0289] The glasses control unit (250) controls the voice input / output state in conjunction with the voice input / output unit (220). For example, when a speech activation command is confirmed from the user terminal (100), the glasses control unit (250) activates the microphone input and controls the state so that it is visually displayed on the transparent display unit (210). In addition, when a listening-only switching command is received, the microphone input is deactivated and controls the state to maintain only the voice reception state.

[0290] Additionally, the glasses control unit (250) dynamically controls the display method of the transparent display unit (210). For example, the display position, display intensity, display duration, or display priority can be adjusted according to effect information transmitted from the effect control unit (340) or status information transmitted from the user terminal (100). Through this, even when multiple events occur simultaneously, confusion regarding information transmitted to the user is minimized.

[0291] The glasses control unit (250) transmits status information of the smart glasses (200) to the user terminal (100) through the communication unit (230). For example, the wearing status, voice input / output status, or environmental information detected by the sensor unit (240) is transmitted to the user terminal (100) and can be used for voice command processing or control with the intercom server (300).

[0292] In this way, the glasses control unit (250) mediates the control flow between the user terminal (100) and the internal components of the smart glasses (200), and dynamically adjusts the operation of the smart glasses (200) by reflecting sensor-based situation awareness, thereby enabling stable and intuitive intercom usage even in a hands-free environment.

[0293] The power supply unit (260) is configured to supply power to the smart glasses (200) and manage the power consumption of each component, and operates by including a rechargeable battery and a power management circuit. The power supply unit (260) is designed to enable low-power operation considering the long-term wearing environment of the smart glasses (200), and dynamically controls the power usage status in conjunction with the glasses control unit (250).

[0294] The power supply unit (260) distributes the necessary power to each component when power is applied to the smart glasses (200). At this time, the power supply unit (260) can set priorities by considering the power demand status of the communication unit (230), sensor unit (240), voice input / output unit (220), and transparent display unit (210), and can control to supply power starting from the essential components.

[0295] The power supply unit (260) adjusts the power supply status according to a control signal transmitted from the glasses control unit (250). For example, if the wear detection sensor detects that the user is not wearing the smart glasses (200), the power supply unit (260) may cut off the power supply to the transparent display unit (210) and the voice input / output unit (220) or switch to a low-power standby mode. Conversely, if the wear state is detected, the power supply to the corresponding components is resumed.

[0296] Additionally, the power supply unit (260) can optimize power consumption according to the communication status and usage pattern. For example, if voice communication is not performed for a certain period of time, the power consumption of the communication unit (230) can be reduced, or the display brightness of the transparent display unit (210) can be lowered to reduce battery consumption. This power control can be performed by reflecting environmental information detected by the sensor unit (240) or user settings.

[0297] The power supply unit (260) continuously monitors the remaining battery level and, when the remaining level decreases below a reference value, transmits the status to the glasses control unit (250). Based on the transmitted battery status information, the glasses control unit (250) can transmit a low battery notification to the user terminal (100) or provide visual guidance to the user through the transparent display unit (210).

[0298] Additionally, the power supply unit (260) manages the charging state. For example, when an external power source is connected and charging begins, the power supply unit (260) detects the charging state and charges the battery while stably supplying power to each component. During charging, the current can be controlled through a power management circuit so that heat generation or overcurrent does not occur.

[0299] In this way, the power supply unit (260) adjusts the power supply to each component of the smart glasses (200) according to the situation and induces low-power operation, thereby extending the usage time in a wearable environment and improving the stability of the system.

[0301] FIG. 5 is a diagram illustrating voice-based intercom control and visual information display status using smart glasses according to an embodiment of the present invention.

[0302] Referring to FIG. 5, the user performs voice command-based communication control by linking with an intercom application running through a user terminal while wearing smart glasses (200). The smart glasses (200) display information such as the intercom channel status, call target, and connection status in the form of a virtual layer through a transparent display located in front of the user's field of vision.

[0303] The transparent display of the smart glasses (200) outputs an icon representing a currently active communication group or client node, a graphic element representing the number of users, and a visual display that allows for intuitive recognition of the communication status.

[0304] This visual information is rendered in a translucent form so as not to obstruct the user's actual field of vision, and its position can be fixed or corrected depending on the user's head movement or gaze direction.

[0305] The user can perform speech in the form of natural language through the voice input / output unit and control the operating state of the intercom by uttering voice commands such as “select,” “channel switching,” and “speaking activation.” The voice command is converted into a control signal through voice recognition and command mapping processes at the user terminal and then reflected in the smart glasses (200).

[0306] The smart glasses (200) change the UI state on the transparent display in real time in response to the control signal and provide feedback to the user, for example, by highlighting the selected channel or visually displaying the speaking activation state. At the same time, the user can intuitively recognize the current intercom status through voice output or visual effects.

[0308] FIG. 6 is a block diagram illustrating an embodiment in which a management server and an AI learning device according to one embodiment of the present invention are configured as a single integrated server.

[0309] Referring to FIG. 6, the management server (400) includes a user management unit (410), a channel policy management unit (420), a security event analysis unit (430), an AI learning and model creation unit (440), a model management and distribution unit (450), and an integrated control unit (460).

[0310] The user management unit (410) manages user accounts, user rights, device mapping, and user status in the integrated management server (400) to perform the function of controlling the operation of the intercom server (300) and the user terminal (100) based on the legal user standard.

[0311] The user management unit (410) does not stop at simple account storage, but operates in a way that dynamically maintains the linkage between user, device, and authority by considering the possibility of device theft, account sharing, and fraudulent use that may occur in a smart glasses (200)-based environment.

[0312] The user management unit (410) performs a user registration procedure when a request for registration of a new user is received. The user registration request may be entered through an administrator terminal or a user terminal (100), and the user management unit (410) creates and stores a user profile including user identification information, organization information, security level, available channel range, and speaking authority.

[0313] For example, in a special operations environment, the accessible channels may be restricted according to the security level of each user, so the user management unit (410) manages the user-specific channel access range by including it in the profile.

[0314] The user management unit (410) manages mapping information between user accounts and devices. Here, the devices may include a user terminal (100) and smart glasses (200), and the user management unit (410) may store the device identification information and the user account in conjunction so that the user can use the intercom function only on the device where the user is registered.

[0315] For example, if a specific smart glasses (200) is configured to be assigned only to a registered user, the user management unit (410) maps the device identification information of the smart glasses to a user account and restricts any attempt to use the device from another account.

[0316] The user management unit (410) updates the user account status by reflecting the user authentication status received from the user terminal (100) or the intercom server (300). For example, when authentication success is confirmed at the user terminal (100), the user management unit (410) sets the corresponding user session to an active state and provides active state information so that the intercom server (300) can process communication requests normally. Conversely, when authentication failure is detected, the user management unit (410) may switch the corresponding user account to a restricted state, or switch the account to a locked state if the number of authentication failures accumulates and exceeds a certain standard.

[0317] The user management unit (410) manages the session-based user status by considering the possibility of equipment hijacking or malicious use. For example, if a user logs in normally and a communication session is in progress, and the re-authentication result during communication repeatedly fails, the user management unit (410) determines that there is a possibility of session hijacking and can switch the user session to a forced termination state. In this case, the user management unit (410) transmits a request for session termination or authorization revocation to the integrated control unit (460), and the integrated control unit (460) can control the transmission blocking or session termination command to the intercom server (300).

[0318] Additionally, the user management unit (410) provides user-specific permission information in real time. For example, even if a specific user requests “speaking activation,” if that user is set to receive-only, the user management unit (410) can confirm that the user’s permission does not allow transmission and provide permission information to the intercom server (300) to refuse transmission activation. Conversely, for a user with administrator privileges, a permission flag can be set so that priority speaking permission is granted in emergency situations.

[0319] In another embodiment, the user management unit (410) may provide policy information to the user terminal (100) based on the user profile. For example, for a user with a high security level, the policy may be provided to increase the authentication threshold or to set a short periodic re-authentication cycle. Through this, security levels can be applied differently to each user even within the same system.

[0320] In this way, the user management unit (410) performs user registration, authorization, device mapping, reflection of user authentication status, and session status update in an integrated manner, thereby playing a role in simultaneously improving security and management efficiency so that only legitimate users can perform communication functions in a smart glasses (200)-based intercom environment.

[0321] The channel policy management unit (420) performs the function of defining and managing channel configuration, channel access rights, speaking priority, and communication mode control policies within the integrated management server (400). The channel policy management unit (420) is designed to go beyond simply storing a list of channels and to dynamically perform communication control at the channel level in conjunction with user authority information and security status.

[0322] The channel policy management unit (420) creates multiple channels during the initial system setup and defines attributes for each channel. Channel attributes may include channel identification information, channel type, accessible user group, list of transmittable users, whether it is set to receive-only, whether it is in emergency mode, and priority level. For example, different policies can be applied to each channel by classifying them into general work channels, command channels, emergency channels, etc.

[0323] When a user terminal (100) transmits a request to participate in a specific channel, the intercom server (300) may query the channel policy management unit (420) for the request. The channel policy management unit (420) compares the user authority information provided by the user management unit (410) with the channel policy to determine whether the user is allowed to access the channel. If access is permitted, it transmits participation approval information to the intercom server (300), and if access is not permitted, it returns a rejection response.

[0324] For example, when an actual registered user requests a channel switch while wearing smart glasses (200) and successfully completing authentication, the channel policy management unit (420) checks whether the user has access rights to the requested channel. If the rights are confirmed, the channel switch is approved, and the intercom server (300) connects the user to the channel and performs voice stream relay. Additionally, if the channel is a channel with restricted speaking rights, the ability to transmit is additionally checked, and the speaking function is activated only if transmission rights are present.

[0325] On the other hand, if a malicious user steals the smart glasses (200) and attempts to join the channel, the following actions are performed. If an authentication failure state occurs on the user terminal (100) or if the account is displayed as restricted by the user management unit (410), the channel policy management unit (420) rejects the channel access request for the user.

[0326] Additionally, if a security event occurs when a re-authentication failure is detected during communication, the channel policy management unit (420) can change the policy to immediately revoke the transmission authority on the channel and communicate with the integrated control unit (460) to transmit a transmission blocking command to the intercom server (300).

[0327] The channel policy management unit (420) also controls the communication mode for each channel. For example, a specific channel can be set to a receive-only mode so that all users can only receive, and a specific channel can be set so that only one person can speak sequentially. In a multi-party communication environment, when multiple users attempt to speak simultaneously, the channel policy management unit (420) can apply a policy to determine the right to speak according to a pre-set priority or authority level.

[0328] Additionally, when an emergency situation is detected, the channel policy management unit (420) can automatically activate the emergency channel and dynamically change the policy to grant priority speaking rights to specific users. For example, an administrator or commander account can be set to have priority speaking rights for all channels in an emergency situation.

[0329] The channel policy management unit (420) can store channel access records, transmission authority change records, and policy change history to be used for security audits and post-analysis. If repeated unauthorized channel access attempts occur, additional security measures for the relevant user or device can be performed in conjunction with the user management unit (410).

[0330] The security event analysis unit (430) performs the function of detecting abnormal behavior or security threats by comprehensively analyzing user authentication status, channel access records, communication session information and terminal operation logs within the integrated management server (400).

[0331] The security event analysis unit (430) operates as an active security module that does not simply store logs, but distinguishes between normal user behavior and abnormal behavior to calculate the risk level and, if necessary, immediately induces communication restriction measures.

[0332] The security event analysis unit (430) receives various event information from the user terminal (100), the intercom server (300), and the user management unit (410). The received event information may include records of user login success or failure, re-authentication results, records of channel access requests, records of transmission activation requests, session duration, whether device identification information has changed, and information on changes in speech patterns.

[0333] When an actual registered user wears the smart glasses (200) and uses them normally, the security event analysis unit (430) accumulates authentication success records and normal channel access records to form a normal usage pattern.

[0334] For example, if a specific user primarily uses a specific channel during a specific time period, that pattern can be learned as normal behavior or stored as reference data. If the re-authentication result remains within the normal range even during a communication session, the security event analysis unit (430) maintains the session in a stable state.

[0335] On the other hand, if a malicious user steals and uses the smart glasses (200), the following analysis is performed. First, when an authentication failure result is reported from the user terminal (100), the security event analysis unit (430) accumulates the number of failures and classifies it as a security risk event if it exceeds a certain threshold.

[0336] Additionally, if the login is successful but the speaker's voice characteristics change rapidly during communication, information regarding re-authentication failure or a sudden drop in suitability may be transmitted, and the security event analysis unit (430) determines this as a suspected session hijacking event.

[0337] As another example, if a user suddenly attempts to access a high-security channel that they do not normally access, or repeatedly switches between multiple channels within a short period of time, it may be judged as abnormal behavior. The security event analysis unit (430) raises the risk score if the deviation from the normal usage history of each user exceeds a certain standard. If the risk score exceeds a threshold, the session is classified as a high-risk state.

[0338] When an abnormal state is detected, the security event analysis unit (430) transmits a request for security measures to the integrated control unit (460). For example, if a state suspected of session hijacking is determined, a transmission blocking or session forced termination command may be transmitted to the intercom server (300). Additionally, the user account may be switched to a temporary lock state by the user management unit (410), and a warning notification may be sent to the administrator terminal.

[0339] The security event analysis unit (430) can make judgments not only on a single event but also on a combination of multiple events. For example, if authentication failure, device change, and abnormal channel access occur simultaneously, they can be evaluated as having a higher risk than individual events. Through this, it is possible to distinguish between simple misidentification and actual theft attempts.

[0340] Additionally, the security event analysis unit (430) can continuously accumulate event data and utilize it to improve security policies. Patterns of repeated spoofing attempts can be transmitted to the AI ​​learning and model generation unit (440) and reflected in future authentication model learning. Through this, the system's security detection accuracy improves over time.

[0341] The AI ​​learning and model generation unit (440) is a core AI module that generates a user-customized authentication model based on user voice data and behavior data within the integrated management server (400) and performs learning to detect impersonation or equipment theft. The AI ​​learning and model generation unit (440) is configured not to be a simple voice recognition model, but to generate a security-specialized model capable of distinguishing between registered and unregistered users and continuously determining user suitability during communication.

[0342] The AI ​​learning and model generation unit (440) first receives user voice data collected through a user terminal (100) or smart glasses (200). The collected voice data undergoes preprocessing steps such as speech segment separation, noise removal, volume normalization, and silent segment removal. In particular, considering environments with high background noise, such as industrial sites or special operations environments, data below a certain quality standard may be processed by excluding it from learning or lowering its weight.

[0343] A user feature vector is extracted from preprocessed voice data. The feature vector may include the voice's frequency distribution, pitch variation, formant structure, phonation intensity curve, speech rate, intersyllable spacing, intonation patterns, and repetitive speech habits. Additionally, if speech text information is available, vocabulary selection tendencies, sentence length distribution, and the frequency of repetition of specific expressions may also be included as feature elements. These features are used as elements to form the user's unique voice signature.

[0344] The AI ​​learning and model generation unit (440) learns a user authentication model using the feature vector. The authentication model may be configured to calculate the similarity between the input voice feature vector and the registered user data, and is designed to determine the user as a legitimate user if the similarity score is greater than or equal to a threshold value. Additionally, by including voice samples of others or sham speech data as fraud samples during training, the ability to detect voice imitation or intonation shaming can be enhanced.

[0345] When an actual registered user communicates repeatedly through smart glasses (200), the AI ​​learning and model generation unit (440) can continuously accumulate voice data of the user and gradually improve the model. For example, even if the user's voice state changes over time, the model can be updated to maintain authentication accuracy by reflecting the latest data.

[0346] On the other hand, if a malicious user steals the equipment and attempts to impersonate the voice, the voice data may exhibit a low suitability score during the authentication process. The AI ​​learning and model generation unit (440) receives such failure cases from the security event analysis unit (430), classifies them as impersonation attempt patterns, and can add them to future training data as negative samples. This can improve the detection performance for the same type of impersonation attempt.

[0347] When the AI ​​learning and model generation unit (440) completes model learning, it transmits the model to the model management and distribution unit (450). The model is distributed to a user terminal (100) and used for real-time authentication. At this time, the model may include version information and integrity verification information, and the user terminal (100) performs a verification procedure when loading the model to check for forgery or alteration.

[0348] In addition, the AI ​​learning and model generation unit (440) can learn not only single-user unit learning but also group unit abnormal behavior patterns. For example, if a specific type of impersonation attempt occurs repeatedly, the pattern can be defined as a common risk model to strengthen security standards for all users.

[0349] The model management and distribution unit (450) performs the function of storing, verifying, and versioning the user authentication model and anomaly detection model generated by the AI ​​learning and model generation unit (440), and securely distributing them to the user terminal (100). The model management and distribution unit (450) operates not as a simple file transfer function, but as an integrated model management module that includes model integrity guarantee, model version tracking, and user-specific customized model branching and rollback control.

[0350] When a new model is created from the AI ​​learning and model creation unit (440), the model management and distribution unit (450) registers the model in the repository and records the model identifier, version information, creation time, training data range, and target user information together. This allows for the separate management of user-specific authentication models and common security models.

[0351] The model management and distribution unit (450) generates integrity verification information when registering a model. For example, it generates a hash value or digital signature for the model file to verify whether the model has been tampered with or forged. Subsequently, when a user terminal (100) requests a model update, the model management and distribution unit (450) checks the user account and device identification information of the terminal, and then selects and transmits the latest version of the model applicable to the user.

[0352] For example, when an actual registered user uses smart glasses (200), the user terminal (100) can send a model update request to the management server (400) at a certain period or at the time of authentication model renewal. The model management and distribution unit (450) checks whether the user's account status is normal and transmits the latest authentication model to the user terminal (100) through an encrypted communication channel. The user terminal (100) verifies the integrity information of the received model and applies it to the local storage.

[0353] On the other hand, if a malicious user steals the equipment and requests a model using a forged terminal, the model management and distribution unit (450) can check the device identification information or user authentication status and refuse to provide the model if it is not a legitimately registered device. Additionally, if the account is locked or classified as high-risk by the security event analysis unit (430), the model distribution can be restricted or configured to provide only the least privilege model.

[0354] The model management and distribution unit (450) performs model version management. For example, if the authentication failure rate increases rapidly after applying the latest model, it can be designed to automatically roll back to the previous stable version. This allows for the maintenance of system stability and user authentication accuracy.

[0355] Additionally, the model management and distribution unit (450) can collect model performance feedback data from the user terminal (100). For example, statistical information such as the authentication success rate, misrecognition rate, and re-authentication failure rate can be collected and transmitted to the AI ​​learning and model generation unit (440) to be used for future model improvement.

[0356] The integrated control unit (460) operates as a final execution control module that transmits actual control commands to the intercom server (300) and user terminal (100) by synthesizing the judgment results of the user management unit (410), channel policy management unit (420), security event analysis unit (430), AI learning and model generation unit (440), and model management and distribution unit (450) within the integrated management server (400). The integrated control unit (460) is not merely a simple policy transmission function, but a core control device that dynamically controls communication sessions by integrally judging security status, user authority, and channel policy.

[0357] The integrated control unit (460) first receives user account status and authority information from the user management unit (410). If the user is in a normal authentication state, the channel access authority and speaking authority granted to the user are transmitted to the intercom server (300) to ensure that communication is maintained normally. For example, if an actual registered user wears smart glasses (200) and performs a “speaking activation” command, the integrated control unit (460) checks whether the user’s transmission authority is valid and then transmits a transmission permission signal to the intercom server (300).

[0358] On the other hand, if information regarding account restriction or lock status is transmitted from the user management unit (410), the integrated control unit (460) transmits a command to block channel access or terminate session for the corresponding user to the intercom server (300). This ensures that even if a malicious user steals the equipment and logs in, communication is immediately blocked according to the account status.

[0359] The integrated control unit (460) receives channel unit policy information from the channel policy management unit (420) and controls the communication mode. For example, if a specific channel is set to a receive-only mode, the integrated control unit (460) transmits a transmission disable command to the intercom server (300) to block user speech on that channel. Additionally, if an emergency mode is activated, a control signal can be transmitted to the intercom server (300) to grant priority speaking rights to a specific user.

[0360] When an abnormal behavior detection signal is transmitted from the security event analysis unit (430), the integrated control unit (460) immediately executes a security response procedure. For example, if re-authentication failures are repeated during communication and it is determined that the session hijacking is suspected, the integrated control unit (460) may forcibly terminate the session or apply a restriction mode that blocks only the transmission function while maintaining reception. If classified as a high-risk state, a command may be transmitted to the intercom server (300) to block access to all channels of the device.

[0361] In addition, the integrated control unit (460) can perform a step-by-step security response. For example, if the risk score exceeds a certain threshold, a level 1 warning can be applied, followed by transmission blocking upon further increase, and finally session termination and account locking. This allows for distinguishing between simple misidentification and actual threats while preventing excessive blocking.

[0362] The integrated control unit (460) performs control functions in conjunction with the model management and distribution unit (450) even at the time of model updates. For example, it can instruct the intercom server (300) to apply an enhanced re-authentication policy for a certain period immediately after a new authentication model is distributed. This minimizes security gaps that may occur during the model transition process.

[0363] In the case of an actual registered user, the integrated control unit (460) stably maintains the communication session while the normal authentication state and normal usage pattern are maintained, and provides a delay-free communication environment by promptly approving channel switching and speech requests. On the other hand, if a malicious user attempts to use the goggles, the communication session is immediately restricted or terminated by integrating information such as authentication failure, detection of abnormal behavior, and device mismatch, thereby blocking information leakage.

[0365] Although the present invention has been described by the embodiments and drawings described above, the present invention is not limited to the above embodiments, and various modifications and variations are possible from this description by those skilled in the art to which the present invention pertains. Accordingly, the concept of the present invention should be understood only by the claims set forth below, and all equivalent or analogous variations thereof shall be considered to fall within the scope of the concept of the present invention. Explanation of the symbols

[0367] 100: User terminal, 200: Smart glasses, 210: Transmissive display section, 220: Audio input / output unit, 230: Communications Department, 240: Sensor section, 250: Glasses control unit, 260: Power supply, 300: Intercom server, 310: Communications Department, 320: Channel Management Department, 330: Input / Output Control Unit, 340: Effect control unit, 350: Central control unit, 400: Management Server,

Claims

Claim 1 An AI-based bidirectional intercom system characterized by comprising: a user terminal that collects user voice input, performs voice-to-text conversion and natural language understanding to generate intercom control commands; smart glasses that display communication status or control results within the user's field of vision through a transparent display and perform voice input and output; an intercom server that establishes voice communication sessions between multiple users and relays voice data; a management server that manages user authorization information and channel policy information, and controls the operation of the intercom server according to user authentication results or security events; and a user-customized AI learning device that collects user voice data to learn and generate a user-customized AI model, generates a user authentication model and an anomalous utterance detection model using learning data including registered user voice data and spoofed utterance data, updates the user authentication model by retraining authentication failure data generated during the real-time authentication process as fraud samples, and provides version information and integrity verification information for the generated authentication model to the management server or user terminal. Claim 2 An AI-based bidirectional intercom system according to claim 1, characterized in that the AI ​​learning device is configured to analyze spoofed speech pattern data collected from multiple users to generate a common risk pattern model and to update security standards for all users using the common risk pattern model. Claim 3 An AI-based bidirectional intercom system according to claim 1, characterized in that the management server is configured to determine whether the model has been tampered with or forged by verifying the hash value or electronic signature of the distributed authentication model or the user terminal. Claim 4 An AI-based bidirectional intercom system according to claim 1, wherein the user authentication model is continuously updated by reflecting the latest voice data collected during the user's repetitive intercom communication process. Claim 5 A step in which a user terminal collects the user's voice input, performs voice-to-text conversion and natural language understanding to generate an intercom control command; a step in which the user terminal transmits the generated intercom control command to an intercom server or a management server; a step in which the intercom server establishes a voice communication session between multiple users and relays voice data; a step in which smart glasses display the communication status or control results within the user's field of vision through a transparent display and perform voice input and output; a step in which an AI learning device extracts acoustic feature vectors from user voice data; a step in which the AI ​​learning device trains a user authentication model and an anomalous speech detection model using training data including registered user voice data and spoofed speech data; a step in which the user authentication model is trained to calculate the similarity between the input voice feature vector and the registered user data; a step in which the AI ​​learning device classifies authentication failure data generated during the real-time authentication process as fraudulent samples and retrains the user authentication model; a step in which the AI ​​learning device continuously updates the user authentication model using the latest user voice data collected during the repetitive intercom communication process; a step in which the AI ​​learning device generates version information and integrity verification information for the generated authentication model; and a step in which the AI ​​learning device distributes the generated authentication model to the management server or the user terminal. An AI-based bidirectional intercom method characterized by including the step of the management server controlling the operation of the intercom server according to a user authentication result or a security event.

Images