An augmented reality-based close-range awareness social interaction system and method

Abstract

This invention discloses a near-field perception social interaction system and method based on augmented reality, belonging to the field of human-computer interaction and augmented reality. The system includes AR glasses and incorporates a communication module, a SLAM positioning module, an AR rendering module, a real-time interaction module, a real-time destruction module, a matching module, and an eye-tracking module. Users bind an activity identifier, and the AR glasses acquire and automatically match identifiers of surrounding devices via short-range wireless communication. Upon successful matching, the virtual information of the other party is overlaid and displayed. A real-time data channel is established after the user gazes at and nods in confirmation, and the data is transmitted in a streaming manner. The system is immediately destroyed from memory after transmission is complete. This invention is applicable to scenarios such as fan meetings, concerts, and sporting events.

Description

Technical Field

[0001] This invention relates to the fields of human-computer interaction, augmented reality, and near-field communication technology, specifically to an augmented reality-based near-field perception social interaction system and method, which can be applied to offline gathering scenarios such as fan meetings, concerts, music festivals, sports events, offline game activities, and exhibitions. Background Technology

[0002] Current social interaction methods largely rely on smartphone screens, requiring users to look down at their phones and preventing face-to-face communication. Augmented reality (AR) technology offers the possibility of head-free social interaction, but existing AR social solutions are mostly focused on remote virtual meetings or gaming scenarios, lacking specific designs for offline face-to-face social interaction. While existing near-field social solutions achieve Bluetooth-based discovery and matching, they cannot visually display the other person's information, requiring users to still look at their phone screen or perform physical actions. Furthermore, existing solutions lack features such as AR spatial positioning, gaze confirmation, virtual information overlay, and shared AR space group interaction, failing to provide an immersive offline social experience. Summary of the Invention

[0003] To overcome the shortcomings of existing technologies, this invention provides an augmented reality-based near-field perception social interaction system and method, enabling users to intuitively see virtual information of others through AR glasses in offline gathering scenarios, complete confirmation through eye contact and nodding, establish real-time data interaction, and destroy interaction data in real time to protect user privacy.

[0004] The technical solution of this invention is as follows: An augmented reality-based near-field perception social interaction system includes several AR glasses terminal devices. The AR glasses have a built-in communication module, a SLAM spatial positioning module, an AR content rendering module, a real-time interaction module, a real-time destruction module, a matching module, and an eye-tracking module. The AR glasses are used to bind the user's currently participating activity identifier or interest identifier, and store these identifiers locally. The activity identifiers or interest identifiers include one or more of the following: fan meeting name, concert name, artist name, sports event name, team name, game name, and exhibition name. The communication module is used for AR glasses to acquire activity identifiers or interest identifiers from other nearby AR glasses via short-range wireless communication technologies (including one or more of Bluetooth, Wi-Fi Direct, UWB, NFC, ultrasound, and millimeter wave). The matching module compares the acquired activity identifiers or interest identifiers with the locally stored activity identifiers or interest identifiers; if they match, a matching relationship is automatically established. The SLAM spatial positioning module is used to track the position and posture of the AR glasses in three-dimensional space in real time, anchoring virtual information to a specific location in the real world. Optionally, the SLAM spatial positioning module also incorporates UWB centimeter-level ranging technology to assist positioning. When visual SLAM feature points are insufficient, the spatial position of the AR glasses is corrected using UWB ranging data. The AR content rendering module is used to overlay and display the other party's virtual information in the real world after a successful match. The virtual information includes one or more of the following: nickname, fan level, identifier color, virtual avatar, distance to the other party, direction, shared interests, entry time, and seating area. The virtual information is overlaid on the other party's real body as a floating label and moves synchronously with the other party's movement. The eye-tracking module is used to track the user's gaze point in real time and detect whether the user is looking at the overlaid virtual information. When the user's gaze point falls within the virtual information area and the duration exceeds a preset threshold (e.g., 0.5 seconds), the virtual information is highlighted. When the user nods while looking at the other party, the system determines that the user has confirmed agreement. The real-time interaction module is used to establish a real-time data interaction channel after the user looks at the virtual information and confirms with a nod. Data sent by the user is collected, transmitted, and output to the other party's AR glasses in real time. Data is transmitted in a streaming manner and does not form a complete file. The data includes one or more of voice, text, images, and video. The real-time destruction module is used to: ensure that all data generated during the interaction exists only briefly in volatile memory during transmission; after transmission, regardless of whether the other party has received, viewed, or played the data, the data is immediately destroyed from memory and not written to any non-volatile storage medium; after the interaction is completed, the virtual information superimposed by the AR content rendering module is immediately cleared from the display cache; after the group interaction ends, all shared AR space data is released from the memory of each participant, and no historical records are retained.The system also includes an AR group interaction module, used to establish a shared AR space group interaction channel after multiple users have successfully matched. The shared AR space supports one or more of the following interactions: shared virtual glow stick linkage, synchronized display of cheering slogans, virtual gift sending and receiving, and group virtual effects. The shared virtual glow stick linkage includes: when any user in the group waves their AR glasses, the color and direction of their virtual glow stick are synchronized to the AR displays of all users in the group. The synchronized display of cheering slogans includes: when any user in the group sends out a cheering slogan, the slogan text is simultaneously displayed in the field of vision of all users in the group as an AR overlay. The AR glasses also support a guest mode. The AR glasses worn by guests or organizers can broadcast virtual information to all user AR glasses within a preset range or batch confirm user interaction requests. The broadcast virtual information includes one or more of the following: virtual wave, virtual signature, and virtual group photo frame. This invention also provides a near-field perception social interaction method based on augmented reality, comprising the following steps: AR glasses are bound to an activity identifier or interest identifier currently being participated in by the user, and the activity identifier or interest identifier is stored locally; the AR glasses acquire activity identifiers or interest identifiers of other AR glasses in the vicinity through short-range wireless communication technology; the acquired activity identifiers or interest identifiers are compared with the locally stored activity identifiers or interest identifiers, and if they are the same, a matching relationship is automatically established; after successful matching, the position and posture of the AR glasses are tracked in real time through a SLAM spatial positioning module; and the virtual information of the other party is overlaid and displayed in the real world through an AR content rendering module, wherein the virtual information packet... The system includes one or more of the following: nickname, fan level, support color, and virtual avatar. The eye-tracking module tracks the user's gaze in real time, detecting whether the user is looking at the overlaid virtual information. Once the system detects the user looking at the virtual information and confirms with a nod, a real-time data interaction channel is established. Data sent by the user is collected, transmitted, and output to the other party's AR glasses in real time. Data is transmitted in a streaming manner and does not form a complete file. All data generated during the interaction only exists briefly in volatile memory during transmission. After transmission, regardless of whether the other party has received, viewed, or played the data, it is immediately destroyed from memory and not written to any non-volatile storage medium. When the AR glasses are a combination of a mobile phone and AR glasses accessories, both parties need to install the corresponding application. When the AR glasses are a standalone device, the corresponding functions can be directly implemented through hardware firmware.

[0005] The beneficial effects of this invention include: eliminating the need to look down to operate, allowing for social interaction while maintaining face-to-face communication; displaying virtual information of the other party through AR overlay, providing an intuitive and immersive experience; completing confirmation through gaze and nod, which is natural and seamless; sharing AR space for group interaction, supporting virtual glow stick linkage and synchronized cheering slogans; point-to-point direct connection and self-destructing messages to protect privacy; and a guest-side mode that supports broadcasting and batch confirmation by the organizer. Attached Figure Description

[0006] Figure 1 This is a structural block diagram of the system of the present invention.

[0007] Detailed Description of Embodiments: The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0008] Example 1 (Fan Meet-and-Greet - AR Virtual Information Overlay + Gaze Confirmation) At a fan meeting, fans wore AR glasses. The system automatically matched fans in the venue who were associated with the same idol. Fan A saw an AR floating tag superimposed above Fan B's real body, displaying "Nickname: Xiao Zhou," "Fan Level: Die-hard Fan," and "Identification Color: Blue." Fan A stared at the floating tag for 0.5 seconds, and the tag highlighted. Fan A nodded, and the system automatically established a real-time voice call with Fan B. Fan A said, "Hello, I also like this idol!" Fan B heard it in real time. After the call ended, all voice data was immediately destroyed from the memory of both AR glasses. The AR floating tag was cleared from the display cache.

[0009] Example 2 (Concert - Shared AR Space Virtual Glow Stick Linkage) At a concert, multiple fans wore AR glasses. The system automatically created a shared AR space group for fans linked to the same concert logo. When any fan in the group waved their AR glasses, the color and direction of their virtual glow stick were simultaneously displayed in the AR view of all other fans. When the singer sang the chorus, a fan in the group posted a slogan, which was simultaneously displayed as an AR overlay in the view of all fans in the group. After the concert, the shared AR space was automatically destroyed, and all data was released from memory.

[0010] Example 3 (Sports Events - AR Information Overlay + Point-to-Point Calls) At a football match, fans are wearing AR glasses. The system automatically matches fans who support the same team. Fan A sees an AR floating label on the opposing fan B that reads "Supported Team: Manchester United" and "Fan Level: Gold Member." Fan A looks at the label and nods, establishing a real-time communication. Fan A says "Good shot!", which Fan B hears immediately. After the match, all data is destroyed.

[0011] Example 4 (Guest Mode - Virtual Waving Broadcast) At a music festival, a singer wore AR glasses. With a single click, the singer broadcast a virtual wave animation to all the fans' AR glasses, and all fans saw the singer waving at them in their AR view. The singer could also confirm fan interaction requests in batches, accepting friend requests from up to 50 fans at once.

[0012] Example 5 (UWB-assisted SLAM localization) In a dimly lit concert venue, visual SLAM feature points are insufficient. AR glasses use UWB ranging data to correct spatial position, ensuring the stability of virtual information anchoring. The AR floating tag of fan B seen by fan A will not drift due to changes in lighting.

[0013] Example 6 (Verification of self-destructing statements) User A sends a virtual glow stick waving message in a shared AR space group, which other members see in real time. After the group chat ends, no records are saved on any of the participants' devices. Technicians attempted to recover the data from the AR glasses' storage but found no remnants.

[0014] Other implementation details Depending on the hardware capabilities of the terminal device, virtual information may also include: the distance to the other party (e.g., "1.5 meters"), direction (e.g., "left front"), shared interests (e.g., "both like a certain actor"), entry time (e.g., "enter at 18:30"), seating area (e.g., "Area A, row 3"), etc. Attached Figure

[0015] Figure 1 This is a structural block diagram of the system of the present invention. The left side is an AR glasses terminal device, which internally includes a communication module, a SLAM spatial positioning module, an AR content rendering module, a real-time interaction module, a real-time destruction module, a matching module, an eye-tracking module, and an AR group interaction module; the right side is another AR glasses terminal device, which communicates directly with the left AR glasses through bidirectional arrows.

Claims

1. A proximity-based social interaction system based on augmented reality, characterized in that, The system includes several AR glasses terminal devices; each AR glasses has a built-in communication module, SLAM spatial positioning module, AR content rendering module, real-time interaction module, real-time destruction module, matching module, and eye-tracking module; the AR glasses are used to bind the user's currently participating activity identifier or interest identifier, and store the activity identifier or interest identifier locally; the communication module is used for AR glasses to obtain activity identifiers or interest identifiers of other AR glasses in the vicinity through short-range wireless communication technology; the matching module is used to compare the obtained activity identifiers or interest identifiers with the locally stored activity identifiers or interest identifiers, and automatically establish a matching relationship if they are the same; the SLAM spatial positioning module is used to track the position and posture of the AR glasses in three-dimensional space in real time, and anchor the virtual information to a specific location in the real world. The AR content rendering module is used to overlay and display the other party's virtual information in the real world after a successful match; the virtual information includes one or more of the following: nickname, fan level, identifier color, and virtual avatar; the eye-tracking module is used to track the user's gaze point in real time and detect whether the user is looking at the overlaid virtual information of the other party; the real-time interaction module is used to establish a real-time data interaction channel after the user looks at the other party's virtual information and completes the nod confirmation; the data sent by the user is collected, transmitted, and output to the other party's AR glasses in real time; The data is transmitted in a streaming manner and does not form a complete file; The real-time destruction module is used to ensure that all data generated during the interaction process only exists briefly in volatile memory during transmission; Once the transmission is complete, regardless of whether the recipient has received, viewed, or played the data, the data is immediately destroyed from memory and not written to any non-volatile storage medium.

2. The system according to claim 1, characterized in that, The virtual information also includes one or more of the following: distance to the other party, direction, common interests, entry time, and seating area; the virtual information is superimposed on the other party's real body in the form of a floating label and moves synchronously with the other party's movement.

3. The system according to claim 1, characterized in that, The eye-tracking module is also used to: highlight the virtual information when the user's gaze falls within the virtual information area of ​​the other party for a duration exceeding a preset threshold; and determine that the user's nodding action is a confirmation of consent when the user completes the nodding action while gazing.

4. The system according to claim 1, characterized in that, The system also includes an AR group interaction module, which is used to establish a shared AR space group interaction channel after multiple users are successfully matched; the shared AR space supports one or more of the following interactions: shared virtual glow stick linkage, synchronized display of slogan text, virtual gift sending and receiving, and group virtual special effects.

5. The system according to claim 4, characterized in that, The shared virtual glow stick linkage includes: when any user in the group waves their AR glasses, the color and swing direction of their virtual glow stick are synchronized to the AR display of all users in the group; the synchronized display of slogan text includes: when any user in the group sends out a slogan text, the slogan text is simultaneously displayed in the field of vision of all users in the group in the form of an AR overlay.

6. The system according to claim 1, characterized in that, The AR glasses also support a guest mode; the AR glasses worn by guests or organizers can broadcast virtual information or batch confirm user interaction requests to all AR glasses within a preset range; the broadcast virtual information includes one or more of the following: virtual wave, virtual signature, and virtual group photo frame.

7. The system according to claim 1, characterized in that, The SLAM spatial positioning module is also used to: combine UWB centimeter-level ranging technology to assist positioning; when visual SLAM feature points are insufficient, correct the spatial position of AR glasses through UWB ranging data to ensure the stability of virtual information anchoring.

8. The system according to claim 1, characterized in that, The short-range wireless communication technology includes one or more of Bluetooth, Wi-Fi Direct, UWB, NFC, ultrasound, and millimeter wave.

9. The system according to claim 1, characterized in that, The real-time destruction module is also used to: immediately clear the virtual information superimposed by the AR content rendering module from the display cache after the interaction is completed; and release all shared AR space data from the memory of each participant after the group interaction ends, without retaining any historical records.

10. An augmented reality-based method for proximity-based social interaction, applied to the system described in any one of claims 1-9, characterized in that, Includes the following steps: AR glasses are bound to the user's currently participating activity identifier or interest identifier, and the activity identifier or interest identifier is stored locally; AR glasses acquire activity or interest identifiers from other AR glasses in the vicinity via short-range wireless communication technology; they compare the acquired activity or interest identifiers with locally stored activity or interest identifiers, and automatically establish a matching relationship if they match; after successful matching, the AR glasses' position and posture are tracked in real time by the SLAM spatial positioning module; the AR content rendering module overlays virtual information of the other party onto the real world, including one or more of nickname, fan level, identifier color, and virtual avatar; the eye-tracking module tracks the user's gaze point in real time, detecting whether the user is looking at the overlaid virtual information; when the user is detected looking at the other party's virtual information and nodding in confirmation, a real-time data interaction channel is established; data sent by the user is collected, transmitted, and output to the other party's AR glasses in real time. Data is transmitted in a streaming manner and does not form a complete file; all data generated during the interaction only exists briefly in volatile memory during the transmission process. Once the transmission is complete, regardless of whether the recipient has received, viewed, or played the data, the data is immediately destroyed from memory and not written to any non-volatile storage medium.

Images