Method for synchronously playing video content through augmented reality AR glasses and smart phone based on multi-device cooperation

Abstract

The invention discloses a method for synchronously playing video content by augmented reality AR glasses and a smart phone based on multi-device collaboration. The method specifically comprises the following five steps: 1, initializing and connecting devices; 2, video content transmission and display; 3, synchronizing user interaction and operation; 4, seamless switching between the devices supports user active or system automatic triggering switching, synchronous playing progress, volume and other parameters, and the target device directly jumps to synchronous progress playing without re-buffering; and 5, data cloud synchronization: uploading a user watching history, a collection list and preference setting to a cloud end, pushing and updating the user watching history, the collection list and the preference setting to associated equipment in real time, temporarily storing data in an offline state, and performing automatic synchronization after networking. Real-time parameter synchronization, seamless switching and full-device data consistency of the smart phone and the AR glasses are realized, the operation complexity is reduced, the fluency and immersive experience of cross-device video watching are improved, and the method is suitable for multiple scenes such as home entertainment and mobile watching.

Description

Technical Field

[0001] This invention relates to the fields of video playback technology, AR glasses technology, and multi-device collaboration technology, specifically a method for synchronously playing video content between augmented reality (AR) glasses and a smartphone based on multi-device collaboration. Background Technology

[0002] With the development of AR technology, AR glasses have become an important smart device for providing immersive experiences, but existing cross-device video playback solutions have the following problems: Inconsistent information synchronization: The video playback progress, volume and other parameters of the smartphone and AR glasses are not synchronized, requiring manual adjustment when switching devices, resulting in a poor user experience; High interaction complexity: The two types of devices have very different control methods (touch for mobile phones, gestures / voice for AR glasses), and users need to adapt to different operating logics, which increases the cost of use; Device switching delay: When switching playback devices, the video is prone to stuttering and interruptions, making it impossible to achieve a smooth transition. Summary of the Invention

[0003] The purpose of this invention is to provide a method for synchronously playing video content between augmented reality (AR) glasses and smartphones based on multi-device collaboration, aiming to: 1. Enables rapid connection and data synchronization between smartphones and AR glasses, maintaining the initial state consistency without manual configuration; 2. Ensure smooth transmission of video content between the two devices, and synchronize playback progress, volume, and other parameters in real time to avoid switching delays; 3. Unified cross-device operation logic: user control commands on any device can be instantly synchronized to another device, reducing operational complexity; 4. By synchronizing user data (viewing history, favorites, preferences) in the cloud, a consistent viewing experience can be achieved across multiple devices.

[0004] This invention provides a method for synchronously playing video content between augmented reality (AR) glasses and smartphones based on multi-device collaboration. This is achieved through five core steps: device connection, content transmission, operation synchronization, device switching, and data cloud synchronization. The system includes a user device module, a real-time synchronization module, a communication and data transmission module, a video playback management module, a user interaction and control module, and a backend service module. The specific process is as follows: (a) Step 1: Device initialization and connection Communication protocol selection: Smartphones and AR glasses establish a connection through wireless communication protocols such as Bluetooth, Wi-Fi or NFC. Wi-Fi is preferred to ensure high bandwidth transmission requirements, while Bluetooth is used as an auxiliary connection method to ensure basic communication in low power consumption scenarios. Application and data synchronization: Launch the Tianyi Ultra HD APP on both devices at the same time. The APP will automatically perform initial data synchronization with the backend server, synchronizing user account information and current viewing settings (such as picture quality and default volume) to both devices to ensure that the initial state is consistent. Connection status monitoring: The communication and data transmission module monitors connection stability in real time. If Wi-Fi connection is lost, it automatically switches to Bluetooth to maintain basic synchronization (only transmitting control commands and temporarily storing video segments). Complete data transmission will resume after Wi-Fi is restored.

[0005] (II) Step 2: Video content transmission and display Video preprocessing: The video content of the China Telecom Ultra HD platform is pre-segmented according to the HLS or DASH streaming media protocol to generate index files (HLS corresponds to .m3u8, DASH corresponds to .mpd), and the segment file format is .ts (HLS) or .mp4 (DASH). Transmission architecture: The smartphone is used as a local streaming media server, and video data is pushed to the AR glasses in the form of HTTP requests via Wi-Fi LAN; if in a public network environment, it can be relayed through the China Telecom Ultra HD backend server to ensure transmission stability across network scenarios. AR playback processing: The AR glasses parse the index file path through the built-in streaming media player module, load video segments as needed, and render the video content to the virtual display interface using hardware acceleration decoding. At the same time, the playback management module compares the playback timestamps of the two devices in real time and adjusts the playback rate (±0.1x speed fine-tuning) through the real-time synchronization module to ensure that the progress deviation is less than 100ms.

[0006] (III) Step 3: Synchronization of User Interaction and Operation Command capture and encapsulation: The user interaction and control module captures operation commands (play / pause, fast forward / rewind, volume adjustment, image quality switching) from any device and encapsulates the commands into standardized data packets, including operation type (such as "volume adjustment"), timestamp (accurate to milliseconds), parameter value (such as volume "50%)" and device identifier (such as "phone-device ID"). Command transmission and parsing: Data packets are transmitted to the target device via the communication module (direct transmission for local connections, relaying via the background service module for remote connections). The target device's interaction and control module parses the data packets and extracts key information. Synchronized execution of status: The video playback management module updates the local playback status based on the parsing results. For example, after adjusting the volume on the mobile phone, the AR glasses automatically synchronize the same volume value, and the status update delay between the two devices does not exceed 50ms, ensuring that the operation takes effect immediately.

[0007] (iv) Step 4: Seamless handover between devices Switching trigger methods: Supports two switching modes - user-initiated triggering (such as clicking the "Switch to AR Glasses" button in the APP) and system-initiated triggering (such as when the AR glasses detect the user wearing the glasses and automatically request to switch playback); Switching process: After the switch is triggered, the real-time synchronization module immediately obtains the progress timestamp, volume, and picture quality parameters of the current playback device, generates a switching data packet, and transmits it to the target device; the playback management module of the target device quickly loads the corresponding video segment based on the data packet and directly jumps to the synchronization progress to start playback without re-buffering; Post-switch state maintenance: After the switch is completed, the communication module continues to maintain parameter synchronization between the two devices. If the user performs an operation on the new device, it will still be synchronized to the original device according to the logic in step 3 to ensure state consistency when switching again.

[0008] (v) Step 5: Data cloud synchronization Synchronized data scope: The backend service module uploads the user's viewing history (such as recent viewing progress and viewing duration), favorites list (videos marked as "favorites"), and preference settings (such as default video quality, volume, and subtitle language) to the cloud database; Real-time synchronization mechanism: When a user updates data on any device (such as adding a favorite or modifying preferences), the APP immediately sends an update request to the cloud. After the cloud database is updated, a data update notification is pushed to all devices associated with the user (smartphones and AR glasses that are logged into the same account). Offline data processing: If the device is offline, the APP temporarily stores local data change records and automatically synchronizes them to the cloud after reconnecting to the network, ensuring that offline operations are not lost and data consistency is quickly restored after reconnection.

[0009] In summary, due to the adoption of the above-mentioned technologies, the beneficial effects of this invention are: 1. Synchronization optimization: Through real-time timestamp comparison and fast command transmission, the playback progress deviation between two devices is less than 100ms, and the synchronization delay of parameters such as volume and picture quality is less than 50ms, completely solving the problem of inconsistent parameters across devices.

[0010] 2. Improved smoothness of experience: No need to re-buffer when switching devices. Based on segmented preloading and precise progress jump, a "zero-latency" connection is achieved, avoiding lag or progress interruption.

[0011] 3. Reduced operational complexity: Unified cross-device operation logic allows users' gesture control on AR glasses and touch operation on mobile phones to be synchronized, eliminating the need to adapt to different operation methods.

[0012] 4. Data consistency guarantee: Cloud synchronization covers all user data. Whether switching devices or using cross-platform, historical viewing status and preference settings can be quickly restored, achieving a "seamless" full-scene experience. Attached Figure Description

[0013] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention, making other features, objects, and advantages of the invention more apparent. The illustrative embodiments of the invention illustrated in the drawings and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a diagram of the method of the present invention. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to represent selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] In the description of this invention, it should be understood that the terms indicating orientation or positional relationship are based on the orientation or positional relationship shown in the drawings and are only for the convenience of describing the invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention.

[0016] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific context of the specification.

[0017] This invention provides a method for synchronously playing video content between augmented reality (AR) glasses and a smartphone based on multi-device collaboration. Example 1: Switching from smartphone to AR glasses for playback Scenario: A user is watching a 4K movie on their smartphone through the China Telecom Ultra HD APP. Midway through the movie, they want to get an immersive experience by switching to AR glasses, triggering the "Switch to AR Glasses" operation. Implementation process: The mobile phone and AR glasses have established a connection via Wi-Fi. After the APP detects the switching command, it generates a switching data packet containing the current playback progress (such as "01:23:45"), volume (60%), and image quality (4K), and transmits it to the AR glasses via Wi-Fi. The AR glasses' player module parses the data packets, locates the corresponding .ts segment based on the .m3u8 index file, loads the video segment corresponding to "01:23:45", and uses hardware acceleration decoding for fast rendering. The switching is completed within 30ms. The AR glasses start playing from the target progress, and the volume is kept at 60%. During the playback, the user can adjust the volume to 70% through gestures on the AR glasses, and the volume on the mobile phone is updated to 70% in sync.

[0018] Example 2: Cross-device operation synchronization Scenario: A user is wearing AR glasses to watch a live sports event, with their phone placed aside. They need to pause the video to answer a phone call. Implementation process: When a user touches the phone screen and taps the "Pause" button, the mobile interaction module captures the command and generates a data packet containing "Operation Type: Pause, Timestamp: 16:45:30, Device Identifier: Mobile Phone"; The data packet is quickly transmitted to the AR glasses via Bluetooth, and the AR glasses immediately pause playback after parsing it. After answering the call, the user can use the voice command "continue playing" on the AR glasses. This command is then synchronized to the mobile phone, and both devices resume playback simultaneously, maintaining the same progress.

[0019] Example 3: Cloud Data Synchronization Scenario: A user saves a TV series to their favorites using AR glasses in a home Wi-Fi environment. The next day, while commuting, they use their smartphone to watch the series and need to quickly locate the saved content. Implementation process: When AR glasses collect TV series, the APP uploads the "collection identifier, video ID, and collection time" to the cloud database, and the cloud immediately pushes an update notification to the associated smartphone; The next day, when the user opens the China Telecom Ultra HD APP on their mobile phone, the APP automatically retrieves the latest collection list from the cloud and displays the TV series in the "My Collections" section. When a user clicks to play, the app synchronizes the user's recent viewing progress of the TV series from the cloud (if there is a previous viewing history), and jumps directly to the corresponding progress, achieving "seamless continuation of viewing across devices".

Claims

1. A method for synchronously playing video content by an augmented reality (AR) glasses and a smart phone based on multi-device collaboration, characterized in that: The method comprises the following steps: S1: device initialization and connection The smart phone and the AR glasses establish a connection through a Bluetooth, Wi-Fi or NFC wireless communication protocol, and simultaneously start the Tianyi Ultra HD APP on both devices, which synchronizes data with the background server to keep the initial settings and content of both devices consistent; S2: video content transmission and display The video content in the Tianyi Ultra HD APP is pre-fragmented according to the HLS or DASH streaming media protocol to generate corresponding index files, wherein the HLS protocol corresponds to.m3u8 index files and.ts fragment files, and the DASH protocol corresponds to.mpd index files and.mp4 fragment files; the smart phone is used as a local streaming media server to push video data in the form of an HTTP request to the AR glasses through a Wi-Fi network or a local area network, or through the Tianyi Ultra HD background server for transfer; The AR glasses parse the index file path through a built-in streaming media player module, load video fragments on demand, render streaming media content to a virtual display interface using hardware acceleration decoding function, and keep the video playback progress of the AR glasses and the smart phone synchronized; S3: user interaction and operation synchronization Capture the play, pause, volume adjustment or fast forward and rewind operations performed by the user on either device, and convert the operations into formatted instruction data packets containing operation type, timestamp, operation parameter and device identification; transmit the instruction data packets to the other device through a communication protocol or background service node, and the other device updates its own playback state after analyzing the instruction data packets; S4: seamless switching between devices Receive device switching instructions triggered by the user or automatically triggered by the system, obtain the video playback progress, volume and quality parameters of the current playback device, generate a switching data packet and transmit it to the target device; the target device loads the corresponding video fragments and starts playing from the playback progress in the switching data packet; S5: data cloud synchronization Upload the user's viewing history, collection list and preference settings to the cloud database through the Tianyi Ultra HD APP; when the user updates the data on either device, the cloud database updates and pushes data update notifications to all devices associated with the user; if the device is offline, temporarily store the local data change record, and automatically synchronize to the cloud when the device is reconnected.

2. The method for synchronously playing video content of an augmented reality (AR) glass and a smart phone based on multi-device cooperation according to claim 1, characterized in that, In step 1, Wi-Fi is preferred for connection to meet the bandwidth requirements of video transmission, and Bluetooth is used as an auxiliary connection method to maintain basic synchronization when Wi-Fi is disconnected; at the same time, the connection stability is detected in real time, and the video data transmission is automatically switched back to Wi-Fi when Wi-Fi is restored.

3. The method for synchronously playing video content of an augmented reality (AR) glass and a smart phone based on multi-device cooperation according to claim 1, characterized in that, In step 2, the AR glasses and the smart phone adjust the playback rate to achieve progress synchronization by comparing the playback timestamps in real time, and the playback progress deviation of both devices is less than 100 ms.

4. The method for synchronously playing video content of an augmented reality (AR) glass and a smart phone based on multi-device cooperation according to claim 1, characterized in that, In step 3, the transmission delay of the instruction data packet is not more than 50 ms, ensuring the immediacy of the operation state synchronization of both devices.

5. The method for playing video content synchronously by AR glasses and a smart phone based on multi-device cooperation according to claim 1, characterized in that, In step 4, the user initiates the switching by clicking the switching button in the Tianyi Ultra HD APP, and the system automatically triggers the switching when the AR glasses detect the user's wearing action; and the target device loads the video segments without rebuffering and directly jumps to the synchronized progress for playing.

6. The method for playing video content synchronously by AR glasses and a smart phone based on multi-device cooperation according to claim 1, characterized in that, In step 5, the user preference settings include default quality, default volume, and subtitle language; when the cloud database pushes an update notification to the user's associated device, it only pushes data formats that are compatible with the device type.

7. The method for playing video content in synchronization with a smart phone based on multi-device collaboration of the augmented reality (AR) glasses according to claim 1, wherein In step 2, when the smartphone and AR glasses are in a public network environment, the Tianyi Ultra HD background server is used to transfer video data, ensuring transmission stability in cross-network scenarios.

8. The method for playing video content in synchronization with a smart phone based on multi-device collaboration of the augmented reality (AR) glasses according to claim 1, wherein In step 3, the device identifier is a unique ID for the device, used to distinguish between the instruction initiating device and the instruction receiving device, preventing instruction transmission confusion.

9. The method for playing video content in synchronization with a smart phone based on multi-device collaboration of the augmented reality (AR) glasses according to claim 1, wherein In step 1, the data synchronized between the Tianyi Ultra HD APP and the background server includes user account information and current viewing settings, including default quality parameters and default volume parameters.

10. The method for synchronously playing video content of an augmented reality (AR) glass and a smart phone based on multi-device cooperation according to claim 1, wherein In step 2, when the AR glasses' built-in streaming player module loads video segments on demand, it prioritizes loading 1-2 video segments after the current playing progress, ensuring smooth playback.

Images