Cross-platform VRMR shopping data synchronization and payment system

Abstract

The invention discloses a cross-platform VRMR shopping data synchronization and payment system and method, and the system comprises a multi-terminal synchronization module which is used for the data interaction and scene switching of a cellphone AR terminal, a VR terminal and MR equipment, and comprises a data synchronization unit and a scene switching unit; the payment processing module is used for a user to carry out cross-platform payment operation, payment is supported to be completed at MR equipment, a VR end and a mobile phone AR end, and the payment processing module comprises a unified account unit and a security verification unit; and the disaster recovery processing module is used for automatically degrading the operation to the local mode of the mobile phone terminal or the MR equipment when the VR / MR equipment is disconnected from the network, and synchronizing the local cache data. According to the method, seamless connection from AR scanning to VR / MR deep experience is realized, operation interruption caused by state loss is avoided, multi-modal security verification of an MR equipment touch interface and a VR head-mounted display is supported, payment convenience and security are improved, and multi-terminal collaborative immersive experience is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of shopping payment, and in particular to a cross-platform VRMR shopping data synchronization and payment system. Background Art

[0002] With the rapid development of mixed reality (MR), virtual reality (VR), and augmented reality (AR) technologies, the retail industry is accelerating its transformation toward immersive, cross-platform shopping experiences. Consumers' shopping needs have evolved from simple commodity transactions to a 3D shopping experience characterized by seamless integration across multiple devices and real-time data synchronization. In line with this trend, cross-platform MR / VR shopping systems, by integrating offline physical experiences with online data resources to build a shopping ecosystem that "virtualizes physical scenes and makes virtual experiences real," have become a key direction for the retail industry's digital upgrade. For example, the introduction of MR touch terminals, VR headsets, and mobile AR devices in store settings allows consumers to switch freely between different devices, synchronizing product browsing, interactive operations, and shopping cart information in real time. This approach preserves the spatial perception advantages of offline scenarios while leveraging the data integration capabilities of online platforms, providing users with an efficient and convenient cross-platform shopping experience.

[0003] However, existing cross-platform shopping systems still face significant bottlenecks in both technical implementation and user experience. First, multi-terminal data synchronization is inefficient. Traditional solutions rely on centralized servers for data transfer, resulting in significant delays in interactions between mobile AR and VR devices, and data inconsistencies are particularly common in complex scenarios. Furthermore, cross-platform session switching logic is complex. When users switch from AR scanning to VR in-depth experience, they often experience interruptions or loss of interaction history due to lost session IDs or incomplete scene state synchronization. Furthermore, cross-platform compatibility in the payment process is insufficient. Existing systems struggle to adapt to the multimodal interactive features of MR devices. Security verification methods are limited, lacking convenient identity authentication methods in immersive VR scenarios, and weak disaster recovery capabilities for abnormal situations such as network outages, which can easily lead to transaction failures or data loss. Furthermore, existing solutions fail to fully utilize the spatial interaction advantages of MR devices and fail to achieve scene integration through multi-terminal collaboration (such as the virtual-reality linkage between MR large screens and VR headsets), limiting the depth and fluidity of the cross-platform shopping experience. Summary of the Invention

[0004] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of this application to avoid obscuring the purpose of this section, the abstract and the title of the invention, and such simplifications or omissions should not be used to limit the scope of the present invention.

[0005] In view of the above existing problems, the present invention is proposed. Therefore, the present invention provides a cross-platform VR / MR shopping data synchronization and payment system to solve the problems of multi-terminal data synchronization delay and session switching interruption in cross-platform MR / VR shopping scenarios.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a cross-platform VRMR shopping data synchronization and payment system, comprising:

[0008] A multi-terminal synchronization module is used for data interaction and scene switching between the mobile AR terminal, VR terminal and MR device. The multi-terminal synchronization module includes a data synchronization unit and a scene switching unit;

[0009] The data synchronization unit realizes real-time data synchronization between the mobile AR terminal, VR terminal and MR device through the WebRTC protocol, and supports dynamic update of shopping cart status, user location information, product browsing history and interactive operation history among the three terminals;

[0010] When a user enters VR / MR mode after scanning a product on a mobile AR or MR device through the scene switching unit, the user switches to the VR or MR device to continue browsing by maintaining the same session ID;

[0011] A payment processing module, which is used by users to perform cross-platform payment operations and supports payment on MR devices, VR terminals, and mobile AR terminals. The payment processing module includes a unified account unit and a security verification unit;

[0012] Performing facial recognition and calling an authorized payment interface through the unified account unit;

[0013] The security verification unit performs eye tracking verification or touch password input in the MR / VR scene to ensure payment security;

[0014] The disaster recovery processing module is used to automatically downgrade operations to the local mode of the mobile phone or MR device when the VR / MR device loses network connection, and synchronize local cached data.

[0015] As a preferred solution for the cross-platform VRMR shopping data synchronization and payment system described in the present invention, the real-time data includes the shopping cart status, the user's UWB positioning coordinates in the store, the browsing time of the product in the MR / VR scene, and the MR device touch operation record.

[0016] As a preferred solution of the cross-platform VRMR shopping data synchronization and payment system of the present invention, the data synchronization unit realizes real-time data synchronization between the mobile AR terminal, VR terminal and MR device through the WebRTC protocol, including:

[0017] The WebRTC protocol is used to establish point-to-point connections between the mobile AR terminal, VR terminal and MR device. The signaling server implements three-terminal session initialization and media negotiation based on the WebSocket protocol.

[0018] Encapsulates real-time data into JSON format packets and transmits them between the three terminals through the WebRTC data channel, supporting low-latency synchronization of touch operation commands of MR devices;

[0019] Adopting a version number mechanism and incremental update strategy, we prioritize synchronization of key data that affects visual interaction, targeting the high refresh rate screen characteristics of MR devices.

[0020] After the three ends establish a connection, they exchange heartbeat packets regularly to maintain the connection status. When the MR device is used as a fixed terminal in the store, it supports wired network priority connection to improve stability.

[0021] As a preferred solution of the cross-platform VR / MR shopping data synchronization and payment system of the present invention, when a user enters VR / MR mode after scanning a product on a mobile AR terminal or MR device through the scene switching unit, the user switches to the VR terminal or MR device to continue browsing by maintaining the same session ID, including:

[0022] When a user scans a product through an AR device on their mobile phone or a MR device in a store, a unique session ID is generated, which is bound to the user account, current product information, and the selected interactive terminal type;

[0023] When the user switches from the MR device to the VR side, the VR side obtains the session ID from the distributed cache after authentication. The MR device encodes the current scene state into a lightweight scene description file and transmits it to the VR side through the WebSocket channel;

[0024] Using a progressive loading strategy, the MR device first transmits basic product information and interaction history to the VR end. The VR end preloads general scene resources and then asynchronously supplements the transmission of MR-specific touch interaction data through WebRTC.

[0025] As a preferred solution of the cross-platform VRMR shopping data synchronization and payment system of the present invention, it also includes:

[0026] After the scene reconstruction is completed on the VR / MR side, the validity of the session ID is verified through a two-way authentication mechanism. When the MR device acts as a management terminal, it can actively push the updated scene data to the VR side.

[0027] During the session switching process, the mixed reality transition animation is displayed on the MR device or VR end, and the loading progress is displayed simultaneously

[0028] As a preferred solution of the cross-platform VRMR shopping data synchronization and payment system of the present invention, the payment processing module is used for users to perform cross-platform payment operations and supports payment completion on MR devices, VR terminals and mobile AR terminals. The payment processing module includes a unified account unit and a security verification unit, including:

[0029] The MR device's depth camera and eye tracking module are combined with blockchain evidence storage technology to verify the user's identity and generate a digital identity that is bound to the MR device.

[0030] When the user confirms payment on the touch screen of the MR device, the system calls the liveness detection engine to perform 3D face recognition and verifies whether the user is looking at the payment confirmation area through eye tracking.

[0031] After payment is completed, an interactive NFT digital invoice is generated on the MR device, which supports zooming in and out to view logistics information through gestures, or directly projecting the screen to the store's large screen for display.

[0032] As a preferred solution of the cross-platform VR / MR shopping data synchronization and payment system described in the present invention, the disaster recovery processing module is used to automatically downgrade the operation to the mobile phone or MR device local mode and synchronize the local cache data when the VR / MR device is disconnected from the network, including:

[0033] Deploy dual network modules (wired plus 5G) on the MR device side. When the primary network is disconnected, it automatically switches to the backup network. If both networks are disconnected, the local cache mode is activated to support offline browsing of loaded product 3D models.

[0034] When both VR / MR devices are disconnected from the network, the disaster recovery module uses BLE low-power Bluetooth to achieve temporary communication between the mobile phone and the MR device, prioritizing the synchronization of shopping cart and payment data.

[0035] After the network is restored, the MR device acts as a central node to coordinate the merging of data from the three ends, and uses the CRDT algorithm to resolve conflicts between MR touch operations and VR handle operations to ensure the consistency of interaction records.

[0036] In a second aspect, the present invention provides a method for synchronizing cross-platform VRMR shopping data and payment systems, comprising:

[0037] Users interact with data and switch scenes through a multi-terminal synchronization module, which includes a data synchronization unit and a scene switching unit. The data synchronization unit uses the WebRTC protocol to synchronize data between the mobile AR terminal, VR terminal, and MR device in real time, supporting dynamic updates of the shopping cart status, user location information, product browsing history, and interaction operation history among the three terminals. When a user enters VR / MR mode after scanning a product on the mobile AR terminal or MR device through the scene switching unit, they can switch to the VR terminal or MR device to continue browsing by maintaining the same session ID.

[0038] Cross-platform payment operations will be performed through the payment processing module, which supports payment on MR devices, VR terminals, and mobile AR terminals. The payment processing module includes a unified account unit and a security verification unit. The unified account unit performs facial recognition and calls the authorized payment interface. The security verification unit performs eye tracking verification or touch password input in the MR / VR scene to ensure payment security.

[0039] When the VR / MR device loses network connection, the operation will be automatically downgraded to the local mode of the mobile phone or MR device, and the local cached data will be synchronized.

[0040] In a third aspect, the present invention provides a computing device, comprising:

[0041] memory and processor;

[0042] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of synchronizing the cross-platform VRMR shopping data and the payment system are implemented.

[0043] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the cross-platform VRMR shopping data synchronization and payment system.

[0044] Compared with existing technologies, this invention offers the following advantages: It uses the WebRTC protocol to establish a point-to-point, real-time data synchronization channel between mobile AR, VR, and MR devices. It employs a signaling server and incremental update strategy to reduce data synchronization latency to milliseconds, ensuring dynamic consistency across multiple devices for key information such as shopping cart status and interactive operations. An innovative cross-platform scene switching mechanism based on a unique session ID, combined with distributed caching and progressive loading, enables a seamless transition from AR scanning to VR / MR in-depth experiences, avoiding operational interruptions caused by state loss. In the payment process, it integrates blockchain evidence storage, 3D facial recognition, and eye tracking technologies to support multimodal security authentication between the MR device touch interface and VR headset, improving payment convenience and security. The disaster recovery module utilizes dual network switching, BLE temporary communication, and the CRDT algorithm to ensure operational degradation and bidirectional data merging in the event of a network outage, ensuring transaction continuity. Furthermore, the interaction logic is optimized to address the characteristics of MR devices, such as prioritizing touch operation data synchronization and supporting virtual-to-real scene transition animations, enhancing the immersive multi-terminal collaborative experience and providing an efficient, secure, and smooth technical solution for cross-platform retail systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort. Among them:

[0046] Figure 1 Schematic diagram of a cross-platform VRMR shopping data synchronization and payment system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the following detailed description of the specific embodiments of the present invention is given in conjunction with the accompanying drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in this field without creative work should fall within the scope of protection of the present invention.

[0048] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0049] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0050] The present invention is described in detail with reference to schematic diagrams. For ease of illustration, cross-sectional views of device structures may be partially enlarged and not to scale when describing embodiments of the present invention. Furthermore, the schematic diagrams are merely illustrative and should not limit the scope of the present invention. Furthermore, in actual production, the three-dimensional dimensions of length, width, and depth should be included.

[0051] Furthermore, in the description of the present invention, it should be noted that the terms "upper, lower, inner, and outer" and other references to orientations or positional relationships are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the systems or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first, second, or third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0052] In this disclosure, unless otherwise specified or limited, the terms "mounted," "connected," and "connected" should be interpreted broadly. For example, they may refer to fixed, removable, or integral connections. They may also refer to mechanical, electrical, or direct connections, indirect connections through an intermediary, or internal communication between two components. Those skilled in the art will understand the specific meanings of these terms in this disclosure.

[0053] Example 1

[0054] Reference Figure 1 , as one embodiment of the present invention, provides a cross-platform VRMR shopping data synchronization and payment system, including:

[0055] A multi-terminal synchronization module is used for data interaction and scene switching between the mobile AR terminal, VR terminal and MR device. The multi-terminal synchronization module includes a data synchronization unit and a scene switching unit;

[0056] The data synchronization unit realizes real-time data synchronization between the mobile AR terminal, VR terminal and MR device through the WebRTC protocol, and supports dynamic update of shopping cart status, user location information, product browsing history and interactive operation history among the three terminals;

[0057] When a user enters VR / MR mode after scanning a product on a mobile AR or MR device through the scene switching unit, the user switches to the VR or MR device to continue browsing by maintaining the same session ID;

[0058] A payment processing module, which is used by users to perform cross-platform payment operations and supports payment on MR devices, VR terminals, and mobile AR terminals. The payment processing module includes a unified account unit and a security verification unit;

[0059] Performing facial recognition and calling an authorized payment interface through the unified account unit;

[0060] The security verification unit performs eye tracking verification or touch password input in the MR / VR scene to ensure payment security;

[0061] The disaster recovery processing module is used to automatically downgrade operations to the local mode of the mobile phone or MR device when the VR / MR device loses network connection, and synchronize local cached data.

[0062] Preferably, the real-time data includes shopping cart status, the user's UWB positioning coordinates in the store, the browsing time of the product in the MR / VR scene, and the MR device touch operation record.

[0063] Preferably, the data synchronization unit realizes real-time data synchronization between the mobile AR terminal, VR terminal and MR device through the WebRTC protocol, including:

[0064] The WebRTC protocol is used to establish point-to-point connections between the mobile AR terminal, VR terminal and MR device. The signaling server implements three-terminal session initialization and media negotiation based on the WebSocket protocol.

[0065] Encapsulates real-time data into JSON format packets and transmits them between the three terminals through the WebRTC data channel, supporting touch operation commands of MR devices, such as zooming and dragging 3D models with low-latency synchronization.

[0066] Adopting a version number mechanism and incremental update strategy, we prioritize synchronization of key data that influences visual interaction, such as model perspective and animation status, targeting the high refresh rate screen characteristics of MR devices.

[0067] After the three ends establish a connection, they exchange heartbeat packets regularly to maintain the connection status. When the MR device is used as a fixed terminal in the store, it supports wired network priority connection to improve stability.

[0068] Preferably, when the user enters VR / MR mode after scanning a product on the mobile AR terminal or MR device through the scene switching unit, the user switches to the VR terminal or MR device to continue browsing by maintaining the same session ID, including:

[0069] When a user scans a product through an AR phone or a MR device in a store, a unique session ID is generated, which is bound to the user account, current product information, and the selected interactive terminal type (AR / VR / MR);

[0070] When the user switches from the MR device to the VR end, the VR end obtains the session ID from the distributed cache after authentication. The MR device encodes the current scene state, such as the 3D model perspective and the marked product parameters, into a lightweight scene description file and transmits it to the VR end through the WebSocket channel.

[0071] Using a progressive loading strategy, the MR device first transmits basic product information and interaction history to the VR end. The VR end preloads common scene resources, such as the exhibition hall environment model, and then asynchronously supplements the transmission of MR-specific touch interaction data, such as annotation notes and operation trajectories, through WebRTC.

[0072] Furthermore, it also includes:

[0073] After the scene reconstruction is completed on the VR / MR side, the validity of the session ID is verified through a two-way authentication mechanism. When the MR device acts as a management terminal, it can actively push the updated scene data to the VR side.

[0074] During the session switching process, mixed reality transition animations are displayed on the MR device or VR terminal, such as virtual and real scene fusion effects, and loading progress is displayed simultaneously

[0075] Preferably, the payment processing module is used for users to perform cross-platform payment operations and supports payment on MR devices, VR terminals and mobile AR terminals. The payment processing module includes a unified account unit and a security verification unit, including:

[0076] The MR device's depth camera and eye tracking module are combined with blockchain evidence storage technology to verify the user's identity and generate a digital identity that is bound to the MR device.

[0077] When the user confirms payment on the touch screen of the MR device, the system calls the liveness detection engine to perform 3D face recognition and verifies whether the user is looking at the payment confirmation area through eye tracking.

[0078] After payment is completed, an interactive NFT digital invoice is generated on the MR device, which supports zooming in and out to view logistics information through gestures, or directly projecting the screen to the store's large screen for display.

[0079] Preferably, the disaster recovery processing module is used to automatically downgrade the operation to the mobile phone or MR device local mode and synchronize the local cache data when the VR / MR device is disconnected from the network, including:

[0080] Deploy dual network modules on the MR device side: wired + 5G. When the primary network is disconnected, it automatically switches to the backup network. If both networks are disconnected, the local cache mode is activated to support offline browsing of loaded product 3D models.

[0081] When both VR / MR devices are disconnected from the network, the disaster recovery module uses BLE low-power Bluetooth to achieve temporary communication between the mobile phone and the MR device, prioritizing the synchronization of shopping cart and payment data.

[0082] After the network is restored, the MR device acts as a central node to coordinate the merging of data from the three ends, and uses the CRDT algorithm to resolve conflicts between MR touch operations and VR handle operations to ensure the consistency of interaction records.

[0083] The cross-platform VR shopping data synchronization and payment method in this embodiment includes:

[0084] Users interact with data and switch scenes through a multi-terminal synchronization module, which includes a data synchronization unit and a scene switching unit. The data synchronization unit uses the WebRTC protocol to synchronize data between the mobile AR terminal, VR terminal, and MR device in real time, supporting dynamic updates of the shopping cart status, user location information, product browsing history, and interaction operation history among the three terminals. When a user enters VR / MR mode after scanning a product on the mobile AR terminal or MR device through the scene switching unit, they can switch to the VR terminal or MR device to continue browsing by maintaining the same session ID.

[0085] Cross-platform payment operations will be performed through the payment processing module, which supports payment on MR devices, VR terminals, and mobile AR terminals. The payment processing module includes a unified account unit and a security verification unit. The unified account unit performs facial recognition and calls the authorized payment interface. The security verification unit performs eye tracking verification or touch password input in the MR / VR scene to ensure payment security.

[0086] When the VR / MR device loses network connection, the operation will be automatically downgraded to the local mode of the mobile phone or MR device, and the local cached data will be synchronized.

[0087] It should be noted that this invention uses the WebRTC protocol to establish a point-to-point real-time data synchronization channel between mobile AR, VR, and MR devices. It employs a signaling server and incremental update strategy to reduce data synchronization latency to milliseconds, ensuring dynamic consistency across multiple devices for key information such as shopping cart status and interactive operations. An innovative cross-platform scene switching mechanism based on a unique session ID, combined with distributed caching and progressive loading, enables a seamless transition from AR scanning to a deep VR / MR experience, avoiding operational interruptions caused by state loss. In the payment process, blockchain evidence storage, 3D facial recognition, and eye tracking technologies are integrated to support multimodal security authentication between the MR device touch interface and the VR headset, enhancing payment convenience and security. The disaster recovery module uses dual network switching, BLE temporary communication, and the CRDT algorithm to ensure operational degradation and bidirectional data merging in the event of network outages, ensuring transaction continuity. Furthermore, the interaction logic is optimized to address the characteristics of MR devices, such as prioritizing the synchronization of touch operation data and supporting virtual-to-real scene transition animations, enhancing the immersive multi-terminal collaborative experience and providing an efficient, secure, and smooth technical solution for cross-platform retail systems.

[0088] This embodiment further provides a computing device suitable for cross-platform VRMR shopping data synchronization and payment system, including:

[0089] Memory and processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the cross-platform VRMR shopping data synchronization and payment system proposed in the above embodiment.

[0090] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the cross-platform VRMR shopping data synchronization and payment system proposed in the above embodiment.

[0091] The storage medium proposed in this embodiment and the cross-platform VRMR shopping data synchronization and payment system proposed in the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0092] Through the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented with the help of software and necessary general-purpose hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer's floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of various embodiments of the present invention.

[0093] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A cross-platform VRMR shopping data synchronization and payment system, characterized by: include: A multi-terminal synchronization module is used for data interaction and scene switching between the mobile AR terminal, VR terminal and MR device. The multi-terminal synchronization module includes a data synchronization unit and a scene switching unit; The data synchronization unit realizes real-time data synchronization between the mobile AR terminal, VR terminal and MR device through the WebRTC protocol, and supports dynamic update of shopping cart status, user location information, product browsing history and interactive operation history among the three terminals; When a user enters VR / MR mode after scanning a product on a mobile AR or MR device through the scene switching unit, the user switches to the VR or MR device to continue browsing by maintaining the same session ID; A payment processing module, which is used by users to perform cross-platform payment operations and supports payment on MR devices, VR terminals, and mobile AR terminals. The payment processing module includes a unified account unit and a security verification unit; Performing facial recognition and calling an authorized payment interface through the unified account unit; The security verification unit performs eye tracking verification or touch password input in the MR / VR scene to ensure payment security; The disaster recovery processing module is used to automatically downgrade operations to the local mode of the mobile phone or MR device and synchronize local cache data when the VR / MR device loses network connection.

2. The cross-platform VRMR shopping data synchronization and payment system according to claim 1, characterized in that: The real-time data includes shopping cart status, the user's UWB positioning coordinates in the store, the browsing time of the product in the MR / VR scene, and the MR device touch operation record.

3. The cross-platform VRMR shopping data synchronization and payment system according to claim 1 or 2, characterized in that: The data synchronization unit realizes real-time data synchronization between the mobile AR terminal, VR terminal and MR device through the WebRTC protocol, including: The WebRTC protocol is used to establish point-to-point connections between the mobile AR terminal, VR terminal and MR device. The signaling server implements three-terminal session initialization and media negotiation based on the WebSocket protocol. Encapsulates real-time data into JSON format packets and transmits them between the three terminals through the WebRTC data channel, supporting low-latency synchronization of touch operation commands of MR devices; Adopting a version number mechanism and incremental update strategy, we prioritize synchronization of key data that affects visual interaction, targeting the high refresh rate screen characteristics of MR devices. After the three ends establish a connection, they exchange heartbeat packets regularly to maintain the connection status. When the MR device is used as a fixed terminal in the store, it supports wired network priority connection to improve stability.

4. The cross-platform VRMR shopping data synchronization and payment system according to claim 3, characterized in that: When a user enters VR / MR mode after scanning a product on a mobile AR or MR device through the scene switching unit, the user switches to the VR or MR device to continue browsing by maintaining the same session ID, including: When a user scans a product through an AR device on their mobile phone or a MR device in a store, a unique session ID is generated, which is bound to the user account, current product information, and the selected interactive terminal type; When the user switches from the MR device to the VR side, the VR side obtains the session ID from the distributed cache after authentication. The MR device encodes the current scene state into a lightweight scene description file and transmits it to the VR side through the WebSocket channel; Using a progressive loading strategy, the MR device first transmits basic product information and interaction history to the VR end. The VR end preloads general scene resources and then asynchronously supplements the transmission of MR-specific touch interaction data through WebRTC.

5. The cross-platform VRMR shopping data synchronization and payment system according to claim 4, characterized in that: Also includes: After the scene reconstruction is completed on the VR / MR side, the validity of the session ID is verified through a two-way authentication mechanism. When the MR device acts as a management terminal, it can actively push the updated scene data to the VR side. During the session switching process, the mixed reality transition animation is displayed on the MR device or VR end, and the loading progress is displayed synchronously.

6. The cross-platform VRMR shopping data synchronization and payment system according to claim 5, characterized in that: The payment processing module is used for users to perform cross-platform payment operations and supports payment on MR devices, VR terminals, and mobile AR terminals. The payment processing module includes a unified account unit and a security verification unit, including: The MR device's depth camera and eye tracking module are combined with blockchain evidence storage technology to verify the user's identity and generate a digital identity that is bound to the MR device. When the user confirms payment on the touch screen of the MR device, the system calls the liveness detection engine to perform 3D face recognition and verifies whether the user is looking at the payment confirmation area through eye tracking. After payment is completed, an interactive NFT digital invoice is generated on the MR device, which supports zooming in and out to view logistics information through gestures, or directly projecting the screen to the store's large screen for display.

7. The cross-platform VRMR shopping data synchronization and payment system according to claim 6, characterized in that: The disaster recovery processing module is used to automatically downgrade operations to the mobile phone or MR device local mode when the VR / MR device loses network connection, and synchronize local cache data, including: Deploy dual network modules (wired plus 5G) on the MR device side. When the primary network is disconnected, it automatically switches to the backup network. If both networks are disconnected, the local cache mode is activated to support offline browsing of loaded product 3D models. When both VR / MR devices are disconnected from the network, the disaster recovery module uses BLE low-power Bluetooth to achieve temporary communication between the mobile phone and the MR device, prioritizing the synchronization of shopping cart and payment data. After the network is restored, the MR device acts as a central node to coordinate the merging of data from the three ends, and uses the CRDT algorithm to resolve conflicts between MR touch operations and VR handle operations to ensure the consistency of interaction records.

8. A method for cross-platform VRMR shopping data synchronization and payment system, characterized in that: include, Users interact with data and switch scenes through a multi-terminal synchronization module, which includes a data synchronization unit and a scene switching unit. The data synchronization unit uses the WebRTC protocol to achieve real-time data synchronization between the mobile AR terminal, VR terminal, and MR device, supporting dynamic updates of shopping cart status, user location information, product browsing history, and interactive operation records among the three terminals. When a user enters VR / MR mode after scanning a product on a mobile AR or MR device through the scene switching unit, the user switches to the VR or MR device to continue browsing by maintaining the same session ID; Cross-platform payment operations will be performed through the payment processing module, which supports payment on MR devices, VR terminals, and mobile AR terminals. The payment processing module includes a unified account unit and a security verification unit. The unified account unit performs facial recognition and calls the authorized payment interface. The security verification unit performs eye tracking verification or touch password input in the MR / VR scene to ensure payment security. When the VR / MR device loses network connection, the operation will be automatically downgraded to the local mode of the mobile phone or MR device, and the local cached data will be synchronized.

9. An electronic device comprising: memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the cross-platform VRMR shopping data synchronization and payment system as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the steps of the cross-platform VRMR shopping data synchronization and payment system according to any one of claims 1 to 7.

Images