Wearable device and element universe fusion interaction system and method

Abstract

The invention discloses a wearable device and meta universe fusion interaction system and method. The method comprises the following steps: S10, collecting user data through a wearable device; s20, data processing and analysis: analyzing the data collected in the step S10, and generating personalized content through an analysis result; s30, interactive experience generation, wherein the personalized content generated in the step S20 is applied to the meta universe environment; and S40, performing feedback circulation, and optimizing the system by analyzing the user experience data. The system comprises a server, an integrated software platform, a meta universe platform, a wearable device and a sensor network. The server completes information interaction between the user and the meta universe platform through the wearable device; the integrated software platform supports the API butt joint with the meta universe platform; digital display is constructed on the meta universe platform; a wearable device and a sensor network capture actions and physiological responses of a user. Technical support and innovative thoughts are provided for development of the intelligent exhibition hall, and the exhibition industry is promoted to develop in the more intelligent, interactive and immersive direction.

Description

Technical Field

[0001] The present invention relates to the field of mixed reality, and in particular to a wearable device and metaverse fusion interaction system and method. Background Art

[0002] This section merely provides background information related to the present disclosure and is not necessarily prior art.

[0003] At present, the exhibition hall field is actively trying to integrate wearable technology and mixed reality (MR) to enhance the interactive experience of visitors, but the actual results are still insufficient. The current experience is mostly superficial, lacking depth and personalization. The functions of smart devices such as smart watches and AR glasses are limited to basic information push and navigation, and fail to fully tap the immersive potential of the metaverse. Although data collection has started, analysis and application are still in the early stages, and it is not possible to provide a customized experience based on the collected physiological indicators and behavioral patterns. The personalized service and content recommendation effects are limited.

[0004] In terms of technology integration, the combination of wearable devices and the metaverse is still immature. There is a lack of an efficient and unified technology platform to integrate various advanced technologies, which leads to low efficiency in coordination between devices and affects the smoothness and continuity of the interactive experience. The exhibition hall design focuses too much on technological innovation and ignores the overall user experience. The interface friendliness and ease of operation are often ignored, making it difficult to popularize high-tech equipment and greatly reducing its efficiency.

[0005] In the modernization process of smart exhibition halls, although wearable devices and metaverse technology have shown amazing potential in specific fields, their single applications have also exposed certain limitations. Wearable devices are known for their portability and personalized data collection capabilities, but they are often limited by the limited computing resources and data island problems of the devices, making it difficult to achieve in-depth data analysis and cross-platform applications. Although metaverse technology has built a grand and rich virtual digital space, it still has technical bottlenecks and experience gaps in achieving seamless connection between the physical world and the virtual world, and enhancing the realism and immersion of the virtual experience.

[0006] Therefore, how to find a balance between technological innovation and user experience will be an urgent problem to be solved in future exhibition hall design. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide a wearable device and metaverse fusion interaction system and method in view of the existing deficiencies.

[0008] A wearable device and metaverse fusion interaction method, comprising the following steps:

[0009] S10: Data collection, collecting user data through wearable devices.

[0010] S20: Data processing and analysis, analyzing the data collected in S10, and generating personalized content based on the analysis results.

[0011] S30: Interactive experience generation, applying the personalized content generated by S20 to the metaverse environment.

[0012] S40: Feedback loop, optimizing the system by analyzing user experience data.

[0013] Furthermore, the S20 data processing and analysis includes:

[0014] S201: Fusing the data through a data fusion algorithm and analyzing the user behavior pattern of the fused data. The data fusion algorithm integrates the data collected by the wearable device with the exhibition hall environment data to analyze the user behavior pattern, including the user's action path in the exhibition hall, the length of stay, the degree of attention paid to specific exhibits, etc.

[0015] S202: Build a deep learning model based on the analysis results. Train the model based on the user's historical behavior data so that it can learn and predict user preferences.

[0016] S203: Analyze user behavior preferences and historical interaction data based on deep learning models to make dynamic content recommendations. Achieve highly personalized content display, including identifying topics of interest to users and pushing relevant digital display content.

[0017] Furthermore, the data fusion formula in S201 is as follows:

[0018]

[0019] Among them, y is the fused data, x i is the data of the ith sensor, w i is the corresponding weight, satisfying The weights are determined based on factors such as the sensor's accuracy, reliability, and historical performance.

[0020] Furthermore, the data weight determination in the data fusion in S201 includes:

[0021] S2011: Initialize weights, assign an initial weight to each sensor based on the theoretical accuracy of the sensor;

[0022] S2012: Historical data analysis: analyze sensor data over a period of time, evaluate the reliability of sensors based on the consistency between the analyzed data and the actual situation, and adjust the weights based on the evaluation results;

[0023] S2013: User feedback, indirectly evaluate the effect of the sensor through user feedback and adjust the weight;

[0024] S2014: Dynamic adjustment, regularly re-evaluate the performance of sensors and dynamically adjust the weights based on the evaluation results.

[0025] Furthermore, the step S202 of building a deep learning model includes:

[0026] S2021: Customize and design the model network structure according to the application scenario and data characteristics. The network structure includes the number of layers, number of neurons, activation function, etc. to optimize the model's expressiveness and generalization performance.

[0027] S2022: Train the model and adjust the model parameters through the back propagation algorithm. Use a large amount of user behavior data to train the model and adjust the model parameters through the back propagation algorithm to minimize the gap between the predicted value and the true value.

[0028] S2023: Measure the prediction accuracy of the model through a loss function, and iteratively update the model parameters through an optimization algorithm. The optimization algorithm may be a gradient descent method.

[0029] S2024: Regularly evaluate model performance, including indicators such as accuracy and recall, to ensure the effectiveness of the model. Optimize and adjust the model based on the evaluation results, such as increasing the number of network layers, changing the activation function, etc., to improve the model's predictive performance and generalization ability. Continuously explore new deep learning technologies and algorithms to adapt to changing market demands and user preferences.

[0030] Furthermore, the S30 interactive experience generation includes:

[0031] S301: updating the virtual tour path in real time according to the user's location information, including generating a personalized tour route in the metaverse environment, so that the user can visit the exhibits in the order of interest.

[0032] S302: By building a system compatible with multiple types of wearable devices, we integrate advanced mixed reality (MR) and augmented reality (AR) technologies. Combined with the perception capabilities of wearable devices, we create a highly immersive smart experience scene and achieve seamless integration with the digital environment of the Metaverse.

[0033] S303: interacting with users by driving digital avatars and digital exhibits, including enabling users to touch digital exhibits to obtain detailed information through gesture recognition, or interacting with users through voice recognition to enhance the naturalness and fluency of the interactive experience.

[0034] Further, the S40 feedback loop includes:

[0035] S401: Establish a multi-channel feedback mechanism to identify user needs and pain points. Ensure that the system can continuously adapt to user needs and provide a better user experience by continuously collecting user feedback and optimizing system performance and content, including online questionnaires, real-time voting, etc., to collect user evaluations of exhibition content and interactive experience.

[0036] S402: Use A / B testing and iterative design to adjust display content and interaction methods based on user feedback, including replacing unpopular exhibits, optimizing user interface design, and improving user experience.

[0037] S403: Monitoring and Optimization: Continuously monitor user feedback and system performance in each iteration to optimize the design.

[0038] A wearable device and metaverse fusion interaction system, comprising a server, an integrated software platform, a metaverse platform, a wearable device, and a sensor network;

[0039] The server completes information interaction between the user and the Metaverse platform through wearable devices.

[0040] The integrated software platform supports API docking with the Metaverse platform.

[0041] The Metaverse platform builds a digital display.

[0042] The wearable device and sensor network capture the user's movements and physiological responses.

[0043] Furthermore, the integrated software platform includes a data acquisition module, a content recommendation system, a data processing and analysis module, and a user interface design module; the data acquisition module collects user behavior data, the data processing and analysis module analyzes user behavior patterns through the collected data, and the content recommendation system trains a deep learning model based on the analysis results to recommend user content.

[0044] Furthermore, the metaverse platform includes a digital scene construction module and a digital character construction module; the digital scene construction module constructs a metaverse digital scene, and the scene includes an exhibition hall, an exhibit showroom, and showroom lights; the digital character component module processes user data captured by wearable devices, and the digital avatar in the metaverse environment synchronously updates personalized information.

[0045] The process of integrating wearable devices and Metaverse interactive systems and methods involves interactive ecosystems and data flow and information interaction mechanisms, aiming to ensure real-time data exchange and accurate transmission of instructions. By integrating wearable devices and Metaverse technology, the linkage between devices is significantly enhanced, providing a new form of digital, virtual, and personalized display for smart exhibition halls. As a result, user interaction in the exhibition hall becomes smoother and more coherent, improving the overall experience; it provides technical support and innovative ideas for the development of smart exhibition halls, and promotes the exhibition industry to develop in a more intelligent, interactive, and immersive direction.

[0046] The interactive ecosystem looks like this:

[0047] The array of sensors on wearable devices is the key to connecting the real world with the digital world. These include but are not limited to accelerometers, gyroscopes, magnetometers, heart rate monitors, temperature sensors, cameras, and ambient light sensors. These sensors can continuously monitor the user's movements, postures, physiological states, and environmental changes, convert the collected physical world information into digital signals, and provide real-time data input for the digital environment. For example, accelerometers and gyroscopes can track the user's movement trajectory, heart rate monitors can reflect the user's emotional state, and ambient light sensors help adjust the lighting conditions of the digital environment to achieve synchronization with the real environment.

[0048] To ensure real-time data transmission, wearable devices are equipped with a range of wireless communication technologies, including Bluetooth, Wi-Fi, 5G / 6G networks, and even ultra-wideband (UWB) technology. These modules not only enable high-speed, low-latency connections between devices and Metaverse servers, but also ensure stable communication with other wearable devices or Internet of Things (IoT) devices, building a seamless network ecosystem. For example, 5G and future 6G networks provide extremely high data transmission rates and lower latency, making large-scale multiplayer online interactions possible and enhancing the immersion of virtual reality.

[0049] The user interface is the window through which wearable devices communicate with users, and its design is directly related to the quality of the user experience. Whether it is the compact display of a smart watch, the transparent display of smart glasses, or the panoramic view of a mixed reality helmet, these interfaces need to be intuitive and easy to use, allowing users to switch freely between the real and digital worlds. In addition, the integration of natural user interfaces (NUIs) such as touch, gesture control, and voice commands enables users to interact with virtual environments in a more natural and intuitive way, enhancing the authenticity of the immersive experience.

[0050] The operation of the Metaverse relies on powerful computing and data processing capabilities, and back-end servers and cloud computing platforms bear this heavy responsibility. They not only receive massive amounts of data from wearable devices, but are also responsible for real-time processing, analysis, and storage of this data, dynamically adjusting the digital environment based on the user's actions and behaviors, and ensuring a high degree of real-time and interactivity. For example, cloud computing platforms can use distributed computing and storage resources to handle complex data analysis tasks such as emotion recognition, behavior prediction, and personalized content recommendations, providing users with customized digital experiences.

[0051] At this point, the interface design of the wearable device and the Metaverse Exhibition Hall fusion technology has built a complete chain integrating data collection, transmission, processing and feedback.

[0052] The data flow and information interaction mechanism are as follows:

[0053] At the source of the data stream, the sensor array on the wearable device continuously monitors the user's physiological state and environmental changes, capturing information such as heart rate, number of steps, location coordinates, ambient light intensity, etc. These raw data are pre-processed on the device's microprocessor, including data cleaning, denoising, format conversion, and feature extraction to ensure the accuracy and validity of the data. The pre-processing step can significantly reduce the bandwidth requirements for data transmission, while also reducing the computing burden on the back-end server, making the data flow more efficient.

[0054] Efficient and reliable data transmission protocols such as MQTT (Message Queuing Telemetry Transport) and CoAP (Constrained Application Protocol) are used to ensure the secure and fast transmission of data between wearable devices and the Metaverse platform. These protocols support data transmission in low-power and low-bandwidth environments, which are suitable for the use scenarios of wearable devices. At the same time, they provide data encryption and authentication functions to enhance the security of data transmission.

[0055] After receiving the data, the Metaverse platform conducts in-depth analysis, integrating the data from wearable devices with data from other sources (such as social media activities, historical behavioral data, etc.) to build a comprehensive user status model. This model includes not only the user's real-time physiological and behavioral data, but also the user's historical preferences, social networks, and environmental background information, laying the foundation for the provision of personalized services.

[0056] The core advantage of the Metaverse platform lies in its real-time feedback capability. Based on the parsed user state model, the platform can dynamically adjust the digital environment, such as updating the lighting and sound effects in the digital scene, or adjusting the interactive exhibit experience rules to match the user's actions and decisions. At the same time, the platform will also send feedback information to wearable devices, such as tactile feedback (vibration prompts), visual updates (such as augmented reality information in smart glasses) or auditory instructions, forming a closed-loop interaction process. This instant feedback mechanism not only enhances the realism and immersion of the user experience, but also enables users to interact deeply with the digital world and feel that they are part of the digital world, not just a bystander.

[0057] Security and privacy protection are an integral part of the entire data flow and information interaction mechanism. We use advanced encryption technology and privacy protection measures, such as data desensitization, anonymization, and differential privacy, to ensure the security of user data during transmission and processing, while complying with relevant laws and regulations to protect user privacy.

[0058] The present invention can accurately capture the user's movements and physiological reactions by integrating high-precision sensors and advanced data processing algorithms, thereby achieving more natural and in-depth interactions in the metaverse environment, and the personalized recommendation algorithm ensures that each user can obtain a customized exhibition experience. The effective use of the data fusion algorithm enables the data from wearable devices and sensor networks to be deeply integrated and analyzed, which not only improves the utilization value of the data, but also provides data support for accurate user behavior analysis and personalized services. By building a unified and efficient platform, the wearable device and the metaverse technology are integrated, the linkage between devices is significantly enhanced, and the innovative integration of wearable devices and the metaverse technology is realized, providing a new form of digital, virtualized and personalized display for the smart exhibition hall, so that the user interaction in the exhibition hall becomes smoother and more coherent, and the overall experience is improved. Through continuous user feedback collection and system optimization, the continuous improvement of user experience is ensured, and the careful consideration of interface friendliness and ease of operation makes high-tech equipment easier to be accepted and effectively used by the public. Through personalized content recommendation, the interactivity and user satisfaction of the exhibition are improved, and the diversification and in-depth of the exhibition content are promoted. The present invention brings users a richer, more personalized and more natural interactive experience of smart exhibition halls through the dual improvement of technological innovation and user experience; it provides technical support and innovative ideas for the development of smart exhibition halls, and promotes the exhibition and display industry to develop in a more intelligent, interactive and immersive direction. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0060] Figure 1 It is a structural block diagram of a wearable device and metaverse fusion interaction system of the present invention.

[0061] Figure 2 It is a flow chart of a wearable device and metaverse fusion interaction method of the present invention.

[0062] Figure 3 It is the virtual tour path generated by the present invention.

[0063] Figure 4 This is an effect diagram of the interaction between the user and the exhibits of the present invention. DETAILED DESCRIPTION

[0064] See also Figure 1 As an embodiment of the present invention, the digital scene construction module creates a metaverse digital scene through a digital display tool, wherein the metaverse digital scene includes an exhibition hall, an exhibit showroom, and showroom lighting.

[0065] like Figure 1 As shown in the figure, it shows the interaction between the Metaverse platform and the server and how the server communicates with wearable devices and sensor networks. The following is a detailed explanation of several key modules involved in the figure:

[0066] Browsing Path Module: This module is responsible for tracking and analyzing the user's movement path in the digital scene. It receives data from high-precision sensors, which may include location coordinates, direction, speed and other information. By processing and analyzing these data, the Browsing Path Module can understand the user's behavior patterns in the digital world, thereby helping to improve the design and navigation functions of the digital scene.

[0067] Exhibit recommendation module: This module is mainly responsible for recommending relevant exhibits based on the user's interests and behavior patterns. It combines the results of the data fusion algorithm and the output of the personalized recommendation algorithm to provide users with customized visit suggestions. By analyzing the user's preferences and behaviors, the exhibit recommendation module can intelligently push relevant content to enhance the user's sense of participation and satisfaction.

[0068] User Interface Design Module: This module is responsible for creating and maintaining an intuitive and easy-to-use user interface that allows users to easily operate and interact in a digital environment. The user interface design module should take into account user needs and feedback, provide a friendly graphical interface, easy-to-understand operation guides, and necessary prompts and help information. In addition, it should support multiple devices and operating systems to meet the needs of different user groups.

[0069] Information transmission path: The data flow of the entire system starts from the wearable device, which collects the user's behavior data and environmental data through high-precision sensors. These data are sent to the server, and after being cleaned, integrated and analyzed by the data processing / analysis module, the content recommendation engine generates personalized recommended content. At the same time, the digital scene component module and the digital character component module also send data to the server, which will be used to improve the experience of the digital world. Finally, the server will return the processed data and recommended content to the user interface design module and display it to the user. Users can provide their opinions and suggestions through the user feedback channel, and these feedbacks will be used to continuously improve the system.

[0070] Through the collaborative work of the above modules and information transmission paths, the Metaverse platform is able to provide a highly immersive and personalized virtual reality experience to meet the different needs of users.

[0071] In the above scheme, by determining the theme, layout and style of the Metaverse digital scene, including the design of the exhibition hall and the exhibition showroom, the various elements in the scene, such as walls, floors, ceilings, and exhibit placement, are planned according to the design requirements. 3D modeling software is used to create various elements in the digital scene, including the architectural structure of the exhibition hall, exhibit models, etc. The built 3D models are textured and rendered to ensure the realism and delicacy of the scene's visual effects; the lighting layout in the showroom is designed to achieve the ideal lighting effect and atmosphere, and the brightness, color and projection angle of the light are adjusted to highlight the characteristics of the exhibits while ensuring the overall visual comfort of the scene.

[0072] Set up interaction points in the digital scene to allow users to interact with exhibits by clicking, dragging, etc., realize the navigation function within the scene, help users easily browse different exhibition halls and showrooms, optimize the digital scene to ensure that it can run smoothly on different devices, conduct multiple tests, check for bugs in the scene and fix them, improve user experience, integrate the produced digital scene into the Metaverse platform, and publish the digital scene for users to explore and interact in the Metaverse.

[0073] Through the above steps, the digital scene construction module can use digital display tools to create realistic and interactive metaverse digital scenes, including exhibition halls, exhibit showrooms, and lighting designs in showrooms, which not only enriches the user's visual experience, but also enhances the user's immersion and participation in the metaverse.

[0074] See also Figure 1As an embodiment of the present invention, a wearable device has a built-in high-precision sensor for capturing user movements and physiological data; the wearable device includes a smart watch, AR glasses or smart control gloves, wherein the high-precision sensor includes a gyroscope, an accelerometer or a biometric sensor; the user interface of the wearable device supports touch, voice or gesture control, wherein the wearable device captures user behavior through sensor technology and synchronizes it to the metaverse environment in real time through a digital character component module; the interaction between the wearable device and the metaverse environment enables the user's digital avatar to experience and finely operate virtual exhibits.

[0075] In the above-mentioned scheme, the wearable device has built-in high-precision sensors, including gyroscopes, accelerometers and biometric sensors, which can accurately capture the user's movements, such as gestures, body postures, and physiological data, such as heart rate, blood pressure, etc.; the user interface design of the wearable device supports touch, voice or gesture control, and provides a variety of interactive means. The user can operate the device by touching virtual buttons, issuing voice commands or making specific gestures; the wearable device captures the user's behavior data in real time through sensor technology. These data are processed by the digital character component module on the server and synchronized to the metaverse environment in real time. In the metaverse, the user's digital avatar will update personalized information based on the synchronized data to reflect the user's real movements and emotions; due to the real-time interaction between the wearable device and the metaverse environment, the user's digital avatar can accurately calculate the user's real movements. This technology enhances the user's immersion and participation in the metaverse, making the digital experience more real and vivid, thereby enhancing the user's interactive experience and immersion in the metaverse.

[0076] See also Figure 1 As an embodiment of the present invention, the data fusion algorithm is a key component of the data processing and analysis stage. Its main task is to comprehensively process the data from different sensors to generate more accurate and reliable fused data. This process includes multiple steps such as data cleaning, integration, and user behavior pattern analysis.

[0077] Specific steps:

[0078] Data cleaning: First, remove duplicate, invalid or abnormal data to ensure the accuracy and reliability of the data. For missing data, interpolation, estimation or deletion methods are used to process it.

[0079] Data integration: Integrate user data collected by wearable devices (such as heart rate, number of steps, activity type, etc.) with exhibition hall environment data (such as temperature, humidity, light, etc.) to ensure that data from different sources are consistent in time and space for subsequent analysis.

[0080] Data fusion: The weighted average method is used to fuse the integrated data. This method can effectively integrate the data of multiple sensors to obtain more accurate fused data. The formula is as follows:

[0081] in

[0082] Among them, y is the fused data, x i is the data of the ith sensor, w i are the corresponding weights, and the sum of all weights is equal to 1.

[0083] Weight determination: weight w i The determination of is based on factors such as the accuracy, reliability, and historical performance of the sensor. In specific implementation, an initial weight value can be set in advance and dynamically adjusted according to the actual performance of the sensor in use. For example, if a sensor has performed stably and the data is accurate over a period of time, its weight value can be appropriately increased; otherwise, its weight value can be reduced.

[0084] User behavior pattern analysis: Utilize the integrated data to analyze the user's movement path in the exhibition hall, the length of stay, the degree of attention paid to specific exhibits, etc. Through pattern recognition technology, extract the characteristics of user behavior, such as preferences and habits.

[0085] In order to ensure the effectiveness of the data fusion algorithm, the following optimization designs are also required:

[0086] Dynamic weight adjustment mechanism: Dynamically adjust the weight value according to the actual performance of the sensor to ensure that the accuracy of the fused data is not affected by the poor performance of individual sensors.

[0087] Anomaly detection and processing: An anomaly detection mechanism is added in the data cleaning stage. Once abnormal data is found, it is immediately isolated and processed to prevent it from interfering with the final fusion result.

[0088] Data quality monitoring: Regularly evaluate the data quality of the sensors and adjust the weight allocation strategy in the fusion algorithm accordingly.

[0089] Through the above steps, the data fusion algorithm can effectively integrate data from different sensors and generate high-quality fused data, thereby providing a solid foundation for subsequent user behavior analysis and personalized content recommendation.

[0090] In the above-described scheme, wearable devices are used to collect the user's motion, physiological and other data, including the user's heart rate, number of steps, activity type, etc.; at the same time, the sensor network deployed in the exhibition hall is used to sense and collect environmental data in the exhibition hall, such as temperature, humidity, light intensity, etc.; the collected raw data is cleaned to remove outliers, noise and duplicate data to ensure data quality; the data is standardized to make the data collected by different sensors comparable.

[0091] The weighted average method is used as the data fusion algorithm, which can effectively integrate the data of multiple sensors. According to the formula:

[0092]

[0093] Integrate data from multiple sensors to obtain more accurate fused data.

[0094] In the smart exhibition hall project, the following sensors are used to collect data:

[0095] Smartwatch: used to collect physiological and sports data such as heart rate and number of steps of the user. Temperature sensor: deployed inside the exhibition hall to measure the temperature inside the exhibition hall. Humidity sensor: also deployed inside the exhibition hall to measure the humidity level. Light sensor: used to measure the light intensity in the exhibition hall.

[0096] Multiply the data of each sensor by the corresponding weight and sum them to get the fused data; use the fused data in combination with user behavior analysis models (such as event analysis, funnel analysis, user path analysis, etc.) to conduct in-depth analysis of user behavior patterns. Through analysis, we can understand the user's activity habits, preferences and possible needs in the exhibition hall, and provide a basis for optimizing the exhibition hall layout and improving user experience; according to the analysis results, adjust the exhibition hall's operating strategy, such as optimizing the placement of exhibits, improving environmental comfort, etc., to improve user satisfaction and visiting experience.

[0097] See also Figure 1 As an embodiment of the present invention, the personalized recommendation algorithm is based on a deep learning model, and dynamically adjusts and recommends exhibition content according to user behavior preferences. In personalized content recommendation, commonly used loss functions include cross entropy loss and mean square error. In this system, cross entropy loss is used as the main loss function, especially when dealing with binary classification problems. Its formula is as follows:

[0098]

[0099] Among them, y i is the true label of the i-th sample, is the predicted probability of the model for the i-th sample, and N is the total number of samples.

[0100] The reason for choosing cross entropy loss as the loss function is that it is suitable for classification problems, especially binary classification problems. In our application scenario, we need to distinguish the user's interest in certain exhibition content, that is, whether the user will click or pay attention to a certain content. Cross entropy loss can effectively measure the difference between the model's predicted probability distribution and the true label, and make the probability distribution of the model output as close to the actual label as possible.

[0101] like Figure 2 As shown, the steps of this method are as follows:

[0102] S10: Data collection, collect user behavior data, including browsing history, click behavior, activity type, etc. These data will be used as input for the deep learning model.

[0103] S20: Data processing and analysis, data preprocessing, by preprocessing the data, such as cleaning, deduplication, standardization, etc., to ensure the quality and consistency of the data. Model selection and design, select a suitable deep learning model, such as convolutional neural network (CNN) or recurrent neural network (RNN), and customize it according to the specific application scenario and data characteristics. Design the network structure of the model, including the number of layers, number of neurons, activation function, etc., to optimize the model's expressiveness and generalization performance; model training, use a large amount of user behavior data to train the model, and adjust the model parameters through the back propagation algorithm to minimize the gap between the predicted value and the true value; loss function optimization, during the training process, use the cross entropy loss function to measure the accuracy of the model's prediction, and continuously iterate and update the model parameters through optimization algorithms (such as gradient descent) to improve the model's prediction accuracy.

[0104] S30: Interactive experience generation recommendation strategy adjustment: Based on the trained deep learning model, analyze and predict the user's behavior preferences, and dynamically adjust and recommend exhibition content according to the user's personalized needs. For example, if the user often browses a certain type of exhibition or artwork, the model will recommend more related content.

[0105] S40: Feedback loop, real-time feedback and adjustment, real-time tracking of user feedback and behavior changes, and continuous adjustment of recommendation strategies to improve user satisfaction and engagement; Model performance evaluation and optimization: Regularly evaluate the performance of the model, including indicators such as accuracy and recall, to ensure the effectiveness of the model. Optimize and adjust the model based on the evaluation results, such as increasing the number of network layers, changing the activation function, etc., to improve the model's predictive performance and generalization ability.

[0106] Through the above steps, an efficient personalized recommendation system is built to continuously adapt to the changing market demands and user preferences, providing users with a more satisfactory and personalized exhibition experience.

[0107] See also Figure 1 As an embodiment of the present invention, the server also includes a user feedback channel for collecting users' evaluations of exhibition content and interactive experience, wherein the server can optimize the user experience by using A / B testing and iterative design based on the feedback information.

[0108] In the above-mentioned scheme, by setting up multiple user feedback channels, such as online survey forms, user comment areas, email feedback, etc., it is ensured that users can conveniently provide evaluations of exhibition content and interactive experience. These channels are regularly monitored and managed to ensure timely collection and response of user feedback; the collected user feedback is classified and sorted to identify user satisfaction, problems and suggestions for exhibition content and interactive experience; data analysis tools are used to conduct in-depth analysis of user feedback to discover the commonalities and differences in user needs, as well as potential problems and improvement points; based on user feedback and analysis results, the optimization points that need to be A / B tested are determined, such as page layout, interactive functions, etc., and A / B testing plans are designed and implemented to divide users into different groups. Display the original version and the optimized version separately, track and collect user behavior data during the test period, such as click-through rate, conversion rate, etc., to evaluate the pros and cons of different versions; based on the results of A / B testing and continuously collected user feedback, iteratively design the product, adjust the presentation of exhibition content, optimize the interactive experience process, and improve the design of the user interface to enhance user satisfaction and engagement; release the new version after iterative design, and continuously monitor user feedback and data indicators to ensure that the improvement effect meets expectations; repeat the above process, continuously optimize and improve the product based on user feedback and market changes; through continuous iterative design and A / B testing, ensure that the product always meets user needs and remains competitive in the market.

[0109] See also Figure 1 As an embodiment of the present invention, in step S20, the following steps are also specifically included:

[0110] S201: Data fusion algorithm includes data cleaning, integration and analysis processes, aiming to extract useful user behavior patterns and preferences. The algorithm integrates the data collected by wearable devices with the exhibition hall environment data to analyze the user's movement path in the exhibition hall, the length of stay, the degree of attention paid to specific exhibits, etc.

[0111] Specifically, the data fusion algorithm includes the following steps:

[0112] Data cleaning: Remove duplicate, invalid or abnormal data to ensure data accuracy and reliability. For missing data, interpolation, estimation or deletion methods are used.

[0113] Data integration: Integrate user data collected by wearable devices (such as heart rate, number of steps, activity type, etc.) with exhibition hall environment data (such as temperature, humidity, light, etc.). Ensure that data from different sources are consistent in time and space.

[0114] User behavior pattern analysis: Utilize the integrated data to analyze the user's movement paths in the exhibition hall and identify their frequently used routes and stopping points; count the time users stay in each exhibition area and determine their degree of attention to different exhibits; and extract the characteristics of user behavior, such as preferences and habits, through pattern recognition technology.

[0115] The data fusion formula is as follows:

[0116]

[0117] Among them, y is the fused data, x i is the data of the i-th sensor, w i is the corresponding weight, satisfying

[0118] Weight determination process:

[0119] S2011: Initialize weights, assign an initial weight to each sensor based on the theoretical accuracy of the sensor. For example, if the accuracy of a smartwatch is 90%, a higher initial weight can be assigned to it; if the accuracy of a temperature sensor is relatively low, its initial weight is correspondingly lower.

[0120] S2012: Historical data analysis, collect sensor data over a period of time, and evaluate the reliability of the sensor by comparing and analyzing the consistency of these data with the actual situation. For example, you can compare the number of steps recorded by the smart watch with the actual distance walked by the user. If the two are highly consistent, it means that the reliability of the sensor is good, and you can consider increasing its weight.

[0121] S2013: User feedback, through the user's feedback on the exhibition hall experience to indirectly evaluate the effect of the sensor. If users generally reflect that the temperature in a certain area is suitable, then it can be considered that the temperature sensor in this area is more accurate, and its weight can be appropriately increased.

[0122] S2014, Dynamic Adjustment: Regularly re-evaluate the performance of sensors over time and dynamically adjust weights based on the evaluation results. For example, if a temperature sensor frequently reports abnormal values ​​over a period of time, its weight should be reduced until the cause is identified and resolved.

[0123] Assume that there are three sensors, namely temperature sensor, humidity sensor and light sensor, with initial weights of 0.4, 0.3 and 0.3 respectively. After a period of observation, it is found that the readings of the temperature sensor occasionally fluctuate greatly, while the data of the humidity and light sensors are relatively stable. Therefore, the weights of the three sensors are 0.3, 0.4 and 0.3, thereby reducing the influence of the temperature sensor in data fusion.

[0124] S202: The step S202 includes the construction of a personalized recommendation algorithm, which is based on a deep learning model and can dynamically adjust the recommended content according to the user's real-time behavior. Specifically, this step includes but is not limited to the following sub-steps:

[0125] S2021: Customize and design the model network structure according to the application scenario and data characteristics. The network structure includes the number of layers, number of neurons, activation function, etc. to optimize the model's expressiveness and generalization performance.

[0126] S2022: Train the model and adjust the model parameters through the back propagation algorithm. Use a large amount of user behavior data to train the model and adjust the model parameters through the back propagation algorithm to minimize the gap between the predicted value and the true value.

[0127] S2023: Measure the prediction accuracy of the model through a loss function, and iteratively update the model parameters through an optimization algorithm. The optimization algorithm may be a gradient descent method.

[0128] S2024: Regularly evaluate model performance, including indicators such as accuracy and recall, to ensure the effectiveness of the model. Optimize and adjust the model based on the evaluation results, such as increasing the number of network layers, changing the activation function, etc., to improve the model's predictive performance and generalization ability. Continuously explore new deep learning technologies and algorithms to adapt to changing market demands and user preferences.

[0129] S203: Based on the deep learning algorithm model, analyze user behavior preferences and historical interaction data, and dynamically recommend content suitable for personal interests based on the analysis results to achieve highly personalized content display, including identifying topics of interest to users and pushing related digital display content.

[0130] User behavior preference analysis: Use deep learning algorithms to analyze users’ historical interaction data, such as comments, likes, and shares, and use these data to identify topics and categories that users are interested in.

[0131] Personalized content recommendation: Based on the analysis results of user behavior preferences, digital display content related to users' interests is pushed to users; recommended content may include specific exhibits, exhibition activities, related information, etc.

[0132] Continuous optimization and feedback: Continuously optimize deep learning models and recommendation algorithms through user feedback and interaction data; ensure that recommended content is always consistent with user interests, and improve user satisfaction and engagement.

[0133] In step S30, the following steps are also specifically included:

[0134] S301: Figure 3 As shown, the virtual tour path is updated in real time according to the user's location information, including generating a personalized tour route in the metaverse environment, allowing users to visit exhibits in the order of their interests.

[0135] User location tracking: Utilize the positioning technology in wearable devices to track the user’s location in the metaverse environment in real time, and combine it with the virtual map of the exhibition hall to determine the user’s precise coordinates.

[0136] Personalized tour route generation: Dynamically generate personalized tour routes based on the user's historical tour data and preferences, and use algorithms to optimize the tour order to ensure that users can visit exhibits in the order of their interests.

[0137] Virtual tour path update: As the user's location moves, the virtual tour path is updated in real time to provide navigation guidance to help users reach their destination efficiently and accurately.

[0138] like Figure 4 As shown, the user's interaction steps are as follows:

[0139] S302: By building a system compatible with multiple types of wearable devices, integrating advanced virtual reality (VR) and mixed reality (MR) technologies, and combining the perception capabilities of wearable devices, a highly immersive smart space experience scene is created to achieve seamless integration with the metaverse digital environment.

[0140] Device compatibility development: Design and develop a system platform that is compatible with a variety of wearable devices to ensure that different types of wearable devices (such as smart watches, AR glasses, etc.) can be smoothly connected.

[0141] VR / AR technology integration: Integrate advanced virtual reality (VR) and augmented reality (AR) technologies into the system and use these technologies to create an immersive smart city experience scenario for users.

[0142] Connect with the Metaverse digital environment: Realize seamless connection between wearable devices and the Metaverse digital environment, and ensure that users’ actions and interactions in the digital environment can be reflected on wearable devices in real time and accurately.

[0143] S303: Improving the naturalness and fluency of the interactive experience by driving the digital avatar and the digital exhibits to interact with the user, including enabling the user to touch the digital exhibits to obtain detailed information through gesture recognition, or interacting with the user through voice recognition.

[0144] Digital avatar drive: Create a digital avatar for the user in the metaverse environment, capture the user's movements and expressions in real time through the perception capabilities of wearable devices, and drive the digital avatar to perform corresponding actions.

[0145] Gesture recognition and interaction: Gesture recognition technology enables users to touch and operate digital exhibits through gestures, providing detailed exhibit information and enhancing users' understanding of the exhibits and interactive experience.

[0146] Voice recognition and interaction: Integrate voice recognition technology to enable users to interact with the digital environment through voice, design natural and smooth dialogue processes, and enhance the communication experience between users and the digital environment.

[0147] Optimize interactive experience: Continuously collect and analyze user interaction data, optimize interaction logic and processes, and ensure that users have a natural and smooth experience when interacting with digital avatars and digital exhibits.

[0148] See also Figure 1 As an embodiment of the present invention, in step S40, the following steps are also specifically included:

[0149] S401: Establish user feedback channels to ensure that the system can continuously adapt to user needs and provide a better user experience by continuously collecting user feedback and optimizing system performance and content, including online questionnaires, real-time voting, etc., to collect user evaluations of exhibition content and interactive experience;

[0150] Establish a multi-channel feedback mechanism: set up an online questionnaire system, regularly push questionnaires to users, collect their comments and suggestions on exhibition content and interactive experience, and provide real-time voting functions so that users can instantly provide feedback on their preferences for specific exhibits or interactive functions.

[0151] Continuously collect and analyze user feedback: organize and analyze feedback provided by users through online questionnaires, real-time voting and other channels to identify user needs and pain points, as well as their satisfaction with the current system;

[0152] Ensure that the system adapts to user needs: adjust system performance and content based on user feedback to ensure that the system can continuously meet users' changing needs, and regularly evaluate user experience to ensure that the system provides high-quality services.

[0153] S402: Use A / B testing and iterative design to adjust display content and interaction methods based on user feedback, including replacing unpopular exhibits, optimizing user interface design, and improving user experience.

[0154] Conduct A / B testing: Design two versions of the A / B test with different display content and interaction methods (for example, version A is the current version, and version B is the new version adjusted based on user feedback); randomly assign users to the two versions, and collect their behavior and satisfaction data under different versions.

[0155] Analyze A / B test results: Compare the user data of the two versions and analyze which version performs better in terms of user experience, engagement, conversion rate, etc. Based on the data analysis results, determine whether it is necessary to adjust the displayed content or interaction method.

[0156] Iterative design and optimization: Based on the results of A / B testing, unpopular exhibits are replaced or improved, and the user interface design is optimized to make it more in line with user intuition and habits, thereby improving user experience. The optimized version is used as a new control group, and A / B testing is repeated to continuously improve the product.

[0157] S403: Monitoring and Optimization: After each iteration, continuously monitor user feedback and system performance, and make further optimizations and adjustments based on new user feedback and data analysis results.

[0158] The present invention provides a wearable device and metaverse fusion interaction system and method. There are many methods and ways to implement the technical solution. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the scope of protection of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. A wearable device and metaverse fusion interaction method, characterized in that: The following steps are involved: S10: Data collection, collecting user data through wearable devices; S20: data processing and analysis, analyzing the data collected in S10 and generating personalized content based on the analysis results; S30: Interactive experience generation, applying the personalized content generated in S20 to the metaverse environment; S40: Feedback loop, optimizing the system by analyzing user experience data.

2. The wearable device and metaverse fusion interaction method according to claim 1, characterized in that: The S20 data processing and analysis includes: S201: Fusing data through a data fusion algorithm, and analyzing user behavior patterns on the fused data; S202: Build a deep learning model based on the analysis results; S203: Analyze user behavior preferences and historical interaction data based on deep learning models to perform dynamic content recommendations.

3. The wearable device and metaverse fusion interaction method according to claim 2, characterized in that: The S201 data fusion formula is as follows: Among them, y is the fused data, x i is the data of the i-th sensor, w i is the corresponding weight, satisfying 4. The wearable device and metaverse fusion interaction method according to claim 3, characterized in that: The data weight determination in the data fusion step S201 includes: S2011: Initialize weights, assign an initial weight to each sensor based on the theoretical accuracy of the sensor; S2012: Historical data analysis: analyze sensor data over a period of time, evaluate the reliability of sensors based on the consistency between the analyzed data and the actual situation, and adjust the weights based on the evaluation results; S2013: User feedback, indirectly evaluate the effect of the sensor through user feedback and adjust the weight; S2014: Dynamic adjustment, regularly re-evaluate the performance of sensors and dynamically adjust the weights based on the evaluation results.

5. The wearable device and metaverse fusion interaction method according to claim 2, characterized in that: The step S202 of building a deep learning model includes: S2021: Customize and design the model network structure according to the application scenario and data characteristics; S2022: Train the model and adjust the model parameters through the back propagation algorithm; S2023: Measure the prediction accuracy of the model through the loss function and iteratively update the model parameters through the optimization algorithm; S2024: Regularly evaluate model performance.

6. The wearable device and metaverse fusion interaction method according to claim 1, characterized in that: The S30 interactive experience generation includes: S301: updating the virtual tour guide path in real time according to the user location information; S302: Integrate virtual reality and mixed reality technologies by building a system compatible with multiple types of wearable devices; S303: Interact with users by driving digital avatars and digital exhibits.

7. The wearable device and metaverse fusion interaction method according to claim 1, characterized in that: The S40 feedback loop includes: S401: Establish a multi-channel feedback mechanism to identify user needs and pain points; S402: Use A / B testing and iterative design to adjust display content and interaction methods based on user feedback; S403: Monitoring and Optimization: Continuously monitor user feedback and system performance in each iteration to optimize the design.

8. A wearable device and metaverse fusion interactive system, characterized in that: A wearable device and metaverse fusion interaction method comprising any one of claims 1 to 6.

9. The wearable device and metaverse fusion interaction system according to claim 7, characterized in that: Including servers, integrated software platforms, metaverse platforms, wearable devices, and sensor networks; The server completes information interaction between the user and the Metaverse platform through the wearable device; The integrated software platform supports API docking with the Metaverse platform; the integrated software platform includes a data acquisition module, a content recommendation system, a data processing and analysis module, and a user interface design module; The data collection module collects user behavior data, the data processing and analysis module analyzes user behavior patterns through the collected data, and the content recommendation system trains a deep learning model based on the analysis results to recommend user content; The said metaverse platform constructs a digital display; The wearable device and sensor network capture the user's movements and physiological reactions, and the wearable device and sensor network include an accelerometer, a gyroscope, a magnetometer, a heart rate monitor, a temperature sensor, a camera, and an ambient light sensor.

10. The wearable device and metaverse fusion interaction system according to claim 8, characterized in that: The metaverse platform includes a digital scene construction module and a digital character construction module; The digital scene construction module constructs a virtual digital scene of the metaverse, which includes an exhibition hall, an exhibit showroom, and showroom lights; the digital character component module processes user data captured by wearable devices, and the digital avatars in the metaverse environment synchronously update their movements and expressions.

Images