Intelligent convenience service system based on XR technology and operation method

Abstract

The invention provides an intelligent convenience service system based on an XR technology and an operation method, user information of a current user is acquired through an information acquisition module of an immersive interaction layer, and an application service demand of the current user is acquired through a central control layer according to the acquired user information; the configurable application layer is coordinated to configure a corresponding application module, so that a scene display module of the immersive interaction layer is controlled based on immersive technical service support, and an immersive service scene of the application module is provided for a current user; therefore, rich application services are provided for the user, diversified requirements of the user are met, the user can perform interaction and experience in an immersive service scene, the interactivity and humanized experience of the system are improved, and the user experience is improved. The technical problems that an existing intelligent convenient service kiosk is not matched with scene requirements in function, low in humanized service degree and poor in interactivity are solved.

Description

Technical Field

[0001] This application relates to the technical field of convenient services, and particularly to a smart convenient service system and an operation method based on XR technology. Background Art

[0002] A smart convenient service kiosk is a comprehensive service system that utilizes modern information technology with the goals of improving service efficiency, optimizing service experience, and meeting the diverse needs of the public, covering multiple fields such as social security, medical health, cultural education, and employment services.

[0003] Existing smart convenient service kiosks generally have problems such as single functions, complex operations, insufficient adaptation of functions to scenarios, and difficulty in meeting the diverse needs of users; moreover, the degree of humanized service is low, especially unfriendly to the operation of the elderly and those unfamiliar with smart devices, making users unable to feel the interactive and humanized service; as a result, the utilization rate of smart convenient service kiosks built in many places has dropped sharply, and most of them cannot be used normally, becoming a mere formality, leading to a large amount of resource waste. Summary of the Invention

[0004] In view of the above-mentioned disadvantages of the prior art, the purpose of this application is to provide a smart convenient service system and an operation method based on XR technology, which are used to solve the technical problems of the mismatch between the functions and scenario requirements of existing smart convenient service kiosks, low degree of humanized service, and poor interactivity.

[0005] To achieve the above purpose and other related purposes, the first aspect of this application provides a smart convenient service system based on XR technology. The smart convenient service system based on XR technology includes: an immersive interaction layer, including: an information collection module and a scene display module; the information collection module is used to collect user information, and the scene display module is used to provide an immersive service scene; a configurable application layer, including: one or more application modules, each application module is respectively used to provide corresponding application services; an immersive technology layer, including: a virtual reality technology module, an augmented reality technology module, and a mixed reality technology module, which are respectively used to provide corresponding immersive technology services; a central control layer, which is respectively connected to the immersive interaction layer, the configurable application layer, and the immersive technology layer, and is used to obtain the application service requirements of the current user according to the user information collected by the information collection module, and coordinate the configurable application layer to configure the corresponding application module, and then promote the joint operation of the virtual reality technology module, the augmented reality technology module, and the mixed reality technology module, so as to provide support based on immersive technology services, control the scene display module, provide an immersive service scene of the application module for the current user, and generate service data of the current user.

[0006] In some embodiments of the first aspect of the present application, the information collection module includes: a visual collection unit for capturing the facial expressions and movement postures of the current user; an audio collection unit for collecting the voice information of the current user; a human body perception unit for collecting the movement information and physiological information of the current user; a biometric recognition unit for collecting the biometric information of the current user, where the biometric information includes one or a combination of more than one of fingerprint information, palmprint information, face information, and iris information; and a digital terminal unit for collecting the text input information of the current user.

[0007] In some embodiments of the first aspect of the present application, the scene display module includes: a display unit including one or more display devices for constructing an immersive virtual space and providing an immersive virtual display scene based on the support of immersive technology services to provide immersive interaction services for the current user; and a voice playback unit for providing voice services for the current user and providing an immersive service scene.

[0008] In some embodiments of the first aspect of the present application, the scene display module further includes: a lighting control unit, a temperature control unit, a humidity control unit, a ventilation control unit, and a door control unit, which are respectively used to adjust the lighting brightness, temperature, humidity, ventilation, and door control of the current environment in real time to jointly provide an immersive service scene for the current user in combination with the display unit and the voice playback unit.

[0009] In some embodiments of the first aspect of the present application, the application module includes one or a combination of more than one of a consultation module, a payment module, an entertainment module, an advertising module, a storage module, a retail module, a rental module, a garbage recycling module, and a charging module.

[0010] In some embodiments of the first aspect of the present application, based on the support of immersive technology services, the method of controlling the scene display module to provide an immersive service scene of the application module for the current user includes: scanning the current physical environment through a depth sensor to generate a digital model of the current real space, and constructing a virtual reality scene based on a physics engine and real-time rendering technology; real-time identifying the objects in the current physical environment and synchronously tracking the head, hands, and whole body movements of the current user to dynamically align the virtual objects in the virtual reality scene with the current real space, so as to superimpose the virtual objects on the current real space to construct an augmented reality scene; combining the immersion of the virtual reality scene and the superimposition ability of the augmented reality scene to realize real-time interaction between the virtual objects and the current physical environment to construct a mixed reality scene; and processing the interaction logic of the virtual reality scene, the augmented reality scene, and the mixed reality scene simultaneously through a hybrid rendering pipeline, and then providing the corresponding immersive service scene for the current user based on the scene display module.

[0011] In some embodiments of the first aspect of the present application, the central control layer further includes: an information security module, connected to the processor module and respectively connected to the immersive interaction layer, the configurable application layer, and the immersive technology layer, enabling the processor module to be respectively connected to the immersive interaction layer, the configurable application layer, and the immersive technology layer through the information security module; wherein, the information security module includes: an access control unit, configured to allow legitimate users to access data within their access rights and restrict illegal users from accessing sensitive data, thereby building an access control barrier; a privacy protection unit, configured to protect the privacy information of legitimate users, where the privacy information includes: personal information, account information, and transaction information; a data encryption unit, configured to encrypt and decrypt the interaction data between the processor module and other modules to protect the interaction data and prevent the interaction data from being illegally stolen and maliciously tampered with.

[0012] In some embodiments of the first aspect of the present application, the XR technology-based intelligent and convenient service system further includes: an environmental monitoring layer, connected to the central control layer, including: an environmental video monitoring module, an environmental sound monitoring module, an environmental condition monitoring module, and an environmental personnel monitoring module, configured to collect video information and audio information of the current environment, monitor the environmental conditions and personnel conditions of the current environment in real time, and generate environmental monitoring data to be sent to the central control layer for the central control layer to judge the risk situation of the current environment and generate early warning information.

[0013] In some embodiments of the first aspect of the present application, the XR technology-based intelligent and convenient service system further includes: an information communication layer, connected to the central control layer and connected to an external operation center, configured to send the generated service data, environmental monitoring data, and early warning information to the external operation center.

[0014] To achieve the above and other related purposes, the second aspect of the present application provides an XR technology-based intelligent and convenient service operation method. The XR technology-based intelligent and convenient service operation method is applied to the XR technology-based intelligent and convenient service system described in any of the above embodiments. The method includes: collecting user information of the current user through the information collection module of the immersive interaction layer; obtaining the application service requirements of the current user according to the collected user information through the central control layer, and coordinating the configurable application layer to configure the corresponding application module; promoting the joint operation of the virtual reality technology module, the augmented reality technology module, and the mixed reality technology module of the immersive technology layer through the central control layer, so as to provide immersive service scenarios of the application module for the current user based on immersive technology service support, and generating service data of the current user by controlling the scene display module.

[0015] As described above, the present application provides an intelligent and convenient service system and an operation method based on XR technology. The information collection module of the immersive interaction layer collects the user information of the current user, and the central control layer obtains the application service requirements of the current user according to the collected user information, coordinates the configurable application layer to configure the corresponding application module, and then controls the scene display module of the immersive interaction layer based on the support of immersive technology services to provide the immersive service scene of the application module for the current user. The present application has the following beneficial effects: providing rich application services for users, meeting the diverse needs of users, enabling users to interact and experience in the immersive service scene, improving the interactivity and user-friendly experience of the system, and solving the technical problems of the mismatch between the functions and scene requirements of existing intelligent and convenient service kiosks, low level of user-friendly service, and poor interactivity. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It shows a schematic structural diagram of an intelligent and convenient service system based on XR technology in an embodiment of the present application.

[0017] Figure 2 It shows a schematic flowchart of a method for realizing an immersive service scene in an embodiment of the present application

[0018] Figure 3 It shows a schematic flowchart of a method for monitoring the intelligent and convenient service environment in an embodiment of the present application.

[0019] Figure 4 It shows a schematic flowchart of an operation method of an intelligent and convenient service based on XR technology in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] The following specific examples illustrate the implementation manners of the present application. Those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific implementation manners, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0021] In the embodiments of the present application, terms such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and roles. For example, the first batch normalization layer and the second batch normalization layer are only used to distinguish different batch normalization layers, and their sequence is not limited. Those skilled in the art can understand that terms such as "first" and "second" do not limit the quantity and execution order, and "first", "second", etc. do not necessarily mean different.

[0022] Smart convenience service kiosks are grassroots stations that integrate multiple public service functions to meet the daily needs of residents, and use modern information technology to provide residents with convenient, diversified, and personalized public services. They are widely set up in communities, streets, and other areas, greatly improving the convenience of residents' lives. However, existing smart convenience service kiosks generally have problems such as single functions, complex operations, insufficient adaptation of functions and scenarios, and difficulty in meeting the diverse needs of users; in addition, the level of humanized services is low, especially for the elderly and people who are not familiar with smart devices, so that users cannot feel the interactive and humanized services; therefore, the utilization rate of smart convenience service kiosks that have been built in many places has dropped sharply, and most of them cannot be used normally, which is equivalent to a mere formality, resulting in a large waste of resources.

[0023] In order to adapt to the diverse needs of users of smart convenience service kiosks and solve the problems in the above-mentioned background technology, the present application provides a smart convenience service system and operation method based on XR technology, which aims to control the connected immersive interaction layer, configurable application layer and immersive technology layer through the central control layer, and provide the current user with an immersive service scenario of the application module based on immersive technology service support, thereby solving the technical problems of existing smart convenience service kiosks, such as mismatch between functions and scenario requirements, low level of humanized service and poor interactivity.

[0024] Before further explaining the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations:

[0025] <1> VR (Virtual Reality): Virtual reality is a technology that uses computer technology to generate a simulated virtual environment. It provides users with an immersive experience of multiple senses such as vision, hearing, and touch, making users feel as if they are in a virtual space different from the real world.

[0026] <2> AR (Augmented Reality): Augmented reality is a technology that cleverly integrates virtual information with the real world. It generates virtual objects, information or images through computer technology and superimposes them on real scenes, allowing users to see the real world and virtual elements at the same time, and the two can interact, thereby enhancing users' perception and experience of the real world.

[0027] <3> MR (Mixed Reality): Mixed reality is a technology that combines virtual reality (VR) and augmented reality (AR). By integrating the virtual world with the real world, it creates a hybrid environment that contains both real and virtual elements, allowing users to interact and experience it naturally.

[0028] <4>XR (Extended Reality): Extended Reality combines multiple technologies such as virtual reality (VR), augmented reality (AR), and mixed reality (MR). Through computer technology and wearable devices, etc., it creates a virtual environment that integrates with the real world, providing users with an immersive experience.

[0029] In order to make the invention purpose, technical solutions and advantages of this application clearer, the technical solutions in the embodiments of this application are further described in detail through the following embodiments in combination with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit this application.

[0030] As Figure 1 shown, it shows a schematic structural diagram of a smart and convenient service system 100 based on XR technology in the embodiments of this application. The smart and convenient service system 100 based on XR technology in this embodiment includes: an immersive interaction layer 110, a configurable application layer 120, an immersive technology layer 130, and a central control layer 140.

[0031] Specifically, the immersive interaction layer 110 includes: an information collection module 111 for collecting user information and a scene display module 112 for providing an immersive service scene.

[0032] In one embodiment, the information collection module 111 includes: a visual collection unit, an audio collection unit, a human body perception unit, a biometric recognition unit, and a digital terminal unit. Among them, the visual collection unit, such as devices like cameras and cameras, is used to capture the facial expressions and action postures of the current user; the audio collection unit is used to collect the voice information of the current user; the human body perception unit is used to collect the motion information and physiological information of the current user, such as motion information such as the motion posture, motion speed, and acceleration of the current user, and physiological information such as the weight, body temperature, and heart rate of the current user; the biometric recognition unit is used to collect the biometric information of the current user, and the biometric information includes: one or a combination of fingerprint information, palmprint information, face information, and iris information; the digital terminal unit is used to collect the text input information of the current user. It should be noted that the digital terminal unit includes but is not limited to: touchscreens, keyboards, and mice, etc., and this application does not limit them.

[0033] In this embodiment, the information collection module 111 collects the facial expressions, voice information, action postures, motion information, physiological information, biological information, and text input information entered by the current user through the visual collection unit, the audio collection unit, the human perception unit, the biometric unit, and the digital terminal unit, so as to obtain the user information of the current user, for the central control layer to obtain the application service requirements of the current user according to the collected user information, and thus provide an adapted application module for the current user. The information collection module 111 of the present application collects user information in multiple modes. The current user can actively input through the digital terminal unit, so that the central control layer directly obtains the user information and application service requirements; the current user can also convey their application service requirements through voice commands and action expressions, enabling the elderly or people who are not familiar with smart devices to operate conveniently, and providing more user-friendly services for users.

[0034] In one embodiment, the scene display module 112 includes: a display unit and a voice playback unit. Among them, the display unit includes one or more display devices, such as an LED display screen. In this embodiment, the display unit can construct an immersive virtual space through each display device, and based on the support of immersive technology services, can provide an immersive virtual display scene for the current user, provide an immersive interaction service for the current user, enable the current user to interact and experience therein, and bring an immersive virtual experience to the user. And, the voice playback unit is used to provide voice services for the current user, can conduct voice interaction with the current user, and improve the interactivity of the current user; at the same time, the voice playback unit is also used to provide an immersive service scene for the current user, further improving the interactivity and user experience of the intelligent and convenient service system 100 based on XR technology. In a specific embodiment, the voice playback unit uses a speaker.

[0035] In one embodiment, the scene display module 112 further includes: a lighting control unit, a temperature control unit, a humidity control unit, a ventilation control unit, and a door control unit, which are respectively used to adjust the lighting brightness, temperature, humidity, ventilation, and door control of the current environment in real time, so as to jointly with the display unit and the voice playback unit, further provide an immersive service scene for the current user, and further enhance the immersion of the current user.

[0036] In this embodiment, the immersive interaction layer 110 provides an interactive platform containing information such as images, voice, video, and text, and cooperates with multiple display devices to build an immersive virtual space, and can also build a virtual human image to provide interactive services. Therefore, the immersive interaction layer 110 can achieve immersive interaction, and can also carry out corresponding application services by linking with the configurable application layer, making the corresponding immersive service scene more comfortable, and providing the current user with perfect, more humane, and more interactive application services.

[0037] The configurable application layer 120 includes: one or more application modules, each of which is used to provide corresponding application services. In one embodiment, the application module includes: a consultation module, a payment module, an entertainment module, an advertising module, a storage module, a retail module, a rental module, a garbage collection module and a charging module. One or more combinations thereof can provide information consultation services, fee submission services, entertainment game services, advertising push services, temporary storage services, automatic retail services, smart rental services, garbage collection services and charging services for the current user. Among them, it should be noted that the charging module also includes a battery module charging and replacement unit, a battery vehicle charging unit, an electric vehicle charging unit and a drone charging unit, which can provide the current user with battery module charging services, battery module charging replacement services, battery vehicle charging services, electric vehicle charging services and drone charging services. In this embodiment, the smart convenience service system 100 based on XR technology is rich in functions and can meet the diverse needs of users. In addition, other types of application modules can be added or corresponding application modules can be deleted according to needs, and this application is not limited.

[0038] The immersive technology layer 130 includes: a virtual reality technology module 131, an augmented reality technology module 132 and a mixed reality technology module 133, which are respectively used to provide corresponding immersive technology services.

[0039] Among them, the virtual reality technology module 131 is used to build a completely virtualized digital space; the augmented reality technology module 132 is used to superimpose virtual information on the real space to combine virtual and reality; the mixed reality technology module 133 is used to enhance the real space through the virtual space, so that users can get a richer and more intuitive experience.

[0040] The central control layer 140 is respectively connected to the immersive interaction layer 110, the configurable application layer 120, and the immersive technology layer 130. It is used to obtain the application service requirements of the current user according to the user information collected by the information collection module 111, coordinate the configurable application layer 120 to configure the corresponding application module, and then promote the joint operation of the virtual reality technology module 131, the augmented reality technology module 132, and the mixed reality technology module 133. Based on the support of immersive technology services, it controls the scene display module 112 to provide an immersive service scene of the application module for the current user and generate service data of the current user. The specific working method is as Figure 2 shown.

[0041] Step S101: Collect the user information of the current user through the information collection module 111.

[0042] In a preferred embodiment, the information collection module 111 includes a visual collection unit, an audio collection unit, a human body perception unit, a biometric recognition unit, and a digital terminal unit, and can provide multi-modal interaction means such as voice, gesture, expression, and full-body movement, so as to provide more user-friendly and better interactive services for users.

[0043] Step S102: Through the central control layer 140, obtain the application service requirements of the current user according to the collected user information, so as to coordinate the configurable application layer 120 to configure the corresponding application module.

[0044] In a preferred embodiment, the central control layer 140 can deeply analyze the collected multi-modal user information based on AI intelligent data analysis technology, so as to accurately grasp the application service requirements of users, and match and present corresponding application services to users according to the application service requirements of users, realizing efficient linkage with various types of application modules such as social security, medical and health, culture and education, and employment services.

[0045] Step S103: Through the central control layer 140, promote the joint operation of the virtual reality technology module 131, the augmented reality technology module 132, and the mixed reality technology module 133 to provide immersive technology service support.

[0046] Step S104: Through the central control layer 140, based on the support of immersive technology services, control the scene display module 112 to provide an immersive service scene of the application module for the current user and generate service data of the current user.

[0047] In one embodiment, the central control layer 140 includes: a processor module 141. The processor module 141 is respectively connected to the immersive interaction layer 110, the configurable application layer 120, and the immersive technology layer 130. Specifically, the processor module 141 is connected to the information collection module 111, and is configured to receive various types of user information collected by the information collection module 111, and based on the user information, obtain the application service requirements of the current user; the processor module 141 is connected to each application module, and is configured to coordinate and configure the corresponding application module according to the obtained application service requirements, so as to provide the corresponding application service for the current user; the processor module 141 is further connected to the virtual reality technology module 131, the augmented reality technology module 132, and the mixed reality technology module 133, and is configured to promote the joint operation of the virtual reality technology module 131, the augmented reality technology module 132, and the mixed reality technology module 133, and implement the extended reality technology that integrates virtual reality technology, augmented reality technology, and mixed reality technology, so as to provide immersive technology service support; the processor module 141 is connected to the scene display module 112, and is configured to control the scene display module based on the immersive technology service support, provide the immersive service scene of the application module for the current user, and generate the service data of the current user.

[0048] In a specific embodiment, the manner in which the processor module 141 controls the scene display module based on the immersive technology service support to provide the immersive service scene of the application module for the current user includes the following steps.

[0049] ① Scan the current physical environment through a visual sensor and a depth sensor, generate a digital twin model of the current real space, and construct a virtual reality scene based on a physics engine and real-time rendering technology.

[0050] It should be understood that the digital twin model is a virtual model created in a virtual space that is highly similar to a physical object in a physical environment. In a specific embodiment, the visual sensor may be a high-definition camera, and the depth sensor may be a laser radar, a time-of-flight sensor, and an inertial measurement unit. In this embodiment, a virtual reality scene can be constructed based on naked-eye 3D technology and an advanced real-time rendering engine to achieve an intuitive presentation of the real space. The naked-eye 3D technology uses special optical designs, such as light barriers, cylindrical lenses, or pointing light sources, so that users can feel the stereoscopic visual effect without wearing additional equipment. It uses the binocular parallax principle of the human eye to project images of different perspectives to the left and right eyes, respectively, so as to synthesize a three-dimensional virtual reality scene in the brain. The real-time rendering engine is responsible for efficiently and high-quality generation of virtual display images of virtual reality scenes, and can calculate and output virtual display images in real time according to dynamic changes in real scenes, such as movement of objects, changes in lighting, etc., to ensure the fluency and fidelity of virtual reality scenes. By combining the two, the naked-eye 3D technology provides an immersive stereoscopic visual foundation, and the real-time rendering engine ensures the dynamic presentation and visual details of the virtual reality scene, thereby constructing a virtual reality scene that is extremely realistic and interactive.

[0051] ②Real-time identification of objects in the current physical environment, and synchronous tracking of the current user's head, hands, and whole body movements, so that the virtual objects in the virtual reality scene are dynamically aligned with the current real space, so as to superimpose the virtual objects into the current real space and construct an augmented reality scene.

[0052] In a specific embodiment, the visual sensor and the depth sensor, such as a high-definition camera and an inertial measurement unit, can be used to respectively obtain image data and motion information of each object in the current physical environment, and the image data and the motion information can be fused through a visual positioning algorithm to calculate the posture information of each object; each virtual object created in the real-time rendering engine is associated with the posture information of each object in the current physical environment to ensure that each virtual object can be accurately superimposed on the correct position in the current real space; and each virtual object is fused with the object image in the current real space using an image fusion algorithm to make the virtual object more coordinated with the current real space.

[0053] ③ Combine the immersiveness of virtual reality scenes and the superposition capability of augmented reality scenes to achieve real-time interaction between virtual objects and the current physical environment and build a mixed reality scene.

[0054] In this embodiment, a mixed reality scene is further constructed through multimodal interaction means such as voice, gestures, expressions and whole body movements to form a mixed reality experience.

[0055] ④Through the hybrid rendering pipeline, the interaction logic of virtual reality scenarios, augmented reality scenarios, and mixed reality scenarios is processed simultaneously, and then based on the scenario display module, an immersive service scenario corresponding to the current user is provided.

[0056] It should be understood that the hybrid rendering pipeline combines the advantages of various rendering technologies such as traditional rasterization rendering and ray tracing rendering, including: using rasterization rendering to quickly render a large number of geometric models in virtual reality scenarios, and using ray tracing rendering to simulate the light propagation and reflection of virtual objects in augmented reality scenarios and mixed reality scenarios, accurately calculating the shadow effects of virtual objects in the real scenario, and being able to present lighting consistent with the real space, making it realistic.

[0057] In a preferred embodiment, based on the support of immersive technology services, the manner of controlling the scenario display module to provide an immersive service scenario of this application module for the current user further includes: through the model context protocol technology, integrating the user information obtained by the information collection module 111 through multi-modal interaction means, and jointly with the corresponding application module, constructing a unified context data structure and transmission mechanism, and then providing support for immersive technology services.

[0058] It should be understood that the model context protocol technology, namely Model Context Protocol, abbreviated as MCP, is a standardized protocol for artificial intelligence systems, aiming to define, organize, and transmit the context information required by AI models when processing multi-source and multi-modal data. This protocol constructs a unified context data structure and transmission mechanism, forming a complete set of efficient data flow management systems from data collection, preprocessing to context encoding, priority sorting, and dynamic update, enabling AI models to obtain and integrate information such as historical interactions, environmental states, and user behaviors in real time, thereby improving accuracy and response speed in semantic understanding and decision-making processes. Through MCP technology, the central control layer 140 can not only optimize the resource allocation of the internal attention mechanism to ensure that the user information of multi-source multi-modal interactions and the application data of each application module can be called immediately, but also achieve interoperability and data collaboration in multi-module and high-concurrency scenarios in distributed applications, providing solid technical support for immersive technology services.

[0059] In this embodiment, the real-time transmission technology is also adopted to provide a corresponding immersive service scenario for the current user. Specifically, based on the low-latency and high-bandwidth characteristics of the 5G network, the immersive service scenario can be transmitted to the display device in real time, or transmitted to wearable devices such as smart glasses, smart bracelets, and smart watches, so as to provide a responsive and personalized immersive service experience for users.

[0060] In one embodiment, the central control layer 140 further includes an information security module 142. The information security module 142 is connected to the processor module 141 and is respectively connected to the immersive interaction layer 110, the configurable application layer 120, and the immersive technology layer 130, enabling the processor module 141 to be respectively connected to the immersive interaction layer 110, the configurable application layer 120, and the immersive technology layer 130 through the information security module 142.

[0061] Among them, the information security module 142 includes an access control unit, a privacy protection unit, and a data encryption unit, which are used to enhance the overall security of the XR technology-based intelligent and convenient service system 100.

[0062] The access control unit is used to allow legitimate users to access data within their permissions and restrict illegal users from accessing sensitive data, thereby building an access control barrier. The specific methods include: judging the legitimacy of the current user; if the current user is a legitimate user, judging the user level of the current user, and according to the user level, granting the current user access permissions corresponding to their user level, and only allowing them to access data within their access permissions; if the current user is an illegal user, restricting the illegal user from accessing sensitive data and building an access control barrier; thus effectively preventing malicious users or external attackers from performing illegal operations.

[0063] The privacy protection unit is used to protect the privacy information of legitimate users, and the privacy information includes personal information, account information, and transaction information.

[0064] The data encryption unit is used to encrypt and decrypt the interaction data between the processor module and other modules, protect the interaction data, and prevent the interaction data from being illegally stolen and maliciously tampered with.

[0065] Specifically, the encryption algorithms adopted mainly include AES (Advanced Encryption Standard), RSA (Rivest-Shamir-Adleman, an asymmetric encryption algorithm based on large integer factorization), and SM (national cryptographic algorithm).

[0066] Among them, AES is a symmetric encryption algorithm that uses the same key for encryption and decryption and is suitable for efficient encryption. The AES encryption calculation formula is: C = AES Encrypt (P, K); the decryption calculation formula is: C = AES Decrypt (C, K); where C is the ciphertext, P is the plaintext, and K is the key.

[0067] RSA is an asymmetric encryption algorithm. It generally uses a public key for encryption and a private key for decryption, and is applicable to key exchange and digital signature. The RSA encryption calculation formula is: C = P e mod(n); The decryption calculation formula is: P = C d mod(n); In the formula, C is the ciphertext, P is the plaintext, e and n are the public keys, and d is the private key.

[0068] The SM series is a series of cryptographic algorithm standards formulated by the State Cryptography Administration of China, applicable to various applications, including SM1, SM2, SM3, SM4, etc. Among them, SM1 is symmetric encryption, similar to AES; SM2 is asymmetric encryption based on elliptic curves; SM3 is a hashing algorithm, similar to SHA-256; SM4 is symmetric encryption with a block length of 128 bits. The SM4 encryption calculation formula is: C = SM4 Encrypt (P, K); The SM4 decryption calculation formula is: P = SM4 Decrypt (C, K); In the formula, C is the ciphertext, P is the plaintext, and K is the key.

[0069] In one embodiment, the XR technology-based intelligent convenient service system 100 further includes: an environment monitoring layer 150. The environment monitoring layer 150 is connected to the central control layer 140 and includes: an environment video monitoring module 151, an environment sound monitoring module 152, an environment condition monitoring module 153, and an environment personnel monitoring module 154, which are used to collect video information and audio information of the current environment, monitor the environment condition and personnel condition of the current environment in real time, and generate environment monitoring data and send it to the central control layer 140 for the central control layer 140 to judge the risk situation of the current environment and generate early warning information.

[0070] In this embodiment, through the environment video monitoring module 151, the environment sound monitoring module 152, the environment condition monitoring module 153, and the environment personnel monitoring module 154, the environment monitoring layer 150 can monitor the temperature, humidity, brightness, smoke concentration, air quality, and water depth in case of floods or heavy rains in the current environment in real time, so as to timely discover environmental abnormal events and equipment failure states in the service kiosk, etc.; at the same time, it can monitor the action state, speech expression, and facial expression of the personnel in the current environment in real time, so as to timely discover personnel abnormal events such as excessive personnel gathering, personnel making trouble, and sudden illness of personnel in the service kiosk.

[0071] In one embodiment, the XR technology-based intelligent convenient service system 100 further includes: an information communication layer 160. The information communication layer 160 is connected to the central control layer 140 and is also connected to an external operation center, and is used to send the generated service data, environment monitoring data, and early warning information to the external operation center.

[0072] Specifically, the information communication layer 160 includes: an Internet communication module 161, an Internet of Things communication module 162, and a wireless communication module 163. The Internet communication module 161 uses Internet protocol technology to achieve data transmission and information exchange between multiple computer devices through various network devices and communication lines. The Internet of Things communication module 162 collects information about various objects in the physical world through various types of information sensors, radio frequency identification technology, global positioning system, and other technologies and devices, and then transmits it to an external operation center through network devices and communication lines to achieve intelligent connection and communication between things and between things and people, thereby realizing intelligent identification, positioning, tracking, monitoring, and management of objects. The wireless communication module 163 uses a communication method that transmits information by the propagation of electromagnetic waves in free space, without the need for physical cable connection, enabling communication devices to move freely within a certain range to achieve communication anytime and anywhere.

[0073] In one embodiment, as Figure 3 shown, the intelligent and convenient service system 100 based on XR technology can implement environmental monitoring of the service kiosk, and the specific process includes the following steps.

[0074] Step S105: Through the environmental monitoring layer 150, collect video information and audio information of the current environment, monitor the environmental conditions and personnel conditions of the current environment in real time, and generate environmental monitoring data and send it to the central control layer 140.

[0075] Step S106: Through the central control layer 140, judge the risk situation of the current environment. When an abnormal event in the current environment is identified, generate a warning message.

[0076] Step S107: Through the information communication layer 160, send the generated warning message to an external operation center.

[0077] In this embodiment, through the information communication layer 160 of the present application, the central control layer 140 can send the generated service data, environmental monitoring data, and warning information obtained by analyzing the service data and the environmental detection data to an external operation center for the staff to monitor various application services of the service kiosk, monitor the status of various devices, environmental monitoring, etc., and promptly discover and handle abnormal events.

[0078] As Figure 4 shown, it shows a schematic flowchart of an intelligent and convenient service operation method based on XR technology in an embodiment of the present application. The intelligent and convenient service operation method based on XR technology is applied to the intelligent and convenient service system 100 provided in the above embodiments.

[0079] As Figure 1 shown, the XR technology-based intelligent and convenient service system 100 includes: an immersive interaction layer 110, a configurable application layer 120, an immersive technology layer 130, and a central control layer 140 that is respectively connected to the immersive interaction layer 110, the configurable application layer 120, and the immersive technology layer 130. Among them, the immersive interaction layer 110 includes: an information collection module 111 and a scene display module 112; the configurable application layer 120 includes: one or more application modules; the immersive technology layer 130 includes: a virtual reality technology module 131, an augmented reality technology module 132, and a mixed reality technology module 133.

[0080] As Figure 4 shown, the XR technology-based intelligent and convenient service operation method includes steps S401 to S403.

[0081] Step S401: Collect the user information of the current user through the information collection module 111 of the immersive interaction layer 110.

[0082] Step S402: Through the central control layer 140, according to the collected user information, obtain the application service requirements of the current user, and coordinate the configurable application layer 120 to configure the corresponding application module.

[0083] Step S403: Through the central control layer 140, promote the joint operation of the virtual reality technology module 131, the augmented reality technology module 132, and the mixed reality technology module 133 of the immersive technology layer 130, so as to provide support based on immersive technology services, control the scene display module 112 of the immersive interaction layer 110, provide an immersive service scene of the application module for the current user, and generate service data of the current user.

[0084] It should be understood that the specific processes of each step have been described in detail in the above system embodiments. For the sake of brevity, they will not be repeated here.

[0085] The functions and implementation methods of the XR technology-based intelligent and convenient service system provided by the embodiments of the present application can be implemented on the terminal side or the server side, or completed through hardware related to computer programs.

[0086] The above are only specific implementation manners of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0087] In summary, the present application provides an intelligent and convenient service system and an operation method based on XR technology. The information collection module of the immersive interaction layer collects the user information of the current user, and the central control layer obtains the application service requirements of the current user according to the collected user information, and coordinates the configurable application layer to configure the corresponding application module. Furthermore, based on the support of the immersive technology service, the scene display module of the immersive interaction layer is controlled to provide the immersive service scene of the application module for the current user; thereby providing rich application services for users, meeting the diverse needs of users, enabling users to interact and experience in the immersive service scene, enhancing the interactivity and user-friendly experience of the system, and solving the technical problems of the mismatch between the functions and scene requirements, low degree of user-friendly service, and poor interactivity existing in the existing intelligent and convenient service kiosks.

[0088] Therefore, the present application effectively overcomes various disadvantages in the prior art and has high industrial utilization value.

[0089] The above embodiments are only illustrative of the principles and effects of the present application, and are not used to limit the present application. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical ideas disclosed in the present application should still be covered by the claims of the present application.

Claims

1. A smart and convenient public service system based on XR technology, characterized in that, Including: An immersive interaction layer, including: an information collection module and a scene display module; the information collection module is used to collect user information, and the scene display module is used to provide an immersive service scene; A configurable application layer, including: one or more application modules, each application module is respectively used to provide corresponding application services; An immersive technology layer, including: a virtual reality technology module, an augmented reality technology module, and a mixed reality technology module, which are respectively used to provide corresponding immersive technology services; A central control layer, which is respectively connected to the immersive interaction layer, the configurable application layer, and the immersive technology layer, and is used to obtain the application service requirements of the current user according to the user information collected by the information collection module, and coordinate the configurable application layer to configure the corresponding application module, and then promote the joint operation of the virtual reality technology module, the augmented reality technology module, and the mixed reality technology module, so as to control the scene display module based on the support of immersive technology services, provide an immersive service scene of this application module for the current user, and generate service data of the current user.

2. The intelligent and convenient service system based on XR technology according to claim 1, characterized in that The information collection module includes: A visual collection unit, which is used to capture the facial expressions and movement postures of the current user; An audio collection unit, which is used to collect the voice information of the current user; A human body perception unit, which is used to collect the movement information and physiological information of the current user; A biometric recognition unit, which is used to collect the biometric information of the current user, and the biometric information includes: one or more combinations of fingerprint information, palmprint information, face information, and iris information; A digital terminal unit, which is used to collect the text input information of the current user.

3. The intelligent and convenient service system based on XR technology according to claim 1, wherein The scene display module includes: A display unit, including one or more display devices, which is used to construct an immersive virtual space, and based on the support of immersive technology services, provide an immersive virtual display scene to provide immersive interaction services for the current user; A voice playback unit, which is used to provide voice services for the current user and provide an immersive service scene.

4. The intelligent and convenient public service system based on XR technology according to claim 3, wherein, The scene display module further includes: a lighting control unit, a temperature control unit, a humidity control unit, a ventilation control unit, and a door control unit, which are respectively used to adjust the lighting brightness, temperature, humidity, ventilation, and door control of the current environment in real time, so as to jointly provide an immersive service scene for the current user with the display unit and the voice playback unit.

5. The intelligent and convenient public service system based on XR technology according to claim 1, wherein The application module includes: one or more combinations of a consultation module, a payment module, an entertainment module, an advertising module, a storage module, a retail module, a rental module, a garbage recycling module, and a charging module.

6. The intelligent and convenient public service system based on XR technology according to claim 1, wherein, Based on the support of immersive technology services, the ways to control the scene display module to provide an immersive service scene of this application module for the current user include: Scanning the current physical environment through a depth sensor to generate a digital model of the current real space, and constructing a virtual reality scene based on a physics engine and real-time rendering technology; Recognize objects in the current physical environment in real time, and synchronously track the head, hands, and full-body movements of the current user, so that virtual objects in the virtual reality scene are dynamically aligned with the current real space, in order to superimpose the virtual objects onto the current real space and construct an augmented reality scene; Combine the immersion of the virtual reality scene and the superimposition ability of the augmented reality scene to achieve real-time interaction between virtual objects and the current physical environment, and construct a mixed reality scene; Through a hybrid rendering pipeline, simultaneously process the interaction logics of the virtual reality scene, the augmented reality scene, and the mixed reality scene, and then based on the scene display module, provide the corresponding immersive service scene for the current user.

7. The intelligent and convenient service system based on XR technology according to claim 6, characterized in that, The central control layer further includes: An information security module, connected to the processor module, and respectively connected to the immersive interaction layer, the configurable application layer, and the immersive technology layer, so that the processor module is connected to the immersive interaction layer, the configurable application layer, and the immersive technology layer through the information security module; Among them, the information security module includes: An access control unit, used to allow legitimate users to access data within their permissions and restrict illegal users from accessing sensitive data, to build an access control barrier; A privacy protection unit, used to protect the privacy information of legitimate users, and the privacy information includes: personal information, account information, and transaction information; A data encryption unit, used to encrypt and decrypt the interaction data between the processor module and other modules, to protect the interaction data and prevent the interaction data from being illegally stolen and maliciously tampered with.

8. The intelligent and convenient public service system based on XR technology according to claim 1, characterized in that It further includes: An environment monitoring layer, connected to the central control layer, including: an environment video monitoring module, an environment sound monitoring module, an environment condition monitoring module, and an environment personnel monitoring module, used to collect video information and audio information of the current environment, monitor the environment condition and personnel condition of the current environment in real time, and generate environment monitoring data and send it to the central control layer for the central control layer to judge the risk situation of the current environment and generate early warning information.

9. The intelligent and convenient public service system based on XR technology according to claim 8, wherein It further includes: An information communication layer, connected to the central control layer and connected to an external operation center, used to send the generated service data, environment monitoring data, and early warning information to the external operation center.

10. A method for operating a smart and convenient public service based on XR technology, characterized in that, Applied to the intelligent and convenient service system based on XR technology as described in any one of claims 1 to 9, the method includes: Collect the user information of the current user through the information collection module of the immersive interaction layer; Through the central control layer, according to the collected user information, obtain the application service requirements of the current user, and coordinate the configurable application layer to configure the corresponding application module; Through the central control layer, promote the joint operation of the virtual reality technology module, the augmented reality technology module, and the mixed reality technology module of the immersive technology layer, so as to control the scene display module based on the support of immersive technology services, provide the immersive service scene of the application module for the current user, and generate the service data of the current user.

Images