Modularized upgradable capsule type element universe office system and method

Abstract

The invention provides a modular upgradable capsule type meta-universe office system and method. The system comprises a physical office cabin layer and a meta-universe platform layer, the physical office cabin layer comprises a standardized capsule office cabin body, a pluggable core calculation module and an upgradable perception interaction module; the internal space of the standardized capsule office cabin serves as a credible office area, and the standardized capsule office cabin is provided with a module interface backboard; the pluggable core calculation module is detachably connected with the module interface backboard, and the pluggable core calculation module comprises a local security processing unit and an encryption communication unit; the upgradable perception interaction module is detachably connected with the module interface backboard and is in communication connection with the pluggable core calculation module, and the upgradable perception interaction module comprises a multi-mode credible data acquisition module; the encryption communication unit is in communication connection with the meta universe platform layer; and improvement of enterprise management efficiency and optimization of remote cooperation experience are realized.

Description

Technical Field

[0001] This invention relates to the field of metaverse office management technology, and in particular to a modular and upgradeable capsule-type metaverse office system and method. Background Technology

[0002] Currently, the traditional centralized office model centered on major cities faces multiple structural challenges: companies bear high office rents and operating costs; employees endure significant commuting time and housing pressures; the "siphoning effect" of talent and job opportunities to large cities further exacerbates the imbalance in regional economic development; remote work is seen as a solution, but the existing work-from-home model has significant flaws: 1. Space and efficiency issues: The home environment is full of distractions, lacks a sense of ritual and dedicated space for work, resulting in low work efficiency.

[0003] 2. Management and collaboration issues: Enterprises find it difficult to effectively grasp the true working status of their employees, and simple video conferencing tools cannot provide an immersive team collaboration experience.

[0004] 3. Privacy and Trust Crisis: The monitoring measures taken by enterprises to ensure work efficiency (such as keeping cameras on continuously) are in serious conflict with the privacy rights of employees' families, lacking a technological foundation of trust.

[0005] In summary: First, integrated office equipment (such as smart desks and office pods) cannot keep up with rapid technological iterations, which can easily lead to the obsolescence of the entire system; second, cloud-based remote monitoring solutions pose a risk of original privacy data leakage, and the monitoring boundaries are blurred, making it impossible to guarantee privacy from a technical perspective; third, the existing metaverse office system lacks linkage with trusted physical entities, resulting in the user status in the virtual world being unknowable and uncontrollable. Therefore, the industry urgently needs a systematic solution that can fundamentally break the above deadlock. It can not only provide an immersive and efficient work experience, but also, through the technical architecture itself, ensure the efficiency of enterprise management while achieving absolute respect for employee privacy. This application provides a modular and upgradeable capsule-type metaverse office system and method. Summary of the Invention

[0006] This invention provides a modular and upgradeable capsule-style metaverse office system and method. By establishing a closed-loop technology system encompassing space authentication, privacy classification, local computation and destruction, and metaverse synchronization, it fundamentally solves the challenges of privacy security and management efficiency in remote work. It authenticates physical office pods as trusted work areas, providing a secure boundary for subsequent data processing. Real-time local data classification and processing ensures that privacy-sensitive data is destroyed immediately after feature extraction, achieving "data without leaving the domain, privacy without leakage," thus eliminating the risk of privacy infringement from a technical architecture perspective. Simultaneously, the method encrypts and synchronizes desensitized trusted status data (such as dress code compliance and work posture) to the metaverse, providing enterprises with authentic and reliable management data and ensuring the real-time nature and immersive experience of virtual avatar states. Ultimately, while absolutely respecting employee privacy, it improves enterprise management efficiency and optimizes the remote collaboration experience.

[0007] A modular and upgradeable capsule-type metaverse office system, comprising: a physical office pod layer and a metaverse platform layer; The physical office cabin layer includes: a standardized capsule office cabin body, a pluggable core computing module, and an upgradeable sensing and interaction module; The standardized capsule office pod has an internal space serving as a trusted office area and is equipped with a module interface backplane. The pluggable core computing module is detachably connected to the module interface backplane and includes a local security processing unit and an encrypted communication unit. The upgradeable perception and interaction module is detachably connected to the module interface backplane and communicates with the pluggable core computing module. The upgradeable perception and interaction module includes a multimodal trusted data acquisition module. The encrypted communication unit communicates with the metaverse platform layer. The multimodal trusted data acquisition module is used to acquire raw data streams. The local security processing unit includes: a data classification engine and a privacy computing module; The data classification engine is used to perform real-time analysis and classification of the raw data stream collected by the multimodal trusted data acquisition module, and obtain privacy-sensitive data and non-privacy state data based on the classification results; the privacy computing module is used to process the privacy-sensitive data locally to generate de-identified data, and at the same time, forcibly destroy the privacy-sensitive data. The encrypted communication unit is used to encrypt the non-privacy state data and / or desensitized data and transmit it to the metaverse platform layer.

[0008] Preferably, a modular and upgradeable capsule-type metaverse office system is provided, wherein the module interface backplane integrates a standardized data bus, power bus, mechanical locking mechanism and module identity authentication circuit; after the pluggable core computing module and the upgradeable sensing interaction module are physically connected, they must pass the verification of the module identity authentication circuit before they can be powered on and the driver program can be loaded.

[0009] Preferably, a modular and upgradeable capsule-type metaverse office system, wherein the upgradeable sensing and interaction module is a multimodal trusted data acquisition module, and the data path is controlled by a physical switch or a sensor linked to the hatch.

[0010] Preferably, a modular and upgradeable capsule-type metaverse office system is provided, wherein the privacy computing module runs in a secure enclave of the processor and is used to immediately trigger a permanent deletion mechanism of the original data after completing the feature extraction of privacy-sensitive data.

[0011] A modular and scalable capsule-style metaverse office method includes: Register the capsule office pod on the management platform and logically authenticate the internal space of the registered capsule office pod as a trusted office area; When the trusted office area is in office mode, raw data is collected and classified for privacy to obtain non-privacy state data and privacy-sensitive data. For privacy-sensitive data, features are extracted locally in a trusted office area using privacy computing to obtain de-identified data, and the corresponding original data is then destroyed. Non-privacy state data and / or de-identified data are encrypted and synchronously transmitted to the metaverse platform layer to update the virtual avatar state of the corresponding user.

[0012] Preferably, a modular and scalable capsule-style metaverse office method includes: When a target functional module to be upgraded is detected in the system, the target functional module includes: a pluggable core computing module and an upgradeable perception and interaction module; Complete the physical replacement on the interface backplane of the target functional module to be upgraded, and start the system after the physical replacement is completed; Once the system starts up, it automatically identifies the upgraded target function module installed based on the pluggable core computing module and loads the driver corresponding to the upgraded target function module.

[0013] Preferably, a modular and scalable capsule-style metaverse office method involves collecting raw data and classifying the raw data for privacy when the trusted office area is in office mode, obtaining non-privacy state data and privacy-sensitive data, including: When the trusted office area is in office mode, the upgradeable sensing and interaction module is activated, and multimodal data streams of the trusted office area are collected based on the upgradeable sensing and interaction module. The multimodal data streams include: video stream data, audio stream data, and environmental sensing data. The multimodal data streams are fused and correlated, and the data are classified into non-privacy state data, first-category privacy-sensitive data, second-category privacy-sensitive data, or third-category privacy-sensitive data based on the analysis results.

[0014] Preferably, a modular and scalable capsule-style metaverse office method, wherein the fusion and correlation analysis of multimodal data streams includes: Extract voiceprint features from audio stream data, and separate user speech from environmental background noise based on voiceprint features in audio stream data; Detect spatial environmental anomalies using environmental sensor data and output reliable environmental status characteristics of the office area; The video stream data is input into a neural network model deployed in a pluggable core computing module for real-time semantic segmentation, to obtain multiple sub-semantic regions, and to add semantic labels to each sub-semantic region. At the same time, the regional features of each sub-semantic region are determined based on the semantic labels. The regional features of each sub-semantic region, the voiceprint features of the user's voice, and the environmental state features of the trusted office area are input into the multimodal feature fusion matrix; Spatiotemporal alignment is performed on each modal feature in the multimodal feature fusion matrix to establish temporal correlations between features; Based on temporal correlation, the facial regions in the video stream data are matched with the voiceprint features captured synchronously in the audio stream data to identify the person. When the face region and voiceprint feature are successfully matched, the target region feature and voiceprint feature of the current face region are marked to generate the first type of privacy-sensitive data; otherwise, non-privacy state data is generated. When the environmental anomaly detection outputs abnormal environmental state characteristics, the multimodal data collected during the abnormal period is marked as Category II privacy-sensitive data; otherwise, non-privacy state data is generated.

[0015] Preferably, a modular and upgradeable capsule-style metaverse office method, wherein the fusion and correlation analysis of multimodal data streams further includes: performing temporal consistency verification on the lip movement region in video data and the speech content in audio data; When the time sequence consistency verification passes, the corresponding voice content text features are marked as non-privacy state data; If the time sequence consistency verification fails, the corresponding voice data and associated facial features will be jointly marked as Category III privacy-sensitive data.

[0016] Preferably, a modular and upgradeable capsule-style metaverse office method, wherein the temporal consistency verification of the lip movement region in video data and the speech content in audio data includes: Extract motion feature sequences of the lip movement region from video data, the motion features including lip opening and closing frequency and shape change trajectory; The acoustic feature sequence of speech content is extracted from audio data, wherein the acoustic features include pitch period and formant transitions; Dynamic time warping is performed on the motion feature sequence and the acoustic feature sequence to eliminate time axis deviation caused by differences in speech rate; Calculate the cross-correlation function between the normalized motion feature sequence and the acoustic feature sequence, and obtain the maximum correlation coefficient between the sequences; The maximum correlation coefficient is compared with a preset threshold to determine whether the time series consistency verification is successful. When the maximum correlation coefficient exceeds a preset threshold, the timing consistency verification is deemed successful.

[0017] Preferably, a modular and scalable capsule-style metaverse office method extracts features from privacy-sensitive data locally in a trusted office area through privacy computing to obtain de-identified data, and destroys the corresponding original data, including: The privacy-sensitive data obtained from the classification is transmitted to the privacy computing module of the pluggable core computing module; In the secure enclave environment of the privacy computing module, a feature extraction algorithm is performed on the input privacy-sensitive data, and a corresponding feature vector is generated from the privacy-sensitive data based on the feature extraction algorithm; After the feature vector is generated, the original data destruction mechanism is immediately triggered to permanently delete the original data of the privacy-sensitive data.

[0018] Preferably, a modular and upgradeable capsule-style metaverse office method encrypts and synchronously transmits non-privacy state data and / or anonymized data to the metaverse platform layer, updating the corresponding user's virtual avatar state, including: Add timestamps and spatial coordinate labels to non-privacy state data to generate state data packets; Add user identification and data integrity verification codes to the anonymized data to generate a feature data package; An encrypted communication unit based on a pluggable core computing module encrypts status data packets and feature data packets according to a preset key; Establish a secure communication link between the encrypted communication unit and the metaverse platform layer, and transmit the encrypted status data packets and feature data packets to the metaverse platform; When the metaverse platform layer receives the state data packet and feature data packet, it updates the action state and environmental interaction data of the corresponding user virtual avatar according to the user identity identifier.

[0019] Preferably, a modular and upgradeable capsule-style metaverse office method, wherein updating the corresponding user avatar's action state and environmental interaction data includes: using the non-privacy state data to drive the avatar's basic action model, and using the desensitized data to drive the avatar's refined facial expression or lip-reading model.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows: By establishing a closed-loop technology system encompassing space authentication, privacy classification, local computation and destruction, and metaverse synchronization, this approach fundamentally solves the challenges of privacy security and management efficiency in remote work. Authentication of physical office pods as trusted work areas provides a secure boundary for subsequent data processing. Real-time local data classification and processing ensures that privacy-sensitive data is destroyed immediately after feature extraction, achieving "data without leaving the domain, privacy without leakage," thus eliminating the risk of privacy violations from a technical architecture perspective. Simultaneously, this method encrypts and synchronizes de-identified trusted status data (such as dress code compliance and work posture) to the metaverse, providing enterprises with authentic and reliable management data and ensuring the real-time nature and immersive experience of virtual avatar states. Ultimately, while absolutely respecting employee privacy, it enhances enterprise management efficiency and optimizes the remote collaboration experience.

[0021] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in this application.

[0022] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0023] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram illustrating the overall architecture of a modular and upgradeable capsule-type metaverse office system and its connection with the metaverse platform layer in an embodiment of the present invention. Figure 2 This is a flowchart of a modular and upgradeable capsule-type metaverse office method in an embodiment of the present invention; Figure 3 This is a flowchart illustrating the modular upgrade process in a modular and upgradeable capsule-type metaverse office method according to an embodiment of the present invention. Detailed Implementation

[0024] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0025] Example 1: This example provides a modular and upgradeable capsule-style metaverse office system, such as... Figure 1 As shown, it includes: the physical office module layer and the metaverse platform layer; The physical office cabin layer includes: a standardized capsule office cabin body, a pluggable core computing module, and an upgradeable sensing and interaction module; The standardized capsule office pod has an internal space serving as a trusted office area and is equipped with a module interface backplane. The pluggable core computing module is detachably connected to the module interface backplane and includes a local security processing unit and an encrypted communication unit. The upgradeable perception and interaction module is detachably connected to the module interface backplane and communicates with the pluggable core computing module. The upgradeable perception and interaction module includes a multimodal trusted data acquisition module. The encrypted communication unit communicates with the metaverse platform layer. The multimodal trusted data acquisition module is used to acquire raw data streams. The local security processing unit includes: a data classification engine and a privacy computing module; The data classification engine is used to perform real-time analysis and classification of the raw data stream collected by the multimodal trusted data acquisition module, and obtain privacy-sensitive data and non-privacy state data based on the classification results; the privacy computing module is used to process the privacy-sensitive data locally to generate de-identified data, and at the same time, forcibly destroy the privacy-sensitive data. The encrypted communication unit is used to encrypt the non-privacy state data and / or desensitized data and transmit it to the metaverse platform layer.

[0026] In this embodiment, the module interface backplane integrates a standardized data bus, power bus, mechanical locking mechanism, and module authentication circuit. After the pluggable core computing module and the upgradeable sensing interaction module are physically connected, they must pass the verification of the module authentication circuit before they can be powered on and the driver can be loaded.

[0027] In this embodiment, the multimodal trusted data acquisition module of the upgradeable sensing interaction module has its data path controlled by a physical switch or a sensor linked to the hatch. Both the pluggable core computing module and the upgradeable sensing interaction module have pre-installed signed device digital certificates on their interfaces. When the module is inserted into the backplane and a physical connection is established, the security chip on the backplane reads the module's device certificate through dedicated contacts and verifies it against the trusted root certificate pre-installed in the security chip. Only when the certificate chain verification is successful and the device is not on the revocation list will the security chip send an enable signal to the power management chip on the backplane, allowing power to be supplied to the module. Subsequently, when the operating system kernel of the pluggable core computing module detects new hardware, it loads the corresponding driver from the protected driver memory area based on the verified device ID.

[0028] In this embodiment, the privacy computing module runs in a secure enclave of the processor and is used to immediately trigger a permanent deletion mechanism for the original data after completing the feature extraction of privacy-sensitive data.

[0029] In this embodiment, the standardized capsule office pods can be deployed in employees' homes. By deploying them in a fixed area at home, privacy issues are reduced, and employees can enter a working state as soon as they enter the work pod. Furthermore, the standardized capsule office pods can be re-registered. For example, when an employee's job changes, the registration information of the standardized capsule office pod can be changed to manage other employees or other companies.

[0030] The working principle of the above technical solution is as follows: the module interface backplane is the core connection hub fixed inside the cabin, providing standard snap-fit ​​rails, high-speed data interfaces (such as USB4), and high-power power interfaces. Both the pluggable core computing module and the upgradeable sensing and interaction module obtain power and communicate with the system through this backplane.

[0031] Employees enter the cabin, close the physical switch, and the system starts up. The cameras in the upgradable perception and interaction module begin working, and the video stream is sent to the pluggable core computing module. The data classification engine within the module runs in real time, marking faces and backgrounds in the video footage as "privacy-sensitive data," and areas such as clothing collars as "non-privacy data." Subsequently, within a secure enclave, the privacy computing module analyzes only the non-privacy data, and upon determining a "compliant / non-compliant" conclusion, immediately destroys the original image of that frame. The conclusion information is encrypted and sent to the metaverse platform, driving the employee's virtual avatar state.

[0032] When an upgrade is needed, users can purchase a new generation of pluggable core computing modules (such as versions with more powerful AI chips) or a higher-resolution upgradeable perception and interaction module separately. To replace the module, simply disconnect the power, remove the old module, insert the new module into the backplane, and lock it in place. The system will automatically recognize and enable the new module after a reboot.

[0033] The beneficial effects of the above technical solutions are: providing a reliable distributed office infrastructure, effectively reducing costs for enterprises and individuals; modular design allows hardware to iterate like software, and users can upgrade computing power or sensors in a targeted manner, greatly reducing long-term ownership costs; through the technical closed loop of "space definition," "data classification," and "local destruction," it ensures that sensitive data never leaves the domain, complies with the most stringent privacy regulations in terms of architecture, and establishes a cornerstone of technological trust; it provides enterprises with reliable status data from a trusted space, while providing an immersive collaborative experience through the metaverse interactive terminal, achieving a balance between management efficiency and employee experience.

[0034] Example 2: A modular and scalable capsule-style metaverse office method, used to implement the following steps based on a modular and scalable capsule-style metaverse office system as described in Example 1, such as... Figure 2 As shown, it includes: Step 1: Register the capsule office pod on the management platform and logically authenticate the internal space of the registered capsule office pod as a trusted office area; Step 2: When the trusted office area is in office mode, collect raw data and classify the raw data for privacy to obtain non-privacy state data and privacy-sensitive data; Step 3: Extract features from privacy-sensitive data locally in a trusted office area using privacy computing to obtain de-identified data, and then destroy the corresponding original data; Step 4: Encrypt non-privacy state data and / or de-identified data and transmit them synchronously to the metaverse platform layer to update the virtual avatar state of the corresponding user.

[0035] In this embodiment, the trusted office area refers to the physical and logical space inside the standardized capsule office pod, which has a clear data security policy after being logically authenticated by the management platform; data collected in this area is considered to be generated in a controlled and trusted environment.

[0036] In this embodiment, privacy-sensitive data refers to data in the raw data stream that has been classified and identified and can directly or indirectly identify a specific natural person or reflect their private activities. Examples include facial images, voiceprint features, and background environment information.

[0037] In this embodiment, non-privacy state data refers to data identified through classification in the raw data stream that is unrelated to personal identity, does not involve private activities, and is only used to represent work status. Examples include preset office dress codes and key points of body posture.

[0038] In this embodiment, desensitized data refers to data that cannot be reversed to restore the original information after localized privacy calculations (such as feature extraction) are performed on privacy-sensitive data; for example, pose feature vectors extracted from face images, or text content converted from speech.

[0039] In this embodiment, when a user uses the capsule office pod for the first time, they need to log in to the enterprise management platform through the in-pod terminal and complete two-way identity authentication. The platform issues a unique, encrypted space credential to the secure storage area of ​​the pod's pluggable core computing module. Simultaneously, the system initiates an automatic space calibration procedure: guiding the user to stand at a designated location inside the pod, the camera captures a reference image, which is matched with a pre-stored 3D model of the pod to calibrate the camera's internal and external parameters, and this calibration data is bound to the space credential. Afterward, the internal space of the pod is logically authenticated as a trusted office area belonging to the corresponding user. Only when the system detects a valid space credential and the sensor data conforms to the calibration range will it enter office mode and initiate the data acquisition process.

[0040] The beneficial effects of the above technical solution are as follows: By establishing a closed-loop technology of spatial authentication, privacy classification, local computation and destruction, and metaverse synchronization, the problem of privacy security and management efficiency in remote work is fundamentally solved; authenticating physical office pods as trusted office areas provides a secure boundary for subsequent data processing; by classifying and processing data locally in real time, it ensures that privacy-sensitive data is destroyed immediately after feature extraction, achieving "data does not leave the domain, privacy is not leaked," thus eliminating the risk of privacy infringement from a technical architecture perspective; at the same time, this method encrypts and synchronizes desensitized trusted status data (such as dress compliance, work posture, etc.) to the metaverse, providing enterprises with real and credible management basis and ensuring the real-time nature and immersive experience of virtual avatar status; ultimately, while absolutely respecting employee privacy, it achieves improved enterprise management efficiency and optimized remote collaboration experience.

[0041] Example 3: Based on Example 2, this example provides a modular and upgradeable capsule-style metaverse office method, such as... Figure 3 As shown, it includes: S1: When a target functional module to be upgraded is detected in the system, the target functional module includes: a pluggable core computing module and an upgradeable perception and interaction module. S2: Complete the physical replacement on the interface backplane of the target functional module to be upgraded, and start the system after the physical replacement is completed; S3: After the system starts, it automatically identifies the upgraded target function module installed based on the pluggable core computing module and loads the driver corresponding to the upgraded target function module.

[0042] In this embodiment, the target functional module refers to the hardware unit in the system that supports hot-swappable replacement, specifically the pluggable core computing module or the upgradeable perception and interaction module.

[0043] In this embodiment, physical replacement refers to the hardware replacement operation of removing the old module from the module interface backplane and installing the new module onto the backplane.

[0044] In this embodiment, automatic identification refers to the ability of the pluggable core computing module to automatically detect newly installed hardware by enumerating bus devices and reading device IDs when the system starts up.

[0045] The beneficial effects of the above technical solution are: through a closed-loop process of module identification, physical replacement and automatic driving, on-demand upgrades of hardware functions are realized, significantly extending the product life cycle and reducing the long-term usage costs for users.

[0046] Example 4: Based on Example 2, this example provides a modular and upgradeable capsule-style metaverse office method. When the trusted office area is in office mode, raw data is collected and the raw data is classified for privacy to obtain non-privacy state data and privacy-sensitive data, including: When the trusted office area is in office mode, the upgradeable sensing and interaction module is activated, and multimodal data streams of the trusted office area are collected based on the upgradeable sensing and interaction module. The multimodal data streams include: video stream data, audio stream data, and environmental sensing data. The multimodal data streams are fused and correlated, and the data are classified into non-privacy state data, first-category privacy-sensitive data, second-category privacy-sensitive data, or third-category privacy-sensitive data based on the analysis results.

[0047] Among them, the fusion and correlation analysis of multi-modal data streams includes: extracting voiceprint features from audio stream data, separating user speech from environmental background noise in audio stream data based on voiceprint features, and extracting voiceprint features from the separated user speech. Simultaneously, the spatial environment anomaly detection is performed on the environmental sensor data, and the environmental status characteristics of the reliable office area are output. The video stream data is input into a neural network model deployed in a pluggable core computing module for real-time semantic segmentation, to obtain multiple sub-semantic regions, and to add semantic labels to each sub-semantic region. At the same time, the regional features of each sub-semantic region are determined based on the semantic labels. The regional features of each sub-semantic region, the voiceprint features of the user's voice, and the environmental state features of the trusted office area are input into the multimodal feature fusion matrix; Spatiotemporal alignment is performed on each modal feature in the multimodal feature fusion matrix to establish temporal correlations between features; Based on temporal correlation, the facial regions in the video stream data are matched with the voiceprint features captured synchronously in the audio stream data to identify the person. When the face region and voiceprint feature are successfully matched, the target region feature and voiceprint feature of the current face region are marked to generate the first type of privacy-sensitive data; otherwise, non-privacy state data is generated. When the environmental anomaly detection outputs abnormal environmental state characteristics, the multimodal data collected during the abnormal period is marked as Category II privacy-sensitive data; otherwise, non-privacy state data is generated.

[0048] The fusion and correlation analysis of multimodal data streams also includes: verifying the temporal consistency between the lip movement region in video data and the speech content in audio data; If the time sequence consistency verification passes, the corresponding voice content text features are marked as non-privacy state data; otherwise, they are marked as third-category privacy-sensitive data. Based on the aforementioned non-privacy state data and the first type of privacy-sensitive data, the second type of privacy-sensitive data, and the third type of privacy-sensitive data, the original data is classified for privacy.

[0049] In this embodiment, a trusted office area refers to a physical or virtual office space designed to be secure and controlled for metaverse office activities, ensuring that data collection and processing are carried out in a trusted environment.

[0050] In this embodiment, the upgradeable sensing and interaction module refers to an upgradeable hardware or software module responsible for collecting multimodal data streams (including video, audio, and environmental sensor data) to enhance sensing and interaction capabilities.

[0051] In this embodiment, the pluggable core computing module refers to a core processing unit that can be easily inserted or removed for deploying and running computational tasks such as neural network models, facilitating system maintenance and performance upgrades.

[0052] In this embodiment, the multimodal feature fusion matrix refers to a data structure or algorithm framework used to integrate features from different modalities (such as video, audio, and environment) and establish the correlation between features through spatiotemporal alignment operations. In essence, it is a time alignment module that uses a Kalman filter to smoothly align the timestamps of visual and auditory features, and concatenates the aligned feature vectors into a joint feature vector, which is then input into a fully connected layer for classification decision.

[0053] In this embodiment, the first type of privacy-sensitive data refers to personal biometric data that is confirmed by successfully matching facial regions with voiceprint features, and has high privacy sensitivity.

[0054] In this embodiment, the second type of privacy-sensitive data refers to multimodal data collected during periods of abnormal environmental conditions, which may contain sensitive information and require special protection.

[0055] In this embodiment, the third type of privacy-sensitive data refers to the data that fails to verify the temporal consistency between lip movements and speech content, indicating that there may be inconsistencies or anomalies, which require privacy attention.

[0056] In this embodiment, non-privacy state data refers to data that has been verified and is deemed not to contain sensitive information, and can be used or shared securely without special privacy protection.

[0057] In this embodiment, the data classification engine runs on an AI-dedicated processor (such as an NPU) with a pluggable core computing module; the neural network model it implements is a lightweight multi-task learning model, with the backbone network using MobileNetV3 and connected to three output heads: one for semantic segmentation of video frames (outputting pixel-level labels for faces, bodies, backgrounds, etc.), one for voiceprint feature extraction of audio (using a pre-trained x-vector model), and one for anomaly detection of sensor data (based on a gated recurrent unit GRU).

[0058] The beneficial effects of the above technical solution are as follows: Multimodal data streams are collected in real time through an upgradeable sensing and interaction module, and processed efficiently using a pluggable core computing module. Combined with a multimodal feature fusion matrix, spatiotemporal alignment and feature association of video, audio, and environmental data are achieved, thereby accurately identifying and classifying privacy data. This effectively distinguishes between non-privacy data and various types of privacy-sensitive data, significantly improving data security and user privacy protection in the metaverse office scenario. Simultaneously, the modular design supports flexible system upgrades and expansion, adapting to constantly changing office needs, enhancing the reliability and adaptability of the overall office experience, and avoiding the risk of privacy leaks.

[0059] Example 5: Based on Example 4, this example provides a modular and upgradeable capsule-style metaverse office method to verify the temporal consistency between the lip movement region in video data and the speech content in audio data, including: Extract motion feature sequences of the lip movement region from video data, the motion features including lip opening and closing frequency and shape change trajectory; The acoustic feature sequence of speech content is extracted from audio data, wherein the acoustic features include pitch period and formant transitions; Dynamic time warping is performed on the motion feature sequence and the acoustic feature sequence to eliminate time axis deviation caused by differences in speech rate; Calculate the cross-correlation function between the normalized motion feature sequence and the acoustic feature sequence, and obtain the maximum correlation coefficient between the sequences; The maximum correlation coefficient is compared with a preset threshold to determine whether the time series consistency verification is successful. When the maximum correlation coefficient exceeds a preset threshold, the time series consistency verification is deemed successful. Establish association rules between verification results and data classification. When the time-series consistency verification passes, the corresponding voice content text features are marked as non-privacy state data. Voice data that fails verification is categorized together with the corresponding facial features as privacy-sensitive data.

[0060] In this embodiment, the lip movement region refers to the facial lip region captured in the video data, which is used to analyze the lip shape changes during speech.

[0061] In this embodiment, the motion feature sequence refers to the feature data extracted continuously from the lip movement area and arranged in chronological order, including the frequency of lip opening and closing and the trajectory of shape changes.

[0062] In this embodiment, the acoustic feature sequence refers to speech feature data extracted from audio data and arranged in chronological order, including parameters such as pitch period and formant transitions.

[0063] In this embodiment, dynamic time warping refers to a time alignment algorithm used to eliminate time axis deviations caused by rate differences between two time series.

[0064] In this embodiment, the cross-correlation function refers to a mathematical function used to measure the similarity between two normalized sequences. The correlation between the sequences can be obtained by calculating its maximum value.

[0065] The beneficial effects of the above technical solution are as follows: by aligning the time series of lip movements and speech features through dynamic time warping, and using cross-correlation functions to quantify the degree of synchronization between the two, accurate temporal consistency verification of lip reading and speech content is achieved. It can effectively identify abnormal situations where speech data and visual performance are inconsistent, thereby accurately distinguishing between non-privacy state data and privacy-sensitive data, significantly improving the accuracy of speech data classification in the metaverse office environment, and enhancing the reliability of user privacy protection.

[0066] Example 6: Based on Example 2, this example provides a modular and upgradeable capsule-style metaverse office method. This method extracts features from privacy-sensitive data locally in a trusted office area using privacy computing to obtain de-identified data and destroys the corresponding original data, including: The privacy-sensitive data obtained from the classification is transmitted to the privacy computing module of the pluggable core computing module; In the secure enclave environment of the privacy computing module, a feature extraction algorithm is performed on the input privacy-sensitive data, and a corresponding feature vector is generated from the privacy-sensitive data based on the feature extraction algorithm; After the feature vector is generated, the original data destruction mechanism is immediately triggered to permanently delete the original data of the privacy-sensitive data.

[0067] In this embodiment, video data classified as privacy-sensitive is input into a secure enclave of the privacy computing module; within the secure enclave, facial region blurring is performed on the video data, and the coordinates of key points of human posture are extracted; audio data classified as privacy-sensitive is input into the secure enclave; within the secure enclave, voiceprint feature removal is performed on the audio data, and the text features of the speech content are extracted; the coordinates of the key points of human posture and the text features of the speech content are combined into desensitized data for output; and the original data of the privacy-sensitive video and audio data is destroyed immediately after feature extraction is completed.

[0068] In this embodiment, the privacy computing module refers to a hardware or software unit within the pluggable core computing module specifically responsible for securely processing privacy data, possessing encryption and secure isolation capabilities. It relies on a secure enclave technology provided by the processor. Within this secure world, a protected trusted application runs. When privacy-sensitive data (such as face region image blocks) identified by the classification engine is input, the trusted application invokes a built-in feature extraction algorithm (e.g., a lightweight face recognition network trained using the ArcFace loss function, retaining only the feature vectors before the fully connected layers). The feature extraction algorithm completes all calculations within the secure enclave. After feature extraction is complete, the trusted application immediately sends an instruction to the static memory controller within the secure enclave to execute a physical memory erase instruction (such as memset_s) on the secure memory area storing the original image data and disable caching, ensuring the original data cannot be recovered. The generated desensitized feature vector is then transmitted to the ordinary world through the enclave's internal and external communication interface for subsequent encrypted transmission.

[0069] In this embodiment, the secure enclave environment refers to an isolated execution environment built through hardware security technology, which can ensure that code and data are protected from external attacks or unauthorized access during the computation process.

[0070] In this embodiment, the feature extraction algorithm refers to a specific computational method for extracting representative feature vectors from raw privacy data, aiming to retain data utility while removing personal identification information.

[0071] In this embodiment, the original data destruction mechanism refers to a physical or logical security process that can permanently delete the original privacy data, ensuring that the data is unrecoverable.

[0072] The beneficial effects of the above technical solution are: by performing feature extraction and immediately destroying the original data in a secure enclave environment, localized desensitization processing of privacy-sensitive data is achieved, which not only preserves the effective features of the data, but also fundamentally eliminates the risk of leakage of original privacy data, ensuring the security and compliance of the data processing process.

[0073] Example 7: Based on Example 2, this example provides a modular and upgradeable capsule-style metaverse office method, which encrypts and synchronously transmits non-privacy state data and / or de-identified data to the metaverse platform layer, updating the virtual avatar state of the corresponding user, including: Add timestamps and spatial coordinate labels to non-privacy state data to generate state data packets; Add user identification and data integrity verification codes to the anonymized data to generate a feature data package; An encrypted communication unit based on a pluggable core computing module encrypts status data packets and feature data packets according to a preset key; Establish a secure communication link between the encrypted communication unit and the metaverse platform layer, and transmit the encrypted status data packets and feature data packets to the metaverse platform; When the metaverse platform layer receives the state data packet and feature data packet, it updates the action state and environmental interaction data of the corresponding user virtual avatar according to the user identity identifier.

[0074] In this embodiment, the state data packet refers to a structured data unit formed by adding timestamps and spatial coordinate labels to non-privacy state data.

[0075] In this embodiment, the feature data packet refers to the structured data unit formed by adding user identity identifiers and data integrity verification codes to de-identified data.

[0076] In this embodiment, the encrypted communication unit refers to a hardware or software component in the pluggable core computing module that is specifically responsible for data encryption and secure transmission.

[0077] In this embodiment, the secure communication link refers to the encrypted data transmission channel established between the encrypted communication unit and the metaverse platform layer.

[0078] In this embodiment, the metaverse platform layer refers to the infrastructure platform that provides metaverse virtual environment services and is responsible for the rendering and state management of virtual avatars.

[0079] In this embodiment, updating the action state and environmental interaction data of the corresponding user avatar includes: using the non-privacy state data to drive the basic action model of the avatar, and using the desensitized data to drive the refined expression or lip-reading model of the avatar.

[0080] The beneficial effects of the above technical solution are as follows: by encrypting the transmission of status data packets and feature data packets through the encrypted communication unit, a complete data security link is established from the terminal to the metaverse platform layer. While ensuring the security of data transmission, by adding timestamps, spatial coordinates and integrity verification codes, the accurate restoration of data at the platform layer and the real-time synchronization of virtual avatars are ensured, thereby improving the realism of the metaverse office environment and the reliability of interaction.

[0081] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A modular and upgradeable capsule-style metaverse office system, characterized in that, include: Physical office module layer and metaverse platform layer; The physical office cabin layer includes: a standardized capsule office cabin body, a pluggable core computing module, and an upgradeable sensing and interaction module; The standardized capsule office pod has an internal space serving as a trusted office area and is equipped with a module interface backplane. The pluggable core computing module is detachably connected to the module interface backplane and includes a local security processing unit and an encrypted communication unit. The upgradeable perception and interaction module is detachably connected to the module interface backplane and communicates with the pluggable core computing module. The upgradeable perception and interaction module includes a multimodal trusted data acquisition module. The encrypted communication unit communicates with the metaverse platform layer. The multimodal trusted data acquisition module is used to acquire raw data streams. The local security processing unit includes: a data classification engine and a privacy computing module; The data classification engine is used to perform real-time analysis and classification of the raw data stream collected by the multimodal trusted data acquisition module, and obtain privacy-sensitive data and non-privacy state data based on the classification results; the privacy computing module is used to process the privacy-sensitive data locally to generate de-identified data, and at the same time, forcibly destroy the privacy-sensitive data. The encrypted communication unit is used to encrypt the non-privacy state data and / or desensitized data and transmit it to the metaverse platform layer.

2. The modular and upgradeable capsule-type metaverse office system according to claim 1, characterized in that, The module interface backplane integrates a standardized data bus, power bus, mechanical locking mechanism, and module authentication circuit. After the pluggable core computing module and the upgradeable sensing interaction module are physically connected, they must pass the verification of the module authentication circuit before they can be powered on and the driver can be loaded.

3. The modular and upgradeable capsule-type metaverse office system according to claim 1, characterized in that, The multimodal trusted data acquisition module of the upgradeable sensing and interaction module has its data path controlled by physical switches or sensors linked to the hatch.

4. The modular and upgradeable capsule-type metaverse office system according to claim 1, characterized in that, The privacy computing module runs in a secure enclave of the processor and is used to immediately trigger a permanent deletion mechanism for the original data after completing the feature extraction of privacy-sensitive data.

5. A modular and upgradeable capsule-style metaverse office method, comprising the following steps based on a modular and upgradeable capsule-style metaverse office system as described in any one of claims 1-4, characterized in that, include: Register the capsule office pod on the management platform and logically authenticate the internal space of the registered capsule office pod as a trusted office area; When the trusted office area is in office mode, raw data is collected and classified for privacy to obtain non-privacy state data and privacy-sensitive data. For privacy-sensitive data, features are extracted locally in a trusted office area using privacy computing to obtain de-identified data, and the corresponding original data is then destroyed. Non-privacy state data and / or de-identified data are encrypted and synchronously transmitted to the metaverse platform layer to update the virtual avatar state of the corresponding user.

6. The modular and upgradeable capsule-style metaverse office method according to claim 5, characterized in that, include: When a target functional module to be upgraded is detected in the system, the target functional module includes: a pluggable core computing module and an upgradeable perception and interaction module; Complete the physical replacement on the interface backplane of the target functional module to be upgraded, and start the system after the physical replacement is completed; Once the system starts up, it automatically identifies the upgraded target function module installed based on the pluggable core computing module and loads the driver corresponding to the upgraded target function module.

7. A modular and upgradeable capsule-style metaverse office method according to claim 5, characterized in that, When the trusted office area is in office mode, raw data is collected and classified for privacy reasons to obtain non-privacy data and privacy-sensitive data, including: When the trusted office area is in office mode, the upgradeable sensing and interaction module is activated, and multimodal data streams of the trusted office area are collected based on the upgradeable sensing and interaction module. The multimodal data streams include: video stream data, audio stream data, and environmental sensing data. The multimodal data streams are fused and correlated, and the data are classified into non-privacy state data, first-category privacy-sensitive data, second-category privacy-sensitive data, or third-category privacy-sensitive data based on the analysis results.

8. A modular and upgradeable capsule-style metaverse office method according to claim 7, characterized in that, The fusion and correlation analysis of multimodal data streams includes: Extract voiceprint features from audio stream data, and separate user speech from environmental background noise based on voiceprint features in audio stream data; Detect spatial environmental anomalies using environmental sensor data and output reliable environmental status characteristics of the office area; The video stream data is input into a neural network model deployed in a pluggable core computing module for real-time semantic segmentation, to obtain multiple sub-semantic regions, and to add semantic labels to each sub-semantic region. At the same time, the regional features of each sub-semantic region are determined based on the semantic labels. The regional features of each sub-semantic region, the voiceprint features of the user's voice, and the environmental state features of the trusted office area are input into the multimodal feature fusion matrix; Spatiotemporal alignment is performed on each modal feature in the multimodal feature fusion matrix to establish temporal correlations between features; Based on temporal correlation, the facial regions in the video stream data are matched with the voiceprint features captured synchronously in the audio stream data to identify the person. When the face region and voiceprint feature are successfully matched, the target region feature and voiceprint feature of the current face region are marked to generate the first type of privacy-sensitive data; otherwise, non-privacy state data is generated. When the environmental anomaly detection outputs abnormal environmental state characteristics, the multimodal data collected during the abnormal period is marked as Category II privacy-sensitive data; otherwise, non-privacy state data is generated.

9. A modular and upgradeable capsule-style metaverse office method according to claim 8, characterized in that, The fusion and correlation analysis of multimodal data streams also includes: verifying the temporal consistency between the lip movement region in video data and the speech content in audio data; When the time sequence consistency verification passes, the corresponding voice content text features are marked as non-privacy state data; If the time sequence consistency verification fails, the corresponding voice data and associated facial features will be jointly marked as Category III privacy-sensitive data.

10. A modular and upgradeable capsule-style metaverse office method according to claim 9, characterized in that, The temporal consistency verification of the lip movement region in the video data and the speech content in the audio data includes: Extract motion feature sequences of the lip movement region from video data, the motion features including lip opening and closing frequency and shape change trajectory; The acoustic feature sequence of speech content is extracted from audio data, wherein the acoustic features include pitch period and formant transitions; Dynamic time warping is performed on the motion feature sequence and the acoustic feature sequence to eliminate time axis deviation caused by differences in speech rate; Calculate the cross-correlation function between the normalized motion feature sequence and the acoustic feature sequence, and obtain the maximum correlation coefficient between the sequences; The maximum correlation coefficient is compared with a preset threshold to determine whether the time series consistency verification is successful. When the maximum correlation coefficient exceeds a preset threshold, the timing consistency verification is deemed successful.

11. A modular and upgradeable capsule-style metaverse office method according to claim 5, characterized in that, For privacy-sensitive data, features are extracted locally in a trusted office area using privacy-preserving computation to obtain de-identified data, and the corresponding original data is then destroyed, including: The privacy-sensitive data obtained from the classification is transmitted to the privacy computing module of the pluggable core computing module; In the secure enclave environment of the privacy computing module, a feature extraction algorithm is performed on the input privacy-sensitive data, and a corresponding feature vector is generated from the privacy-sensitive data based on the feature extraction algorithm; After the feature vector is generated, the original data destruction mechanism is immediately triggered to permanently delete the original data of the privacy-sensitive data.

12. A modular and upgradeable capsule-style metaverse office method according to claim 5, characterized in that, Encrypt non-privacy state data and / or anonymized data and transmit them synchronously to the metaverse platform layer to update the virtual avatar state of the corresponding user, including: Add timestamps and spatial coordinate labels to non-privacy state data to generate state data packets; Add user identification and data integrity verification codes to the anonymized data to generate a feature data package; An encrypted communication unit based on a pluggable core computing module encrypts status data packets and feature data packets according to a preset key; Establish a secure communication link between the encrypted communication unit and the metaverse platform layer, and transmit the encrypted status data packets and feature data packets to the metaverse platform; When the metaverse platform layer receives the state data packet and feature data packet, it updates the action state and environmental interaction data of the corresponding user virtual avatar according to the user identity identifier.

13. A modular and upgradeable capsule-type metaverse office method according to claim 12, characterized in that, The update of the corresponding user avatar's action state and environmental interaction data includes: using the non-privacy state data to drive the avatar's basic action model, and using the desensitized data to drive the avatar's refined facial expression or lip-reading model.

Images