Intelligent sensing control system and method for magnetic levitation holographic display

Abstract

This invention relates to an intelligent perception control system and method for magnetic levitation holographic displays. The method operates on a display system integrating magnetic levitation drive, air projection holography, and multimodal perception units, and includes: identifying audience interaction intentions and generating structured descriptions through multimodal perception data fusion and context analysis; dynamically generating personalized display narrative lines that define entity motion sequences, holographic content, and their linkage relationships based on these descriptions; decoupling the narrative lines into time-synchronized magnetic levitation control command streams and holographic rendering command streams to drive precise collaborative display of entity motion and holographic images; the system includes perception fusion, intelligent decision-making, collaborative control, magnetic levitation drive, and air projection holography modules; this invention achieves a transformation from passive playback to active intelligent narrative, enhancing the interactivity, personalization, and immersion of the display.

Description

Technical Field

[0001] This invention relates to the field of intelligent interaction and augmented reality display technology, and in particular to an intelligent perception control system and method for magnetic levitation holographic display. Background Technology

[0002] Magnetic levitation displays and holographic projection technologies offer novel approaches to high-end exhibitions. However, existing technologies often simply combine the two or only achieve pre-programmed fixed interactions, lacking true intelligence. The main problem lies in: The presentation process is static or pre-planned and cannot be changed in real time according to the audience and environment.

[0003] The interaction is one-way or simple, and the system cannot understand the complex, impromptu intentions of the audience.

[0004] The virtual and real content are separated, and the physical movement and holographic images lack a semantically based, dynamically generated collaborative relationship, failing to achieve the enhancement effect of "1+1>2".

[0005] Lacking continuous optimization capabilities, the system's performance becomes stagnant and its display effects fail to evolve with the accumulation of usage data.

[0006] Therefore, existing solutions have significant bottlenecks in terms of intelligence, personalization, and adaptability, resulting in dull display effects and limited user experience. To address this, this invention proposes an intelligent sensing control system and method for magnetic levitation holographic display. Summary of the Invention

[0007] This invention addresses the technical problems existing in the prior art by providing an intelligent perception and control method and system for magnetic levitation holographic displays, thus solving the problems of rigid displays, stiff interactions, and a disconnect between the virtual and real worlds in existing technologies. The core of this method lies in constructing an intelligent closed loop of "perception-understanding-decision-generation-execution-optimization," enabling the display system to act like an "intelligent guide," proactively understanding the audience's intentions and dynamically generating personalized display narratives that deeply integrate virtual and real content.

[0008] The technical solution of this invention to solve the above-mentioned technical problems is as follows: an intelligent sensing and control method for magnetic levitation holographic display; comprising the following steps: The method, operating on an intelligent display system integrating a magnetic levitation drive unit, an air projection holographic unit, and a multimodal sensing unit, includes: Contextual understanding and intent recognition steps: Based on the multi-source data of the audience, environment and suspended entity status collected in real time by the multimodal perception unit, data fusion and contextual analysis are performed to identify the audience's interaction intent and generate a structured description containing intent type and target object; Dynamic content strategy generation steps: Based on the structured description, a personalized display narrative line is dynamically generated. The narrative line defines the expected motion sequence of the suspended entity, the associated holographic image content, and the spatiotemporal linkage between the two. Virtual-real collaborative execution steps: The personalized display narrative is decoupled into a time-synchronized magnetic levitation control command stream and a holographic rendering command stream, and then sent to the magnetic levitation drive unit and the air projection holographic unit for execution, so as to achieve precise collaborative display of the movement of the suspended entity and the changes of the holographic image.

[0009] Furthermore, the context understanding and intent recognition steps specifically include: Identify the audience's basic attributes, spatial location, and focus of attention through multimodal perception data; Recognize audience interaction gestures and / or voice commands; By integrating the semantics of the attention focus, interactive gestures, and / or voice commands, a multimodal reasoning model is used to determine the viewer's core interactive intent, which includes at least one of the following types: requesting detailed explanation, comparing and viewing, and story narration.

[0010] Furthermore, in the dynamic content strategy generation step, different content generation strategies are matched according to different intent types: When the intent type is "Request for details", a strategy is generated to control the suspended entity to maintain or move to the optimal viewing pose, while the holographic image presents the target object's internal structure, working principle animation, or related data charts. When the intent type is "comparison and viewing", a strategy is generated to control the floating entity to switch between different states, while the holographic image presents multiple comparison items or parameter change curves. When the intent type is "storytelling", a strategy is adopted to generate and control the suspended entity to move in anthropomorphic manner in coordination with the narrative, while holographic images present virtual characters and plot animations.

[0011] Furthermore, in the virtual-real collaborative execution step, a pre-established unified spatial coordinate system is used to ensure that the relative positional relationship between the spatial anchor point of the holographic image and the key parts of the suspended entity in three-dimensional space is accurate; and a precision clock synchronization protocol is used to ensure that the timestamp deviation between the magnetic levitation control command stream and the holographic rendering command stream is within a preset threshold range.

[0012] Furthermore, the method also includes: Closed-loop interaction and optimization steps: While the virtual-real collaborative execution steps are being performed, new feedback data from the audience is continuously collected; based on the new feedback data, the display parameters of the personalized display narrative line being executed are adjusted in real time; and the complete interaction process data is used to update the model used in the context understanding and intent recognition steps and / or the dynamic content strategy generation steps.

[0013] Furthermore, the real-time adjustment of display parameters includes at least one of the following: adjusting the playback speed of the holographic animation, changing the volume and speed of the narration, highlighting or enlarging the virtual component that the viewer is currently gazing at, and repeating the previous display content.

[0014] On the other hand, an intelligent sensing and control system for magnetic levitation holographic display is provided, for implementing the intelligent sensing and control method for magnetic levitation holographic display as described in any one of the claims, comprising: The perception fusion module is used to collect and fuse multimodal data from the audience, environment, and suspended entities; The intelligent decision-making module, connected to the perception fusion module, is used to execute the context understanding and intent recognition steps and the dynamic content strategy generation steps, and outputs the personalized display narrative line. The collaborative control module, connected to the intelligent decision-making module, is used to decouple the personalized display narrative line into synchronized magnetic levitation control commands and holographic rendering commands; A magnetic levitation drive module, connected to the collaborative control module, is used to drive the movement of the suspended entity according to the magnetic levitation control command; An air projection holographic module, connected to the collaborative control module, is used to generate holographic images that coordinate with the movement of the suspended entity according to the holographic rendering instructions.

[0015] Furthermore, the perception fusion module includes at least one of a depth camera for acquiring audience visual information, a microphone array for acquiring audience voice information, and a lidar or visual marker tracking system for high-precision measurement of the pose of suspended entities.

[0016] Furthermore, the intelligent decision-making module has a built-in multimodal large language model for reasoning on the fused perceptual data to identify user intent; the intelligent decision-making module is also connected to a local or cloud-based content database and rule engine for dynamically generating the display narrative line based on the identified intent.

[0017] The beneficial effects of this invention are: This invention solves the problems of rigid presentation, stiff interaction, and disconnect between virtual and real content in existing technologies by constructing an intelligent closed loop of "perception-understanding-decision-generation-execution". Its core advantages are: Deep contextual understanding and dynamic response: Through multimodal perception and AI reasoning, it can accurately understand the audience's intentions and dynamically generate matching presentation content, achieving personalized narratives for each individual. Precise virtual-real-time spatiotemporal coordination: Through unified spatiotemporal calibration and command synchronization, it ensures a high degree of consistency between the movement of suspended entities and changes in holographic images in physical space and logic, creating a seamlessly integrated augmented reality experience. Closed-loop evolution capability: The system possesses a complete optimization closed loop from real-time interactive fine-tuning to long-term data learning, enabling continuous iteration of presentation strategies and continuous improvement in intelligence and user experience over time. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating an intelligent sensing and control method for magnetic levitation holographic display. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0021] In the description of this application, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0022] In one embodiment, an intelligent sensing and control method for magnetic levitation holographic displays includes the following steps: The method, operating on an intelligent display system integrating a magnetic levitation drive unit, an air projection holographic unit, and a multimodal sensing unit, includes: S1: Contextual understanding and intent recognition steps: Based on the multi-source data of the audience, environment and suspended entity status collected in real time by the multimodal perception unit, data fusion and contextual analysis are performed to identify the audience's interaction intent and generate a structured description containing intent type and target object. S2: Dynamic content strategy generation step: Based on the structured description, a personalized display narrative line is dynamically generated. The narrative line defines the expected motion sequence of the suspended entity, the associated holographic image content, and the spatiotemporal linkage between the two. S3: Virtual-Real Collaborative Execution Steps: The personalized display narrative line is decoupled into a time-synchronized magnetic levitation control command stream and a holographic rendering command stream, and then sent to the magnetic levitation drive unit and the air projection holographic unit for execution, so as to achieve precise collaborative display of the movement of the suspended entity and the changes of the holographic image. S4: Closed-loop interaction and optimization steps: While the virtual-real collaborative execution steps are being performed, new feedback data from the audience is continuously collected; based on the new feedback data, the display parameters of the personalized display narrative line being executed are adjusted in real time; and the complete interaction process data is used to update the model used in the context understanding and intent recognition steps and / or the dynamic content strategy generation steps.

[0023] Specifically: Environment setup and initialization.

[0024] Establish a unified spatiotemporal benchmark: jointly calibrate and register the spatial coordinate system of the magnetic levitation drive unit, the imaging spatial coordinate system of the air projection holographic unit, and the world coordinate system of the multimodal perception unit to form a unified "display space-perception space-control space" mapping model; load basic AI models and content assets.

[0025] Multimodal perception and data fusion.

[0026] Parallel and synchronous acquisition of multi-source heterogeneous sensing data, including: Audience data: Visual information including audience identity features, spatial location, head posture, gaze focus, gestures, and facial expressions is acquired through visual sensors; audience voice commands and ambient sounds are acquired through audio sensors.

[0027] Environmental data: Ambient light intensity and color temperature, and ambient noise level are collected.

[0028] Entity state data: Real-time acquisition of high-precision six-degree-of-freedom pose and vibration spectrum information of the suspended entity.

[0029] The above data are aligned and fused in the spatiotemporal dimension to construct a panoramic perception field at the current moment.

[0030] Contextual understanding and deep recognition of user intent.

[0031] Based on the aforementioned panoramic perception field, multi-level contextual understanding is performed: First layer: Objective state recognition. Identify the number of audience members, group classification (e.g., adults / children, single / multiple people), the relative layout of the audience and the core exhibition area, and the level of ambient light and noise.

[0032] The second layer: Behavior and focus recognition. It identifies the audience's macro-behavior (such as approaching, moving away, or pausing) and micro-interaction intentions (such as pointing, magnifying gestures, or staring at a certain spot); and accurately calculates the three-dimensional coordinates of the audience's attention focus in the suspended physical or virtual space.

[0033] The third layer: Semantic intent reasoning. This layer integrates speech recognition text, gaze focus, gesture semantics, and historical interaction context. A pre-trained multimodal large language model is used for reasoning to generate a structured description of the viewer's current core intent. This structured description includes the intent type (e.g., "Request detailed explanation," "Compare and view," "Listen to the story"), the target object (e.g., entity A component, holographic B function demonstration), and the desired interaction depth.

[0034] Dynamic content strategy generation is decoupled from instructions.

[0035] Based on the structured description and the current context, a personalized narrative is dynamically generated and decoupled into executable collaborative control instructions: Content strategy generation: Based on intent type and target object, content is instantly combined or created from a preset rule base or through a generative AI model. The strategy clearly defines: ① the motion sequence of the suspended entity in the subsequent time period (such as rotating to a specific angle, simulating opening and closing, slight trembling); ② holographic image content semantically related to the entity's motion (such as exploded diagrams of internal structures, dynamic data streams, virtual character animations, and contrast models); ③ the spatiotemporal linkage and triggering logic between the two (such as when the entity rotates to the X angle, the holographic image highlights the Y component and plays an explanatory animation).

[0036] Command decoupling: The narrative line is decomposed into two time-synchronized command streams: magnetic levitation control command stream (including position, speed, and posture sequences) and holographic rendering command stream (including 3D models, animations, special effects, and spatial anchor point sequences).

[0037] Precise execution and real-time rendering of virtual-real collaboration.

[0038] The decoupled dual instruction streams are then sent to the magnetic levitation drive unit and the air projection holographic unit for execution in a strictly synchronous manner: The magnetic levitation drive unit drives the electromagnetic array through a closed-loop control algorithm based on its command flow, so that the suspended entity can accurately and smoothly reproduce the preset motion sequence.

[0039] The air projection holographic unit renders three-dimensional images in real time according to its command stream, and generates holographic images at specified three-dimensional coordinates in the air that are synchronized in time and anchored in space with the movement of the suspended entity through beam scanning and modulation, forming a dynamic display effect that integrates the virtual and the real.

[0040] Interactive closed loop and adaptive optimization.

[0041] In the process of executing the narrative, we continuously return to perception, forming a closed loop: Real-time fine-tuning: Based on new audience feedback (such as whether their eyes follow the speaker, whether they show confused expressions, or new audio interruptions), dynamically adjust the ongoing narrative line within milliseconds to seconds. For example, change the narration speed, switch the level of detail shown, or repeat a certain segment.

[0042] Long-term learning: Anonymized complete interactive session data (from intent recognition to the end of display) is uploaded to the cloud. This data is then used to optimize the intent recognition model and the policy generation model through reinforcement learning or supervised learning, making the system's interaction increasingly accurate and natural.

[0043] Example: Taking a smartwatch as an example Assume the suspended entity is a smartwatch (real device or model).

[0044] The system sensed a visitor (visitor data: young male, single person) approaching. His gaze first fell on the floating watch, and then he made a general magnification gesture of "spreading five fingers and pulling back" (behavior recognition), while saying, "I want to see how its heart rate sensor works?" (voice data).

[0045] First layer: Recognized as "single person, adult, quiet environment".

[0046] The second layer recognizes the gesture as "request zoom / details" and focuses the gaze on the back of the watch.

[0047] The third layer: The multimodal large model integrates speech semantics ("heart rate sensor", "how it works") and visual focus (back of the watch) to infer structured intent: {Type: request for detailed explanation, Object: optical heart rate sensor module on the back of the watch, Depth: principle-level dynamic demonstration}.

[0048] Based on this intent, the strategy engine generates a narrative thread: Magnetic levitation command flow: Control the watch to slowly rotate 180 degrees, so that the back of the watch faces the audience stably.

[0049] Holographic command flow: A magnified, semi-transparent 3D model of a heart rate sensor is generated above the watch back. When the watch back is rotated into position, an animation is triggered: several virtual beams are emitted from the sensor and projected onto a cross-sectional model of the wrist skin. The animation dynamically displays the entire process of how the beams receive reflected light signals, convert them into photoplethysmography (PPG) signals, and then calculate the heart rate value using an algorithm. Simultaneously, a dynamic heart rate curve is generated on the side.

[0050] Dual command streams are issued simultaneously. The physical watch rotates precisely, and the holographic image appears at the correct position and time to play animation. The two work together perfectly to clearly demonstrate the working principle.

[0051] During execution, the system detected that the audience member leaned forward and stared at the animation inside the sensor (new feedback). The system made real-time fine-tuning, slightly slowing down the animation playback speed and making the virtual beam blink to highlight the key points. The data from this successful interactive session (from recognizing the "heart rate sensor" request to the audience member's final nod and leave) was encrypted and uploaded to optimize the response strategy for future requests based on similar technical principles.

[0052] In another embodiment, an intelligent sensing and control system for implementing the above-described method of magnetic levitation holographic display is provided, comprising: Perception fusion layer: includes a multi-sensor array and a data fusion processor, used to execute step S200 to construct the panoramic perception field.

[0053] Intelligent decision-making layer: includes edge AI computing unit, embedded with multimodal large language model and policy engine, used to execute steps S1 and S2 to complete context understanding, intent recognition and dynamic content policy generation.

[0054] Collaborative Execution Layer: Includes a high-precision magnetic levitation drive controller and an air projection holographic rendering controller, which receive synchronous command streams from the intelligent decision layer to execute step S3 and achieve accurate display of virtual-real collaboration.

[0055] Feedback optimization layer: This includes a local feedback loop and a cloud-based data analysis platform, used to execute step S4 and achieve real-time fine-tuning and long-term iterative optimization of the system.

[0056] In this embodiment, the implementation of key system units is as follows: The perception fusion layer can use RGB-D cameras and millimeter-wave radar to achieve accurate skeletal tracking and gaze estimation.

[0057] The AI ​​model for the intelligent decision-making layer can adopt a multimodal model based on the Transformer architecture and run on edge computing devices.

[0058] In the collaborative execution layer, the magnetic levitation drive adopts high-frequency response PID combined with feedforward control; the holographic rendering adopts real-time rendering with a GPU cluster and maintains time synchronization with the magnetic levitation controller through the network PTP protocol.

[0059] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0060] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for intelligent sensing control of a magnetic levitation holographic display, characterized in that, Includes the following steps: The method, which operates on an intelligent display system integrating a magnetic levitation drive unit, an air projection holographic unit, and a multimodal sensing unit, comprises: Contextual understanding and intent recognition steps: Based on the multi-source data of the audience, environment and suspended entity status collected in real time by the multimodal perception unit, data fusion and contextual analysis are performed to identify the audience's interaction intent and generate a structured description containing intent type and target object; Dynamic content strategy generation steps: Based on the structured description, a personalized display narrative line is dynamically generated. The narrative line defines the expected motion sequence of the suspended entity, the associated holographic image content, and the spatiotemporal linkage between the two. Virtual-real collaborative execution steps: The personalized display narrative is decoupled into a time-synchronized magnetic levitation control command stream and a holographic rendering command stream, and then sent to the magnetic levitation drive unit and the air projection holographic unit for execution, so as to achieve precise collaborative display of the movement of the suspended entity and the changes of the holographic image.

2. The intelligent sensing control method of the magnetic levitation holographic display according to claim 1, characterized in that, The context understanding and intent recognition steps specifically include: Identify the audience's basic attributes, spatial location, and focus of attention through multimodal perception data; Recognize audience interaction gestures and / or voice commands; By integrating the semantics of the attention focus, interactive gestures, and / or voice commands, a multimodal reasoning model is used to determine the viewer's core interactive intent, which includes at least one of the following types: requesting detailed explanation, comparing and viewing, and story narration.

3. The method of claim 1, wherein the magnetic levitation holographic display is an intelligent sensing control method, characterized in that, In the dynamic content strategy generation step, different content generation strategies are matched according to different intent types: When the intent type is "Request for details", a strategy is generated to control the suspended entity to maintain or move to the optimal viewing pose, while the holographic image presents the target object's internal structure, working principle animation, or related data charts. When the intent type is "comparison and viewing", a strategy is generated to control the floating entity to switch between different states, while the holographic image presents multiple comparison items or parameter change curves. When the intent type is "storytelling", a strategy is adopted to generate and control the floating entity to move in anthropomorphic manner in coordination with the narrative, while holographic images present virtual characters and plot animations.

4. The intelligent sensing control system and method for magnetic levitation holographic display according to claim 1, wherein, In the virtual-real collaborative execution step, a pre-established unified spatial coordinate system is used to ensure that the relative positional relationship between the spatial anchor point of the holographic image and the key parts of the suspended entity in three-dimensional space is accurate; and a precision clock synchronization protocol is used to ensure that the timestamp deviation between the magnetic levitation control command stream and the holographic rendering command stream is within a preset threshold range.

5. The method of intelligent sensing control for magnetic levitation holographic display according to claim 1, wherein, The method further includes: Closed-loop interaction and optimization steps: While the virtual-real collaborative execution steps are being performed, new feedback data from the audience is continuously collected; based on the new feedback data, the display parameters of the personalized display narrative line being executed are adjusted in real time; and the complete interaction process data is used to update the model used in the context understanding and intent recognition steps and / or the dynamic content strategy generation steps.

6. The intelligent sensing and control method for magnetic levitation holographic display according to claim 1, characterized in that, The real-time adjustment of display parameters includes at least one of the following: adjusting the playback speed of holographic animation, changing the volume and speed of the narration, highlighting or enlarging the virtual component that the viewer is currently gazing at, and repeating the previous display content.

7. An intelligent sensing and control system for magnetic levitation holographic display, used to implement the intelligent sensing and control method for magnetic levitation holographic display as described in any one of claims 1-6, characterized in that, include: The perception fusion module is used to collect and fuse multimodal data from the audience, environment, and suspended entities; The intelligent decision-making module, connected to the perception fusion module, is used to execute the context understanding and intent recognition steps and the dynamic content strategy generation steps, and outputs the personalized display narrative line. The collaborative control module, connected to the intelligent decision-making module, is used to decouple the personalized display narrative line into synchronized magnetic levitation control commands and holographic rendering commands; A magnetic levitation drive module, connected to the collaborative control module, is used to drive the movement of the suspended entity according to the magnetic levitation control command; An air projection holographic module, connected to the collaborative control module, is used to generate holographic images that coordinate with the movement of the suspended entity according to the holographic rendering instructions.

8. The intelligent sensing and control system for magnetic levitation holographic display according to claim 7, characterized in that, The perception fusion module includes at least one of a depth camera for acquiring audience visual information, a microphone array for acquiring audience voice information, and a lidar or visual marker tracking system for high-precision measurement of the pose of suspended entities.

9. The intelligent sensing and control system for magnetic levitation holographic display according to claim 7, characterized in that, The intelligent decision-making module has a built-in multimodal large language model, which is used to reason about the fused perception data to identify user intent; the intelligent decision-making module is also connected to a local or cloud-based content database and rule engine, which is used to dynamically generate the display narrative line according to the identified intent.

Images