A materialized interaction method and a materialized interaction system

Abstract

This application belongs to the field of human-computer interaction technology, and provides a physical interaction method and a physical interaction system. The method is applied to a control system including a 3D display module, a sensor module, and a physical construction module, including: controlling the 3D display module to continuously present a dynamic 3D image of a target object, so that the target object exists in a virtual form in the interaction space; monitoring the interaction space in real time through the sensor module, and determining the target area required for the current interaction when a preset interaction intention triggered by the user is detected; and generating a local physical structure with realistic tactile feedback from the 3D image within the target area through a preset construction method. This allows the system to dynamically maintain the coexistence and switching between virtual and physical states in the same interaction space, avoiding resource redundancy and space constraints caused by full-domain physicalization, and overcoming the fundamental defect of pure virtual interaction lacking tactile anchoring.

Description

Technical Field

[0001] This application belongs to the field of human-computer interaction technology, and in particular relates to a physical interaction method and a physical interaction system. Background Technology

[0002] Human-computer interaction (HCI) technology, serving as a bridge connecting users and the digital world, has been widely applied in virtual reality, augmented reality, and smart devices in recent years. Holographic projection technology achieves the floating display of three-dimensional images in mid-air through light field reconstruction, providing users with an immersive visual experience. Simultaneously, advancements in sensor technology enable systems to capture user movements and environmental changes in real time, providing a foundation for interactive intent recognition. In existing technologies, a typical implementation uses a holographic projection device to continuously display virtual objects and monitors user behavior through cameras and infrared sensors. When a specific gesture is detected, a preset physical feedback mechanism is activated, such as providing physical contact at a designated location via a robotic arm or fixed haptic device. This superimposes haptic feedback onto the virtual image, achieving a combined virtual and real interactive effect.

[0003] However, in existing technologies, the switching between virtual and physical forms often relies on a pre-fixed hardware structure or a fully physical design, making it difficult for the system to dynamically balance maintaining a lightweight virtual state and providing a realistic tactile experience. Summary of the Invention

[0004] In view of this, the embodiments of this application provide a physical interaction method and a physical interaction system, which can solve the technical problem that existing AI interaction devices are either purely virtual and lack tactile feedback or fixed entities and lack flexibility, making it impossible to achieve seamless switching between virtual and real forms.

[0005] In a first aspect, this application provides a physical interaction method applied to a control system comprising a 3D display module, a sensor module, and a physical construction module. The physical construction module includes a swarm of magnetic particles with switchable magnetic poles, a macroscopic magnetic field coil array, and a material supply device. The 3D display module is used to reconstruct a depth-sensing 3D image in a space without a physical support medium. The method includes: S1: Control the three-dimensional display module to continuously present the dynamic three-dimensional image of the target object, so that the target object exists in the interactive space in a virtual form; S2: The sensor module monitors the interaction space in real time. When the preset interaction intention triggered by the user is identified, the target area required for the current interaction is determined. S3: Within the target area, a guiding magnetic field is generated by the macroscopic magnetic field coil array, and the magnetic particle group is controlled to move under the action of the guiding magnetic field so that the magnetic particles are self-organized and arranged by the attraction of opposite poles of the built-in magnetic poles, generating a local solid structure with a real tactile feel.

[0006] The technical advantages of this application are as follows: a stable virtual space benchmark is established through the S1 normal presentation step; low-latency, high-precision context perception and spatial positioning of user intent are achieved through the S2 interaction trigger step; and the physical entity generation mechanism is initiated on demand in the target area based on the S3 entity construction step. The three constitute a closed-loop link of visual presence-intent recognition-physical response, which enables the system to dynamically maintain the coexistence and switching between virtual and physical states in the same interactive space. This avoids the resource redundancy and spatial pressure brought about by full-domain physicalization, and overcomes the fundamental defect of pure virtual interaction lacking tactile anchoring. Thus, it solves the technical problem in the prior art that virtual and physical forms cannot be obtained at the same time and cannot be switched on demand, and realizes an efficient, natural and cyclical physical interaction paradigm that is light in normal times and physical during interaction.

[0007] In one embodiment, when layer-by-layer deposition is used, step S3 further includes: S31a: Activate the macroscopic magnetic field coil array corresponding to the target area to generate a guiding magnetic field for defining the outline of the entity, wherein the magnetic field strength distribution of the guiding magnetic field is used to indicate the boundary range of the particle deposition. S32a: Control the multiple stored magnetic particles to fall under gravity or be transported by airflow into the target area, and activate the magnetic field boundary layer by layer starting from the bottom first layer, so that the falling magnetic particles can achieve self-organization and arrangement in each layer through automatic switching of built-in magnetic poles, and attract and fix each other with the fixed lower layer particles. S33a: Repeat the S32a layer-by-layer filling step until the target area is completely filled with magnetic particles, forming a local solid structure consistent with the contour of the three-dimensional image.

[0008] The technical advantages of this application's embodiments are as follows: a spatially adjustable guiding magnetic field is generated through a macroscopic magnetic field coil array to define the particle deposition boundary; with the help of gravity feeding and interlayer magnetic field timing control, magnetic particles are driven to fall layer by layer, self-organize and arrange themselves, and be magnetically fixed between layers; finally, the three-dimensional forming of the target area is achieved through repeated filling. On this basis, each layer of particles achieves controllable coupling in the construction stage by relying on the built-in magnetic pole switching capability. Combined with the tactile optimization design of the outer shell layer, the formed local solid structure has both geometric accuracy highly consistent with the three-dimensional image and provides realistic and tangible physical feedback; this collaborative mechanism does not require mechanical support or rely on external bonding materials, significantly improving the system's on-demand response capability, structural robustness, and immersive virtual-real fusion.

[0009] In one embodiment, after step S3, the method further includes: S4: When the interaction is detected to be over, a disengagement signal is sent to the magnetic particles to control the magnetic particles to switch their magnetic poles to a mutually repulsive state, so that the local physical structure is recycled to the bottom storage area under the action of gravity, and the target object is restored to the virtual form of the holographic projection.

[0010] The technical advantage of this embodiment lies in the following: by sending a dispersal signal to the magnetic particles to control their switching to a mutually repelling state, the local physical structure naturally falls and is reclaimed under the influence of gravity, achieving imperceptible dissipation after the interaction ends and ensuring the space quickly returns to its virtual state. Finally, through a context-aware, on-demand physicalization mechanism, while maintaining the normal operation of the virtual form, physical structures are dynamically generated only in the interactive area, effectively balancing visual immersion and tactile realism, and solving the technical bottleneck of seamless switching between virtual and real forms.

[0011] In one embodiment, sending a disengagement signal to the magnetic particles includes: Broadcast a disband command to all magnetic particles, or send disband signals sequentially via timing control; When each magnetic particle receives the disband command or the disband signal, the built-in magnetic field switching module switches the magnetic poles to a state where they repel each other from adjacent particles. This causes the particles to lose their mutual fixing force and fall naturally under the action of gravity, sliding down the preset recycling channel to the bottom storage area to wait for the next construction task.

[0012] The technical effect of this embodiment is as follows: by sending a dispersal signal through broadcasting or timing control, combined with the localized and self-sustaining repulsive state switching of the magnetic poles by the built-in magnetic field switching module of each magnetic particle, the controllable transformation of the particle cluster from spatially ordered fixation to gravity-dominated falling is realized; on this basis, by taking advantage of the geometric guidance and surface characteristics of the preset recycling channel, the unique falling path, stable movement, and accurate collection of particles are further ensured; finally, the imperceptible dissipation, orderly recycling, and immediate reuse of local physical structures are achieved, which not only solves the problem of physical residue affecting the spatial experience after the interaction ends, but also maintains the technical closed loop of seamless switching between the normal virtual state and the instantaneous physical state of the system.

[0013] In the case of the layer-by-layer deposition, step S3 further includes: S31b: Activate the macroscopic magnetic field coil array corresponding to the target region to generate a magnetic trap barrier for defining the outline of the entity; S32b: Controlling multiple stored magnetic particles to fall into the target area under the action of gravity, with the built-in magnetic poles of the magnetic particles preset to be opposite each other, so that there is always a repulsive force between the particles; wherein, the particles fall to the boundary of the magnetic trap under the action of gravity and are trapped and constrained in the target area by the magnetic trap potential barrier. S33b: Repeat the S32b layer-by-layer filling step, and by adjusting the interlayer strength of the magnetic trap barrier, the microparticles are suspended and stacked layer by layer to form a local solid structure consistent with the contour of the three-dimensional image; When the interaction is detected to be over, the magnetic trap barrier only needs to be removed or the magnetic field direction switched, and the local solid structure will naturally disperse and be recycled under the combined action of gravity and the repulsive force between particles.

[0014] The technical advantage of this embodiment is that, in the repulsive force mode, there is no need to actively switch the magnetic poles of each particle when dispersing; simply removing the magnetic field or changing its direction is sufficient, resulting in a faster response and simpler control.

[0015] In one embodiment, in the case of directional spraying, step S3 further includes: The macroscopic magnetic field coil array corresponding to the target area is activated to generate a guiding magnetic field for guiding the flight trajectory of the particles; Magnetic microparticles are sprayed onto the target area using a microparticle spraying device, and the magnetic microparticles are deposited on the surface of the target area under the action of the guiding magnetic field to form a microparticle layer structure; the deposited magnetic microparticles are fixed to each other by the attraction of built-in magnetic poles to form a hollow solid structure consistent with the outline of the three-dimensional image.

[0016] The technical advantages of this embodiment are as follows: compared to solid construction that fills in layers by gravity, this embodiment significantly reduces material usage, substantially lowers the weight of the entity, and significantly improves construction speed. Furthermore, since the microparticles only adhere and deposit on the surface of the magnetic field contour, remaining hollow internally, it ensures the tactile realism of the user-interactive area while avoiding unnecessary material waste, making it suitable for interactive scenarios requiring the rapid construction of large-size or lightweight entities.

[0017] In one embodiment, it also includes: A water mist generating device generates a controllable water mist layer in the interactive space, wherein the water mist layer is located on the surface of the target area or the user's interactive hand area. The water mist generating device is configured to avoid interfering with the light field reconstruction of the 3D display module. The water mist layer is used to form a moist tactile sensation by contacting the surface of the local solid structure when the local solid structure is generated in the target area, and / or to carry temperature-controlled particles to provide temperature feedback. The technical effect of this application embodiment is that by using water mist to enhance the optical performance and tactile cues of 3D images in the virtual state, and by enabling water mist and building materials to work together in the physical state to form a surface layer with humidity, temperature, or odor responsiveness, cross-modal coupling of vision, touch, temperature, and smell is achieved.

[0018] The second invention, this application also provides a physical interactive system, including: A three-dimensional display module is used to continuously present dynamic three-dimensional images of a target object, so that the target object exists in a virtual form in the interactive space. The three-dimensional display module includes a projection unit, a light field display unit, or a volume display unit. The sensor module is used to monitor the interaction space in real time, identify the preset interaction intentions triggered by the user, and determine the target area required for the current interaction. The solid construction module includes a swarm of magnetic particles with switchable magnetic poles, a macroscopic magnetic field coil array, and a material supply device, used to generate local solid structures with realistic tactile feel in the target area by layer-by-layer deposition or directional spraying under the guidance of a magnetic field. The control module, which is connected to the 3D display module, the sensor module, and the entity construction module respectively, is used to execute the steps of the context-aware on-demand entity interaction method of the first aspect mentioned above.

[0019] In one embodiment of the second aspect, the magnetic particles used in the particle deposition method of the entity building module include: The magnetic core, made of magnetic material, is used to generate magnetic force in response to an external magnetic field; A magnetic field switching unit, integrated inside the magnetic microparticles, is used to switch the magnetic polarity according to the received control signal, so that adjacent microparticles generate mutual attraction during the construction stage and mutual repulsion during the disintegration stage. The outer shell, which wraps around the magnetic core and the magnetic field switching unit, is used to provide preset tactile properties for the surface that comes into contact with the user after the local solid structure is formed.

[0020] The beneficial effects of the second aspect of this application are basically the same as those of the first invention, and will not be repeated here. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating the physical interaction method provided in Embodiment 1 of this application; Figure 2 This is a flowchart illustrating the physical interaction method provided in Embodiment 2 of this application; Figure 3 This is a flowchart illustrating the physical interaction method provided in Embodiment 3 of this application; Figure 4This is a flowchart illustrating the materialization interaction method provided in Embodiment 4 of this application; Figure 5 This is a flowchart illustrating the physical interaction method provided in Embodiment 5 of this application; Figure 6 This is a structural block diagram of the physical interactive system provided in the embodiments of this application. Detailed Implementation

[0023] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0024] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0025] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0026] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0027] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0028] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0029] Example 1 Understandably, existing AI interaction devices generally suffer from rigid forms and a disconnect between perception and operation: purely virtual interaction devices (such as holographic projectors and AR glasses) only provide visual feedback and lack real tactile feedback, resulting in a lack of physical anchoring and immersion for user operations; while traditional physical interaction devices (such as robotic arms and 3D-printed physical models) provide tactile feedback, their fixed structure and inability to be reconfigured as needed make them difficult to adapt to dynamic interaction intentions, and maintaining a physical state for extended periods leads to high energy consumption, large space occupation, and rigid interaction. Neither of these two types of solutions can achieve on-demand, partial, and reversible switching between virtual and physical forms within a single system, thus limiting the naturalness, flexibility, and energy efficiency of human-computer interaction.

[0030] Based on the above issues, please refer to Figure 1 As shown, this application provides a physical interaction method, which includes the following steps S1 to S3: Step S1 (Normal Presentation Step): Control the 3D display module to continuously present the dynamic 3D image of the target object, so that the target object exists in the interactive space in a virtual form; As a specific implementation, the three-dimensional display module can be an optical display unit that realizes three-dimensional light field reconstruction based on the principle of interference diffraction. It is used to generate a floating dynamic three-dimensional image with parallax and can be viewed from multiple angles in free space without a physical support medium. The three-dimensional image in this embodiment can refer to a three-dimensional visual presentation with depth perception formed in space through technologies such as holography, light field reconstruction, volume display or high-speed parallax switching, and is not limited to a specific optical principle. The target object can be any modeled object with three-dimensional spatial coordinates, such as a person, animal, machine, or abstract geometric shape; the virtual form can be an object that does not have mass, volume, or physical contact feedback, but only transmits visual information to the user through light signals. In this embodiment, this step continues to run when the system does not detect any interaction intent, serving as the basic state for normal interaction and providing spatial positioning reference and visual guidance for subsequent materialization.

[0031] In specific implementation, the three-dimensional display module used in this application embodiment takes holographic projection technology as an example: For example, one approach is to use the spatial coordinate mapping engine built into the 3D display module to convert the 3D model data of the target object into corresponding light field parameters in real time, and drive a laser array or spatial light modulator to generate dynamic images. Another approach is to use a preset viewpoint tracking mechanism, combined with user eye position information fed back by the sensor module, to dynamically adjust the image projection angle and focal length to maintain a stable stereoscopic viewing experience. Furthermore, another approach is to automatically adjust the projection brightness and contrast according to the ambient light intensity to ensure that the image has clear visibility under different lighting conditions. The embodiments of this application obtain a stable, continuous, and interactive virtual existence state of the target object in the interactive space based on any of the above methods.

[0032] In specific applications, for example, the control system of this embodiment continuously projects a full-body dynamic 3D image of a virtual assistant in a conference room scene. The image updates the perspective in real time as the user moves, maintaining a natural orientation and proportional relationship. When the user does not actively interact, the image always exists in the form of lightweight, low-power pure light, without triggering any entity construction action, and the interactive space remains empty and transparent.

[0033] It should be noted that the three-dimensional images described in this application encompass a variety of three-dimensional image rendering technologies: On the one hand, there are high-precision light field projections based on laser interference and spatial light modulators, capable of presenting photorealistic continuous images, complex textures, and light and shadow gradations, suitable for interactive scenarios with high visual fidelity requirements. On the other hand, there are also "pseudo-holographic" display methods based on water mist screens, light dot arrays, or geometric projections. The latter controls the position and brightness of a small number of light sources (such as laser dots) to form lines, light dots, or simple geometric shapes, constructing abstract, minimalist, and artistic "light composition" images. This method achieves an abstract visual representation of the target object with lower energy consumption and hardware costs, and is especially suitable for everyday companionship scenarios where real-time performance and tactile sensitivity are not critical, aligning with the core concept of "on-demand materialization." The choice between the two projection methods mentioned above can be made according to the specific application, and will not be elaborated further here.

[0034] Step S2 (Interaction Triggering Step): The interaction space is monitored in real time by the sensor module. When the preset interaction intention triggered by the user is identified, the target area required for the current interaction is determined. As a specific implementation, the sensor module may be a multimodal sensing unit that includes one or more combinations of depth cameras, infrared arrays, millimeter-wave radars, or inertial motion capture units, used to collect user limb postures, motion trajectories, distance changes, and spatial relative relationships. The preset interaction intent is a movement pattern with clear spatial orientation and behavioral purpose, such as reaching out, hovering the palm, pointing with the fingertips, simulating grasping, or leaning forward. This embodiment can collect interaction behavior information between the user and the 3D image through a sensor module; based on the spatial location corresponding to the interaction behavior information, it determines the target area required for the current interaction and activates the entity construction module corresponding to the target area. The interaction behavior information includes, but is not limited to, actions such as reaching out, touching, or hugging the 3D image.

[0035] The target region can refer to a local spatial volume in a 3D image that is spatially nearest to the user's interactive action and is located within the normal extension direction of the image surface. Its size and shape are determined by the spatial coverage of the interactive action and the curvature of the image surface. In this embodiment, this step does not rely on global modeling or offline training, but directly completes intent discrimination and region localization based on real-time sensing data stream. The output result is a set of 3D spatial bounding boxes or point clouds represented in the world coordinate system.

[0036] As an example, this embodiment can fit a gesture movement trend vector based on the three-dimensional coordinate sequence of the user's hand joints collected by the sensor module, and perform similarity matching with the reach-and-touch template in the preset intent template library. When the match is successful, a spherical target region is generated by expanding outward with a radius of 5 cm along the normal of the image surface, with the center point of the hand as the origin. Alternatively, this application can determine whether there is active approach behavior based on the minimum distance change rate between the user's torso and the image in the depth image. When the distance is less than 20 cm and the change rate exceeds 0.5 m / s, an ellipsoidal target region is adaptively generated with the nearest point as the center and according to the curvature of the local surface of the image. Furthermore, this application can perform redundant discrimination of the same interaction intent based on the multi-source sensor fusion results, and only activate the target region generation process when at least two sensors simultaneously confirm the intent. This application obtains a target region that is highly consistent with the user's actual interaction intent and has accurate spatial positioning based on any of the above methods. For example, when a user extends their palm towards a holographically projected image of an ancient porcelain vase in the exhibition hall, the sensor module captures the position and orientation of their palm in real time. After the system recognizes the intention to touch, an ellipsoidal space with a diameter of about 8 cm within a 5 cm range extending normally from the surface of the porcelain vase is identified as the target area. This area is precisely aligned with the current spatial position of the user's palm and is updated in real time with the slightest movement of the palm.

[0037] Step S3 (Solid Construction Step): Within the target area, a guiding magnetic field is generated by the macroscopic magnetic field coil array. The magnetic particle group is controlled to move under the action of the guiding magnetic field, so that the magnetic particles are self-organized and arranged by the attraction of opposite poles of the built-in magnetic poles, generating a local solid structure with a realistic tactile feel.

[0038] In this embodiment, within the target area, a guiding magnetic field is generated by the macroscopic magnetic field coil array. This guide magnetic field controls the deposition or directional ejection of the magnetic particle swarm layer by layer under the influence of the guiding magnetic field. The magnetic particles then self-organize and arrange themselves through the attraction of opposite poles of their built-in magnetic poles, generating a localized solid structure with a realistic tactile feel. The aforementioned layer-by-layer deposition refers to a method of guiding particle deposition using a magnetic field. This can refer to using a controllable array of magnetic field coils to generate a guiding magnetic field with a spatial gradient distribution within a target area. This allows particles carrying magnetic cores to be deposited along a preset trajectory and self-organized and fixed by the combined action of Lorentz force and magnetic gradient force during their fall. In specific implementation, the layer-by-layer deposition in this embodiment relies on gravity as the Y-axis driving source. The solid construction module does not contain a precision XY-axis motion mechanism. The deposition position of the magnetic particles in the XY plane is completely determined by the natural fall trajectory of gravity and the boundary constraints of the guiding magnetic field. The guiding magnetic field achieves three-dimensional construction through Z-axis interlayer timing control. The magnetic field interactions include opposite-pole attraction construction mode or like-pole repulsion constraint mode.

[0039] The aforementioned directional spraying refers to a method of guiding particle spraying and attachment. This can be achieved by actively spraying magnetic particles stored in the feeding unit to the target area using a particle spraying device, while simultaneously activating the corresponding macroscopic magnetic field coil array to generate a guiding magnetic field. This guiding magnetic field exerts a magnetic attraction force on the flying magnetic particles, causing them to fly along a preset magnetic field trajectory and adhere to the surface of the target area. After deposition, the magnetic particles automatically switch their built-in magnetic poles, attracting and fixing themselves with the already attached particles, gradually forming a local solid structure consistent with the contour of the 3D image. In practical implementation, the particle movement is propelled by the combined effect of the active spraying force of the spraying device and the magnetic field attraction force, eliminating the need for gravity-driven descent. This enables rapid and directional particle delivery, suitable for interactive scenarios requiring high-speed construction or where the construction direction is limited.

[0040] It should be noted that the local entity structure generated in step S3 is not designed to withstand strong interactions. Because the mutual fixing force formed between magnetic particles through magnetic pole switching has a certain threshold, this structure is suitable for gentle interactions such as light touches, handshakes, and soothing in normal interaction scenarios, providing users with realistic and delicate tactile feedback. However, when the user applies external force exceeding the design threshold (such as excessive gripping or strong pulling), the magnetic fixing force between the particles will be destroyed, causing the local entity structure to naturally disintegrate and the particles to scatter. For example, the magnetic attraction threshold is designed to be 0.5N; exceeding this force will cause the structure to disintegrate. This characteristic is not a technical defect, but rather a physical property of the particle assembly in this solution that simulates the subtle emotional relationships between people: it can withstand gentle interactions, symbolizing understanding and resonance; while when subjected to excessive external force, its structure will naturally disintegrate and scatter, symbolizing the destruction of relationships by possessiveness. This example design in this application makes the AI's body no longer a cold machine, but a poetic response to interaction methods, thereby improving the user experience.

[0041] In a specific implementation, the magnetic microparticle group in this embodiment has a particle size of 50μm-10mm; when the particle size is 100μm-500μm, it is used to achieve high-fidelity matching with the three-dimensional image contour; when the particle size is 1mm-10mm, it is used to achieve coarse-grained entity construction in humid environments or low-cost scenarios.

[0042] The technical effect of this application embodiment is as follows: a stable virtual space benchmark is established through the S1 normal presentation step, low-latency and high-precision context perception and spatial positioning of user intentions are achieved through the S2 interaction trigger step, and the physical entity generation mechanism is activated on demand in the target area based on the S3 entity construction step; the three constitute a closed-loop link of visual presence - intent recognition - physical response, which enables the system to dynamically maintain the coexistence and switching of virtual and physical states in the same interactive space. This avoids the resource redundancy and spatial pressure brought about by full-domain physicalization, and overcomes the fundamental defect of pure virtual interaction lacking tactile anchoring. Thus, it solves the technical problem in the prior art that virtual and physical forms cannot be obtained at the same time and cannot be switched on demand, and realizes an efficient, natural and cyclical physical interaction paradigm that is light when normal and physical when interacting.

[0043] Example 2 Based on the above embodiment one, as Figure 2 As shown, in the case of a layer-by-layer deposition method, step S3 further includes S31a to S33a: S31a (Macroscopic Contour Definition Step): Activate the macroscopic magnetic field coil array corresponding to the target region to generate a guiding magnetic field for defining the entity contour. The magnetic field strength distribution of the guiding magnetic field is used to indicate the boundary range of the particle deposition. In this embodiment, the guiding magnetic field forms a non-uniform magnetic field intensity distribution in the XY plane, forming a magnetic potential barrier only in the boundary region, and maintaining a uniform or weak magnetic field in the central region. This allows the position of the magnetic particles in the XY plane to be determined by the natural fall of gravity and the self-organized arrangement of opposite attraction between particles, without the need for a precise XY axis positioning mechanism. As an example, a macroscopic magnetic field coil array can refer to a combination of multiple sets of independently adjustable electromagnetic coils arranged in the space surrounding the target area, with their spatial arrangement aligned with the interactive spatial coordinate system. The guiding magnetic field can be a spatial magnetic field with a gradient intensity distribution, whose magnetic field intensity reaches a preset threshold at the boundary of the target area and decreases monotonically with increasing distance from the boundary. The magnetic field strength distribution is used to indicate the boundary range of particle deposition. It can refer to the change in direction and magnitude of the resultant force of the Lorentz force and magnetization force on the magnetic particles when they enter the magnetic field region, thereby forming an effective constraint barrier in the region where the magnetic field strength is higher than the threshold, so that the trajectory of the particles is restricted within the boundary range, providing a spatial positioning reference for subsequent layer-by-layer filling.

[0044] In specific implementations, one approach is to define the boundary range of particle deposition by generating a guiding magnetic field based on the spatial distribution of the magnetic field strength gradient; another approach is to define the boundary range of particle deposition by generating a guiding magnetic field based on the coordinated modulation of the current amplitude and phase of each unit in the coil array; furthermore, other approaches may define the boundary range of particle deposition by generating corresponding coil excitation parameters through reverse mapping of the voxel circumscribed envelope of the three-dimensional image contour. This application obtains a non-contact, dynamically adjustable physical fence for the spatial boundary of the target area based on any of the above methods, thereby supporting the subsequent on-demand molding of particles without mechanical molds.

[0045] For example, in a specific application, this embodiment can be implemented such that after a user points to the left shoulder of the holographic image to trigger an interaction, the sensor module identifies the spatial coordinate cluster corresponding to the action, and the control module determines a spherical target area with a radius of 50 mm centered on the left shoulder contour. Then, eight sets of macroscopic magnetic field coils arranged around the spherical area are activated, and a sinusoidal alternating current with a gradient increasing is introduced to form a guiding magnetic field with a peak magnetic field strength of 80 mT on the surface of the spherical area, which decays outward to 20 mT. This magnetic field distribution ensures that the magnetic particles that fall subsequently can only be deposited within the spherical boundary, ensuring that the final local solid structure strictly matches the geometric contour of the left shoulder in the three-dimensional image.

[0046] S32a (layer-by-layer filling step of microparticles): Control the multiple stored magnetic microparticles to fall under gravity or be transported by airflow into the target area, and activate the magnetic field boundary layer by layer starting from the bottom first layer, so that the falling magnetic microparticles can achieve self-organization and arrangement in each layer through automatic switching of built-in magnetic poles, and attract and fix each other with the fixed lower layer microparticles. For example, multiple magnetic particles stored at the top are controlled to fall under gravity or be transported by airflow into the target area. Starting from the first layer at the bottom, the magnetic field boundary is activated layer by layer, so that the falling magnetic particles can achieve self-organization and arrangement in each layer by automatically switching the built-in magnetic poles, and attract and fix each other with the fixed particles in the lower layer. Among them, magnetic microparticles can refer to composite microstructure units with a magnetic core, a magnetic field switching unit and an outer shell; Built-in automatic magnetic pole switching can refer to the process by which magnetic particles, after receiving an interlayer synchronization trigger signal, switch their current magnetic pole state from an initial suspended state to a state with a fixed polarity orientation through their internal magnetic field switching unit. Self-organized arrangement can refer to the spontaneous adjustment of the magnetic moment orientation of multiple magnetic particles according to the direction of the local magnetic field under the constraint of a single-layer magnetic field, and dense arrangement along the magnetic field line under the action of magnetic attraction, forming a two-dimensional layered structure with topological continuity. The attraction and fixation between the current layer particles and the already fixed lower layer particles can refer to the fact that during the falling process, the magnetic pole orientation of the current layer particles is opposite to the magnetic pole of the already stably arranged lower layer particles, thereby generating a net attraction force in the vertical direction and realizing interlayer bonding.

[0047] In specific implementations, one approach of this embodiment is to control magnetic pole switching based on interlayer timing trigger signals to achieve self-organized particle arrangement and interlayer fixation; another approach is to dynamically adjust the magnetic pole switching threshold based on feedback from the current layer's magnetic field boundary strength to achieve self-organized particle arrangement and interlayer fixation; furthermore, other approaches of this application can also be to generate upper-layer magnetic pole calibration commands based on the spatial density distribution of lower-layer particles to achieve self-organized particle arrangement and interlayer fixation. Based on any of the above approaches, this application obtains the ability of particles to autonomously position, orient, and couple between layers without external clamps or binders, thereby constructing a three-dimensional entity with structural integrity and mechanical stability.

[0048] For example, in this embodiment, after the macroscopic contour definition of the target area on the left shoulder is completed, the control system opens the valve of the top storage bin, allowing magnetic microparticles with a diameter of 200μm to fall at a uniform speed under the action of gravity. When the first layer of microparticles reaches the bottom plane of the target area, the control system activates the bottom magnetic field coil to generate a uniform magnetic field with a vertical upward intensity of 60 mT, and simultaneously broadcasts the first layer magnetic pole switching command to all microparticles in the layer. After the microparticles' built-in magnetic field switching unit responds, they uniformly set the bottom magnetic pole to the S pole and the top to the N pole. Subsequently, the second layer of microparticles continues to fall, and when they approach the first layer, they are attracted by its N pole and decelerate and accurately adsorb onto the top of the first layer of microparticles. At the same time, they receive the second layer switching command and set their bottom to the N pole and their top to the S pole, thus forming an opposite pole docking with the first layer. This process is repeated layer by layer, and the thickness of each layer is determined by the particle size and the magnetic field confinement depth to ensure that the layers are dense and the layers are firm.

[0049] S33a (Solid Forming Step): Repeat the S32a layer-by-layer filling step until the target area is completely filled with magnetic particles, forming a local solid structure consistent with the outline of the three-dimensional image.

[0050] It is understandable that complete filling can mean that all voxel positions in the target area are occupied by at least one magnetic particle, and the magnetic attraction between adjacent particles is greater than their detachment threshold under gravity and disturbance. Consistent with the contour of a 3D image, this can refer to a geometric deviation of less than 0.5 mm between the outer envelope of the formed local entity structure and the vertex set of the corresponding part of the surface mesh model in the 3D image in the spatial coordinate system.

[0051] In specific implementations, one approach of this embodiment is to drive a layer-by-layer filling process based on the Z-axis layered slice data of the three-dimensional image voxel model to achieve complete filling of the target area. Another approach is to identify the integrity of the current filling layer edge based on real-time visual feedback and dynamically add microparticles to achieve complete filling of the target area. Furthermore, another approach of this embodiment is to determine the microparticle packing density based on the rate of change of interlayer magnetic flux and adjust the falling rate in a closed loop to achieve complete filling of the target area. Based on any of the above implementation methods, this application embodiment obtains a high-fidelity mapping from the geometric information of the three-dimensional image to the spatial distribution of physical entities, ensuring that the local entity structure remains consistent with the original holographic image in terms of shape, proportion, and spatial orientation.

[0052] The technical advantages of this application's embodiments are as follows: a spatially adjustable guiding magnetic field is generated through a macroscopic magnetic field coil array to define the particle deposition boundary; with the help of gravity feeding and interlayer magnetic field timing control, magnetic particles are driven to fall layer by layer, self-organize and arrange themselves, and be magnetically fixed between layers; finally, the three-dimensional forming of the target area is achieved through repeated filling. On this basis, each layer of particles achieves controllable coupling in the construction stage by relying on the built-in magnetic pole switching capability. Combined with the tactile optimization design of the outer shell layer, the formed local solid structure has both geometric accuracy highly consistent with the three-dimensional image and provides realistic and tangible physical feedback; this collaborative mechanism does not require mechanical support or rely on external bonding materials, significantly improving the system's on-demand response capability, structural robustness, and immersive virtual-real fusion.

[0053] Example 3 Based on the scheme of Embodiment 2 above, after step S3, refer to Figure 3 The method further includes: S4: When the interaction is detected to be over, a disengagement signal is sent to the magnetic particles to control the magnetic particles to switch their magnetic poles to a mutually repulsive state, so that the local physical structure is recycled to the bottom storage area under the action of gravity, and the target object is restored to the projected virtual form.

[0054] It can be understood that the end of the interaction can mean that the sensor module has not detected any user interaction behavior information in the target area for more than a preset time threshold; or that the user has performed a preset end action, such as removing both hands from the interaction space, making a waving goodbye gesture, or issuing an end command via voice; or that the control system has actively triggered a dissipation condition based on the interaction context logic to determine that the current task has been completed. This state determination is based on the interaction behavior information collected by the sensor module, which is from the same source as the information collected in the aforementioned step S2, and does not introduce new sensing channels or data types.

[0055] The dispersal signal can be a wireless broadcast command, synchronously transmitted by the control module to all magnetic particles within the target area via radio frequency or near-field communication; or a time-coded signal, transmitted sequentially by the control module according to spatial hierarchy or particle ID sequence to achieve progressive dispersal control in layers, zones, or batches. After being received by the magnetic field switching unit integrated within the magnetic particles, this signal drives them to perform a magnetic pole polarity reversal operation.

[0056] The repulsive magnetic pole state can refer to a situation where, after the built-in magnetic poles of each magnetic particle are switched, their polarities facing adjacent particles are the same (i.e., NN or SS are opposite), thus generating a net repulsive force between the particles. This repulsive force disrupts the original stable spatial connection network formed by attractive forces, leading to the overall instability of the local solid structure. This state does not change the physical form of the particles themselves, but only their magnetic response properties, and is a reversible electromagnetic state switch.

[0057] The recycling to the bottom storage area refers to the process where, under the premise of dominant repulsive force and the disappearance of attractive force, the particles detach from their original fixed position under the action of gravity and slide down along the system's preset unobstructed recycling channel. This channel has a guiding slope and a low-friction surface treatment to ensure that the particle flow path is unique, without accumulation or jamming. The recycling endpoint is a sealed storage cavity below the top storage area, whose spatial volume matches the total amount of particles used in a single construction, supporting cyclic feeding.

[0058] In specific implementations, one approach is to synchronously send disintegration commands to all magnetic particles via wireless broadcast, enabling each particle to receive a control signal at the same time and synchronously switch its magnetic poles, thereby achieving the instantaneous disintegration of the local physical structure. Another approach is to sequentially send disintegration signals according to spatial layers (from top to bottom or from inside to outside) using a time-series coding method, causing the particles to gradually decouple their magnetic connections layer by layer and region by region, forming a controllable and gradual dissipation process, reducing the impact of falling particles and airflow disturbances. Furthermore, other approaches can be to identify and directionally send disintegration signals based on particle ID sequences, allowing for individual intervention on particles that are abnormally stuck or misaligned, improving the robustness and integrity of the disintegration process. This embodiment can achieve controllable, reversible, and residue-free dissipation capabilities for local physical structures based on any of the above implementation methods, ensuring seamless switching between physical and virtual states in the interactive space.

[0059] This application embodiment can specifically employ the following methods: broadcasting a disband command to all magnetic particles, or sequentially sending disband signals through timing control; when each magnetic particle receives the disband command or the disband signal, the magnetic poles are switched to a state of mutual repulsion with adjacent particles through a built-in magnetic field switching module; causing the particles to lose their mutual fixing force and slide down to the bottom storage area along a preset recycling channel under the action of gravity, waiting for the next construction task.

[0060] For example, in this embodiment, after the user completes the gripping operation of the virtual robotic arm end in the holographic image, if the sensor module does not detect any change in hand posture or contact pressure signal for 2 consecutive seconds, the control system determines that the interaction has ended and then broadcasts a dispersal command to all magnetic particles in the target area via the radio frequency module. After receiving the command, the magnetic field switching unit built into each particle completes the magnetic pole reversal within 10 milliseconds, switching from the original N-S attraction state to the N-N repulsion state. The attractive force network between particles collapses, the physical structure instantly loses its mechanical support, and slides down the smooth inclined recycling channel under the action of gravity, returning to the bottom storage area within 3 seconds. At this time, the three-dimensional display module continues to present dynamic three-dimensional images, the target object is restored to a purely virtual form, the interaction space is reset, and it waits for the next trigger.

[0061] The technical effect of this application embodiment is as follows: by sending a disintegration signal to the magnetic particles at the end of the interaction and controlling their magnetic poles to switch to a mutually repulsive state, the original attractive force connection network is disintegrated by the repulsive force between the particles, so that the local physical structure naturally disintegrates under the action of gravity and is recycled to the bottom storage area along the preset channel, thereby realizing the automatic reset of the target object from the physical state to the virtual state; this process does not require mechanical disassembly, material loss, or external intervention, which significantly improves the closed-loop nature, sustainability, and smoothness of the interaction of the system, and effectively solves the problem of space burden and state switching delay caused by the physical structure residue after the interaction ends.

[0062] Accordingly, the technical solution of this embodiment can be applied to at least the following usage scenarios: Scenario 1: Application in home AI companion robot scenarios: When not in use by the user (holographic projection, lightweight and low power consumption): When the user does not need physical contact, the AI ​​is a "light sprite" floating in the living room (holographic projection). It can play movies, display the weather, and chat with the user as they move around. At this time, its body is light, its energy consumption is extremely low, and it has no physical presence; During user interaction (partial physicalization): When a user is sad, they reach out to it. The AI ​​senses your emotion and the gesture of reaching out (contextual awareness), activating the macroscopic magnetic field of the corresponding area.

[0063] The particles fall from the top and quickly fill in the outline of the holographic image's "hand" layer by layer to build a realistic palm; The hand that the user holds is no longer an illusion. Because the particles are fixed to each other through magnetic pole switching, the user can feel a real grip and body temperature (if the particles integrate a temperature control module). At this point, only the part in contact is solid; the rest of the AI ​​remains light, achieving dynamic energy allocation (consuming energy only where needed to construct a solid). After the user finishes interacting with the AI ​​robot (returning to holography): The user releases their hand, their emotions calming down. The AI ​​robot receives a "disband" signal, and the particles switch to mutual repulsion, flowing back to the bottom recycling area like fine sand under the influence of gravity. The user is once again left with only that warm light. "The physical entity dissipates like sand, leaving no hardware burden, only the memory of warmth."

[0064] Scenario 2: Future Immersive VR / AR Experience Scenarios It's understandable; currently, VR can only provide visual and auditory feedback, and when you touch a wall, the controller still vibrates, offering no real tactile sensation.

[0065] In this specific application, the user can wear thin AR glasses and enter a virtual game. The user sees a stone table (holographic projection) in front of them.

[0066] Materialization process: When the user reaches out to touch the table, the glasses track the user's hand position. The magnetic field of the tabletop area is activated, and countless particles instantly fall from the storage area at the edge of the device (such as the glasses frame or wristband), constructing a hard, cold "stone" texture in the local area where the user's palm touches.

[0067] Interaction and Dissipation: Users can realistically support themselves on the table. When the user removes their hand, these particles lose their magnetic field binding, slide down and retract, and the table returns to its holographic form.

[0068] Scenario 3: Dynamic exhibition booth scene in a museum / shopping mall: It's understandable that with current technology, valuable cultural relics can only be viewed but not touched, resulting in a poor user experience. In this specific application, only the ancient bronze artifacts projected holographically are slowly rotating in the display case.

[0069] The materialization process of this embodiment involves the following: when a user curiously reaches out to touch it, the system recognizes this action. At the location where the user's fingertip is about to touch, a magnetic field is locally activated, and particles fall from the top of the display case in milliseconds, stacking layer by layer to materialize the tactile sensation of that small area. What the user touches is not glass, but a real, textured "bronze" feel.

[0070] The technical effects of this embodiment are: it ensures the safety of cultural relics (they are illuminated normally), satisfies the desire for exploration (they become locally solidified when touched), and each touch produces a unique and natural tactile feedback due to the self-organized arrangement of the particles.

[0071] It should be noted that the application of the aforementioned home AI companion requires consideration of specific usage environments. Since the physical construction module relies on a macroscopic magnetic field coil array to guide particle movement, although the magnetic field strength is strictly controlled and its effective range is limited to the interactive space, in a home environment, if the user is a pregnant woman or a sensitive individual with implanted medical electronic devices (such as a pacemaker), for the highest safety standards, it is recommended to deploy such systems in controlled environments such as commercial exhibition spaces, science and technology experience centers, or professional interactive laboratories. In commercial application mode, the system can activate the physical interactive function through identity recognition and user authorization mechanisms, ensuring that no contraindicated individuals come into contact with it, thereby leveraging the technical advantages of this solution while meeting the safety and ethical requirements of specific scenarios. For ordinary home scenarios, the system can be set to "pure virtual mode," enabling only 3D imaging and water mist enhancement functions, and disabling the magnetic field construction module, to balance safety and user experience.

[0072] Example 4 Furthermore, in other embodiments, based on the above-described embodiment one, such as Figure 4As shown, in the case of the layer-by-layer deposition, step S3 further includes S31b to S33b: S31b: Activate the macroscopic magnetic field coil array corresponding to the target region to generate a magnetic trap barrier for defining the outline of the entity; Specifically, the magnetic trap forms a strong magnetic field boundary in the XY plane and a weak magnetic field or magnetic field gradient in the central region; S32b: Controlling multiple stored magnetic particles to fall into the target area under the action of gravity, with the built-in magnetic poles of the magnetic particles preset to be opposite each other, so that there is always a repulsive force between the particles; wherein, the particles fall to the boundary of the magnetic trap under the action of gravity and are trapped and constrained in the target area by the magnetic trap potential barrier. In a specific implementation, multiple magnetic microparticles stored in the system are controlled to fall into the target area along the Z-axis under the influence of gravity. The magnetic poles of the microparticles are preset to be opposite to each other (NN or SS repulsion), so that there is always a repulsive force between the microparticles. The microparticles fall to the boundary of the magnetic trap under the influence of gravity and are trapped and constrained in the target area by the magnetic trap barrier. The structure is maintained by the constraint of the external magnetic field boundary rather than the attractive force between the microparticles. S33b: Repeat the S32b layer-by-layer filling step, and by adjusting the interlayer strength of the magnetic trap barrier, the microparticles are suspended and stacked layer by layer to form a local solid structure consistent with the contour of the three-dimensional image; Specifically, by repeating the S32b layer-by-layer filling step, the interlayer strength in the Z direction of the magnetic trap barrier can be adjusted to allow the microparticles to be suspended and stacked layer by layer, forming a local solid structure consistent with the contour of the three-dimensional image. When the interaction is detected to be over, the magnetic trap barrier only needs to be removed or the magnetic field direction switched, and the local solid structure will naturally disperse and be recycled under the combined action of gravity and the repulsive force between particles.

[0073] The technical advantage of this embodiment is that, in the repulsive force mode, there is no need to actively switch the magnetic poles of each particle when dispersing (simply remove the magnetic field or change its direction), resulting in faster response and simpler control.

[0074] Example 5 Based on the above embodiment one, as Figure 5 As shown, in the case of directional spraying, step S3 may further include: Step S31b (Magnetic field profile definition): Activate the macroscopic magnetic field coil array corresponding to the target area to generate a guiding magnetic field for guiding the flight trajectory of the particles; Step S32b (Spraying Environment Control): Magnetic microparticles are sprayed onto the target area through a microparticle spraying device, so that the magnetic microparticles are deposited on the surface of the target area under the action of the guiding magnetic field to form a microparticle layer structure. As a specific implementation, the particle spraying device can employ a multi-axis robotic arm. For example, in step S32b, the multi-axis robotic arm can be controlled to drive the particle spraying nozzle to move along a preset path, so that the relative position and attitude of the particle spraying nozzle and the target area change in real time. Then, by adjusting the motion trajectory, spraying angle and spraying parameters of the particle spraying nozzle, the magnetic particles are uniformly deposited under the synergistic effect of the guiding magnetic field and the movement of the robotic arm to form a particle layer structure with controllable thickness.

[0075] As another specific implementation, step S32b can also establish a local vacuum environment by setting a flexible sealing cover around the target area and evacuating the space inside the sealing cover. In the directional spraying step, the magnetic particles are uniformly adsorbed onto the surface of the target area under negative pressure, forming a particle layer structure consistent with the contour of the guiding magnetic field. This example enhances the adsorption force of magnetic particles on the magnetic field contour surface by establishing a local vacuum environment in the target area and utilizing negative pressure. The vacuum environment eliminates the interference of air resistance on the particle flight trajectory, allowing the particles to adhere tightly to the curved surface and avoiding sagging or uneven distribution caused by gravity. At the same time, negative pressure adsorption helps to achieve uniform molding in one go, which is particularly suitable for scenarios where conventional spraying is difficult to cover, such as complex curved surfaces or vertical surfaces, thus improving the quality of solid molding.

[0076] Step S33b (Structure Fixation): The deposited magnetic particles are fixed by mutual attraction through built-in magnetic poles to form a hollow solid structure consistent with the contour of the three-dimensional image.

[0077] The technical advantages of this embodiment are as follows: by constructing a solid structure through "magnetic field contour definition + microparticle jet attachment," rapid prototyping of a hollow thin-shell structure is achieved. Compared to solid construction by gravity-based layer-by-layer filling, this embodiment significantly reduces material usage, substantially lowers the solid weight, and significantly improves construction speed. Simultaneously, since the microparticles only adhere and deposit on the surface of the magnetic field contour, maintaining a hollow interior, it ensures the tactile realism of the user-interactive area while avoiding unnecessary material waste, making it suitable for interactive scenarios requiring the rapid construction of large-size or lightweight solids.

[0078] Example 6 Based on any of the above method embodiments, the method of this application further includes: A water mist generating device generates a controllable water mist layer in the interactive space, wherein the water mist layer is located on the surface of the target area or the user's interactive hand area, and the water mist generating device is configured to avoid interfering with the light field reconstruction of the three-dimensional display module; the water mist layer is used to form a moist tactile sensation by contacting the surface of the local solid structure when the local solid structure is generated in the target area, and / or to carry temperature-controlled particles to provide temperature feedback.

[0079] It is understood that the controllable water mist layer generated by the water mist generating device in this embodiment is distributed on the user's hand interaction area or the surface of a local solid structure. Its function is to provide a moist touch or to serve as a temperature control medium carrier, rather than as a display medium. The mist outlet of the water mist generating device is physically or temporally isolated from the imaging optical path of the three-dimensional display module to ensure that the water mist does not cause scattering interference to the light field reconstruction of the three-dimensional image.

[0080] In specific implementations, the delivery timing of the water mist generator in this application embodiment is staggered with the activation timing of the macroscopic magnetic field coil array to avoid interference from the water mist in the process of establishing the guiding magnetic field. When using millimeter-sized magnetic particles (particle size 1mm-10mm), the droplet size of the water mist layer is 50-200μm, which is much smaller than the particle size of the magnetic particles, to avoid magnetic response failure caused by the droplets enveloping the magnetic particles; the water mist layer forms instantaneous droplets on the surface of the magnetic particles to provide a moist feel, while reducing the electrostatic adsorption of the magnetic particles when they fall.

[0081] Example 6 Figure 6 A physical interaction system is provided for embodiments of this application. The physical interaction system includes: The three-dimensional display module 01 is used to continuously present dynamic three-dimensional images of the target object, so that the target object exists in the interactive space in a virtual form; Sensor module 02 is used to monitor the interaction space in real time, identify the preset interaction intentions triggered by the user, and determine the target area required for the current interaction. The solid construction module 03 includes a magnetic microparticle swarm with switchable magnetic poles, a macroscopic magnetic field coil array, and a material supply device, which are used to generate a local solid structure with a realistic feel in the target area by layer-by-layer deposition or directional spraying under the guidance of a magnetic field. The control module 04 is connected to the 3D display module 01, the sensor module 02 and the entity construction module 03 respectively, and is used to execute the steps of the aforementioned embodiment of the entity interaction method.

[0082] Physical interactive systems can be applied to AI companion robots, VR / AR experience devices, and dynamic display devices in museums and shopping malls.

[0083] In specific implementation, the 3D display module 01 can refer to a 3D light field reconstruction device based on laser interference or digital light field modulation technology. Its output light field can continuously change with the viewing angle, so that the target object presents a naked-eye stereoscopic visual effect in the interactive space without wearing a device. This module can be a desktop holographic stage, a spatial floating light field projection array, or a wall-embedded holographic screen. Its installation position and projection angle can be adapted to the size of the interactive space and the user's movement. When the system is running normally, this module works independently, without relying on the power supply or synchronization signal of the physical construction module. It only receives the image content and playback parameters sent by the control module. Its function is to provide a high-fidelity visual surface, forming a layered collaborative relationship with the subsequently generated local physical structure, where the virtual is the surface and the real is the skeleton. That is, the holographic image defines the spatial existence and semantic outline, and the physical structure is given physical tangibility and mechanical feedback capability on this basis.

[0084] Sensor module 02 can be a multimodal fusion sensing unit, including a depth camera, millimeter-wave radar, capacitive near-field sensing array, and sound source localization microphone array. The depth camera captures the user's limb posture and spatial displacement trajectory, the millimeter-wave radar penetrates lightweight obstructions to continuously track hand micro-movements, and the capacitive sensing array is deployed at the boundary of the interactive space to detect interactive intentions that are approaching but not yet within the field of view. The interactive behavior information collected by this module includes typical gesture sequences such as pointing, hand hovering, fist clenching, simulated touch, or hugging. Its output data, after edge preprocessing, is uploaded to the control module in the form of spatiotemporal coordinate clusters as the spatial basis for target area localization. This module does not communicate directly with the 3D display module; both are controlled by the control module, thus avoiding timing misalignment caused by heterogeneous protocols. Its linkage with the entity construction module is manifested in the following way: the control module only issues a start command to the entity construction module after the sensor module reports a valid interactive intention and completes the target area coordinate calculation, ensuring that the entity construction action strictly responds to the real user intention, rather than being falsely triggered or affected by environmental noise interference.

[0085] The entity construction module 03 can be composed of a magnetic field coil array, a material supply device, and an auxiliary environmental control subsystem. The magnetic field coil array is a group of miniature electromagnetic coils arranged in a two-dimensional planar layout or a three-dimensional distribution. Its spatial density and driving current accuracy can be set according to the minimum resolution scale of the target area, with no fewer than 64 independently controllable coil units arranged within a 20 cm × 20 cm target area. The material supply device includes a top-mounted particle storage bin and a gravity-feed channel, or a spray material storage tank and a piezoelectric nozzle array, either selected or configured in parallel. When using particle deposition, the device also integrates a particle magnetic pole state broadcasting unit to synchronously send construction / dispersion commands to all particles within the target area. When using spray coating, the device is further coupled with a vacuum pump and a curable material atomizer. The module's function is to map the geometric contours of the holographic image into a tangible physical structure within the target area defined by the sensor module. Its cooperation with the 3D display module is crucial. The system is manifested in the following ways: the spatial distribution of the guiding magnetic field generated by the magnetic field coil array strictly matches the cross-sectional contour curvature and normal direction of the holographic image in the target area, so that the particle deposition boundary or spraying trajectory conforms to the geometric features of the image surface; its cooperation with the sensor module is manifested in the following ways: the spatial coordinate system of the target area and the local coordinate system of the magnetic field coil array are pre-calibrated and aligned to ensure that the intent recognition result can drive the entity construction action without deviation; through the above cooperation, the local entity structure generated by the module in the target area has a tactile surface similar to human skin, and maintains a high degree of consistency with the holographic image in terms of spatial position, shape proportion and motion synchronization.

[0086] Control module 04 is an embedded heterogeneous computing platform, including a main control processor, a real-time coprocessor, a multi-protocol communication interface, and a security isolation module. The main control processor runs the system scheduling logic and method flow control program, while the real-time coprocessor is dedicated to processing the raw sensor data stream and executing a lightweight intent recognition model. The multi-protocol communication interface supports MIPI-CSI, SPI, CAN FD, and Gigabit Ethernet, respectively connecting to the video stream channel of the 3D display module, the data bus of the sensor module, and the drive bus of the entity building module. The security isolation module verifies the legality and spatiotemporal validity of the command source before issuing build / dismantle commands, preventing illegal signals from causing particle agglomeration or uncontrolled spraying. This module functions as a central decision-making and coordination node in the system; its operation does not generate new structures or functions, only achieving timing alignment, data routing, and state synchronization between existing modules. Its connection to the 3D display module uses a video frame synchronization trigger signal + metadata channel to ensure that each frame carries the corresponding spatial anchor point information. Its connection to the sensor module uses low-latency interrupts + shared memory buffers to ensure that the interactive intent recognition results are within 50 seconds. The reporting is completed within ms; its connection with the entity building module is a deterministic time-sensitive network (TSN) bus, ensuring that the error of key actions such as magnetic field coil activation, particle release, and spraying start and stop is less than 1 ms; through the above connection relationship and scheduling mechanism, the control module realizes end-to-end deterministic control of the perception-decision-execution closed loop, so that the entire system has no state residue, no action jitter, and no cross-module resource contention when switching between virtual state and physical state.

[0087] It is understood that the embodiments of this application construct a physical interactive system with holography as clothing and microparticles as the skeleton as the architectural paradigm. The system provides contactless visual presence through a three-dimensional display module, provides intention-driven spatial anchoring through a sensor module, provides on-demand activated physical tactileness through a physical construction module, and provides strong real-time collaborative scheduling through a control module. The four components form a technical system with hierarchical decoupling, functional cohesion, and closed-loop response. This system breaks through the limitations of the binary separation between virtual and physical in traditional human-computer interaction, enabling the same target object to dynamically switch its existence form in the same space according to the interaction context. It retains the advantages of low power consumption and high degree of freedom of holographic display, while introducing the real tactile and mechanical feedback capabilities of physical structures.

[0088] For example, the working process and principle of this application embodiment are as follows: Under normal circumstances, the system only runs the three-dimensional display module to maintain the dynamic three-dimensional image presentation of the target object. At this time, the interactive space is in a zero-physical-load state. When the sensor module continuously monitors the user's preset interactive intentions such as pointing, hovering the palm, or simulating touch, the control module immediately parses the spatial coordinate cluster corresponding to the intention, locks the target area, and sends a construction command to the entity construction module. The entity construction module activates the magnetic field coil array corresponding to the target area and simultaneously starts the material supply device, so that the microparticles are directionally deposited under the action of the guiding magnetic field, or the spraying material is attached and formed along the magnetic field trajectory, and finally a local physical structure with a realistic tactile feel is generated in the target area that is consistent with the outline of the holographic image. After the interaction ends, the control module sends a disband command again, and the entity construction module switches the magnetic poles of the microparticles or terminates the spraying and starts the recycling process. The local physical structure naturally dissipates, and the interactive space returns to a pure virtual state. The entire process is uniformly scheduled by the control module, and the synchronization of actions and consistency of states between modules are ensured through a hard real-time communication protocol.

[0089] Furthermore, in some embodiments, the magnetic particles used in the particle deposition method of the entity building module 03 may include: The magnetic core, made of magnetic material, is used to generate magnetic force in response to an external magnetic field; The magnetic field switching unit, integrated inside the magnetic microparticles, is used to switch the magnetic polarity according to the received control signal, so that adjacent microparticles generate mutual attraction during the construction stage and mutual repulsion during the disintegration stage. The outer shell, which wraps around the magnetic core and the magnetic field switching unit, is used to provide preset tactile properties for the surface that comes into contact with the user after the local solid structure is formed.

[0090] In practical implementation, the magnetic core can be composed of ferrite, neodymium iron boron, or soft magnetic alloy, and can be a microscale permanent magnet or soft magnetic particle with a size range of 50 μm to 500 μm. Under the action of an external guiding magnetic field, the magnetic core generates a controllable magnetic moment, which can drive the particles to move along the magnetic field gradient direction and participate in self-organization. Its magnetic properties meet the requirements of magnetic response sensitivity and saturation magnetization intensity in the range of guiding magnetic field strength of 0.01T–0.5T, ensuring that the particles are stably captured and oriented during the layer-by-layer filling process.

[0091] The magnetic field switching unit can be a miniature electromagnetic coil, memristor array, or phase-change magnetic switch structure integrated inside the microparticle. The operating voltage range can be set to 1.8V–5.0V, and the switching response time can be set to 10μs–1ms. The magnetic field switching unit and the control module can receive commands and provide status feedback through wireless power supply and near-field communication links, without relying on physical wire connections. According to the control commands in the construction / disassembly stages, it dynamically changes the overall equivalent magnetic pole orientation of the microparticle. In the construction stage, it makes adjacent microparticles present an N–S opposite pole relative configuration to form an attractive force, and in the disassembly stage, it switches to an N–N or S–S same pole relative configuration to trigger a repulsive force. This switching process does not change the mass distribution and geometric shape of the microparticle itself, but only regulates its response mode to the external magnetic field, without affecting the spatial mapping consistency of the holographic image.

[0092] The outer shell can be made of silicon-based elastomers, hydrogel composites, or biomimetic keratinocyte structures, with a thickness of 1μm–50μm, a surface roughness Ra value of 0.1μm–5μm, and a Young's modulus of 0.1MPa–2MPa. The outer shell isolates the magnetic core from direct contact with the human body, avoiding magnetic stimulation or metallic interference. At the same time, by controlling the surface morphology, mechanical modulus, and hydrophilicity, it simulates the tactile perception characteristics of the inner forearm or palm area of ​​the human hand. The outer shell can be prepared using atomic layer deposition (ALD), microfluidic encapsulation, or vapor deposition processes, forming an integrated microstructure with the magnetic core and magnetic field switching unit, ensuring that it does not peel off or age and fail during repeated build-and-disassemble cycles.

[0093] The magnetic core, magnetic field switching unit, and outer shell are spatially integrated in a concentric nested structure: the magnetic field switching unit is arranged around the magnetic core and coaxially coupled with it, while the outer shell completely covers the outer surfaces of both, forming a single functional microparticle with magnetic response capability, programmable state, and adjustable tactile feel. During the construction phase, the control module applies a spatially encoded guiding magnetic field to the magnetic field coil array and simultaneously broadcasts construction instructions to all microparticles in the target area. After responding, the magnetic field switching unit of each microparticle configures its magnetic poles to an opposite-pole attraction state. Under the combined action of gravity and magnetic field, the microparticles settle layer by layer, automatically align, and attract and fix each other. During the dissolution phase, the control module switches to sending dissolution instructions, and each microparticle simultaneously flips its magnetic poles to a like-pole repulsion state. The binding force between the microparticles is instantly released, the overall structure becomes unstable, and under the action of gravity, it slides orderly along the preset recycling channel to the bottom storage area. Throughout the process, the outer shell maintains physical integrity and tactile consistency, and does not deform, fall off, or chemically degrade due to magnetic pole switching or mechanical movement. It is understood that the magnetic microparticles in this application embodiment integrate a three-layer structure of magnetic core, magnetic field switching unit and outer shell. In the construction stage, high-fidelity contour reproduction and self-organization fixation can be achieved by magnetic field guidance. In the disintegration stage, non-destructive, rapid and controllable structural disintegration can be achieved by magnetic pole reversal. The preset tactile attributes can include a variety of tactile designs, which can mimic human touch or be autonomous AI-generated (such as metallic texture, velvety texture, frosted texture, temperature-sensitive texture, etc.). For example, the outer shell layer uses biocompatible materials with adjustable mechanical and surface properties, which can stably output multi-dimensional tactile feedback close to human skin during physical interaction. For another example, the AI ​​does not have to be completely like a human but has a variety of different free choices. For example, the preset tactile attributes can include, but are not limited to, metallic texture, velvety texture, frosted texture, temperature-sensitive texture, etc.

[0094] Each particle possesses independent state response capabilities, supporting differentiated magnetic pole configuration for any sub-region within the target area, providing the underlying hardware foundation for realizing local stiffness gradient, temperature partition response, or multi-point asynchronous construction.

[0095] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A physical interaction method, characterized in that, It is applied to a control system that includes a 3D display module, a sensor module, and a solid construction module, wherein the solid construction module includes a magnetic particle swarm with switchable magnetic poles, a macroscopic magnetic field coil array, and a material supply device; The three-dimensional display module is used to reconstruct a depth-sensing three-dimensional image in a space without a physical support medium; the method includes: S1: Control the three-dimensional display module to continuously present the dynamic three-dimensional image of the target object, so that the target object exists in the interactive space in a virtual form; S2: The sensor module monitors the interaction space in real time. When the preset interaction intention triggered by the user is identified, the target area required for the current interaction is determined. S3: Within the target area, a guiding magnetic field is generated by the macroscopic magnetic field coil array, and the magnetic particle group is controlled to move under the action of the guiding magnetic field so that the magnetic particles are self-organized and arranged by the attraction of opposite poles of the built-in magnetic poles, generating a local solid structure with a real tactile feel.

2. The method as described in claim 1, characterized in that, Accordingly, step S3 further includes: Within the target area, a guiding magnetic field is generated by the macroscopic magnetic field coil array, which controls the magnetic particle group to be deposited layer by layer or directionally sprayed under the action of the guiding magnetic field. This allows the magnetic particles to self-organize and arrange themselves through the attraction of opposite poles of the built-in magnetic poles, generating a local solid structure with a realistic tactile feel.

3. The method as described in claim 1, characterized in that, Step S2 further includes: The sensor module collects information on the user's interaction with the 3D image. Based on the spatial location corresponding to the interactive behavior information, determine the target area required for the current interaction and activate the entity building module corresponding to the target area.

4. The method as described in claim 2, characterized in that, In the case of the layer-by-layer deposition, step S3 further includes: S31a: Activate the macroscopic magnetic field coil array corresponding to the target area to generate a guiding magnetic field for defining the outline of the entity, wherein the magnetic field strength distribution of the guiding magnetic field is used to indicate the boundary range of the particle deposition. S32a: Control the multiple stored magnetic particles to fall under gravity or be transported by airflow into the target area, and activate the magnetic field boundary layer by layer starting from the bottom first layer, so that the falling magnetic particles can achieve self-organization and arrangement in each layer through automatic switching of built-in magnetic poles, and attract and fix each other with the fixed lower layer particles. S33a: Repeat the S32a layer-by-layer filling step until the target area is completely filled with magnetic particles, forming a local solid structure consistent with the contour of the three-dimensional image.

5. The method as described in claim 4, characterized in that, After step S3, the method further includes: S4: When the interaction is detected to be over, a disengagement signal is sent to the magnetic particles to control the magnetic particles to switch their magnetic poles to a mutually repulsive state, so that the local physical structure is recycled to the bottom storage area under the action of gravity, and the target object is restored to the projected virtual form.

6. The method as described in claim 5, characterized in that, Sending a disengagement signal to the magnetic particles includes: Broadcast a disband command to all magnetic particles, or send disband signals sequentially via timing control; When each magnetic particle receives the dispersal command or the dispersal signal, the built-in magnetic field switching module switches the magnetic poles to a state where they repel each other from adjacent particles; this causes the particles to lose their mutual fixing force and fall naturally under the action of gravity, sliding down the preset recycling channel to the bottom storage area.

7. The method as described in claim 2, characterized in that, In the case of the layer-by-layer deposition, step S3 further includes: S31b: Activate the macroscopic magnetic field coil array corresponding to the target region to generate a magnetic trap barrier for defining the outline of the entity; S32b: Controlling multiple stored magnetic particles to fall into the target area under the action of gravity, with the built-in magnetic poles of the magnetic particles preset to be opposite each other, so that there is always a repulsive force between the particles; wherein, the particles fall to the boundary of the magnetic trap under the action of gravity and are trapped and constrained in the target area by the magnetic trap potential barrier. S33b: Repeat the S32b layer-by-layer filling step, and by adjusting the interlayer strength of the magnetic trap barrier, the microparticles are suspended and stacked layer by layer to form a local solid structure consistent with the contour of the three-dimensional image; When the interaction is detected to be over, the magnetic trap barrier only needs to be removed or the magnetic field direction switched, and the local solid structure will naturally disperse and be recycled under the combined action of gravity and the repulsive force between particles.

8. The method as described in claim 2, characterized in that, In the case of directional spraying, step S3 further includes: Activate the macroscopic magnetic field coil array corresponding to the target area to generate a guiding magnetic field for guiding the flight trajectory of the particles; Magnetic microparticles are sprayed onto the target area using a microparticle spraying device, and the magnetic microparticles are deposited on the surface of the target area under the action of the guiding magnetic field to form a microparticle layer structure; the deposited magnetic microparticles are fixed to each other by the attraction of built-in magnetic poles to form a hollow solid structure consistent with the outline of the three-dimensional image.

9. The method according to any one of claims 1 to 8, characterized in that, Also includes: A water mist generating device generates a controllable water mist layer in the interactive space, wherein the water mist layer is located on the surface of the target area or the user's interactive hand area, and the water mist generating device is configured to avoid interfering with the light field reconstruction of the three-dimensional display module; the water mist layer is used to form a moist tactile sensation by contacting the surface of the local solid structure when the local solid structure is generated in the target area, and / or to carry temperature-controlled particles to provide temperature feedback.

10. A physical interactive system, characterized in that, include: A three-dimensional display module is used to continuously present dynamic three-dimensional images of the target object, so that the target object exists in the interactive space in a virtual form; The sensor module is used to monitor the interaction space in real time, identify the preset interaction intentions triggered by the user, and determine the target area required for the current interaction. The solid construction module includes a swarm of magnetic particles with switchable magnetic poles, a macroscopic magnetic field coil array, and a material supply device, used to generate local solid structures with realistic tactile feel in the target area by layer-by-layer deposition or directional spraying under the guidance of a magnetic field. The control module, which is connected to the 3D display module, the sensor module and the entity construction module respectively, is used to execute the context-aware on-demand entity-based interaction method according to any one of claims 1 to 9.

11. The physical interactive system as described in claim 10, characterized in that, The magnetic particles used in the particle deposition method of the entity building module include: The magnetic core, made of magnetic material, is used to generate magnetic force in response to an external magnetic field; A magnetic field switching unit, integrated inside the magnetic microparticles, is used to switch the magnetic polarity according to the received control signal, so that adjacent microparticles generate mutual attraction during the construction stage and mutual repulsion during the disintegration stage. The outer shell, which wraps around the magnetic core and the magnetic field switching unit, is used to provide preset tactile properties for the surface that comes into contact with the user after the local solid structure is formed.

Images