Space interaction display mobile platform, control method and system thereof and storage medium

Abstract

The application discloses a space interaction display mobile platform and a control method, system and storage medium thereof, which comprises the following steps: acquiring state information of the mobile platform, state information of a user and / or state information of an environment; identifying a task phase currently where the mobile platform is located according to the acquired state information; selecting a target cooperative control mode from a plurality of preset cooperative control modes according to the task phase, wherein different cooperative control modes correspond to different display control strategies and motion control strategies; and controlling the space interaction display module to form display content in the external space of the mobile platform and controlling the mobile platform to execute corresponding motion according to the target cooperative control mode. According to the control method of the space interaction display mobile platform, the task phase currently where the mobile platform is located is identified through the acquisition of the state information, and the corresponding cooperative control mode is selected according to the task phase, so that the contradiction that a single mode cannot meet the requirements of different phases is solved.

Description

Technical Field

[0001] This application relates to the field of mobile platform and spatial display interaction technology, and more specifically, to a spatial interactive display mobile platform and its control method, system and storage medium. Background Technology

[0002] This section provides only background information relevant to this disclosure and is not necessarily prior art.

[0003] With the development of artificial intelligence and mobile platform technology, mobile service platforms with mobility capabilities are increasingly being used in fields such as shopping mall guidance, exhibition hall explanation, restaurant service, and medical companionship. In recent years, spatial interactive display technology, which can create suspended images, holographic projections, or naked-eye 3D displays in the external space of the mobile platform itself, has brought new possibilities to human-computer interaction on mobile platforms.

[0004] However, in existing technologies, the integration of spatial interactive display technology into mobile platforms presents the following main technical problems: First, there is a lack of an adaptive collaborative control architecture for each task stage. A mobile platform, in a complete service flow, goes through several stages with completely different characteristics, such as: long-distance navigation, approaching the target user, stopping to provide detailed explanations, and following the user while they are moving. The collaborative requirements for "display" and "motion" differ greatly at each stage, and may even conflict with each other. Existing technologies typically employ a single, linked control mode, which cannot adaptively handle these stage-specific conflicts, resulting in a poor user experience at some stages.

[0005] Secondly, the stability of the displayed content is severely affected by the motion disturbances of the mobile platform. When the mobile platform walks, turns, crosses obstacles, or stops suddenly, the body will inevitably experience six degrees of freedom of vibration and impact. If the spatial interactive display module is rigidly connected to the body, the image suspended in the air will shake accordingly, making it difficult for users to view clearly, and even causing visual fatigue or dizziness. Furthermore, it cannot solve the problem of display content deviation caused by changes in the body's posture (such as tilting when climbing a slope).

[0006] Secondly, there is an inherent physical delay in interactive response. From the moment a user begins an action (such as turning around, raising a hand, or speaking), to sensor detection, algorithm processing, motion command issuance, and motor execution, there is an unavoidable chain of delays. When a user interacts with a mobile platform while moving, this delay manifests as the displayed content always following the user "a beat behind," disrupting the naturalness and smoothness of the interaction. Summary of the Invention

[0007] The main objective of this application is to provide a control method for a spatial interactive display mobile platform to solve the technical problems of poor task stage adaptability, large impact of motion disturbance on display content, and physical delay in interactive response in existing spatial interactive display mobile platforms. By acquiring state information to identify the current task stage of the mobile platform, and selecting the corresponding collaborative control mode according to the task stage, the method achieves stage adaptation of display control and motion control, and resolves the contradiction that a single mode cannot meet the needs of different stages.

[0008] To achieve the above objectives, the first aspect of this application proposes a control method for a spatial interactive display mobile platform, comprising: Obtain status information of the mobile platform, user status information, and / or environment status information; Based on the acquired status information, identify the current task stage of the mobile platform; According to the task stage, a target collaborative control mode is selected from multiple preset collaborative control modes, wherein different collaborative control modes correspond to different display control strategies and motion control strategies. According to the target collaborative control mode, the control space interactive display module forms display content in the external space of the mobile platform and controls the mobile platform to perform corresponding movements.

[0009] According to the control method of the spatial interactive display mobile platform provided in this application, the task phase includes at least one of the following: navigation movement phase, user approach phase, pausing interaction phase and dynamic follow phase. The collaborative control mode includes: In the first mode, during the navigation movement phase, the displayed content is controlled to maintain directional stability relative to the motion coordinate system of the mobile platform; In the second mode, during the user approach phase, the mobile platform and / or the display module are controlled to actively face the user; In the third mode, during the pause-and-interaction phase, the imaging parameters of the display module are controlled to adapt to the user's real-time viewing posture. In the fourth mode, during the dynamic following phase, the mobile platform is controlled to follow the user's movement with a preset offset, and the display orientation of the displayed content is synchronously adjusted to align with the user and remain aligned.

[0010] According to the control method of the spatial interactive display mobile platform provided in this application, the control method of the spatial interactive display mobile platform further includes: Obtain the real-time motion disturbance parameters of the mobile platform; Based on the motion disturbance parameters, the optical elements and / or mechanical connection mechanisms of the display module are driven to perform reverse compensation motion.

[0011] According to the control method of the spatial interactive display mobile platform provided in this application, the control method of the spatial interactive display mobile platform further includes: Obtain the user's historical movement trajectory; Based on the historical movement trajectory, predict the user's expected location and / or expected line of sight in a preset future time period; Before the user's interaction command or action is completed, the mobile platform and / or the display module are controlled to perform a pre-alignment action in advance based on the prediction result.

[0012] A second aspect of this application proposes a spatial interactive display mobile platform, comprising: Mobile platform core; A spatial interactive display module is installed on the mobile platform body, and the spatial interactive display module is configured to form display content that can be viewed and / or interacted with by the user in the external space of the mobile platform body; A sensing module, configured to collect the state of the mobile platform itself, the state of the user, and / or the state of the environment; A control module is connected to the display module, the mobile platform body, and the sensing module, respectively. The control module is configured to execute the control method of the spatial interactive display mobile platform in any of the above technical solutions.

[0013] The spatial interactive display mobile platform of this invention has the same advantages as the control method of the spatial interactive display mobile platform described above, and will not be repeated here.

[0014] According to the spatial interactive display mobile platform provided in this application, the control module includes: A stage identification unit is configured to identify the current task stage of the mobile platform based on the information collected by the sensing module. A mode switching unit is configured to automatically switch the collaborative control mode according to the identified task stage. The compensation unit is configured to acquire the motion disturbance parameters of the mobile platform body and drive the display module to perform reverse compensation motion; The prediction unit is configured to predict the user's future position and / or line of sight based on the user's historical movement trajectory and trigger a pre-alignment action.

[0015] According to the present application, a spatial interactive display mobile platform is provided, wherein the display module includes a first display sub-module disposed near the end effector of the mobile platform body; The control module is also configured to acquire the real-time pose of the end effector and project the guidance information into the physical operating space through the first display sub-module after coordinate transformation, so that the virtual display position of the guidance information coincides with the operating point of the target physical object in space.

[0016] According to the present application, a spatial interactive display mobile platform is provided, wherein the display module is connected to the mobile platform body through a vibration damping mechanism, and the vibration damping mechanism is configured to reduce the vibration transmitted from the mobile platform body to the display module.

[0017] A third aspect of this application proposes a spatial interactive display system, comprising: At least one spatial interactive display mobile platform in any of the above technical solutions; The edge computing terminal and / or cloud are configured to perform at least one of the following functions: user state recognition, intent prediction, content generation, or strategy optimization. The management platform is configured to manage the mobile platform, distribute content, configure parameters, transmit data back, or monitor its status.

[0018] The spatial interactive display system of this invention has the same advantages as the aforementioned spatial interactive display mobile platform, which will not be repeated here.

[0019] The fourth aspect of this application proposes a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements a control method for a spatial interactive display mobile platform of any of the above-described technical solutions.

[0020] The computer-readable storage medium of this invention has the same advantages as the control method of the above-described spatial interactive display mobile platform, and will not be repeated here.

[0021] The technical solution provided by the embodiments of this application has the following beneficial effects: By acquiring the state information of the mobile platform, the user's state information, and / or the state information of the environment, the context of the interaction scene can be fully perceived, providing multi-dimensional data support for stage identification; by identifying the current task stage of the mobile platform based on the acquired state information, different scenarios such as navigation movement, user approach, stationary interaction, and dynamic following can be accurately distinguished, avoiding poor experience in some stages due to the use of a unified control strategy; by selecting a target mode from multiple preset collaborative control modes according to the task stage, different modes correspond to different display control strategies and motion control strategies, achieving decoupling and adaptive matching of display and motion, and solving the inherent contradiction of needing stable display during navigation and active alignment of display during interaction; by collaboratively controlling the display content and the movement of the mobile platform according to the target mode, the mobile platform can interact with the user with the optimal display posture and motion behavior at different stages, improving the naturalness and smoothness of the overall interaction. Therefore, the present invention realizes the task stage adaptive collaborative control of a spatial interactive display mobile platform, significantly improving the interactive performance and user experience of the mobile platform in complex dynamic environments. Attached Figure Description

[0022] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application. In the drawings: Figure 1 This is a flowchart of a control method for a spatial interactive display mobile platform according to an embodiment of the present invention. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] In this application, the terms "upper," "lower," "left," "right," "front," "rear," "top," "bottom," "inner," "outer," "middle," "vertical," "horizontal," "lateral," and "longitudinal" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are primarily for the purpose of better describing this application and its embodiments, and are not intended to limit the indicated device, element, or component to having a specific orientation, or to be constructed and operated in a specific orientation.

[0026] Furthermore, in addition to indicating location or positional relationship, some of the aforementioned terms may also have other meanings. For example, the term "above" may also be used in some cases to indicate a certain dependency or connection relationship. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances.

[0027] Furthermore, the terms "installation," "setup," "equipped with," "connection," "linked," and "socketing" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral structure; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or an internal connection between two devices, components, or parts. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0028] This invention provides a spatial interactive display mobile platform. The mobile platform includes a movable mobile platform body, a spatial interactive display module, a sensing module, and a control module.

[0029] A mobile platform refers to a physical entity that possesses autonomous or semi-autonomous mobility and can carry or integrate spatial interactive display functional modules. Specific forms of mobile platforms include, but are not limited to: wheeled robots, legged robots (such as bipedal robots and quadrupedal robot dogs), tracked robots, unmanned ground vehicles (such as unmanned vehicles and automated guided vehicles), biomimetic mobile bodies (such as biomimetic robot dogs and biomimetic robot insects), and other intelligent mobile devices with mobile chassis or walking mechanisms.

[0030] The mobile platform body serves as the physical platform and motion execution base of the system. In this embodiment, the mobile platform body is a wheeled mobile platform, including a chassis, a fuselage, and a head. The chassis uses omnidirectional wheels or Mecanum wheels and has a built-in continuous rotation mechanism around a vertical axis, possessing 360° unlimited continuous rotational freedom, enabling translation in any direction and rotation in place. The head is connected to the fuselage via a two-degree-of-freedom gimbal, providing horizontal rotation and pitch adjustment capabilities. The mobile platform body also integrates a power management unit, a wireless communication unit (such as Wi-Fi and 5G modules), and an emergency braking unit.

[0031] The spatial interactive display module is used to create display content that users can view and / or interact with in the external space of the platform. Preferably, it employs an aerial levitation imaging module, such as an aerial imaging device based on the Dirac mirror principle or light field reconstruction technology. Its deployment locations include: a chest interaction module, fixed to the chest cavity of the mobile platform, providing a stable main interactive interface; a head-following module, integrated into the head of the mobile platform and linked to the head gimbal, used to simulate "eye gaze" or display the mobile platform's status (such as battery level or operating mode); and an end effector fine interaction module, integrated into the robotic arm's end effector (in this embodiment, a single-degree-of-freedom gripper), used for close-range operation guidance. The spatial interactive display module is connected to the mobile platform body via a vibration damping mechanism. This vibration damping mechanism can employ elastic elements (such as rubber pads or springs) or damping elements, configured to reduce vibrations transmitted from the mobile platform body to the display module, improving display stability.

[0032] The perception module adopts a multimodal fusion architecture to obtain the mobile platform's own state, user state, and environmental state. Specifically, it includes: a visual perception unit, with at least one set of RGB-D depth cameras to capture key points of the user's entire skeleton, key points of the hands, 6DoF head pose, and gaze direction. The cameras can be mounted on the head or chest of the mobile platform and have automatic exposure and white balance adjustment functions to adapt to different lighting environments; a spatial positioning unit, integrating a UWB module or a visual SLAM module to achieve centimeter-level relative positioning between the mobile platform and the user. UWB base stations can be pre-deployed in the environment, or the mobile platform's own UWB tag can be used to measure distance with the UWB module on the user's handheld terminal; an auditory perception unit, with a microphone ring array, supporting 360° sound source localization and voice command recognition, capable of distinguishing user voices from different directions and performing beamforming; a body state perception unit, including a motor encoder and an IMU inertial measurement unit (collecting three-axis acceleration and angular velocity), used to obtain the mobile platform's own speed, attitude, vibration, and other states; and a motion disturbance detection unit, which can be set at the mounting base of the spatial interactive display module, with an accelerometer specifically collecting high-frequency vibration signals transmitted to the display module for subsequent compensation control.

[0033] The control module is the decision-making center of the system, typically employing an embedded industrial computer or a high-performance microcontroller running a real-time operating system. The control module communicates with the display module, the mobile platform itself (including chassis motors, gimbal motors, etc.), and the sensing module. The control module is configured to execute the collaborative control method of this invention.

[0034] The control module contains multiple functional units. The stage identification unit identifies the current task stage of the mobile platform based on multimodal information collected by the perception module. The identification logic is as follows: when the mobile platform is moving along a planned path and no effective interactive user is detected, it is determined to be in the navigation movement stage; when a user is detected entering a preset distance threshold (e.g., 3 meters) and is approaching the mobile platform, it is determined to be in the user approach stage; when the mobile platform has stopped moving, the user remains within a preset interaction area (e.g., 0.5-1.5 meters), and there is interactive behavior (voice, gesture, gaze), it is determined to be in the stationary interaction stage; when the user issues a follow command or when the system detects that the user is moving and expects the mobile platform to accompany them, it is determined to be in the dynamic following stage.

[0035] The mode switching unit automatically switches the collaborative control mode based on the identified task stage. The mapping relationship between the display control strategy and the motion control strategy differs in different modes. Specifically, in the navigation movement stage (first mode), the control display content maintains a fixed orientation relative to the mobile platform's coordinate system (e.g., always facing forward), and the mobile platform moves along the planned path, prioritizing movement efficiency and obstacle avoidance. In the user approach stage (second mode), the control display content dynamically adjusts its imaging direction to the user's position, while the mobile platform decelerates, adjusts its chassis orientation, and tilts its gimbal to bring the display into the user's field of vision. In the stationary interaction stage (third mode), the control display content follows the user's head movements and line of sight changes to maintain the optimal viewing angle; the mobile platform remains stationary, with only the head gimbal performing micro-angle tracking. In the dynamic following stage (fourth mode), the control display content maintains a preset spatial position relative to the user (e.g., 45 degrees to the user's right front), and the mobile platform follows the user with a constant offset, maintaining relative distance and orientation.

[0036] The compensation unit is used to address motion disturbance issues. It acquires real-time motion disturbance parameters (including vibration, acceleration, angular velocity, displacement, or attitude changes) of the mobile platform and then drives the optical elements and / or mechanical connection mechanisms of the display module to perform reverse compensation motion. For example, for high-frequency vibrations, micron-level rapid displacement compensation can be achieved by driving optical elements such as image source displacement devices and deformable mirrors; for low-frequency attitude changes, reverse angle adjustment can be performed by driving the gimbal mechanism. The compensation objective is to maintain spatial stability of the displayed content relative to a preset reference frame (such as a ground inertial coordinate system or a user's line-of-sight coordinate system).

[0037] The prediction unit addresses the physical latency issue. Based on continuously collected user position, head posture, and gaze direction data from the sensing module, the user's historical movement trajectory is constructed. Predictive models (such as Kalman filtering and LSTM temporal neural networks) are used to predict the user's expected position and gaze direction within a preset future time period. When the prediction indicates that the user is about to enter the interaction area or is about to make an interaction intention, the control module sends pre-alignment commands (such as pre-rotation or pre-translation of the mobile platform, pre-tilt of the gimbal, or pre-deflection of the display module's imaging direction) to the chassis drive motor and head gimbal motor in advance. The pre-alignment action, together with the fine-tuning actions after the user's actual arrival, constitutes a "feedforward + feedback" composite control, significantly reducing the lag caused by the perception-execution delay.

[0038] Furthermore, when the mobile platform is equipped with a robotic arm and there is an end-effector operation task, the control module also includes an end-effector spatial registration unit. This unit acquires the real-time pose of the end-effector (from forward kinematics), transforms the guidance information to be displayed (such as "grasp point" and "installation direction") in real time, and projects it onto the physical operation space through the end-effector fine interaction module, so that the virtual guidance precisely coincides with the physical target. The guidance information may include at least one of the following: target gripping point, installation position, operation sequence, or safety warning area.

[0039] like Figure 1 As shown, the collaborative control method of the present invention is described below with reference to a specific process. This method can be executed by the aforementioned mobile platform and includes the following steps: First, acquiring state information: The perception module collects the state of the mobile platform (position, speed, posture, vibration), the user's state (position, posture, gesture, gaze, voice), and the environmental state (obstacle distribution, ground type, etc.) in real time. Second, identifying the task stage: The stage identification unit determines, based on the above information, whether the current stage is navigation movement, user approach, stationary interaction, or dynamic following. Then, selecting the collaborative control mode: The mode switching unit selects the target mode from multiple preset modes according to the identification result. Next, collaborative control execution: According to the target mode, the control module collaboratively controls the display content of the display module and the movement of the mobile platform, while the compensation unit and prediction unit continue to work, performing reverse compensation and pre-alignment respectively. Finally, acquiring feedback and closed-loop optimization: Information such as user gaze feedback, voice feedback, and interaction completion rate is acquired, control parameters are optimized online, and the process returns to the first step.

[0040] In a specific application scenario, the mobile platform is used as a shopping mall guide platform. In standby mode, the mobile platform is docked at a designated location. When the mobile platform receives a navigation task (such as heading to a store), it enters the navigation movement phase. The control module switches to the first mode, and the displayed content (such as store name arrows) remains forward relative to the mobile platform's coordinate system. The mobile platform moves along the planned path at an economical speed, while using LiDAR for obstacle avoidance. When a user is detected entering within 3 meters and approaching, the user approach phase begins. The control module switches to the second mode, the mobile platform decelerates and adjusts its chassis orientation, while the imaging direction of the chest interaction module dynamically turns towards the user's position, displaying a greeting such as "Hello, do you need help?" The mobile platform's head gimbal slightly tilts, simulating the action of looking at the user. The user stops and interacts with the mobile platform (such as asking "Where is the nearest restaurant?"), entering the standing interaction phase. The control module switches to the third mode, the mobile platform remains stationary, and only the head gimbal tracks the user's head movements with micro-angles. The displayed content dynamically adjusts according to the user's line of sight, for example, projecting the restaurant's location as a highlighted arrow onto the ground in front of the user. Once the user begins walking towards the restaurant and issues the command "Take me there," the dynamic following phase begins. The control module switches to the fourth mode, and the mobile platform follows the user's movement with a 1-meter rear right and a 45-degree offset. Simultaneously, the displayed content (such as virtual landmarks and distance cues) remains aligned with the user and compensates for any disturbances caused by the platform's motion. Throughout the process, the compensation unit operates continuously: when the chassis vibrates due to carpet or seams on the ground, the accelerometer detects high-frequency vibrations, and the compensation unit drives the optical actuators inside the display module to perform micron-level reverse displacement, stabilizing the aerial image; when the mobile platform tilts due to an uphill slope, the compensation unit drives the gimbal to adjust its angle in the opposite direction, keeping the displayed content level relative to the ground.

[0041] In another specific application scenario, the mobile platform is used as a personal assistant on a desktop. The platform is in standby mode, docked at the edge of the desktop. The user approaches from the side. The sensing module detects the user, and the user motion prediction unit, based on the user's walking trajectory over 3 seconds (speed approximately 1.2 m / s, direction pointing 30 degrees to the right of the platform), predicts that the user will reach a point 1 meter in front of the platform in 0.8 seconds. The control module immediately sends a pre-rotation command to the chassis, and the platform completes its turn before the user arrives, aligning the chest-shaped interaction module with the user's expected location. When the user arrives, the displayed content is already stably positioned directly in front of them, without waiting for the platform to "turn around." The user issues a voice command, "Check the weather for me," and the platform responds and displays the weather information.

[0042] In another specific application scenario, the mobile platform is used as an outdoor inspection platform. As the platform moves on a carpeted surface, its chassis generates minute vibrations at a frequency of approximately 20-50Hz. An accelerometer located at the display module mounting base detects that the vibration amplitude exceeds a preset threshold. The compensation unit identifies this vibration as a high-frequency, low-amplitude disturbance, unsuitable for gimbal compensation (gigs have insufficient response speed and are easily damaged). The compensation unit drives the variable-focus liquid lens or deformable mirror inside the spatial interactive display module to change the optical path at a speed synchronized with the vibration frequency, achieving optical image stabilization. The suspended image observed by the user remains stable, without significant jitter, and the text information can be read normally.

[0043] To further improve the adaptability and control accuracy of the present invention in complex environments, the present invention also provides the following improved embodiments. These improvements can be applied as optional features to any of the above embodiments, but do not constitute a limitation on the scope of the claims.

[0044] The first improvement method is to adjust the damping coefficient based on road surface unevenness feedforward. In the aforementioned mobile platform, a road surface recognition unit is added to the perception module. This unit can use visual sensors (such as a front-facing RGB camera or depth camera) to collect real-time ground images within a preset distance (e.g., 1-2 meters) in front of the mobile platform, and identify the road surface type (e.g., smooth ceramic tiles, rough cement, carpet, speed bumps, gravel roads, etc.) or directly extract road surface unevenness parameters (e.g., undulation height, slope). Based on the identified road surface unevenness, the control module pre-calculates the optimal damping coefficient for the vibration damping mechanism and actively adjusts the damping coefficient of the damping adjustment elements (e.g., magnetorheological dampers, electro-hydraulic dampers) of the vibration damping mechanism before the mobile platform enters the uneven road surface. For example, when a speed bump is detected ahead, the damping coefficient is increased in advance to improve rigidity and prevent excessive displacement of the display module; when a gravel road is detected, the damping coefficient is decreased in advance to enhance vibration isolation. Compared to passive or semi-active vibration damping that relies solely on real-time feedback, this feedforward control can more effectively suppress shocks, further protect the display module, and improve display stability.

[0045] The second improvement involves adaptively selecting the compensation actuator and dynamic threshold based on vibration frequency. Building upon the aforementioned compensation unit, this invention further introduces an adaptive selection mechanism. The compensation unit analyzes the spectral characteristics of the motion disturbance parameters in real time and calculates the dominant frequency component of the current vibration. Based on the vibration frequency, the type of compensation actuator is adaptively selected: when the vibration frequency is higher than a preset threshold (e.g., 50Hz), optical element compensation (such as image source displacement devices, deformable mirrors, and liquid lenses) is preferred because optical elements have a fast response speed and are suitable for high-frequency, small vibrations; when the vibration frequency is lower than the preset threshold, mechanical connection mechanism compensation (such as gimbal attitude adjustment and active vibration damping mechanisms) is preferred because mechanical mechanisms have a large stroke and are suitable for low-frequency, large-amplitude movements. Furthermore, this preset threshold is not fixed but dynamically adjusted according to the current speed of the mobile platform or the current ground type. For example, when the mobile platform moves at high speed, the energy of high-frequency vibrations may increase, and the threshold can be appropriately lowered to allow optical compensation to intervene earlier; when the mobile platform moves at low speed, the threshold can be appropriately raised to prioritize mechanical compensation and save the lifespan of optical components. For example, when the ground type is identified as rough stone, the expected vibration frequency is low but the amplitude is large. The threshold can be lowered to allow mechanical compensation to work first. This dynamic threshold adjustment strategy enables the compensation system to maintain optimal performance matching under different operating conditions, improving overall stability and energy efficiency.

[0046] When both of the above improvement methods are applied simultaneously, the front-facing camera identifies the road conditions 1.5 meters ahead in real time when the mobile platform moves on uneven outdoor ground. When a series of speed bumps are detected, the control module pre-adjusts the damping coefficient of the vibration reduction mechanism to a preset value, and dynamically adjusts the adaptive compensation frequency threshold from the default 50Hz to 40Hz based on historical vibration data. When the mobile platform passes over a speed bump, the accelerometer detects low-frequency, high-amplitude vibrations (main frequency approximately 30Hz). The compensation unit determines this is below the dynamic threshold and prioritizes reverse pitch compensation using the gimbal mechanism. Simultaneously, due to the increased damping coefficient of the feedforward adjustment, the vibration amplitude is effectively suppressed, resulting in minimal residual shaking of the display module. When the mobile platform subsequently enters a gravel road surface, the road surface recognition unit detects increased road unevenness. The control module pre-adjusts the damping coefficient to enhance vibration isolation, and simultaneously adjusts the frequency threshold to 60Hz, quickly suppressing high-frequency vibration components through optical compensation. Users observe that the displayed content remains consistently stable and clear, almost unaffected by changes in the ground surface.

[0047] In other embodiments of the present invention, more sensors (such as lidar and ultrasonic sensors) can be added to enhance environmental perception capabilities according to actual needs; the display module can adopt aerial imaging devices based on different principles (such as optical waveguides and microlens arrays); the moving mechanism of the mobile platform can also adopt various forms such as tracked, legged, or spherical. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0048] Finally, those skilled in the art should understand that although this invention has been described in detail using a spatial interactive display mobile platform as an example, its control method can also be applied to other types of mobile display devices, such as display modules mounted on drones and interactive screens on automated guided vehicles (AGVs). Any device employing features such as task-phase adaptive multi-mode collaborative control, display-motion decoupling compensation, and user motion prediction and pre-alignment falls within the protection scope of this invention.

[0049] Embodiments of the present invention also propose a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the control method of the spatial interactive display mobile platform of any of the above embodiments.

[0050] The computer-readable storage medium of this invention has the same advantages as the control method of the above-described spatial interactive display mobile platform, and will not be repeated here.

[0051] The aforementioned computer-readable storage medium may be any combination of one or more computer-readable media. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. Computer-readable storage media include, but are not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory, optical fiber, portable compact disk read-only memory (CDROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0052] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0053] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0054] Computer program code for performing the operations of embodiments of the present invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0055] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0056] Obviously, those skilled in the art should understand that the various units or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps into a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0057] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A control method for a spatial interactive display mobile platform, characterized in that, include: Obtain status information of the mobile platform, user status information, and / or environment status information; Based on the acquired status information, identify the current task stage of the mobile platform; According to the task stage, a target collaborative control mode is selected from multiple preset collaborative control modes, wherein different collaborative control modes correspond to different display control strategies and motion control strategies. According to the target collaborative control mode, the control space interactive display module forms display content in the external space of the mobile platform and controls the mobile platform to perform corresponding movements.

2. The control method for the spatial interactive display mobile platform according to claim 1, characterized in that, The task phases include at least one of the following: navigation movement phase, user approach phase, pause interaction phase, and dynamic follow phase. The collaborative control mode includes: In the first mode, during the navigation movement phase, the displayed content is controlled to maintain directional stability relative to the motion coordinate system of the mobile platform; In the second mode, during the user approach phase, the mobile platform and / or the display module are controlled to actively face the user; In the third mode, during the pause-and-interaction phase, the imaging parameters of the display module are controlled to adapt to the user's real-time viewing posture. In the fourth mode, during the dynamic following phase, the mobile platform is controlled to follow the user's movement with a preset offset, and the display orientation of the displayed content is synchronously adjusted to align with the user and remain aligned.

3. The control method for the spatial interactive display mobile platform according to claim 1, characterized in that, The control method for the spatial interactive display mobile platform further includes: Obtain the real-time motion disturbance parameters of the mobile platform; Based on the motion disturbance parameters, the optical elements and / or mechanical connection mechanisms of the display module are driven to perform reverse compensation motion.

4. The control method for the spatial interactive display mobile platform according to claim 1, characterized in that, The control method for the spatial interactive display mobile platform further includes: Obtain the user's historical movement trajectory; Based on the historical movement trajectory, predict the user's expected location and / or expected line of sight in a preset future time period; Before the user's interaction command or action is completed, the mobile platform and / or the display module are controlled to perform a pre-alignment action in advance based on the prediction result.

5. A spatial interactive display mobile platform, characterized in that, include: Mobile platform core; A spatial interactive display module is installed on the mobile platform body, and the spatial interactive display module is configured to form display content that can be viewed and / or interacted with by the user in the external space of the mobile platform body; A sensing module, configured to collect the state of the mobile platform itself, the state of the user, and / or the state of the environment; A control module is connected to the display module, the mobile platform body, and the sensing module, respectively, and the control module is configured to execute the control method of the spatial interactive display mobile platform according to any one of claims 1 to 4.

6. The spatial interactive display mobile platform according to claim 5, characterized in that, The control module includes: A stage identification unit is configured to identify the current task stage of the mobile platform based on the information collected by the sensing module. A mode switching unit is configured to automatically switch the collaborative control mode according to the identified task stage. The compensation unit is configured to acquire the motion disturbance parameters of the mobile platform body and drive the display module to perform reverse compensation motion; The prediction unit is configured to predict the user's future position and / or line of sight based on the user's historical movement trajectory and trigger a pre-alignment action.

7. The spatial interactive display mobile platform according to claim 5, characterized in that, The display module includes a first display sub-module disposed near the end effector of the mobile platform body; The control module is also configured to acquire the real-time pose of the end effector and project the guidance information into the physical operating space through the first display sub-module after coordinate transformation, so that the virtual display position of the guidance information coincides with the operating point of the target physical object in space.

8. The spatial interactive display mobile platform according to claim 5, characterized in that, The display module is connected to the mobile platform body through a vibration damping mechanism, which is configured to reduce the vibration transmitted from the mobile platform body to the display module.

9. A spatial interactive display system, characterized in that, include: At least one spatial interactive display mobile platform as described in any one of claims 5 to 8; The edge computing terminal and / or cloud are configured to perform at least one of the following functions: user state recognition, intent prediction, content generation, or strategy optimization. The management platform is configured to manage the mobile platform, distribute content, configure parameters, transmit data back, or monitor its status.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 4.

Images