A six-dimensional screen adaptive following system and method for a vehicle cabin

Abstract

This invention belongs to the field of intelligent cockpit technology, and relates to a six-dimensional screen adaptive following system and method for vehicle cockpits. The invention includes a perception module for collecting multimodal data within the cockpit; a decision control unit connected to the perception module for processing the multimodal data, identifying user intent and the current scene, and generating corresponding six-dimensional screen following control commands; a six-dimensional screen actuator connected to the decision control unit for receiving the following control commands and driving the screen to translate forward and backward along the Z-axis, rotate around the X-axis, and rotate around the Y-axis; and a human-computer interaction output module connected to the decision control unit, including a screen UI rendering unit and an audio output unit. The screen UI rendering unit outputs a corresponding virtual avatar animation while the screen moves, and the audio output unit outputs corresponding voice feedback while the screen moves.

Description

Technical Field

[0001] This invention belongs to the field of intelligent cockpit technology, and relates to a six-dimensional screen adaptive following system and method for vehicle cockpits. Background Technology

[0002] With the development of intelligent cockpit technology in vehicles, the functions of in-vehicle screens are becoming increasingly rich, but their interaction methods still have significant shortcomings in terms of intelligence and proactivity. Currently, most in-vehicle screens are fixed or only support limited-angle (such as rotation or tilt) electric adjustment, and their position adjustment mainly relies on manual operation by the user, failing to proactively respond to user needs. While existing technologies can control the screen's display content via voice commands, the screen itself remains static in physical space, lacking multi-dimensional adaptive motion capabilities and physical interactive feedback. Specifically, existing solutions mainly suffer from the following drawbacks: The interaction method is passive; the screen cannot actively sense user needs and follow the position, requiring users to manually adjust it. With a single dimension of motion and limited degrees of freedom of adjustment, it is usually limited to rotation or pitch, and cannot achieve multi-dimensional spatial movement. Lack of scenario understanding: It cannot intelligently switch between different follow modes based on different driving scenarios (such as making calls and entertainment).

[0003] Lack of physical presence: As an interactive terminal, the screen lacks feedback on physical location, making it difficult to create a tangible cabin companion experience. Summary of the Invention

[0004] The purpose of this invention is to provide a six-dimensional screen adaptive following system and method for vehicle cockpits, in order to solve the technical problems of passive interaction mode, single motion dimension, lack of scene understanding and lack of physical sense of in-vehicle screen.

[0005] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, the present invention provides a six-dimensional screen adaptive following system for a vehicle cockpit, comprising: The perception module is used to collect multimodal data in the cockpit, including driver status monitoring data, occupant status monitoring data, voice commands, and vehicle event signals. The decision control unit, connected to the perception module, is used to process the multimodal data, identify user intent and current scene, and generate corresponding six-dimensional screen follow control commands. The six-dimensional screen actuator is connected to the decision control unit and is used to receive the follow control command to drive the screen to translate forward and backward in the Z-axis direction, rotate around the X-axis, and rotate around the Y-axis. The human-computer interaction output module, connected to the decision control unit, includes a screen UI rendering unit and an audio output unit. The screen UI rendering unit is used to output a corresponding virtual character animation while the screen is moving, and the audio output unit is used to output corresponding voice feedback while the screen is moving.

[0006] Secondly, the present invention provides a six-dimensional screen adaptive following method for a vehicle cockpit, applied to a six-dimensional screen adaptive following system for a vehicle cockpit, comprising the following steps: Continuously monitor multimodal data within the cockpit, including driver status monitoring data, occupant status monitoring data, voice commands, and vehicle event signals; Based on the multimodal data, determine the current follow-up scenario that needs to be triggered; Based on the following scenario, a preset following strategy is invoked to generate a control script that includes the screen target pose, motion trajectory, and multimedia feedback content. The six-dimensional screen actuator and human-machine interaction output module are coordinated to execute the control script and complete the adaptive following interaction.

[0007] Compared with the prior art, the present invention has the following beneficial effects: The perception module of this invention is used to collect multimodal data within the cockpit, providing a reliable and rich real-time data foundation for subsequent intelligent decision-making. The decision control unit is connected to the perception module and processes the multimodal data, identifies user intent and the current scene, and generates corresponding six-dimensional screen follow control commands. This enables the system to shift from passively responding to commands to actively anticipating and serving the scene, achieving intelligent and contextualized interaction. The six-dimensional screen actuator is connected to the decision control unit and receives the follow control commands, driving the screen to translate forward and backward along the Z-axis, rotate around the X-axis, and rotate around the Y-axis, providing the hardware foundation for active follow interaction. The human-computer interaction output module is connected to the decision control unit and includes a screen UI rendering unit and an audio output unit. The screen UI rendering unit outputs a corresponding virtual character animation while the screen moves, and the audio output unit outputs corresponding voice feedback while the screen moves. This deep integration of physical motion and digital content creates a multi-sensory interactive experience that blends sight, sound, and touch. The six-dimensional screen adaptive following system for vehicle cockpits of the present invention effectively solves the technical problems of passive interaction mode, single motion dimension, lack of scene understanding and lack of physical sense of in-vehicle screens through the coordinated work of various modules.

[0008] This invention provides a reliable and rich real-time data foundation for subsequent intelligent decision-making by continuously monitoring multimodal data within the cockpit. Based on this multimodal data, it determines the following scenario to be triggered, providing a basis for intelligent decision-making and improving the system's adaptability and intelligence. According to the following scenario, a preset following strategy is invoked to generate a control script containing the screen target pose, motion trajectory, and multimedia feedback content. The screen target pose and motion trajectory clearly define the screen's final position and movement path in space. The multimedia feedback content enhances the fun and practicality of the interaction through screen UI rendering and audio output. The six-dimensional screen actuator and human-computer interaction output module are coordinated to execute the control script, completing the adaptive following interaction. This invention can achieve proactive interactive experience by actively sensing changes in needs and environment, realizing multi-dimensional flexible adjustment of the screen and intelligent scene adaptation, thus improving the sense of physical interaction. Attached Figure Description

[0009] Figure 1 This is a system block diagram of an embodiment of the present invention; Figure 2 This is a diagram of the adaptive following method according to an embodiment of the present invention; Figure 3 This is a structural diagram of the six-dimensional screen execution mechanism according to an embodiment of the present invention.

[0010] Figure 4 This is a flowchart of a method according to an embodiment of the present invention.

[0011] The components include: 1. Linear drive unit; 101. Motor worm gear assembly; 102. Lead screw assembly; 103. Mounting bracket; 104. Mounting base; 105. Damping slide rail; 106. Base; 107. Slider part; 108. Track part; 2. First ball joint structure; 201. Sphere; 202. Spherical shell; 3. Screen; 4. Second ball joint structure; 10. Auxiliary telescopic rod. Detailed Implementation

[0012] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0013] It should be noted that the terms "first," "second," etc., in the specification and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0014] The present invention will now be described in further detail with reference to the accompanying drawings: Example 1: See Figure 1 The present invention discloses a six-dimensional screen adaptive following system for vehicle cockpit, comprising: a perception module, a decision control unit, a six-dimensional screen actuator, and a human-machine interaction output module.

[0015] The perception module is used to collect multimodal data in the cockpit, including driver status monitoring data, occupant status monitoring data, voice commands, and vehicle event signals, providing a reliable and rich real-time data foundation for subsequent intelligent decision-making.

[0016] In a preferred embodiment of the present invention, the sensing module includes: DMS (Driver Monitoring System) cameras used to collect driver status monitoring data; OMS (Occupant Monitoring System) cameras are used to collect passenger status monitoring data. The OMS cameras are integrated with ToF (Time of Flight) depth sensors. A microphone array used to capture voice commands and locate sound sources.

[0017] The decision control unit is connected to the perception module. It processes the multimodal data, identifies user intent and the current scene, and generates corresponding six-dimensional screen-following control commands. This enables the system to shift from passively responding to commands to proactively anticipating service scenarios and generating precise motion control commands. This ensures the screen can adaptively adjust according to user needs and scene requirements, achieving intelligent and contextualized interaction.

[0018] In a preferred embodiment of the present invention, the decision control unit includes a scene decision controller and a multimodal following strategy engine; See Figure 2 The multimodal follow strategy engine has multiple pre-defined follow scenario strategies, which include at least wake-up follow scenario, cockpit butler scenario, entertainment and movie viewing scenario, and incoming call scenario. The scene decision unit is used to process the driver status monitoring data, the passenger status monitoring data, the voice commands and the vehicle event signals, identify the user intent and the current scene, and generate corresponding six-dimensional screen follow control commands based on multiple follow scene strategies preset by the multimodal follow strategy engine.

[0019] In a preferred embodiment of the present invention, when the scene decision controller determines that it is a wake-up following scene, the six-dimensional screen following control command generated by the scene decision controller is used to control the six-dimensional screen execution mechanism to perform the following actions: Drive screen 3 to rotate around the Y-axis so that the front of screen 3 faces the waker identified by sound source localization and face recognition; Drive screen 3 to translate along the Z-axis to a preset distance.

[0020] In a preferred embodiment of the present invention, when the scene decision device determines that it is a cockpit butler scene, the six-dimensional screen follow control command generated by the scene decision device is used to control the six-dimensional screen actuator to perform the following actions: The driver screen 3 first turns towards the target device corresponding to the voice command, stays there for a preset time, and then turns back towards the command issuer.

[0021] In a preferred embodiment of the present invention, the cabin butler scenario further includes: While the screen is being rotated, control commands are sent to the target device via the vehicle's CAN bus; The virtual avatar in the screen UI rendering unit nods and responds with voice.

[0022] In a preferred embodiment of the present invention, when the scene decision device determines that it is an entertainment movie-watching scene, the six-dimensional screen following control command generated by the scene decision device is used to control the six-dimensional screen execution mechanism to perform the following actions: Based on the operator's seat coordinates and the preset optimal ergonomic model for movie viewing, the target pose of screen 3 is calculated. Drive screen 3 to automatically move to the target pose; During the movie viewing, the screen 3 is adjusted to the corresponding preset angle in real time based on the slight changes in the position of the occupants' heads detected by the OMS camera.

[0023] In a preferred embodiment of the present invention, when the scene decision device determines that it is an incoming call scenario, the six-dimensional screen following control command generated by the scene decision device is used to control the six-dimensional screen execution mechanism to perform the following actions: Drive screen 3 to move and rotate towards the driver's side at a preset speed, so that screen 3 enters the driver's field of vision; When the call ends, drive screen 3 returns to its original position.

[0024] In a preferred embodiment of the present invention, the incoming call scenario further includes: The system identifies the driver based on the currently logged-in account or by recognizing the driver's identity using the OMS camera, thus determining the recipient of the incoming call notification. The edges of the screen 3 feature a soft breathing light effect, while a gradually increasing ringtone is played through the audio output unit.

[0025] See Figure 1 The six-dimensional screen actuator is connected to the decision control unit. The six-dimensional screen actuator receives the follow control command and drives the screen 3 to translate forward and backward along the Z-axis, rotate around the X-axis, and rotate around the Y-axis, providing the hardware foundation for active follow interaction. That is, the six-dimensional screen actuator can perform six-dimensional actions: forward movement along the Z-axis, reverse movement along the Z-axis, forward rotation around the X-axis, reverse rotation around the X-axis, forward rotation around the Y-axis, and reverse rotation around the Y-axis.

[0026] See Figure 3 In a preferred embodiment of the present invention, the six-dimensional screen execution mechanism includes: a mounting frame 103, two parallel linear drive units 1, and an auxiliary telescopic rod 10.

[0027] The two linear drive units 1 are fixedly connected to the mounting bracket 103; The auxiliary telescopic rod 10 is hinged to the mounting frame 103; The output end of the linear drive unit 1 is connected to the slider portion 107 of a damping slide rail 105 through the first ball joint structure 2, and the track portions 108 of the two damping slide rails 105 are located on the same straight line and are fixedly connected to the screen 3. The output end of the auxiliary telescopic rod 10 is connected to the screen 3 through the second ball joint structure 4.

[0028] In a preferred embodiment of the present invention, both the linear drive unit 1 and the auxiliary telescopic rod 10 include a motor worm gear assembly 101 and a lead screw assembly 102 driven by the motor worm gear assembly 101. The motor worm gear assembly 101 and the lead screw assembly 102 of the linear drive unit 1 are both fixed to the mounting bracket 103; The auxiliary telescopic rod 10 also includes a base 106. The motor worm gear assembly 101 and the lead screw assembly 102 of the auxiliary telescopic rod 10 are both fixedly connected to the base 106. The base 106 is hinged to the mounting frame 103. Both the first ball joint structure 2 and the second ball joint structure 4 include a ball 201 and a spherical shell 202, meaning that the first ball joint structure 2 and the second ball joint structure 4 have the same structure. The ball 201 is movably embedded in the spherical shell 202. The ball 201 is fixedly connected to the output end of the lead screw assembly 102. The spherical shell 202, which is connected to the linear drive unit 1, is fixedly connected to the damping slide rail 105. The spherical shell 202, which is connected to the auxiliary telescopic rod 10, is fixedly connected to the screen 3.

[0029] The spherical shell 202 of the first ball joint structure 2 is fixedly connected to the slider portion 107 of the damping slide rail 105, and the spherical shell 202 of the second ball joint structure 4 is fixedly connected to the screen 3.

[0030] In a preferred embodiment of the present invention, the track portion 108 is provided with a straight groove, the slider portion 107 is embedded in the groove and is fixedly connected to the spherical shell 202 of the first ball joint structure 2; A rubber damping block is provided between the outer wall of the slider portion 107 and the inner wall of the straight groove, and the damping block is fixed to the slider portion 107 by an interference fit. The damping slide rail 105 also includes a mounting base 104, through which the track portion 108 is fixedly connected to the screen 3.

[0031] See Figure 1 The human-computer interaction output module is connected to the decision control unit. The human-computer interaction output module includes a screen UI rendering unit and an audio output unit. The screen UI rendering unit outputs a corresponding virtual character animation while the screen 3 moves, and the audio output unit outputs corresponding voice feedback while the screen 3 moves. This deep integration of physical motion and digital content creates a multi-sensory interactive experience that combines sight, sound, and touch.

[0032] The perception module of this invention can actively collect various information within the cabin. The decision control unit identifies user intentions and scenarios based on this information, enabling the screen to proactively respond to user needs and follow the user's position without requiring manual adjustment, thus achieving proactive interaction. The six-dimensional screen actuator enables the screen to move in multiple dimensions in space, increasing the freedom of screen adjustment. The system can intelligently switch between different following modes according to different driving scenarios, improving the screen's practicality and adaptability. The human-computer interaction output module, through the screen UI rendering unit and audio output unit, provides accompanying virtual avatar animations and voice feedback while the screen moves, enhancing the screen's sense of presence and creating a personalized cabin companion experience, improving user satisfaction and comfort when interacting with the in-vehicle screen. This invention's six-dimensional screen adaptive following system for vehicle cabins effectively solves the technical problems of passive in-vehicle screen interaction, single-dimensional movement, lack of scenario understanding, and lack of a sense of presence through the collaborative work of various modules.

[0033] See Figure 2 and Figure 4 Based on the above system, the present invention also discloses a six-dimensional screen adaptive following method for vehicle cockpits, comprising the following steps: S1 continuously monitors multimodal data within the cockpit, including driver status monitoring data, occupant status monitoring data, voice commands, and vehicle event signals, providing a reliable and rich real-time data foundation for subsequent intelligent decision-making.

[0034] S2, based on the multimodal data, determines the following scenario that needs to be triggered, providing a basis for intelligent decision-making for the system, enabling the system to take different countermeasures according to different scenario characteristics, thereby improving the system's adaptability and intelligence level.

[0035] In a preferred embodiment of the present invention, the following scenarios include at least a wake-up following scenario, a cockpit butler scenario, an entertainment viewing scenario, and an incoming call scenario; The trigger condition for the wake-up follow-up scenario is the detection of a preset global wake-up word; The trigger condition for the cockpit butler scenario is that a specific voice command involving vehicle control is recognized after the cockpit is woken up. The trigger condition for the entertainment viewing scenario is the recognition of an instruction to start the entertainment system; The trigger condition for the incoming call scenario is that the vehicle information system receives an incoming call request.

[0036] S3, based on the described following scenario, invokes a preset following strategy to generate a control script containing the screen target pose, motion trajectory, and multimedia feedback content. The screen target pose and motion trajectory define the screen's final position and movement path in space. The multimedia feedback content provides users with corresponding virtual avatar animations and voice feedback through screen UI rendering and audio output, enhancing the fun and practicality of the interaction, allowing users to more intuitively feel the screen adjustments and system responses.

[0037] In a preferred embodiment of the present invention, the step of generating the control script includes: The Kalman filter algorithm is used to fuse the driver's head coordinates, passenger coordinates and sound source localization information to obtain target localization data; Based on the target positioning data, a fifth-order polynomial interpolation algorithm is used to plan the motion trajectory of screen 3.

[0038] S4, coordinate the control of the six-dimensional screen actuator and the human-machine interaction output module to execute the control script and complete the adaptive following interaction. Through coordinated control of the six-dimensional screen actuator and the human-machine interaction output module, the six-dimensional screen actuator and the human-machine interaction output module can work closely together and execute precisely according to the requirements of the control script.

[0039] This invention achieves an active interactive experience, with the system proactively sensing needs and environmental changes without requiring manual adjustments from the user; it enables multi-dimensional flexible adjustments, breaking the limitations of traditional screen movement and meeting diverse needs; it achieves intelligent scene adaptation, intelligently switching adjustment and feedback methods according to different driving scenarios to provide personalized services; and it enhances the sense of physical interaction, using multimedia feedback to enhance the screen's physical feel, bringing a tangible cabin companion experience and comprehensively improving the convenience, comfort, and fun of user interaction with the in-vehicle screen.

[0040] Example 2: See Figure 1 This embodiment discloses a six-dimensional screen adaptive following system for vehicle cockpits, including: a perception module, a decision control unit, a six-dimensional screen actuator, and a human-machine interaction output module.

[0041] The perception module is used to collect multimodal data in the cockpit, including driver status data from the DMS, occupant status data from the OMS, voice commands, and vehicle event signals.

[0042] The decision control unit is connected to the perception module and is used to process perception data, identify user intentions and the current scene, and generate corresponding six-dimensional screen-following control commands. Its core includes a scene decision unit and a multimodal following strategy engine.

[0043] The six-dimensional screen actuator is connected to the decision control unit and is used to receive control commands to drive the screen to translate along the X, Y, and Z axes and rotate around the three axes to achieve six-dimensional spatial motion.

[0044] The human-computer interaction output module includes a screen UI rendering unit and an audio output unit, which are used to output corresponding virtual character animations and voice feedback while the screen is moving.

[0045] See Figure 2 and Figure 4 Based on the above system, the present invention also discloses a six-dimensional screen adaptive following method for vehicle cockpits, comprising the following steps: Step 1: Continuously monitor the status data and command events in the cockpit through the sensing module.

[0046] Step 2: Based on the data and events, the decision control unit determines the following scenario that needs to be triggered.

[0047] Step 3: Based on the determined following scenario, invoke the preset following strategy to generate a control script that includes the screen target pose, motion trajectory, and multimedia feedback content.

[0048] Step 4: Coordinate the control of the six-dimensional screen actuator and the human-machine interaction output module, execute the control script, and complete the adaptive following interaction.

[0049] This invention achieves a proactive and intelligent interactive experience: through multimodal perception and scene judgment, the screen can proactively respond to user needs and follow them, transforming passive operation into proactive service, significantly reducing the user's operational burden.

[0050] This invention enhances the naturalness and immersion of the interaction: the six-dimensional motion capability combined with the audiovisual feedback of the virtual image makes the screen an interactive entity with "body language", enhancing the sense of presence and emotional level of the interaction.

[0051] This invention achieves precise scenario-based adaptation: it designs differentiated following strategies for different scenarios such as wake-up follow, cabin butler, entertainment viewing, and incoming call summoning, making the interactive behavior highly consistent with the scenario and more user-friendly.

[0052] This invention enhances the safety and convenience of the cockpit: in situations requiring the driver's attention, such as when receiving a call, the screen can actively and smoothly move into the driver's field of vision, avoiding the driver turning their head sharply or becoming distracted, thus improving driving safety.

[0053] This invention employs a DMS camera and a global shutter sensor with a higher frame rate (60fps) to ensure the accuracy of capturing the driver's micro-movements (such as nodding and rapid eye movements).

[0054] This invention employs an OMS camera: it adds a ToF (Time-of-Flight) depth sensor, which is fused with an RGB (Red, Green, Blue) camera to achieve accurate recognition and positioning of occupant gestures (such as pointing or waving).

[0055] The audio system of this invention adds a microphone array for sound source localization to the original power amplifier, which can help determine the location of the passenger who issued the voice command.

[0056] The core scenarios of this invention include: wake-up follow scenario, cockpit butler scenario, entertainment viewing scenario, and incoming call summoning scenario, and the implementation methods are as follows: The following is an example of the wake-up follow scenario of the present invention: The wake-up condition for the following scenario is that the speech recognition module detects a preset global wake-up word.

[0057] The technical process for waking up a scene is as follows: Sound source localization: The microphone array initially determines the approximate location (left / right) of the person waking up.

[0058] Passenger Confirmation: The OMS system accurately locates the person waking up the user based on the sound source location and facial recognition.

[0059] Motion execution: The screen rotates around the Y-axis, facing the waker; at the same time, the screen moves smoothly forward 100-150mm along the Z-axis, presenting a posture of leaning forward to listen; the motion duration is controlled within 1.2 seconds, using a smooth S-shaped speed curve.

[0060] Awaken multimedia feedback in the scene: The virtual avatar in the screen UI makes animations of looking and listening, accompanied by soft prompting sound effects.

[0061] The following is an example of the cockpit butler feature of this invention: The trigger condition is: after the user is woken up, they issue a specific command involving vehicle control, such as turning on the passenger seat massage.

[0062] The technical process for the cockpit butler scenario is as follows: Command parsing: The natural language processing module parses commands and identifies the target device that needs to be controlled, such as the passenger seat.

[0063] Target localization: The system's built-in cockpit equipment coordinate map is invoked to obtain the spatial location of the target equipment.

[0064] Eye-tracking guidance: The decision control unit generates a control script, which drives the six-dimensional screen to first turn towards the target device, stay for about 1 second to provide visual guidance, and then turn back to the one that issued the command.

[0065] Command execution and feedback: As the screen rotates, control commands are sent to the target device via the vehicle's CAN bus. The UI avatar nods and responds with voice.

[0066] The following are examples of entertainment movie-watching scenarios based on the present invention: The trigger condition for the entertainment viewing scenario is: the user activates the entertainment system (such as "play a movie") via voice or touch.

[0067] The technical process of this invention for the entertainment viewing scenario is as follows: Operator identification: If voice activation is enabled, the operator is identified through sound source localization and OMS; if touch activation is enabled, the operator is identified through seat pressure distribution sensor or gesture tracking of OMS.

[0068] Comfortable posture calculation: Based on the operator's seat coordinates (from DMS / OMS) and the preset optimal viewing ergonomic model, the optimal target posture of the screen (including distance, height and angle) is calculated.

[0069] Automatic steering and confirmation: The screen automatically and smoothly moves to the target pose. Then, a virtual avatar appears and confirms with voice, such as: "Okay, let's watch the movie together now."

[0070] Adaptive adjustment: During the movie viewing process, OMS continuously monitors subtle changes in the position of the occupants' heads and fine-tunes the screen angle in real time to maintain the best viewing angle.

[0071] The following is an embodiment of the call-summoning scenario of the present invention: The trigger condition for the incoming call scenario in this invention is: the vehicle information system receives an incoming call request.

[0072] The technical process for the incoming call summoning scenario in this invention is as follows: Primary User Determination: The system determines the primary recipient of incoming call notifications based on the currently logged-in account or the primary user identity identified by OMS.

[0073] Non-intrusive alert: The screen slowly (movement time approximately 2 seconds) moves towards the driver (Y-axis) and rotates slightly (around the Y-axis) to bring it into the driver's peripheral vision; the movement amplitude is precisely calculated to ensure that the alert is effective but not abrupt.

[0074] Combined with audio: A soft breathing light effect appears at the edge of the screen, while a gradually increasing ringtone plays.

[0075] End Return: When an incoming call is answered or rejected, the screen returns to its original position at the same slow speed.

[0076] The key technical details of this invention are as follows: This invention employs a Kalman filter algorithm to fuse the head coordinates of the DMS, the occupant coordinates of the OMS, and the sound source localization information to obtain stable and reliable target localization data.

[0077] The motion controller of this invention uses a fifth-order polynomial interpolation algorithm for trajectory planning to ensure that the acceleration of all following movements is continuous and shock-free, providing a smooth motion experience.

[0078] The system of this invention monitors the screen motion envelope constructed by ultrasonic sensors in real time. Once an obstacle is detected, the obstacle avoidance algorithm is immediately triggered to replan the path or stop the movement.

[0079] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of this invention.

Claims

1. A six-dimensional screen adaptive following system for vehicle cockpits, characterized in that, include: The perception module is used to collect multimodal data in the cockpit, including driver status monitoring data, occupant status monitoring data, voice commands, and vehicle event signals. The decision control unit, connected to the perception module, is used to process the multimodal data, identify user intent and current scene, and generate corresponding six-dimensional screen follow control commands. The six-dimensional screen actuator is connected to the decision control unit and is used to receive the follow control command and drive the screen (3) to translate forward and backward, rotate around the X axis and rotate around the Y axis in the Z axis direction; The human-computer interaction output module is connected to the decision control unit and includes a screen UI rendering unit and an audio output unit. The screen UI rendering unit is used to output a matching virtual image animation while the screen (3) moves, and the audio output unit is used to output a matching voice feedback while the screen (3) moves.

2. The six-dimensional screen adaptive following system for a vehicle cockpit according to claim 1, characterized in that, The six-dimensional screen actuator includes: Mounting bracket (103); Two parallel linear drive units (1) are fixedly connected to the mounting bracket (103). The auxiliary telescopic rod (10) is hinged to the mounting bracket (103); The output end of the linear drive unit (1) is connected to the slider part (107) of a damping slide rail (105) through the first ball joint structure (2), and the track parts (108) of the two damping slide rails (105) are located on the same straight line and are fixedly connected to the screen (3). The output end of the auxiliary telescopic rod (10) is connected to the screen (3) through the second ball joint structure (4).

3. The system according to claim 2, characterized in that, Both the linear drive unit (1) and the auxiliary telescopic rod (10) include a motor worm gear assembly (101) and a lead screw assembly (102) driven by the motor worm gear assembly (101). The motor worm gear assembly (101) and the lead screw assembly (102) of the linear drive unit (1) are both fixed to the mounting bracket (103). The auxiliary telescopic rod (10) also includes a base (106), and the motor worm gear assembly (101) and the lead screw assembly (102) of the auxiliary telescopic rod (10) are both fixedly connected to the base (106). The base (106) is hinged to the mounting bracket (103). Both the first ball joint structure (2) and the second ball joint structure (4) include a ball (201) and a spherical shell (202). The ball (201) is movably embedded in the spherical shell (202). The ball (201) is fixedly connected to the output end of the lead screw assembly (102). The spherical shell (202) connected to the linear drive unit (1) is fixedly connected to the damping slide rail (105). The spherical shell (202) connected to the auxiliary telescopic rod (10) is fixedly connected to the screen (3). The spherical shell (202) of the first ball joint structure (2) is fixedly connected to the slider portion (107) of the damping slide rail (105), and the spherical shell (202) of the second ball joint structure (4) is fixedly connected to the screen (3).

4. A six-dimensional screen adaptive following system for a vehicle cockpit according to claim 3, characterized in that, The track section (108) is provided with a straight groove, and the slider section (107) is embedded in the groove and fixedly connected to the spherical shell (202) of the first ball joint structure (2). A rubber damping block is provided between the outer wall of the slider part (107) and the inner wall of the straight groove, and the damping block is fixed to the slider part (107) by an interference fit. The damping slide rail (105) also includes a mounting base (104), and the track portion (108) is fixedly connected to the screen (3) via the mounting base (104).

5. A six-dimensional screen adaptive following system for a vehicle cockpit according to claim 1, characterized in that, The sensing module includes: DMS camera used to collect driver status monitoring data; An OMS camera used to collect occupant status monitoring data, wherein the OMS camera integrates a ToF depth sensor; A microphone array used to capture voice commands and locate sound sources.

6. A six-dimensional screen adaptive following system for a vehicle cockpit according to claim 1, characterized in that, The decision control unit includes a scene decision maker and a multimodal following strategy engine; The multimodal follow strategy engine has multiple pre-defined follow scenario strategies, which include at least wake-up follow scenario, cockpit butler scenario, entertainment viewing scenario, and incoming call scenario. The scene decision unit is used to process the driver status monitoring data, the passenger status monitoring data, the voice commands and the vehicle event signals, identify the user intent and the current scene, and generate corresponding six-dimensional screen follow control commands based on multiple follow scene strategies preset by the multimodal follow strategy engine.

7. A six-dimensional screen adaptive following system for a vehicle cockpit according to claim 6, characterized in that, When the scene decision controller determines that the scene is to be woken up and followed, the six-dimensional screen follow control command generated by the scene decision controller is used to control the six-dimensional screen execution mechanism to perform the following actions: Drive the screen (3) to rotate around the Y-axis so that the screen (3) faces the waker identified by sound source localization and face recognition; The drive screen (3) is translated along the Z-axis to a preset distance; Preferably, when the scene decision controller determines that it is a cockpit butler scene, the six-dimensional screen follow control command generated by the scene decision controller is used to control the six-dimensional screen actuator to perform the following actions: The drive screen (3) first turns to the direction of the target device corresponding to the voice command, stays for a preset time, and then turns back to the direction of the command issuer; The cockpit butler scenario also includes: While the screen is being rotated, control commands are sent to the target device via the vehicle's CAN bus; The virtual avatar in the screen UI rendering unit nods and responds with voice; Preferably, when the scene decision controller determines that it is an entertainment viewing scene, the six-dimensional screen following control command generated by the scene decision controller is used to control the six-dimensional screen execution mechanism to perform the following actions: Based on the operator's seat coordinates and the preset optimal ergonomic model for movie viewing, the target pose of the screen (3) is calculated. The drive screen (3) automatically moves to the target pose; During the movie viewing, the screen (3) is adjusted to the corresponding preset angle in real time based on the slight changes in the position of the occupants' heads detected by the OMS camera.

8. A six-dimensional screen adaptive following system for a vehicle cockpit according to claim 6, characterized in that, When the scene decision controller determines that it is an incoming call scenario, the six-dimensional screen follow control command generated by the scene decision controller is used to control the six-dimensional screen execution mechanism to perform the following actions: Drive the screen (3) to move and rotate towards the driver's side at a preset speed, so that the screen (3) enters the driver's field of vision; When the call ends, drive screen (3) returns to its original position; The incoming call scenario also includes: The system identifies the driver based on the currently logged-in account or by recognizing the driver's identity using the OMS camera, thus determining the recipient of the incoming call notification. The screen (3) edge displays a soft breathing light effect, while a gradually increasing ringtone is played through the audio output unit.

9. A six-dimensional screen adaptive following method for a vehicle cockpit, applied to a six-dimensional screen adaptive following system for a vehicle cockpit as described in any one of claims 1 to 8, characterized in that, Includes the following steps: Continuously monitor multimodal data within the cockpit, including driver status monitoring data, occupant status monitoring data, voice commands, and vehicle event signals; Based on the multimodal data, determine the current follow-up scenario that needs to be triggered; Based on the following scenario, a preset following strategy is invoked to generate a control script that includes the screen target pose, motion trajectory, and multimedia feedback content. The six-dimensional screen actuator and human-machine interaction output module are coordinated to execute the control script and complete the adaptive following interaction.

10. A six-dimensional screen adaptive following method for a vehicle cockpit according to claim 9, characterized in that, The following scenarios include at least the wake-up following scenario, the cockpit butler scenario, the entertainment viewing scenario, and the incoming call scenario; The trigger condition for the wake-up follow-up scenario is the detection of a preset global wake-up word; The trigger condition for the cockpit butler scenario is that a specific voice command involving vehicle control is recognized after the cockpit is woken up. The trigger condition for the entertainment viewing scenario is the recognition of an instruction to start the entertainment system; The trigger condition for the incoming call scenario is that the vehicle information system receives an incoming call request; Preferably, the step of generating the control script includes: The Kalman filter algorithm is used to fuse the driver's head coordinates, passenger coordinates and sound source localization information to obtain target localization data; Based on the target positioning data, the motion trajectory of the screen (3) is planned using a fifth-order polynomial interpolation algorithm.

Images