Television person movement following and emperor position self-adaption system and television person movement following and emperor position self-adaption method

Abstract

The embodiment of the invention provides a television human movement following and emperor position self-adaption system and method, and the system comprises a multi-mode sensing module which is used for sensing and tracking the position information of a user through the fusion of an electromagnetic wave radar and an infrared thermal image, and the position information of the user comprises the spatial position coordinates, the movement track and the orientation information of the user; the intelligent decision and control module is used for generating control parameters for adjusting television display and audio output according to the position information of the user, so that the picture visual angle and the sound radiation center of the television are aligned with the user; and the execution module is used for driving a display unit and an audio unit of the television to adjust according to the control parameters. The system can effectively protect the privacy of a user, stably works under various environmental conditions, accurately senses the position, movement and even orientation of the user through a sensor fusion technology, and is linked with a display and audio system to realize automatic adaptation of a picture visual angle and sound radiation.

Description

Technical Field

[0001] This disclosure relates to the fields of smart TV and human-computer interaction technology, and in particular to a TV human motion following and emperor position adaptive system and method. Background Technology

[0002] With the development of display and audio technologies, users have increasingly higher demands for an immersive television experience. The "sweet spot" on a television typically refers to the viewing position directly in front of the screen, offering the best audio-visual experience. However, when multiple family members are watching a movie together, or when users move (e.g., get up, adjust their posture), they often leave this optimal position, resulting in a shift in viewing angle and a deterioration in sound quality. Existing technologies have attempted to address this issue: 1. Camera visual recognition: Tracks users by recognizing their faces or body contours through cameras, but this method fails when the light is dim, the user is facing to the side or with their back to the TV, and it also raises user privacy issues.

[0003] 2. Single sensor positioning: For example, using only electromagnetic wave radar, although it can detect distance and speed, it is difficult to accurately distinguish between human bodies and other moving objects (such as pets or swaying plants), and it is also impossible to identify the user's orientation or posture.

[0004] 3. Simple gimbal rotation: Some high-end TVs can rotate the screen toward the user via a gimbal, but this usually only performs rough position tracking and cannot achieve fine correction of the viewing angle or precise focusing of the sound beam.

[0005] Therefore, there is an urgent need in this field for a technical solution that can track the user's position and posture around the clock with high precision while protecting privacy, and can intelligently adjust the perspective of the screen and the radiation of sound to create a "supreme seat" experience for the user in real time. Summary of the Invention

[0006] This disclosure provides a television human motion tracking and emperor position adaptive system and method to solve the problems of privacy risks, inaccurate recognition, and low audiovisual adaptation in existing television human motion tracking solutions.

[0007] Based on the above problems, in the first aspect, a television human motion tracking and emperor position adaptive system is provided, the system comprising: a multimodal perception module, an intelligent decision-making and control module, and an execution module; The multimodal sensing module is used to track the user's location information by fusing electromagnetic wave radar and infrared thermal imaging. The user's location information includes the user's spatial coordinates, movement trajectory, and orientation information. The intelligent decision-making and control module is used to generate control parameters for adjusting the TV display and audio output based on the user's location information, so that the TV's viewing angle and sound radiation center are aligned with the user. The execution module is used to drive the display unit and audio unit of the television to make adjustments according to the control parameters.

[0008] In conjunction with the first aspect, in one possible implementation, the multimodal sensing module includes an electromagnetic wave radar submodule, an infrared imaging submodule, and a sensor fusion processing unit. The electromagnetic wave radar submodule is used to detect the presence, dynamics, distance, radial velocity and angle information of objects in the space in front of the television, and to identify vital signs through the micro-Doppler effect to distinguish the human body from other objects. The vital signs include breathing and / or heartbeat. The infrared thermal imaging submodule is used to acquire an infrared thermal image of the space in front of the television, identify the human body outline through thermal radiation differences, and determine the orientation of the human body based on the human body outline. The sensor fusion processing unit is used to receive and process electromagnetic wave radar point cloud data and infrared thermal image data, and to perform spatiotemporal registration and fusion of the two types of data through Kalman filtering and deep learning model algorithms to generate the user's location information.

[0009] In conjunction with the first aspect, in one possible implementation, the sensor fusion processing unit is used to: employ a lightweight neural network, taking the distance, angle, velocity and multi-channel features of the radar point cloud and the infrared thermal image as input, and output the bounding box and three-dimensional coordinates of the center point of the target.

[0010] In conjunction with the first aspect, in one possible implementation, the intelligent decision-making and control module is used to generate image rendering parameters, which are used to adjust the image processing engine to perform perspective correction on the image so that the image distortion seen by the user at the current perspective is minimized. The execution module includes a display driver module, which is used to adjust the screen output in real time according to the screen rendering parameters.

[0011] In conjunction with the first aspect, in one possible implementation, the intelligent decision-making and control module is used for: The user's line of sight direction is determined by their spatial location coordinates and orientation information; Calculate the angle between the user's gaze direction and the screen normal, and apply an inverse perspective transformation matrix to correct geometric distortion of the screen image.

[0012] In conjunction with the first aspect, in one possible implementation, the intelligent decision-making and control module is used to generate audio beam imaging parameters, and the speaker array or acoustic processor used to control the television forms a directional sound beam pointing towards the user's current location. The execution module includes an audio driver module, which is used to drive the speaker array of the television according to the audio beamforming parameters.

[0013] In conjunction with the first aspect, in one possible implementation, the intelligent decision-making and control module is used to dynamically adjust the delay and weight of each unit of the speaker array by calculating the distance difference and phase difference between the user and each unit of the speaker array relative to the television, so that the sound energy is coherently superimposed at the user's position.

[0014] In conjunction with the first aspect, in one possible implementation, the intelligent decision-making and control module is further configured to generate screen rotation parameters, which are used to control the rotation of the TV pan-tilt unit so that the central axis of the screen is aligned with the user's position. The execution module also includes a gimbal drive module, which is used to control the motor to rotate the screen according to the screen rotation parameters.

[0015] Secondly, a television human motion tracking and emperor position adaptive method is provided, the method being applied to the system described in any possible embodiment of the first aspect, the method comprising: The system starts up, and the electromagnetic wave radar submodule and infrared thermal imaging submodule continuously perform environmental perception. The sensor fusion processing unit analyzes radar point clouds and infrared thermal images to identify and locate key users within the space. Real-time tracking of the three-dimensional coordinates and movement vectors of the main users; Determine whether the location change of the main user exceeds a preset threshold; If the value is not exceeded, the current parameter will be maintained. If the value exceeds the limit, the intelligent decision-making and control module will calculate new screen rendering parameters and audio beamforming parameters based on the new user location. The execution module applies new image rendering parameters and audio beamforming parameters to achieve adaptive adjustments to the image and sound; Return to the steps of real-time tracking of the main user's 3D coordinates and movement vector.

[0016] In conjunction with the second aspect, in one possible implementation, the sensor fusion processing unit analyzes radar point clouds and infrared thermal images to identify and locate key users within the space, including: The heat source area of ​​the target is determined by the infrared thermal image, the activity frequency of the target is determined by the radar point cloud data, and the main users in the target in space are identified by combining the heat source area and the activity frequency.

[0017] The beneficial effects of the embodiments disclosed herein include: The system uses a fusion of electromagnetic radar and infrared thermal imager to accurately and covertly track the user's position. Based on changes in the user's position, the system intelligently calculates and adjusts the perspective correction parameters and audio beamforming parameters, driving the display and audio systems to ensure that the viewing angle and sound radiation center are always aligned with the user. This dynamically adapts any user position to the optimal "sweet spot," significantly enhancing viewing immersion and comfort. This disclosure boasts advantages in high precision, robustness, and privacy protection.

[0018] 1. All-weather robustness: Combining electromagnetic wave radar (anti-light interference) and infrared thermal imaging (anti-visible light interference), stable tracking is achieved in various complex lighting environments such as bright, dark, and backlight.

[0019] 2. Accurate identification and privacy protection: Infrared thermal imaging senses the outline of the heat source rather than a clear facial image, while electromagnetic radar does not collect optical images, fundamentally avoiding the privacy risks of exposing users' facial features and indoor scenes.

[0020] 3. Complementary and enhanced sensing information: Electromagnetic waves provide accurate distance and speed information, while infrared thermal imaging provides more intuitive shape and orientation information. The fusion of the two can effectively filter false alarms and improve tracking accuracy and reliability.

[0021] 4. Immersive and Adaptive Experience: Not only does it rotate the screen, but it also performs deep software-level adaptation from two dimensions: image perspective and acoustic beams, creating an immersive "emperor's seat" experience for users in any position.

[0022] 5. Supports multi-user scenarios: Through sensor fusion, multiple heat sources and target points can be tracked simultaneously, laying the foundation for subsequent priority judgment (such as tracking the main user) or multi-user audio partitioning (such as sending different audio to different locations through beamforming). Attached Figure Description

[0023] Figure 1 A schematic diagram of the structure of a television human motion tracking and emperor position adaptive system provided in an embodiment of this disclosure; Figure 2 A schematic diagram illustrating the workflow of a sensor fusion processing unit provided in an embodiment of this disclosure; Figure 3 A schematic diagram of the principle of an emperor's position mapping unit provided in an embodiment of this disclosure; Figure 4 A flowchart of a television human motion tracking and emperor position adaptive method provided in an embodiment of this disclosure. Detailed Implementation

[0024] This disclosure provides a television human motion tracking and emperor position adaptive system and method. Preferred embodiments of this disclosure are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustrative and explanatory purposes only and are not intended to limit this disclosure. Furthermore, the embodiments and features described in this application can be combined with each other unless otherwise specified.

[0025] Figure 1 This is a schematic diagram of the structure of a television human motion tracking and emperor position adaptive system provided in an embodiment of the present disclosure, as shown below. Figure 1 As shown, the system includes: a multimodal perception module 100, an intelligent decision-making and control module 200, and an execution module 300.

[0026] The multimodal sensing module 100 is used to track the user's location information by fusing electromagnetic wave radar and infrared thermal imaging. The user's location information includes the user's spatial coordinates, movement trajectory, and orientation information. The intelligent decision-making and control module 200 is used to generate control parameters for adjusting the TV display and audio output based on the user's location information, so that the TV's viewing angle and sound radiation center are aligned with the user. The execution module 300 is used to drive the display unit and audio unit of the television to make adjustments according to the control parameters.

[0027] In this embodiment of the disclosure, the system and method utilize infrared thermal imaging technology and electromagnetic wave radar technology to achieve dynamic tracking of the user and adaptively adjust the orientation of the screen and sound to provide the best viewing experience from the emperor's seat.

[0028] This system can effectively protect user privacy, work stably under various environmental conditions, accurately sense the user's location, movement and even orientation through sensor fusion technology, and link the display and audio systems to achieve automatic adaptation of screen viewing angle and sound radiation.

[0029] As one possible implementation, the multimodal sensing module 100 includes an electromagnetic wave radar submodule, an infrared imaging submodule, and a sensor fusion processing unit.

[0030] The electromagnetic wave radar submodule is used to detect the presence, dynamics, distance, radial velocity, and angle information of objects in the space in front of the television. It also identifies vital signs through the micro-Doppler effect to distinguish human bodies from other objects. Vital signs include breathing and / or heartbeat.

[0031] The advantage of the electromagnetic wave radar submodule is that it is not affected by light, can penetrate some obstructions, and can identify vital signs (such as breathing and heartbeat) through the micro-Doppler effect, thus effectively distinguishing people from other objects.

[0032] For example, the electromagnetic wave radar submodule can employ frequency-modulated continuous wave radar.

[0033] The infrared thermal imaging submodule is used to acquire infrared thermal images of the space in front of the television, identify human body contours through differences in thermal radiation, and determine the orientation of the human body based on the human body contours.

[0034] The advantage of the infrared imaging submodule is that it does not rely on visible light, but senses the human body in the form of heat sources, avoiding the risk of privacy leakage, and can roughly determine the orientation of the human body (such as the face facing the screen or the body turning to the side) through the thermal image outline.

[0035] For example, the infrared thermal imaging submodule can employ an array of uncooled infrared microbolometers.

[0036] The sensor fusion processing unit receives and processes electromagnetic radar point cloud data and infrared thermal image data. It performs spatiotemporal registration and fusion of the two types of data through Kalman filtering and deep learning model algorithms to generate user location information, thereby improving the accuracy of user identification and the stability of tracking.

[0037] The user's location information includes the user's location coordinates (X, Y, Z), movement trajectory, and attitude orientation confidence (maximum probability).

[0038] Figure 2 This is a schematic diagram of the workflow of a sensor fusion processing unit provided in an embodiment of this disclosure, as shown below. Figure 2 As shown, the electromagnetic wave radar submodule provides point cloud data (including distance, angle, and user position coordinates X, Y, Z) and micro-Doppler data (which can identify breathing and heartbeat), while the infrared thermal imaging submodule provides a heat map matrix (including temperature gradient and contour mask) and heat source center (including orientation estimation) data. The point cloud data and heat map matrix first enter the spatiotemporal registration stage to complete timestamp alignment and coordinate system transformation. The data obtained after spatiotemporal registration, along with the micro-Doppler data and heat source center data, enter the feature extraction stage to extract the radar's position and velocity features and the infrared area and orientation features, respectively. Then, a Kalman filter and deep learning matching algorithm are used for fusion decision-making, and finally, the user coordinates and orientation confidence are output, providing accurate user position and attitude basis for subsequent adjustments to the picture and sound of the television.

[0039] As one possible implementation, the sensor fusion processing unit is used to: employ a lightweight neural network, taking the distance, angle, velocity and multi-channel features of the radar point cloud and the infrared thermal image as input, and output the target's bounding box and the three-dimensional coordinates of the center point.

[0040] The sensor fusion processing unit is also used to determine the type of the target as a human body and filter out interference by using the distance accuracy of electromagnetic wave data and the shape information of infrared thermal images.

[0041] The intelligent decision-making and control module 200 is connected to the multimodal perception module 100. The intelligent decision-making and control module 200 includes an "emperor's position mapping unit," used to calculate the optimal audiovisual parameters relative to the television screen based on the fused real-time user location information. These parameters include: image rendering parameters, audio beam parameters, and screen rotation parameters (if the television has a physical rotation function). The execution module 300 receives and executes instructions from the intelligent decision-making and control module 200 to ensure optimized audiovisual effects for the user's location.

[0042] As one possible implementation, the intelligent decision-making and control module 200 is used to generate image rendering parameters, which are used to adjust the image processing engine and perform perspective correction on the image so that the image distortion seen by the user at the current viewpoint is minimized. The execution module 300 includes a display driver module, which is used to adjust the screen output in real time according to the screen rendering parameters.

[0043] After adjusting the image rendering parameters, the image distortion seen by the user at the current viewpoint is minimal, as if viewing the screen directly.

[0044] Adjusting the display output includes performing real-time geometric transformations on the image to compensate for perspective distortion caused by changes in viewing angle.

[0045] As one possible implementation, the intelligent decision-making and control module 200 is used for: The user's line of sight direction is determined by their spatial location coordinates and orientation information; Calculate the angle between the user's gaze direction and the screen normal, and apply an inverse perspective transformation matrix to correct geometric distortion of the screen image.

[0046] As one possible implementation, the intelligent decision and control module 200 is used to generate audio beam imaging parameters to control the speaker array or acoustic processor of the television to form a directional sound beam pointing to the user's current location. The execution module 300 includes an audio driver module, which is used to drive the speaker array of the television according to audio beamforming parameters.

[0047] This ensures the clarity and immersiveness of the user's location sound.

[0048] As one possible implementation, the intelligent decision and control module 200 is used to dynamically adjust the delay and weight of each unit of the speaker array by calculating the distance difference and phase difference between the user and each unit of the speaker array relative to the television, so that the sound energy is coherently superimposed at the user's position.

[0049] Figure 3 This is a schematic diagram of the principle of an emperor's position mapping unit provided in an embodiment of the present disclosure, as shown below. Figure 3 As shown, a speaker array is located next to the TV screen. When the user is in different positions, the emperor's position mapping unit calculates the corresponding rendering matrix and beam angle according to the user's position: when the user is in position A (0° in front, 1.2m away), the picture does not need to be corrected and the sound beam is 0°; when the user is in position B (30° to the left, 1.8m away), the picture undergoes perspective correction and the sound beam is adjusted to -30°; when the user is in position C (45° to the right, 2.5m away), the picture also undergoes perspective correction and the sound beam is adjusted to +45°, thus achieving adaptive optimization of picture and sound under different positions.

[0050] As one possible implementation, the intelligent decision-making and control module 200 is also used to generate screen rotation parameters, which are used to control the rotation of the TV pan-tilt unit so that the central axis of the screen is aligned with the user's position. The execution module 300 also includes a gimbal drive module, which is used to control the motor to rotate the screen according to the screen rotation parameters.

[0051] This disclosure provides a method for television human motion tracking and emperor position adaptation, which is applied to the aforementioned television human motion tracking and emperor position adaptation system. Figure 4 A flowchart of a television human motion tracking and emperor position adaptive method provided in this disclosure embodiment is shown below. Figure 4 As shown, the method includes: Step S1: The system starts up, and the electromagnetic wave radar submodule and infrared thermal imaging submodule continuously perform environmental perception.

[0052] Step S2: The sensor fusion processing unit analyzes the radar point cloud and infrared thermal image to identify and locate the main users in the space.

[0053] As one possible approach, the heat source area of ​​the target is determined by infrared thermal imaging, the activity frequency of the target is determined by radar point cloud data, and the main users of the target in space are identified by combining the heat source area and the activity frequency.

[0054] Step S3: Track the 3D coordinates and movement vectors of the main users in real time; Step S4: Determine whether the location change of the main user exceeds a preset threshold (e.g., moving more than 20 centimeters).

[0055] If the value is not exceeded, the current parameters will be maintained.

[0056] If the number of users exceeds the limit, step S5 is executed: the intelligent decision-making and control module 200 performs emperor position mapping, calculating new screen rendering parameters and audio beamforming parameters based on the new user position.

[0057] Step S6: Execution module 300 applies new image rendering parameters and audio beamforming parameters to complete the adaptive adjustment of image and sound.

[0058] Step S7: Return to "Step S3: Real-time tracking of the main user's 3D coordinates and movement vector" to achieve continuous tracking and closed-loop control.

[0059] This disclosure provides a computer device, including a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of a television human motion tracking and emperor position adaptive method provided in any embodiment of this disclosure.

[0060] The computer device provided in this disclosure includes a processor, a memory, and a bus. The memory, also known as internal memory, stores execution instructions and includes main memory and external memory. The main memory temporarily stores data processed by the processor, as well as data exchanged with external storage devices such as hard disks. The processor exchanges data with external storage devices through main memory. When the electronic device is running, the processor and memory communicate via the bus, enabling the processor to execute the following instructions: Upon system startup, the electromagnetic radar submodule and infrared thermal imaging submodule continuously perform environmental perception. The sensor fusion processing unit analyzes the radar point cloud and infrared thermal image to identify and lock onto the main user in the space. It tracks the three-dimensional coordinates and movement vector of the main user in real time. It determines whether the position change of the main user exceeds a preset threshold. If it does not exceed the threshold, the current parameters are maintained. If it does exceed the threshold, the intelligent decision-making and control module calculates new image rendering parameters and audio beamforming parameters based on the new user position. The execution module applies the new image rendering parameters and audio beamforming parameters to complete the adaptive adjustment of the image and sound. The system then returns to the step of tracking the three-dimensional coordinates and movement vector of the main user in real time.

[0061] This disclosure provides a computer-readable storage medium storing a computer program. When a processor executes the computer program, it performs the steps of a television human tracking and emperor position adaptation method provided in any embodiment of this disclosure. The storage medium can be volatile or non-volatile computer-readable storage.

[0062] Through the above description of the embodiments, those skilled in the art can clearly understand that the embodiments of this disclosure can be implemented in hardware or by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause a computer device (such as a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this disclosure.

[0063] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes in the drawings are not necessarily essential for implementing this disclosure.

[0064] The sequence numbers of the embodiments disclosed above are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0065] Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A television human motion tracking and emperor position adaptive system, characterized in that, The system includes: a multimodal perception module, an intelligent decision-making and control module, and an execution module; The multimodal sensing module is used to track the user's location information by fusing electromagnetic wave radar and infrared thermal imaging. The user's location information includes the user's spatial coordinates, movement trajectory, and orientation information. The intelligent decision-making and control module is used to generate control parameters for adjusting the TV display and audio output based on the user's location information, so that the TV's viewing angle and sound radiation center are aligned with the user. The execution module is used to drive the display unit and audio unit of the television to make adjustments according to the control parameters.

2. The system according to claim 1, characterized in that, The multimodal sensing module includes an electromagnetic wave radar submodule, an infrared imaging submodule, and a sensor fusion processing unit; The electromagnetic wave radar submodule is used to detect the presence, dynamics, distance, radial velocity and angle information of objects in the space in front of the television, and to identify vital signs through the micro-Doppler effect to distinguish the human body from other objects. The vital signs include breathing and / or heartbeat. The infrared thermal imaging submodule is used to acquire an infrared thermal image of the space in front of the television, identify the human body outline through thermal radiation differences, and determine the orientation of the human body based on the human body outline. The sensor fusion processing unit is used to receive and process electromagnetic wave radar point cloud data and infrared thermal image data, and to perform spatiotemporal registration and fusion of the two types of data through Kalman filtering and deep learning model algorithms to generate the user's location information.

3. The system according to claim 2, characterized in that, The sensor fusion processing unit is used to: employ a lightweight neural network, taking the distance, angle, velocity and multi-channel features of the radar point cloud and the infrared thermal image as input, and output the target's bounding box and the three-dimensional coordinates of the center point.

4. The system according to claim 1, characterized in that, The intelligent decision-making and control module is used to generate image rendering parameters, which are used to adjust the image processing engine and perform perspective correction on the image so that the image distortion seen by the user at the current perspective is minimized. The execution module includes a display driver module, which is used to adjust the screen output in real time according to the screen rendering parameters.

5. The system according to claim 4, characterized in that, The intelligent decision-making and control module is used for: The user's line of sight direction is determined by their spatial location coordinates and orientation information; Calculate the angle between the user's gaze direction and the screen normal, and apply an inverse perspective transformation matrix to correct geometric distortion of the screen image.

6. The system according to claim 1, characterized in that, The intelligent decision-making and control module is used to generate audio beam imaging parameters, and the speaker array or acoustic processor used to control the television forms a directional sound beam pointing to the user's current location. The execution module includes an audio driver module, which is used to drive the speaker array of the television according to the audio beamforming parameters.

7. The system according to claim 6, characterized in that, The intelligent decision-making and control module is used to dynamically adjust the delay and weight of each unit of the speaker array by calculating the distance difference and phase difference between the user and each unit of the speaker array relative to the television, so that the sound energy is coherently superimposed at the user's position.

8. The system according to claim 1, characterized in that, The intelligent decision-making and control module is also used to generate screen rotation parameters, which are used to control the rotation of the TV pan-tilt unit so that the central axis of the screen is aligned with the user's position. The execution module also includes a gimbal drive module, which is used to control the motor to rotate the screen according to the screen rotation parameters.

9. A method for television human motion tracking and emperor position adaptive adaptation, characterized in that, The method is applied to the system according to any one of claims 1-8, and the method includes: The system starts up, and the electromagnetic wave radar submodule and infrared thermal imaging submodule continuously perform environmental perception. The sensor fusion processing unit analyzes radar point clouds and infrared thermal images to identify and locate key users within the space. Real-time tracking of the three-dimensional coordinates and movement vectors of the main users; Determine whether the location change of the main user exceeds a preset threshold; If the value is not exceeded, the current parameter will be maintained. If the value exceeds the limit, the intelligent decision-making and control module will calculate new screen rendering parameters and audio beamforming parameters based on the new user location. The execution module applies new image rendering parameters and audio beamforming parameters to achieve adaptive adjustments to the image and sound; Return to the steps of real-time tracking of the main user's 3D coordinates and movement vector.

10. The method according to claim 9, characterized in that, The sensor fusion processing unit analyzes radar point clouds and infrared thermal images to identify and locate key users within the space, including: The heat source area of ​​the target is determined by the infrared thermal image, the activity frequency of the target is determined by the radar point cloud data, and the main users in the target in space are identified by combining the heat source area and the activity frequency.

Images