Interactive decoder system based on holographic projection and modular design and implementation method

Abstract

The invention discloses an interactive decoder system based on holographic projection and modular design and an implementation method, and relates to the technical field of intelligent terminal equipment. Comprising a first step of establishing an interactive decoder system which comprises a main control chip, a holographic projection unit, an expandable module and an interactive assembly, a second step of inserting a magnetic fool-proof connector into the expandable module, triggering Soc Reichlet RK3588 by using interruption, allocating a memory to the expandable module, and returning a ready signal to the system, the gesture of an operator is recognized through the ToF depth camera, instruction analysis is completed through the AI calculation module, and real-time updating of a projection interface is completed through the holographic projection unit.

Description

Technical Field

[0001] The present invention discloses an interactive decoder system based on holographic projection and modular design and an implementation method thereof, and relates to the technical field of intelligent terminal equipment. Background Art

[0002] With the rapid development of smart homes and immersive interactive technologies, users are placing higher demands on the functionality and interactive experience of home entertainment devices. Existing devices primarily rely on remote controls or basic voice control, which can lead to rigid functionality, poor scalability, and a single interactive method. Summary of the Invention

[0003] In view of the problems of the prior art, the present invention provides an interactive decoder system and an implementation method based on holographic projection and modular design, which is suitable for home entertainment, distance education, virtual conferences and the like.

[0004] The specific scheme proposed by the present invention is:

[0005] The present invention provides a method for implementing an interactive decoder system based on holographic projection and modular design, comprising:

[0006] Step 1: Establish an interactive decoder system, which includes a main control chip, a holographic projection unit, expandable modules, and interactive components.

[0007] The main control chip uses the quad-core A76 high-performance SoC Rockchip RK3588 to schedule the operation of the entire system and allocate resources reasonably; the holographic projection unit uses a 532nm / 50W green laser for projection, presenting a suspended interactive interface; the expandable module uses a 16-pin contact magnetic anti-fool connector to expand the AI ​​computing module that supports PCle3.0, the holographic enhancement module that supports HDMI2.1, and the storage expansion module that supports USB4; the interactive components include a ToF depth camera, a microphone array, and an ultrasonic tactile feedback unit.

[0008] Step 2: Insert the expandable module through the magnetic anti-foolproof connector, use the interrupt to trigger the SoC Rockchip RK3588, allocate memory to the expandable module, and return the system ready signal.

[0009] The operator's gestures are recognized through the ToF depth camera, the command analysis is completed by the AI ​​computing module, and then the real-time update of the projection interface is completed by the holographic projection unit.

[0010] Furthermore, in step 2 of the interactive decoder system based on holographic projection and modular design, the expandable module is inserted through a magnetic anti-foolproof connector according to user needs, the expandable module is detected, an interrupt is triggered through the driver layer, and the interrupt is used to trigger Soc Rockchip RK3588, and memory allocation is requested from the resource manager through Soc Rockchip RK3588. A minimum of 256MB of memory is allocated to the AI ​​computing module in the expandable module through the resource manager, and a system ready signal is returned.

[0011] Furthermore, in step 2 of the interactive decoder system based on holographic projection and modular design, the ToF depth camera is used to extract multiple key points of the operator's hand based on an algorithm model that integrates deep learning and computer vision. The AI ​​module predicts the operation intention and maps the key point coordinates to the holographic interface coordinate system. The holographic projection unit completes the command mapping, and finally cooperates with the ultrasonic tactile feedback unit to generate a sense of touch and pressure in the air, completing gesture capture, command parsing, and real-time updating of the projection interface.

[0012] Furthermore, in step 2 of the interactive decoder system based on holographic projection and modular design, the algorithm model based on the fusion of deep learning and computer vision includes a convolutional neural network model CNN, a Transformer-based model and a graph convolutional network model GCN.

[0013] The MediaPipe Hand model is used to predict 21 hand key points from a single-frame depth map or RGB-D data using a convolutional neural network (CNN) model. The self-attention mechanism is used in a Transformer-based model to model long-range dependencies between hand joints for key point detection in complex poses. The hand joints are modeled as a graph structure using a graph convolutional network (GCN) model. The spatial constraints between joints are learned through GCN, such as ST-GCN, to improve the geometric rationality of pose estimation.

[0014] The present invention also provides an interactive decoder system based on holographic projection and modular design, including a main control chip, a holographic projection unit, an expandable module and an interactive component.

[0015] The main control chip uses the quad-core A76 high-performance SoC Rockchip RK3588 to schedule the operation of the entire system and allocate resources reasonably; the holographic projection unit uses a 532nm / 50W green laser for projection, presenting a suspended interactive interface; the expandable module uses a 16-pin contact magnetic anti-fool connector to expand the AI ​​computing module that supports PCle3.0, the holographic enhancement module that supports HDMI2.1, and the storage expansion module that supports USB4; the interactive components include a ToF depth camera, a microphone array, and an ultrasonic tactile feedback unit.

[0016] The interactive decoder system plugs into the expandable module through a magnetic anti-foolproof connector, uses an interrupt to trigger the SoC Rockchip RK3588, allocates memory to the expandable module, and returns a system ready signal.

[0017] The operator's gestures are recognized through the ToF depth camera, the command analysis is completed by the AI ​​computing module, and then the real-time update of the projection interface is completed by the holographic projection unit.

[0018] Furthermore, in the interactive decoder system based on holographic projection and modular design, an expandable module is inserted through a magnetic anti-foolproof connector according to user needs, the expandable module is detected, an interrupt is triggered through the driver layer, and the interrupt is used to trigger the Soc Rockchip RK3588, and memory allocation is requested from the resource manager through the Soc Rockchip RK3588. A minimum of 256MB of memory is allocated to the AI ​​computing module in the expandable module through the resource manager, and a system ready signal is returned.

[0019] Furthermore, in the interactive decoder system based on holographic projection and modular design, the ToF depth camera extracts multiple key points of the operator's hand based on an algorithm model that integrates deep learning and computer vision, and the AI ​​module predicts the operation intention and maps the coordinates of the key points to the holographic interface coordinate system, which is then handed over to the holographic projection unit to complete the command mapping, and finally cooperates with the ultrasonic tactile feedback unit to generate a sense of touch and pressure in the air, completing gesture capture, command parsing, and real-time updating of the projection interface.

[0020] Furthermore, the algorithm model based on the fusion of deep learning and computer vision in the interactive decoder system based on holographic projection and modular design includes a convolutional neural network model CNN, a Transformer-based model and a graph convolutional network model GCN.

[0021] The MediaPipe Hand model is used to predict 21 hand key points from a single-frame depth map or RGB-D data using a convolutional neural network (CNN) model. The self-attention mechanism is used in a Transformer-based model to model long-range dependencies between hand joints for key point detection in complex poses. The hand joints are modeled as a graph structure using a graph convolutional network (GCN) model. The spatial constraints between joints are learned through GCN, such as ST-GCN, to improve the geometric rationality of pose estimation.

[0022] The benefits of the present invention are:

[0023] The present invention uses the main control chip and the holographic projection unit to perform scheduling, aerial imaging, and touch feedback within the entire system; the AI ​​computing module, storage expansion module, communication module, etc. can be expanded through the magnetic anti-fool connector and dynamic resource allocation algorithm of the expandable module; through the interactive component, composed of a depth camera and a microphone array, the user's gestures and voice information are captured and interacted with, realizing a system that supports holographic projection interaction and flexible hardware expansion for the current decoder, solving the problems of current device function rigidity and limited interaction methods, and is suitable for home entertainment, distance education, virtual meetings, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a hardware architecture diagram of the system of the present invention.

[0025] Figure 2 This is the holographic projection interaction flow chart of the present invention.

[0026] Figure 3 This is a flow chart for accessing the expansion module of the present invention. DETAILED DESCRIPTION

[0027] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention.

[0028] Example 1

[0029] The present invention provides a method for implementing an interactive decoder system based on holographic projection and modular design, comprising:

[0030] Step 1: Establish an interactive decoder system, which includes a main control chip, a holographic projection unit, expandable modules, and interactive components.

[0031] The main control chip uses the quad-core A76 high-performance SoC Rockchip RK3588 to schedule the operation of the entire system and allocate resources reasonably; the holographic projection unit uses a 532nm / 50W green laser for projection, presenting a suspended interactive interface; the expandable module uses a 16-pin contact magnetic anti-fool connector to expand the AI ​​computing module that supports PCle3.0, the holographic enhancement module that supports HDMI2.1, and the storage expansion module that supports USB4; the interactive components include a ToF depth camera, a microphone array, and an ultrasonic tactile feedback unit.

[0032] Step 2: Insert the expandable module through the magnetic anti-foolproof connector, use the interrupt to trigger the SoC Rockchip RK3588, allocate memory to the expandable module, and return the system ready signal. According to user needs, insert the expandable module through the magnetic anti-foolproof connector, detect the expandable module, trigger the interrupt through the driver layer, use the interrupt to trigger the SoC Rockchip RK3588, request memory allocation from the resource manager through the SoC Rockchip RK3588, and allocate a minimum of 256MB of memory to the AI ​​computing module in the expandable module through the resource manager, and return the system ready signal.

[0033] The operator's gestures are recognized by the ToF depth camera, the AI ​​computing module completes the command analysis, and the holographic projection unit completes the real-time update of the projection interface.

[0034] The ToF depth camera uses an algorithm model based on the fusion of deep learning and computer vision to extract multiple key points of the operator's hand. The AI ​​module predicts the operation intention and maps the coordinates of the key points to the holographic interface coordinate system. The holographic projection unit completes the command mapping, and finally cooperates with the ultrasonic tactile feedback unit to generate mid-air touch and pressure, completing gesture capture, command parsing, and real-time updating of the projection interface. The algorithm models based on the fusion of deep learning and computer vision include convolutional neural network (CNN) models, Transformer-based models, and graph convolutional network (GCN) models. The convolutional neural network (CNN) model uses the MediaPipe Hand model to predict 21 hand key points from a single-frame depth map or RGB-D data. The Transformer-based model uses the self-attention mechanism to model long-range dependencies between hand joints for key point detection in complex postures. The graph convolutional network (GCN) model models the hand joints as a graph structure. The GCN learns the spatial constraints between joints, such as ST-GCN, to improve the geometric rationality of posture estimation.

[0035] Example 2

[0036] The present invention also provides an interactive decoder system based on holographic projection and modular design, including a main control chip, a holographic projection unit, an expandable module and an interactive component.

[0037] The main control chip uses the quad-core A76 high-performance SoC Rockchip RK3588 to schedule the operation of the entire system and allocate resources reasonably; the holographic projection unit uses a 532nm / 50W green laser for projection, presenting a suspended interactive interface; the expandable module uses a 16-pin contact magnetic anti-fool connector to expand the AI ​​computing module that supports PCle3.0, the holographic enhancement module that supports HDMI2.1, and the storage expansion module that supports USB4; the interactive components include a ToF depth camera, a microphone array, and an ultrasonic tactile feedback unit.

[0038] The interactive decoder system plugs into the expandable module through a magnetic anti-foolproof connector, uses an interrupt to trigger the SoC Rockchip RK3588, allocates memory to the expandable module, and returns a system ready signal.

[0039] The operator's gestures are recognized through the ToF depth camera, the command analysis is completed by the AI ​​computing module, and then the real-time update of the projection interface is completed by the holographic projection unit.

[0040] The information interaction, execution process and other contents between the modules in the above system are based on the same concept as the embodiment of the method of the present invention. For specific contents, please refer to the description in the embodiment of the method of the present invention and will not be repeated here.

[0041] Likewise, the benefits of the system of the present invention are:

[0042] The entire system is dispatched and aerial imaging and touch feedback are carried out through the main control chip and holographic projection unit; the AI ​​computing module, storage expansion module, communication module, etc. are expandable through the magnetic anti-fool connector of the expandable module and the dynamic resource allocation algorithm; through the interactive component composed of a depth camera and a microphone array, the user's gestures and voice information are captured and interacted, and a system that supports holographic projection interaction and flexible hardware expansion for the current decoder is realized, which solves the problem of fixed functions and limited interaction methods of current equipment, and is suitable for home entertainment, distance education, virtual meetings, etc.

[0043] It should be noted that not all steps and modules in the above-mentioned processes and system structures are required, and certain steps or modules can be omitted according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or may be implemented by certain components in multiple independent devices.

[0044] The above embodiments are merely preferred embodiments for the purpose of fully illustrating the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are within the scope of protection of the present invention. The scope of protection of the present invention shall be subject to the claims.

Claims

1. A method for implementing an interactive decoder system based on holographic projection and modular design, characterized by: include: Step 1: Establish an interactive decoder system, which includes a main control chip, a holographic projection unit, expandable modules, and interactive components. The main control chip uses the quad-core A76 high-performance SoC Rockchip RK3588 to schedule the operation of the entire system and allocate resources reasonably; the holographic projection unit uses a 532nm / 50W green laser for projection, presenting a suspended interactive interface; the expandable module uses a 16-pin contact magnetic anti-fool connector to expand the AI ​​computing module that supports PCle3.0, the holographic enhancement module that supports HDMI2.1, and the storage expansion module that supports USB4; the interactive components include a ToF depth camera, a microphone array, and an ultrasonic tactile feedback unit. Step 2: Insert the expandable module through the magnetic anti-foolproof connector, use the interrupt to trigger the SoC Rockchip RK3588, allocate memory to the expandable module, and return the system ready signal. The operator's gestures are recognized through the ToF depth camera, the command analysis is completed by the AI ​​computing module, and then the real-time update of the projection interface is completed by the holographic projection unit.

2. The method for implementing an interactive decoder system based on holographic projection and modular design according to claim 1, characterized in that In step 2, according to user needs, the expandable module is inserted through the magnetic anti-foolproof connector, the expandable module is detected, an interrupt is triggered through the driver layer, and the interrupt is used to trigger the SoC Rockchip RK3588. The SoC Rockchip RK3588 requests memory allocation from the resource manager, and the resource manager allocates a minimum of 256MB of memory to the AI ​​computing module in the expandable module, and returns a system ready signal.

3. The method for implementing an interactive decoder system based on holographic projection and modular design according to claim 1, characterized in that In step 2, the ToF depth camera is used to extract multiple key points of the operator's hand based on an algorithm model that integrates deep learning and computer vision. The AI ​​module predicts the operation intention and maps the coordinates of the key points to the holographic interface coordinate system. The holographic projection unit completes the command mapping, and finally cooperates with the ultrasonic tactile feedback unit to generate a sense of touch and pressure in the air, completing gesture capture, command analysis, and real-time update of the projection interface.

4. The method for implementing an interactive decoder system based on holographic projection and modular design according to claim 1, characterized in that The algorithm models based on the fusion of deep learning and computer vision in step 2 include the convolutional neural network model CNN, the Transformer-based model and the graph convolutional network model GCN. The convolutional neural network (CNN)-based model uses the MediaPipe Hand model to predict 21 hand key points from a single-frame depth map or RGB-D data. The Transformer-based model uses the self-attention mechanism to model the long-distance dependencies between hand joints for key point detection in complex postures. The graph convolutional network (GCN) model is used to model hand joints as a graph structure. The spatial constraints between joints are learned through GCN to improve the geometric rationality of posture estimation.

5. An interactive decoder system based on holographic projection and modular design, characterized by interactive The decoder system includes a main control chip, a holographic projection unit, expandable modules and interactive components. The main control chip uses the quad-core A76 high-performance SoC Rockchip RK3588 to schedule the operation of the entire system and allocate resources reasonably; the holographic projection unit uses a 532nm / 50W green laser for projection, presenting a suspended interactive interface; the expandable module uses a 16-pin contact magnetic anti-fool connector to expand the AI ​​computing module that supports PCle3.0, the holographic enhancement module that supports HDMI2.1, and the storage expansion module that supports USB4; the interactive components include a ToF depth camera, a microphone array, and an ultrasonic tactile feedback unit. The interactive decoder system plugs into the expandable module through a magnetic anti-foolproof connector, uses an interrupt to trigger the SoC Rockchip RK3588, allocates memory to the expandable module, and returns a system ready signal. The operator's gestures are recognized through the ToF depth camera, the command analysis is completed by the AI ​​computing module, and then the real-time update of the projection interface is completed by the holographic projection unit.

6. The interactive decoder system based on holographic projection and modular design according to claim 5, characterized in that According to user needs, the expandable module is inserted through the magnetic anti-foolproof connector, the expandable module is detected, an interrupt is triggered through the driver layer, and the interrupt is used to trigger the SoC Rockchip RK3588. Memory allocation is requested from the resource manager through the SoC Rockchip RK3588. The resource manager allocates a minimum of 256MB of memory to the AI ​​computing module in the expandable module and returns a system ready signal.

7. The interactive decoder system based on holographic projection and modular design according to claim 5 is characterized by: The ToF depth camera extracts multiple key points of the operator's hand based on an algorithm model that integrates deep learning and computer vision. The AI ​​module predicts the operation intention and maps the coordinates of the key points to the holographic interface coordinate system. The holographic projection unit completes the command mapping, and finally cooperates with the ultrasonic tactile feedback unit to generate a sense of touch and pressure in the air, completing gesture capture, command analysis, and real-time update of the projection interface.

8. The interactive decoder system based on holographic projection and modular design according to claim 5 is characterized by: Algorithm models based on the fusion of deep learning and computer vision include convolutional neural network models (CNN), Transformer-based models, and graph convolutional network models (GCN). The convolutional neural network (CNN)-based model uses the MediaPipe Hand model to predict 21 hand key points from a single-frame depth map or RGB-D data. The Transformer-based model uses the self-attention mechanism to model the long-distance dependencies between hand joints for key point detection in complex postures. The graph convolutional network (GCN) model is used to model hand joints as a graph structure. The spatial constraints between joints are learned through GCN to improve the geometric rationality of posture estimation.

Images