Holographic intelligent system based on digital human model technology and equipment thereof
Through a holographic intelligent system based on digital human-big model technology, combined with intelligent interaction modules and neural radiation field algorithms, the problem of single interaction modes of existing AI interactive products and poor adaptability to speech recognition is solved, and an efficient and natural and diverse user interaction experience is achieved.
Patent Information
- Application Number
- CN202411696954.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-25
- Publication Date
- 2025-03-14
AI Technical Summary
The interaction method of existing AI interactive products is single, resulting in low information acquisition efficiency, users may miss conversation content, and voice recognition is difficult to adapt to local accents or fuzzy instructions.
The holographic intelligent system based on digital human mockup technology is adopted, and diversified input processing and AI model processing is realized through intelligent interactive modules. Combined with neural radiation field algorithm and multimodal matching technology, digital humans are driven in real time, and holographic rendering engines are used to optimize holographic images.
It realizes a diversified and highly realistic user interaction experience, enhances the natural fluency of interaction, solves the problems of low information acquisition efficiency and poor adaptability of speech recognition, and provides richer functional support.
Smart Images

Figure CN119620863A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent interaction technology, and in particular to a holographic intelligent system and equipment based on digital human model technology. Background Art
[0002] At present, the interactive AI products on the market are presented in a relatively simple form. For example, AI Bluetooth speakers mainly communicate and transmit information through voice. Users need to spend more time listening to and understanding voice information, and the efficiency of obtaining information is low. And due to the single interaction method, users may miss the previous conversation content and instructions after multiple interactions, which affects the coherence and logic of the conversation and has a poor experience. In the interaction, the speech recognition model of the AI speaker cannot accurately adapt to users with local accents or dialects, resulting in misunderstanding of the instructions of users; and the AI speaker cannot understand the vague and ambiguous functional instructions well. In terms of function, it is mainly in voice-related fields, such as playing songs, querying information, controlling smart appliances, etc., lacking more functional support. These are mainly due to its single interaction method and its underlying model. For this reason, a holographic intelligent system and its equipment based on digital human model technology are proposed. Summary of the invention
[0003] In view of the deficiencies of the prior art, the present invention provides a holographic intelligent system and equipment based on digital human model technology to solve the background technical problems.
[0004] In the first aspect, in order to achieve the above-mentioned purpose, the present invention provides the following technical solutions: a holographic intelligent system based on digital human model technology, the system comprising:
[0005] Intelligent interaction module
[0006] This module connects to the AI large model library to realize the intelligent interaction between digital humans and users. The specific functions include:
[0007] Input processing: The input processing unit receives and processes the user's diverse input, including voice and gestures, and converts the input information into a format that the digital human can understand;
[0008] AI model processing: The AI model processing unit uses the connected AI big model to perform semantic understanding and analysis and generate corresponding replies or instructions;
[0009] Output feedback: The output feedback unit converts the output results of the AI model processing unit into interactive actions and voice responses that can be executed by the digital human, realizing real-time interaction with the user.
[0010] 2.5D digital human real-time driving module based on neural radiation field
[0011] This module uses a combination of neural radiation field algorithm and multi-modal matching technology to achieve real-time driving of digital humans. Specific functions include:
[0012] Audio processing and lip movement prediction: Based on output feedback, the audio processing unit analyzes the pronunciation of the digital human in real time and extracts the phoneme ID, which is input into the NeRF-based lip movement prediction model; this model is trained through deep learning technology and can learn the mapping relationship between phonemes and lip shapes, and predict the corresponding lip shape parameters.
[0013] Action library and action synthesis: Create an action library that contains various predefined actions; based on the context of the output feedback, the digital human selects appropriate actions from the action library for synthesis and presentation.
[0014] Animation generation and rendering: The lip shape parameters and motion parameters are passed to the animation generation unit, which generates real-time lip shape animation and motion animation of the digital human based on these parameters; in addition, this module also uses the NeRF algorithm to render and optimize the generated animation to improve the realism and smoothness of the animation. The mathematical representation of the NeRF model is as follows:
[0015] F(x,θ,φ)→(c,σ)
[0016] Among them, x = (x, y, z) represents the three-dimensional coordinates in the scene, (θ, φ) represents the viewing direction, c = (r, g, b) represents the color, and σ represents the volume density.
[0017] Holographic rendering engine
[0018] The engine uses the neural radiation field algorithm combined with deep learning technology to render and optimize the holographic image of the digital human; specific functions include:
[0019] Image rendering: By training the NeRF model, the 3D shape, texture, and lighting information of the digital human are learned to generate high-quality holographic images and output them for display. The volume rendering formula is as follows:
[0020]
[0021] where C(r) is the color of ray r predicted by the model, T(t) is the cumulative transmittance, σ(r(t)) and c(r(t), d) represent the volume density and color on ray r(t), respectively, and d represents the viewing direction.
[0022] Data transmission and interaction: Receive interaction instructions from the intelligent interaction module and animation data from the 2.5D digital human real-time driving module, and bring them into image rendering to render and optimize holographic images.
[0023] Preferably, the input processing includes speech recognition, specifically:
[0024] Speech recognition: The built-in speech recognition engine can capture and recognize the user's voice in real time, and use deep learning algorithms to extract features and match patterns on voice signals to accurately identify the user's intentions and speech content.
[0025] AI model processing includes semantic understanding, specifically:
[0026] Semantic understanding: Use natural language processing technology to perform semantic analysis on the text input by the user, build a semantic model to understand the keywords, phrases and sentence structures in the user's speech, and thus grasp the user's true intention.
[0027] Output feedback includes dialogue management, specifically:
[0028] Dialogue management: After understanding the user's intention, the digital human system controls and schedules the dialogue process through the dialogue management module. This module generates corresponding replies and interactive instructions based on preset dialogue rules and scenario logic to ensure the continuity and fluency of the dialogue.
[0029] Preferably, the input processing also includes motion capture and recognition:
[0030] Motion capture and recognition: The user's motion information is captured in real time through motion capture devices; these devices usually include cameras and sensors, which can accurately record the user's body movements and facial expressions, and use deep learning algorithms to extract features and match patterns of motion signals to accurately identify the user's body expression purposes.
[0031] Preferably, the intelligent interaction module further includes emotion recognition and personalized interaction, specifically:
[0032] Emotion recognition: The digital human system identifies the user's emotional state through deep learning algorithms and based on motion capture and recognition. By analyzing the user's voice intonation, facial expressions and movement information, the system can judge the user's emotional tendencies and make corresponding emotional responses through output feedback.
[0033] Personalized interaction: Based on the recognition of user emotions and intentions, the digital human system uses deep learning algorithms to generate personalized interactive content; these contents include but are not limited to language responses, action performance, and expression changes, providing an interactive experience that is closer to user needs and preferences.
[0034] Preferably, the underlying layer of the system is developed using the C++ programming language, and the driving time is optimized through lower-level calls of the CPU without relying on GPU computing power.
[0035] Preferably, the underlying C++ programming language of the system is specifically developed as follows:
[0036] CPU multi-threaded parallel processing: Utilize the multi-core and multi-threaded capabilities of the CPU to split the generation and rendering of holographic images into multiple subtasks and process them in parallel to improve efficiency;
[0037] Instruction-level optimization: Through fine control of CPU instructions, the SIMD (single instruction multiple data) instruction set is used to achieve batch processing of data and reduce instruction execution time;
[0038] Memory access optimization: By optimizing memory layout and access patterns, memory latency and bandwidth bottlenecks are reduced and data access efficiency is improved.
[0039] Preferably, the system integrates the "holographic life companion" mode, aiming to provide users with a full range of multi-scenario companionship, teaching and interactive experience. Specific functions include but are not limited to:
[0040] Personalized holographic image customization: The system can generate a highly personalized holographic digital human image based on the user's preferences and needs.
[0041] Companionship and care: Through emotion recognition, the system can perceive the emotional state of the elderly and communicate and interact with them emotionally, such as listening, comforting, sharing, etc., to enhance the elderly's psychological well-being.
[0042] Education and interaction: According to children's preferences and learning needs, the system matches the corresponding digital human, and integrates English learning, math enlightenment, and historical story education content into the interaction.
[0043] Enterprise display: display enterprise products through holographic digital people, and provide interactive question-and-answer functions to help customers better understand product features and usage methods, and enhance customer experience.
[0044] Preferably, motion capture and recognition also has health monitoring and reminder functions. The system can monitor the daily activities and sleep quality health data of the elderly, and provide health advice or medical reminders based on preset health standards.
[0045] In the second aspect, a holographic intelligent device based on digital human model technology includes a storage device, a processor, a computer program stored in the storage device and running on the processor, and an intelligent device carrier. The middle part of the intelligent device carrier is transparent, and a black background board is installed inside it. A laser interferometer for imaging through the black background board is integrated in the intelligent device carrier. A camera module, a sensor module and a receiving module are installed at the upper end of the intelligent device carrier, and an external speaker module is integrated at the lower end of the intelligent device carrier to realize the holographic intelligent system based on digital human model technology as described in any one of the first aspects.
[0046] Compared with the prior art, the present invention has the following beneficial effects:
[0047] The present invention adopts an intelligent interaction module to realize dynamic interaction with users, and renders a data person through a 2.5D digital human real-time driving module based on the neural radiation field. The data person's lip shape and movements are coordinated with the speech form. The holographic rendering engine is also used to render the whole world. Combined with holographic intelligent devices, dynamic interaction is realized. Combined with AI technology, the system realizes a diversified and highly realistic user interaction experience and enhances the natural fluency of interaction. Moreover, the system is based on the technical framework of the digital human large model, which provides the possibility for the system to expand more functions, meets the user's demand for diversified interactive experience, and significantly improves the overall user experience.
[0048] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1 It is a block diagram of the holographic intelligent system based on the digital human model technology of the present invention;
[0050] Figure 2 It is a cross-sectional structural diagram of a holographic intelligent device and an intelligent device carrier based on the digital human model technology of the present invention;
[0051] Figure 3 To invent a holographic intelligent device operation flow chart based on digital human model technology.
[0052] In the figure: 1. Smart device carrier; 2. Black background board; 3. Camera module; 4. Sensor module; 5. Radio module; 6. External speaker module; 7. Laser interferometer. DETAILED DESCRIPTION
[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this technical field without creative work are within the scope of protection of the present invention.
[0054] Embodiment 1:
[0055] See also Figure 2 and Figure 3This embodiment provides a holographic intelligent device based on the digital human model technology, including a storage device, a processor, a computer program stored in the storage device and running on the processor, and an intelligent device carrier 1. The intelligent device carrier 1 is designed with a transparent material, and the middle area is completely transparent to display the holographic image. A black background board 2 is installed inside to enhance the contrast and clarity of the holographic image. A laser interferometer 7 is also integrated in the intelligent device carrier 1. The device presents a delicate and realistic holographic digital human image on the black background board 2 by accurately controlling the interference of the laser.
[0056] A camera module 3 is installed on the upper end of the smart device carrier 1 to capture the user's facial expressions, body movements and surrounding environment information to provide data support for subsequent smart interactions;
[0057] A sensor module 4 is installed on the upper end of the smart device carrier 1. The sensor module 4 includes but is not limited to an infrared sensor, an acceleration sensor, etc., which is used to monitor the user's movements, position, and state changes of the device itself;
[0058] The upper end of the smart device carrier 1 is equipped with a sound receiving module 5 for receiving the user's voice commands or conversation content, and performing real-time processing and analysis through a built-in voice recognition engine;
[0059] The lower end of the smart device carrier 1 is integrated with an external speaker module 6 for playing the digital person's voice reply, background music or prompt sound to enhance the user's auditory experience;
[0060] The overall operation is achieved by the processor calling the computer program in the memory;
[0061] Principle of holography: The holographic object principle of the black background plate 2 is mainly based on the principles of interference and diffraction. Holographic technology uses the principle of interference to record the light wave information of the object. Interference fringes are formed on the holographic film through the interference of two laser beams, namely the laser interferometer 7. These fringes record all the information of the object light wave. Specifically, a laser beam is divided into two beams by a beam splitter: one beam is irradiated on the object to be photographed to form object light, and the other beam is directly irradiated on the holographic film as reference light. The object light and the reference light are superimposed and reduced to form interference fringes and recorded on the film.
[0062] In the process of reproducing the hologram, the diffraction principle is used to reproduce the light wave information of the object. The hologram is like a complex grating. Under the irradiation of coherent laser, each part of the hologram records the light information of each point on the object, so each part can reproduce the entire image of the original object. Through the diffraction effect, the hologram can "summon" the object light of all points on the object, thereby "reorganizing" the object and restoring a three-dimensional image.
[0063] Embodiment 2:
[0064] See also Figure 1-3 Based on the first embodiment, this embodiment further proposes a holographic intelligent system based on the digital human large model technology. The system not only has holographic display and intelligent interaction functions, but also integrates the neural radiation field (NeRF) technology to achieve real-time driving and high-quality rendering of the digital human. The system includes:
[0065] Intelligent interaction module
[0066] This module connects to the AI large model library to realize the intelligent interaction between digital humans and users. The specific functions include:
[0067] Input processing: Receive and process various user inputs, including voice, gestures, and facial expressions. Through the built-in voice recognition engine and deep learning algorithm, the user's intention and speech content can be accurately identified; at the same time, it has motion capture and recognition functions, using motion capture equipment to capture the user's motion information in real time to achieve accurate body expression recognition. In addition, the input processing unit also has a health monitoring function based on motion capture, which analyzes the user's daily activities and sleep quality through motion capture data and provides personalized health suggestions.
[0068] AI model processing: Utilize the docked AI big model for semantic understanding and sentiment analysis; use natural language processing technology to perform semantic analysis on the text input by the user, and build a semantic model to grasp the user's true intention; at the same time, based on motion capture and recognition results, by analyzing the user's voice intonation, facial expressions and motion information, the system can judge the user's emotional tendencies and make corresponding emotional responses through output feedback.
[0069] Output feedback: The output results of the AI model processing unit are converted into interactive actions and voice responses that can be executed by the digital human. It also has dialogue management to control and schedule the ongoing dialogue process to ensure the continuity and fluency of the dialogue. In addition, the output feedback unit also has personalized interaction functions. Based on the recognition of user emotions and intentions, the digital human system uses deep learning algorithms to generate personalized interactive content. These contents include but are not limited to language responses, action performance, and expression changes, providing an interactive experience that is closer to user needs and preferences.
[0070] 2.5D digital human real-time driving module based on neural radiation field
[0071] This module uses a combination of neural radiation field algorithm and multi-modal matching technology to achieve real-time driving of digital humans. Specific functions include:
[0072] Audio processing and lip movement prediction: Based on the number of output feedback, the pronunciation of the digital human is analyzed in real time and the phoneme ID is extracted, which is input into the NeRF-based lip movement prediction model. This model is trained through deep learning technology and can learn the mapping relationship between phonemes and lip shapes, and predict the corresponding lip shape parameters. These parameters are used to generate real-time lip animation of the digital human.
[0073] The mathematical representation of the NeRF model is as follows:
[0074] F(x,θ,φ)→(c,σ)
[0075] Among them, x = (x, y, z) represents the three-dimensional coordinates in the scene, that is, the specific position of the digital human's lips; (θ, φ) represents the observation direction, that is, the user's perspective of observing the digital human; c = (r, g, b) represents the color, that is, the color information of the lip animation; σ represents the volume density. Although it is not directly used in lip movement prediction here, it reflects the NeRF model's ability to accurately describe three-dimensional scenes and object shapes.
[0076] Action library and action synthesis: Create an action library containing various predefined actions. According to the context and emotional state of the output feedback, the digital human selects appropriate actions from the action library for synthesis and presentation.
[0077] Animation generation and rendering: The lip shape parameters and action parameters are passed to the animation generation unit to generate real-time animation of the digital human. At the same time, the NeRF algorithm is used to render and optimize the generated animation.
[0078] When using volume rendering technology to generate images from neural radiation fields, for a camera ray r(t)=o+td (where o is the camera origin and d is the viewing direction), its color C(r) is calculated by the following formula:
[0079] Among them, T(t) is the cumulative transmittance, which represents the probability that light propagates from t1 to t without being intercepted; σ(r(t)) and c(r(t), d) represent the volume density and color on ray r(t), respectively; d represents the viewing direction. This formula ensures the high quality and realism of digital human animation.
[0080] The cumulative transmittance T(t) is calculated by the following formula:
[0081] T(t)=e-∫tt1σ(r(u))·du
[0082] It represents the probability that light is absorbed by matter (such as the skin and clothing of a digital person) during propagation.
[0083] The expected depth d(r) is approximated by the following formula:
[0084] D^(r)=∑i=1NαitiTi
[0085] Among them, αi represents the transparency / opacity at sampling point i; ti represents the distance from the camera to sampling point i; Ti represents the cumulative transmittance before sampling point i. This formula helps to determine the depth position of the digital human in the scene, thereby enhancing the stereoscopic effect.
[0086] Holographic rendering engine
[0087] Image rendering: A method combining neural radiation field algorithm and deep learning technology is used to render and optimize the holographic image of the digital human. The high quality and realism of the holographic image are ensured by real-time calculation and application of the volume rendering formula, cumulative transmittance formula and expected depth formula in the above-mentioned 2.5D digital human real-time driving module based on neural radiation field.
[0088] Data transmission and interaction: Receive interaction instructions from the intelligent interaction module and animation data from the 2.5D digital human real-time driving module, and render and optimize the holographic image accordingly, and interact with the user by updating the holographic image in real time; this rendering mainly renders the entire holographic scene and renders the conversation content next to the digital human.
[0089] System bottom-level development and optimization
[0090] The underlying system is developed in C++ programming language, which optimizes the driving time through lower-level CPU calls without relying on GPU computing power.
[0091] The underlying C++ programming language of this system is specifically developed as follows:
[0092] CPU multi-threaded parallel processing: Utilize the multi-core and multi-threaded capabilities of the CPU to split the generation and rendering of holographic images into multiple subtasks and process them in parallel to improve efficiency;
[0093] Instruction-level optimization: Through fine control of CPU instructions, the SIMD (single instruction multiple data) instruction set is used to achieve batch processing of data and reduce instruction execution time;
[0094] Memory access optimization: By optimizing memory layout and access patterns, memory latency and bandwidth bottlenecks are reduced and data access efficiency is improved.
[0095] The specific rendering process code is:
[0096] import numpy as np
[0097] import torch
[0098] import syssys.path.append('. / nerf-pytorch / ')
[0099] from run_nerf import config_parser,create_nerf
[0100] #Simulate command line parameters and load configuration
[0101] args = config_parser()
[0102] args.config='example_configs / figures.txt'
[0103] #Create NeRF model
[0104] render_kwargs_test=create_nerf(args)
[0105] #Simulate camera parameters
[0106] ray_bundle=render_kwargs_test['ray_bundle']
[0107] #Simulate NeRF model parameters
[0108] results=render_kwargs_test['network_fn'](
[0109] ray_bundle,
[0110] render_kwargs_test['network_query_fn'].squeeze(0),
[0111] **render_kwargs_test['network_input_keys'] )
[0113] #Simulation rendering output
[0114] img=results['rgb_map'].cpu().numpy()
[0115] #Save or process rendering results
[0116] #...
[0117] Embodiment three:
[0118] Based on the first and second embodiments, this embodiment proposes a holographic intelligent system based on digital human model technology, which integrates the "holographic life partner" mode and aims to provide users with all-round, multi-scenario companionship, teaching and interactive experience. Specific functions include but are not limited to:
[0119] Personalized holographic image customization: The system can generate a highly personalized holographic digital human image based on the user's preferences and needs, including appearance, clothing, personality and other aspects.
[0120] Companionship and care: Through emotion recognition, the system can perceive the emotional state of the elderly and communicate and interact with them emotionally, such as listening, comforting, sharing, etc., to enhance the elderly's psychological well-being.
[0121] Education and interaction: The system can match the corresponding digital human according to the children’s preferences and learning needs, and integrate educational content into the interaction, such as English learning, mathematics enlightenment, historical stories, etc.
[0122] Enterprise display: Use holographic digital people to display enterprise products and provide interactive question-and-answer functions to help customers better understand product features and usage methods, thereby improving customer experience. In addition, holographic technology can also be used to create virtual exhibition halls or virtual meeting spaces, providing enterprises with a new way of display and interaction;
[0123] This embodiment expands the usage scenarios of the holographic intelligent device, meets the needs of most markets, and increases the types of interactions based on dynamic and natural interactions.
[0124] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.
Claims
1. A holographic intelligent system based on digital human model technology, characterized in that: The system includes: Intelligent interaction module This module connects to the AI large model library to realize the intelligent interaction between digital humans and users. The specific functions include: Input processing: The input processing unit receives and processes the user's diverse input, including voice and gestures, and converts the input information into a format that the digital human can understand; AI model processing: The AI model processing unit uses the connected AI big model to perform semantic understanding and analysis and generate corresponding replies or instructions; Output feedback: The output feedback unit converts the output results of the AI model processing unit into interactive actions and voice responses that can be executed by the digital human, realizing real-time interaction with the user. 2.5D digital human real-time driving module based on neural radiation field This module uses a combination of neural radiation field algorithm and multi-modal matching technology to achieve real-time driving of digital humans. Specific functions include: Audio processing and lip movement prediction: Based on output feedback, the audio processing unit analyzes the pronunciation of the digital human in real time and extracts the phoneme ID, which is input into the NeRF-based lip movement prediction model; this model is trained through deep learning technology and can learn the mapping relationship between phonemes and lip shapes, and predict the corresponding lip shape parameters. Action library and action synthesis: Create an action library that contains various predefined actions; based on the context of the output feedback, the digital human selects appropriate actions from the action library for synthesis and presentation. Animation generation and rendering: The lip shape parameters and motion parameters are passed to the animation generation unit, which generates real-time lip shape animation and motion animation of the digital human based on these parameters; in addition, this module also uses the NeRF algorithm to render and optimize the generated animation to improve the realism and smoothness of the animation. The mathematical representation of the NeRF model is as follows: F(x,θ,φ)→(c,σ) Among them, x = (x, y, z) represents the three-dimensional coordinates in the scene, (θ, φ) represents the viewing direction, c = (r, g, b) represents the color, and σ represents the volume density. Holographic rendering engine The engine uses the neural radiation field algorithm combined with deep learning technology to render and optimize the holographic image of the digital human; specific functions include: Image rendering: By training the NeRF model, the 3D shape, texture, and lighting information of the digital human are learned to generate high-quality holographic images and output them for display. The volume rendering formula is as follows: where C(r) is the color of ray r predicted by the model, T(t) is the cumulative transmittance, σ(r(t)) and c(r(t), d) represent the volume density and color on ray r(t), respectively, and d represents the viewing direction. Data transmission and interaction: Receive interaction instructions from the intelligent interaction module and animation data from the 2.5D digital human real-time driving module, and bring them into image rendering to render and optimize holographic images.
2. According to claim 1, a holographic intelligent system based on digital human model technology is characterized in that: Input processing includes speech recognition, specifically: Speech recognition: The built-in speech recognition engine can capture and recognize the user's voice in real time, and use deep learning algorithms to extract features and match patterns on voice signals to accurately identify the user's intentions and speech content. AI model processing includes semantic understanding, specifically: Semantic understanding: Use natural language processing technology to perform semantic analysis on the text input by the user, build a semantic model to understand the keywords, phrases and sentence structures in the user's speech, and thus grasp the user's true intention. Output feedback includes dialogue management, specifically: Dialogue management: After understanding the user's intention, the digital human system controls and schedules the dialogue process through the dialogue management module. This module generates corresponding replies and interactive instructions based on preset dialogue rules and scenario logic to ensure the continuity and fluency of the dialogue.
3. The holographic intelligent system based on digital human model technology according to claim 2 is characterized in that: Input processing also includes motion capture and recognition: Motion capture and recognition: The user's motion information is captured in real time through motion capture devices; these devices usually include cameras and sensors, which can accurately record the user's body movements and facial expressions, and use deep learning algorithms to extract features and match patterns of motion signals to accurately identify the user's body expression purposes.
4. The holographic intelligent system based on digital human model technology according to claim 1 is characterized in that: The intelligent interaction module further includes emotion recognition and personalized interaction. Specifically: Emotion recognition: The digital human system identifies the user's emotional state through deep learning algorithms and based on motion capture and recognition. By analyzing the user's voice intonation, facial expressions and movement information, the system can judge the user's emotional tendencies and make corresponding emotional responses through output feedback. Personalized interaction: Based on the recognition of user emotions and intentions, the digital human system uses deep learning algorithms to generate personalized interactive content; these contents include but are not limited to language responses, action performance, and expression changes, providing an interactive experience that is closer to user needs and preferences.
5. The holographic intelligent system based on digital human model technology according to claim 1 is characterized in that: The underlying system is developed using the C++ programming language, which optimizes driving time through lower-level CPU calls without relying on GPU computing power.
6. The holographic intelligent system based on digital human model technology according to claim 5 is characterized in that: The underlying C++ programming language of this system is specifically developed as follows: CPU multi-threaded parallel processing: Utilize the multi-core and multi-threaded capabilities of the CPU to split the generation and rendering of holographic images into multiple subtasks and process them in parallel to improve efficiency; Instruction-level optimization: Through fine control of CPU instructions, the SIMD (single instruction multiple data) instruction set is used to achieve batch processing of data and reduce instruction execution time; Memory access optimization: By optimizing memory layout and access patterns, memory latency and bandwidth bottlenecks are reduced and data access efficiency is improved.
7. The holographic intelligent system based on digital human model technology according to claim 4 is characterized in that: The system integrates the "holographic life companion" mode, aiming to provide users with a full range of multi-scenario companionship, teaching and interactive experience. Specific functions include but are not limited to: Personalized holographic image customization: The system can generate a highly personalized holographic digital human image based on the user's preferences and needs. Companionship and care: Through emotion recognition, the system can perceive the emotional state of the elderly and communicate and interact with them emotionally, such as listening, comforting, sharing, etc., to enhance the elderly's psychological well-being. Education and interaction: According to children's preferences and learning needs, the system matches the corresponding digital human, and integrates English learning, math enlightenment, and historical story education content into the interaction. Enterprise display: display enterprise products through holographic digital people, and provide interactive question-and-answer functions to help customers better understand product features and usage methods, and enhance customer experience.
8. The holographic intelligent system based on digital human model technology according to claim 3 is characterized in that: Motion capture and recognition also has health monitoring and reminder functions. The system can monitor the daily activities, sleep quality and health data of the elderly, and provide health advice or medical reminders based on preset health standards.
9. A holographic intelligent device based on digital human model technology, characterized in that: The invention comprises a storage device, a processor, a computer program stored in the storage device and running on the processor, and an intelligent device carrier (1); the middle part of the intelligent device carrier (1) is transparent and a black background plate (2) is installed therein; a laser interferometer (7) for imaging through the black background plate (2) is integrated and installed in the intelligent device carrier (1); a camera module (3), a sensor module (4) and a sound receiving module (5) are installed at the upper end of the intelligent device carrier (1); and an external speaker module (6) is integrated and installed at the lower end of the intelligent device carrier (1), so as to realize the holographic intelligent system based on the digital human model technology as described in any one of claims 1 to 8.
Citation Information
Cited By
Mistaken touch prevention interaction method based on medium-free holography
CN120994073A