1767 results about "Gesture recognition" patented technology

The invention discloses a distributed intelligent warehouse scheduling system based on artificial intelligence, and belongs to the technical field of warehouse scheduling. Comprising a multi-source environment sensing module, a dynamic inventory management module, a distributed task scheduling module, an intelligent path planning module, a resource dynamic allocation module, an anomaly detection and emergency response module, an energy consumption optimization module, a supply chain collaboration module and a man-machine interaction and visualization module. A warehouse digital twinborn model is constructed, immersive display of a storage state and a scheduling strategy is realized, an AR scene is superposed through a color coding path, a thermodynamic diagram and a particle flow form, a manager can intuitively master inventory distribution, task progress and an abnormal region, eye movement tracking and a gesture recognition technology support an interactive decision, and the workload of the manager is reduced. The AR marking function can mark an abnormal area and synchronize the abnormal area to a decision making system, and through combination of AR and AI, a brand new interaction normal form is provided for intelligence and humanization of warehouse management.

An autonomous environmental regulation and behavioral monitoring system is disclosed, configured to adapt temperature, lighting, and acoustic conditions based on real-time emotional and physiological data. The system includes a dual-redundant central processor, hierarchical communication networks, multi-angle visual acquisition units, infrared thermometers, and modular environmental subsystems. It detects posture, gestures, facial expressions, and thermal signals to classify user states and apply individualized airflow, light, and sound modulation without relying on external internet connectivity. The system also monitors connected appliances using voltage-based pressure analysis to forecast device degradation. With integrated gesture recognition, privacy-preserving data handling, and predictive adaptation, the invention enables multi-user personalization, long-term learning, and uninterrupted operation within residential, administrative, or healthcare infrastructures.

The invention relates to the technical field of power system operation training, and discloses a virtual reality-based power grid fault simulation training system and method, and the system comprises a virtual scene modeling module which is used for constructing a three-dimensional model of power grid equipment and an operation environment thereof, and providing a vivid training environment for subsequent modules; and the interaction and operation module is connected with the virtual scene modeling module and realizes interaction between the trainee and the virtual environment by using a VR head-mounted display, a handle and a motion capture device. By adopting a high-precision three-dimensional modeling technology, PBR material rendering and an LOD optimization strategy, high simulation of power grid equipment and an operation environment is realized, students can experience a vivid visual effect and immersion in a virtual reality scene, and meanwhile, by combining a gesture recognition technology realized by motion capture equipment and a convolutional neural network, the real-time simulation of the power grid equipment and the operation environment is realized. Operation actions of trainees can be accurately captured, a natural and visual interaction mode is provided, and the authenticity and operation experience of the training system are greatly improved.

The invention relates to the technical field of computer vision and human-computer interaction, and discloses an AI digital human intelligent interaction method, system and equipment based on multiple modalities, and the method comprises the steps: pre-awakening a digital human when a human face is detected, and further thoroughly awakening the digital human based on recognized preset voice information or preset gesture information; voice and video information of a user in the interaction process is obtained, a keyword extraction result, a gesture recognition result and an emotional state tag are generated, a pre-constructed knowledge base is utilized to retrieve related information, a big language generation model module is combined to generate an answer text, and the answer text is input into a preset voice synthesis model to generate emotional voice output. And based on the current emotional state label of the user, driving the digital human animation to be output in an emotional manner. According to the method and the system, the digital human for understanding the emotion of the user, generating personalized answers, providing voices with rich emotions and displaying natural expressions and actions can be created, better interaction with the user can be realized, and more humanized and effective services can be provided.

A computer-implemented system and method for pupil-directed retinal projection in near-eye displays are disclosed. The computer-implemented system provides smart glasses with directed physical pixels that project light beams / signals directly onto a user's retina based on pupil position and size. The glasses comprise a frame, lenses with directed pixel layers, and sensors for tracking pupil movement. Each directed pixel may generate multiple virtual pixels by rapidly changing its emission angle. The glasses function as prescription lenses, virtual reality displays, and augmented reality devices without traditional optical systems. Additional features include depth sensors, cameras, and connectivity to peripheral devices. The glasses enable a seamless blend of virtual and real-world experiences, creating an immersive “Mixverse” environment. Various input methods, including gesture recognition and brain-computer interfaces, allow for intuitive control and interaction.

The invention relates to the technical field of gesture recognition, in particular to an anti-interference optimized gesture recognition method. Comprising the following steps: acquiring a gesture video stream through a camera, constructing a dynamic background model by using a frame difference method and a Gaussian mixture model, eliminating a static background and interference, and extracting a target area image; performing local brightness histogram analysis on the target region image, and optimizing the image quality by adopting a region adaptive compensation algorithm and a multi-scale edge enhancement technology; positioning a gesture area in real time by using a color histogram and a feature matching algorithm, and dynamically updating a gesture track in combination with Kalman filtering; and extracting gesture shapes, tracks and dynamic mode features through deep learning, comparing the features with a standard model library, and outputting gesture categories and corresponding function instructions. According to the method, a multi-level optimization strategy is adopted for a complex background, a dynamic target and a changeable illumination environment, so that the anti-interference capability and the recognition precision of gesture recognition are improved.

The invention discloses an intelligent door and window control method and system based on indoor environment analysis, and relates to the technical field of door and window control, in the operation process of the system, temperature, humidity, illumination conditions and noise conditions are collected in real time through sensors, preference settings and operation historical records of a user are collected through voice recognition and gesture recognition, and the user experience is improved. Technical parameters provided by door and window manufacturers are used as a first data set, a second data set and a third data set to carry out preprocessing, analysis and extraction, preference setting and habits of users are recognized, an intelligent door and window control index Znkz is obtained through calculation, and the intelligent door and window control index Znkz is obtained according to indoor environment parameters and user preference information. The intelligent door and window control index Znkz is compared with a preset standard threshold value M and a preset standard threshold value N, the current environment state and user requirements are evaluated, a door and window control strategy is made, the opening and closing states of doors and windows are controlled according to instructions of the decision logic module, and a user interface used for interaction is provided.

The invention relates to the technical field of man-machine interaction, and discloses a touch screen control method and device and electronic equipment. The method comprises the steps that S1, it is detected that a user generates a touch starting event in a screen edge area, touch track data are collected, and the track data comprise touch position coordinates, pressure values and corresponding timestamps; s2, constructing the touch track data into a track function with continuous time, and obtaining speed information, acceleration information and track steering change characteristics of a track; and S3, according to the speed information, the acceleration information and the track steering change characteristics, calculating a group of track dynamic characteristic indexes representing track naturalness. By adopting a combined judgment technology of track naturalness and structural complexity, the naturalness and the complexity of the touch track can be accurately recognized, and a high-precision gesture recognition effect is achieved. Compared with the scheme depending on the touch position and speed in the prior art, the method can effectively distinguish mistaken touch and effective touch, and the problem of frequent misoperation is solved.

The invention discloses a micro posture recognition method based on multi-modal feature fusion and fine adjustment, and relates to the field of computer vision and action recognition. The method is characterized in that a universal cross-modal knowledge fusion framework is provided, and multi-modal features are respectively extracted through a fine tuning network by using three kinds of modal information of video, skeleton and text. Meanwhile, a video-skeleton and text-skeleton fusion module is introduced to enhance the interactivity between modals. Compared learning is adopted to align feature distribution, and model training is optimized in combination with a freezing-fine tuning strategy, so that the calculation complexity is reduced, and the recognition efficiency is improved. According to the method provided by the invention, the problem of single-mode information loss can be solved, the perception capability of tiny attitude change is enhanced, the recognition precision and robustness are improved, and the method is suitable for multiple application scenes such as behavior monitoring, human-computer interaction and safety protection.

Methods and systems for human gesture recognition and activity detection using augmented data. A computer-implemented method includes receiving motion capture data of a target from a camera, generating a first set of motion trajectories from the motion capture data, generating a first set of augmented motion trajectories using a set of data augmentation functions on the first set of motion trajectories, generating a radar cross-section of the target using the motion capture data to perform at least one of gesture recognition or activity detection, generating one or more synthetic electromagnetic (EM) signatures of one or more activities of the target using the first set of augmented motion trajectories and the radar cross-section, and training a machine learning model configured for EM signature-based gesture recognition or activity detection with a domain adaptation process using the one or more synthetic EM signatures.

The invention relates to the technical field of computer graphic interactive processing, in particular to a real-time interactive image generation system based on a multi-point touch canvas. The input acquisition unit is used for acquiring original touch data from an operating system and preprocessing the original touch data; the gesture recognition and analysis unit is used for performing high-level semantic behavior analysis on the contact data sequence processed by the preprocessing module; the interaction control and parameter mapping unit is used for receiving the semantic event output by the gesture recognition and analysis unit, analyzing the semantic event into an image control command and generating an executable command sequence; and the image generation unit generates interactive image content dynamically responded in real time based on an internal graph state management mechanism. By introducing the adaptive Kalman filtering and trajectory prediction auxiliary mechanism, the filtering intensity of the contact data can be dynamically adjusted, the efficient suppression of finger jitter and the consistent reconstruction of the contact ID are realized, and the input stability and data continuity under the multi-point touch operation are remarkably improved.

The invention provides a virtual reality interaction and content generation method and system based on a large model. According to the method, original point cloud data are collected to be aligned with semantic tags, and a topological graph containing object positions, categories and spatial constraints is constructed; dynamically calibrating the point cloud and mapping the point cloud to a unified coordinate system, and extracting a shielding boundary to generate a parallax mapping table; recognizing a blank area delimited by a user through a gesture, and generating a new object parameter candidate set of physical compliance in combination with the topological graph; screening parameters by using a large model, and reserving physical effective parameters after collision response, gravity and illumination verification; and generating the geometry and texture of the virtual object in the occlusion area based on the incremental neural radiation field, optimizing the depth and the rendering precision in combination with the mapping table, outputting the depth and the rendering precision to an interactive interface, and updating topological parameters. According to the method and the device, a new object which is physically consistent and rendered and optimized is dynamically generated based on user interaction in a virtual reality scene, and the sense of reality and interaction experience are improved.

The invention belongs to the technical field of new energy automobile teaching, and particularly relates to a VR interactive new energy automobile high-voltage electrical principle teaching method, system and equipment, and the method comprises the steps: obtaining the physical parameter data and dynamic circuit simulation data of a new energy automobile high-voltage electrical system; constructing a high-precision 3D model according to the physical parameter data, inputting dynamic circuit simulation data into the model, and generating interactive principle annotations and particle effect animations; based on a gesture recognition model and a new energy automobile VR interaction technology, circuit state changes in the particle effect animation are obtained, a new energy automobile high-voltage system principle and a signal flow are displayed, a practical operation simulation training scene is constructed, student operation data are recorded according to the practical operation simulation training scene, an AI evaluation model is trained, and an evaluation report is obtained. And adjusting interaction feedback parameters of the training scene, and meanwhile, generating a personalized learning report of the trainee. Therefore, the problems of safety risk, teaching lag, abstraction obstacle and the like in the prior art are solved.

The invention discloses a robot sign language communication method, a related device and a storage medium. The method comprises the steps of collecting each frame of gesture image of a current user and performing feature extraction; inputting the features of each frame of gesture image into a trained gesture recognition model, and recognizing a current gesture recognition result; based on the mapping relation between the gesture symbols and vocabularies, mapping the current gesture recognition result to obtain a vocabulary sequence; forming the vocabulary sequence into a current complete recognition text according to a preset grammar rule; correcting the current complete recognition text by combining the multi-modal large language model with the current complete recognition text, the current scene understanding information and the current action understanding information; inputting the corrected current complete recognition text, the current scene picture and the historical communication information into a third visual language model, and analyzing a current response text; converting the current response text into a current sign language sequence; and driving the robot to simulate sign language actions according to the current sign language word order.

The invention relates to the technical field of electromagnetic valves, in particular to an intelligent control faucet and a faucet body, the faucet carries a control system, and the system comprises an environment interference suppression module, a gesture recognition processing module, an electromagnetic hysteresis compensation module, a multi-parameter coupling optimization module and a residual magnetism elimination control module. According to the method, the infrared compensation proportion is dynamically adjusted by fusing the environment light intensity and the hand position information, the interference suppression capability is improved, the recognition stability under complex illumination is enhanced, the action consistency is extracted in combination with the gesture track waveform and the dynamic response feature, and the recognition accuracy is improved; electromagnetic hysteresis is dynamically corrected based on real-time current and displacement deviation combined with a temperature factor, motion feedback precision is improved, multi-parameter coupling is fused with multi-physical field data to construct a stable control reference, system sensitivity is enhanced, a reverse pulse sequence is designed by capturing a current attenuation waveform, residual magnetism interference is effectively eliminated, and system performance is improved. And multi-dimensional improvement of identification precision, control stability and response efficiency is integrally realized.

The invention provides an intelligent remote controller control method, device and equipment based on gesture recognition, and the method comprises the steps: obtaining multi-modal sensor data through an integrated time of flight (ToF) camera, a millimeter wave radar and an inertial measurement unit (IMU), carrying out the feature extraction to generate gesture spatial-temporal features, and carrying out the dimension exchange of the spatial-temporal features to generate exchange dimension features; constructing a candidate gesture probability superposition state based on the exchange dimension features, establishing a semantic negative hypothesis and generating a reverse verification parameter; performing weight correction on the probability superposition state by using the inversion verification parameter to generate a correction probability state, and determining a target gesture through probability aggregation; re-extracting verification feature data based on the target gesture, selecting a grading classification model according to a preset grading condition to perform secondary verification, and outputting a gesture recognition result; and detecting a communication environment state, constructing a satellite flash and UWB dual-mode communication race state, aggregating into an optimal communication mode, and completing intelligent control of the remote controller.

The invention discloses a dynamic gesture recognition method based on space-time interaction and rate perception. The method comprises a space-time decoupling feature extraction module, a rate feature injection module and a space-time feature injection interaction module. The space-time decoupling feature extraction is formed by three-dimensional convolution, an Inception module and a multi-scale Transform, and the rate feature injection module realizes visual rhythm feature extraction based on similarity calculation and an optical flow estimation algorithm, performs feature enhancement on an initial feature by using the visual rhythm feature, and performs feature extraction on the initial feature. The spatial-temporal feature injection interaction module is composed of a spatial feature injection module and a time feature interaction module, and the spatial feature injection module and the time feature interaction module are realized based on an attention mechanism. The space-time decoupling feature extraction is a backbone network, and other modules are embedded in the backbone network and play roles of feature injection, feature enhancement and the like. The method relates to the fields of man-machine interaction, dynamic gesture recognition algorithms, deep learning, computer vision and the like, and has the advantages of comprehensive features and accurate recognition.

The invention relates to the technical field of virtual reality, in particular to a centimeter-level VR real-scene real-time synchronous rendering method for multi-source data fusion and physical field coupling. Comprising the following steps: eliminating cloud layer interference and expanding Kalman filtering to improve positioning precision through multi-source acquisition of satellite remote sensing data, meteorological data and an equipment sensor; dynamically adjusting process noise by using Kalman filtering, and complementing delay data; constructing a ray tracing and U-Net noise reduction model, carrying out hierarchical loading of textures in combination with dynamic texture management, modeling dynamic illumination and wind field simulation based on a solar declination angle, and realizing scene rendering; a virtual ground is aligned through WebXR, gesture recognition and a force feedback mechanism are integrated, and fingertip curvature and movement speed features are extracted in combination with a Leap Motion sensor. According to the invention, through multi-source data fusion, efficient rendering and accurate interactive design, the authenticity and real-time synchronization capability of the VR scene are significantly improved.

The invention discloses a gesture recognition method and device based on deep learning, and the method comprises the steps: collecting a video stream, and obtaining a hand key point data set in the video stream; performing data enhancement and preprocessing on the hand key point data set; establishing a gesture recognition model based on deep learning, and performing training optimization; and carrying out lightweight processing on the trained gesture recognition model, and outputting a gesture recognition result in real time based on the lightweight gesture recognition model. According to the method, through pre-segmentation of the video image, a progressive polynomial attenuation pruning strategy, collaborative optimization of pruning and quantification and a comprehensive callback mechanism, real-time, accurate and efficient operation of a gesture recognition model on mobile equipment and an embedded system is achieved, and the high-standard requirement of a modern intelligent interaction system is met.

The invention discloses an intelligent household electrical appliance interaction control system based on embedded software. A system operation process specifically comprises the following steps: acquiring a user input signal; analyzing the user input signal based on an embedded software architecture to generate a standardized control instruction; establishing communication connection with a plurality of intelligent household electrical appliances through an Internet of Things protocol, and obtaining real-time environment data and real-time operation states of the intelligent household electrical appliances; performing conjoint analysis on the user historical behavior data and the user input signal; adjusting operation parameters of the intelligent household electrical appliance; and returning interaction response information to the user through at least one mode of a voice feedback module, a touch interface, a gesture recognition interface or a mobile phone APP. The method has the following advantages and effects: multi-mode instructions can be adaptively fused, the equipment capability and the user intention can be dynamically matched, and cross-equipment and cross-protocol efficient and accurate control is realized through standardized instruction generation, real-time load sensing and priority dynamic calculation.

The invention discloses a real-time dynamic trajectory tracking method and system for millimeter wave radar gesture recognition, and relates to the technical field of gesture recognition tracking, and the method comprises the steps: receiving an echo signal reflected by a gesture, extracting a potential target point cloud, and carrying out the clustering generation of a gesture point cloud sequence; establishing a multi-modal motion model library, dynamically selecting an optimal motion model by adopting graph matching, and generating a prediction state in combination with a gesture point cloud sequence; on the basis of a Poisson multi-Bernoulli hybrid filtering framework, according to the signal-to-noise ratio and spatial distribution of the current gesture point cloud sequence, dynamically adjusting the observation weight, optimizing the observation point cloud, and carrying out optimal association by combining Mahalanobis distance with dynamic time warping; a multi-hypothesis tracking strategy is adopted to maintain trajectory hypothesis, an optimal trajectory is selected through a trajectory scoring mechanism, and Kalman filtering smoothing processing is performed on the optimal trajectory. According to the method, high-precision and low-delay tracking of gesture motion is realized, gesture habits of different users and complex environment interference can be adapted, and meanwhile, relatively high track precision is kept.

The invention discloses a hotel guest room intelligent room selection auxiliary method based on an augmented reality technology. According to the method, multi-dimensional data, including a 3D model, a panoramic image, environment data and a facility state, of a guest room are collected, a dynamically updated digital twin model is constructed, and model data are optimized by adopting a lightweight algorithm. Through an indoor positioning technology, the digital twinborn model is accurately superposed in a real scene, and immersive house selection experience is provided for a user. The system supports various interaction modes such as gesture recognition and voice instructions, information display is dynamically adjusted according to user behaviors, user preferences are analyzed in combination with a machine learning algorithm, and a personalized recommendation result is generated. According to the method, the problems of non-visual information display, poor interactivity and insufficient personalized recommendation of a traditional house selection mode are solved, and an efficient and intelligent house selection auxiliary tool is provided for the user.

The present invention provides a solution of a human-computer interaction method and system based on three-dimensional coordinate measurement, and a plurality of solutions for applying same to solve technical problems. An operation body and / or an operator performs input and operation interaction by means of a virtual keyboard and a virtual mouse, or integrates a display picture space and carries out various participatory and interactive direct operations and experiences on object components and objects in virtual and real scenes. Virtual handwriting input, body movement posture recognition, and lip recognition interaction methods are further provided, and virtual mode control and interaction methods such as gesture recognition and virtual touch screen interaction are supported. Contact is not needed and there is no need to touch the screen, and mobility is achieved. The present invention is suitable for the two-dimensional conventional computing field, and particularly suitable for three-dimensional space computing fields (comprising VR, MR, AR, meta-universe, virtual simulation, software games, three-dimensional graphics, three-dimensional animation, and other three-dimensional scenes). Human-computer interaction operations in work and life of people can be greatly facilitated and optimized.

The invention discloses a high-precision multi-modal twin model assembly method based on a large language model and a multiple docking optimization mechanism. The whole system framework is composed of three modules: a database, an interaction system and human factors. Wherein the database module integrates a model library module, a user interaction data recording module and a temporary data recording module, and is used for storing and managing various data information required for constructing a virtual scene; the interactive system module is used as a core part, integrates a model dynamic loading module, a model accurate positioning module and a model adaptive assembly module, realizes instruction analysis and model generation by relying on a large language model, and combines a multi-docking optimization mechanism of eye movement tracking, gesture recognition, automatic docking and logic rule binding to realize multi-docking. Efficient and accurate interaction between the user and the database is effectively guaranteed; the method not only provides a solid foundation for virtual experiments and virtual-real symbiosis, but also significantly improves the construction efficiency and precision of digital twin model assembly.

The invention relates to the technical field of intelligent power grid control, and discloses an intelligent power grid control system and method based on gesture interaction, and the system comprises a multi-modal data collection and preprocessing module which is used for collecting gesture data of a user, including images, radar and inertial measurement unit data, the gesture data acquisition module is used for acquiring gesture data of a user, filtering, denoising and normalizing the acquired gesture data of the user, and transmitting the processed data to the multi-modal data synchronization and alignment module; and the multi-modal data synchronization and alignment module refers to the output data of the multi-modal data acquisition and preprocessing module, and performs time and space alignment on the multi-modal data through a timestamp synchronization and space calibration technology. According to the method, features of the image, the radar and the inertial measurement unit are subjected to weighted fusion to realize efficient integration of multi-modal information, so that the gesture recognition precision of the system in a complex and variable environment is remarkably improved, and the robustness and the recognition stability of the system are enhanced.

The invention discloses a transformer substation control screen cabinet pressing plate mistaken touch prevention method and system, and the method comprises the steps: the system automatically detects the approaching of the hand of an operator through infrared induction and gesture recognition, and judges whether the recognition operation is effective or not; according to environmental changes such as high humidity or temperature, the system automatically adjusts the sensitivity of the touch screen to optimize the operation experience; before key operation, the system starts a multiple confirmation mechanism, and an operator needs to confirm an operation intention through a touch screen button and voice recognition; for high-risk operation, the system pops up an alarm and carries out secondary confirmation; meanwhile, the system dynamically adjusts the authority according to the identity of the operator and the task requirement, and the low-authority operator is limited to access the key component and only can execute the operation within the authority range of the low-authority operator; according to the method and the system for preventing the pressing plate of the transformer substation control screen cabinet from being touched mistakenly, the accuracy and the flexibility of preventing mistakenly touching are effectively improved, and the defects in the prior art are overcome through an intelligent means.

There are described various embodiments for devices, systems, and methods for security threat detection and reaction using multi-layered decentralized federated learning approach, for use in various combinations of security systems, including facial recognition systems, biometric recognition systems, gesture recognition systems, voice recognition systems, network traffic pattern monitoring systems, security systems using Internet of Things (IoT) sensors, and home automation security systems. By combining federated learning system with a local interpretation layer, it is possible for each local node in a system to interpret anonymized federated learning results into highly personalized results, which can then be used to trigger highly personalized actions. The multi-layered federated learning threat detection and response system therefore optimizes for both enhanced privacy and customization.

The invention discloses a skeleton sign language recognition method of a double-flow space-time dynamic graph convolutional network fused with residual learning, and belongs to the technical field of artificial intelligence and gesture recognition. According to the method, an input gesture skeleton sequence relative to a face is divided into two data streams, namely a hand form data stream and a wrist track data stream through double-reference-system differential homeomorphic mapping; the method comprises the following steps: firstly, processing hand posture data, and capturing a hand joint spatial topological relation by combining a spatial-temporal dynamic graph convolutional network (STDGCNN) with a residual convolutional block; meanwhile, a Finsler trajectory dynamics encoder (FTDE) is adopted to carry out differential geometric modeling on the wrist trajectory, and the direction sensitivity characteristic of the trajectory is captured through a multi-scale causal convolutional network. Then, mutual enhancement of double-flow features is realized through a bidirectional cross feature enhancer (BCFE), and the problem of geometric inconsistency of a heterogeneous feature space is solved through a geometric-driven optimal transmission fusion device (Geometric-OT). The method solves the challenge that a traditional sign language recognition method processes complex space-time correlation of gesture forms and motion tracks at the same time, the technical problems of insufficient relation between hands and faces, insufficient feature expression ability and low space-time feature extraction efficiency, and the problems of geometric inconsistency, single reference system and the like. And the identification accuracy and the real-time performance are obviously improved. Experiments show that the accuracy rate of the method in complex hundreds of sign language vocabulary recognition tasks reaches 95% or above on average, the reasoning speed is only 17ms on average, high-precision real-time sign language recognition is achieved, and the method has higher robustness in complex environments such as noise and shielding.

The invention discloses a dynamic millimeter-wave radar point cloud human arm tracking system and method based on joint learning, and the method comprises the steps: obtaining a millimeter-wave point cloud generated by human motion through millimeter-wave radar equipment, and enabling the generated point cloud to serve as the input of an arm point cloud generation module based on heuristic and clustering; decoupling a dynamic millimeter wave point cloud generated by arm movement; taking the dynamic arm point cloud as the input of a spatial-temporal feature encoder, and carrying out feature extraction; and point cloud features are output as input of a multi-task joint learning module, gesture recognition is introduced as an auxiliary task, learning assistance is provided for an arm tracking task, and finally three-dimensional motion trajectory tracking of main joints of the arm is achieved. According to the method, the technical challenges of arm semantic information loss and dynamic change in millimeter wave point cloud are overcome, and the human arm motion robust tracking with privacy protection is realized.

The invention discloses a vehicle-mounted VR back-row entertainment method, and relates to the field of VR, and the method comprises the steps: processing image data transmitted by a controller through a DCT plate negative reflection plate lens, and generating aerial imaging data; reflecting a virtual image corresponding to the aerial imaging data by using a plane mirror to generate real image data; performing voice recognition processing on the input voice of the user to obtain voice control instruction data; acquiring image data of user gestures through an infrared sensor, and performing image recognition to obtain gesture control instruction data; generating interaction control data according to the three-dimensional coordinates in the real image data, the voice control instruction and the gesture control instruction; aiming at low image imaging precision in a vehicle-mounted VR entertainment system in the prior art, high-quality aerial imaging is generated by using a DCT plate negative reflection plate lens and a plane mirror, multi-modal interaction and the like are realized by adopting voice recognition and gesture recognition, and a vivid imaging effect is provided.