65 results about "Motion perception" patented technology

The invention discloses a high-efficiency time sequence optical flow method, system and device for fusing event and image information and a medium. The method comprises the following steps: constructing a time sequence feature extraction module; constructing a double-branch collaborative iteration optical flow module of the event and the image; constructing a fusion modeling and iterative optimization optical flow module based on cross attention; training a dual-branch collaborative iteration optical flow module of the event and the image and a fusion modeling and iterative optimization optical flow module based on cross attention on a training set of an MVSEC public data set to obtain weights, and testing on a test set of the MVSEC public data set to obtain an optical flow calculation result; the system, the equipment and the medium are used for implementing the method. The method has the advantages of high time resolution, high dynamic modeling capability, fine multi-modal fusion, high estimation precision and the like, the motion sensing performance in a complex scene is remarkably improved, and powerful technical support is provided for an intelligent visual system.

The invention relates to an action perception and reconstruction method and device, equipment and a storage medium. The method comprises the following steps: carrying out space-time synchronization and heterogeneous graph modeling on asynchronous data of an electromagnetic sensor and an inertial measurement unit (IMU) through multi-source sensor data preprocessing and fusion base construction; performing deep feature extraction and hierarchical fusion by using a gated attention time-space diagram neural network (ST-GNN); through a dynamic interference confrontation and feature purification mechanism, the robustness of the system in a complex environment is improved; generating a high-fidelity and physiologically reasonable action sequence by adopting a multi-scale space-time generation network (MSTGN) and combining biomechanical constraints; a physical information neural network (PINN) constraint is embedded, and high-precision six-degree-of-freedom (6DoF) pose estimation is realized through a decoupling collaborative framework. According to the method, the precision, robustness and real-time performance of motion sensing and reconstruction are effectively improved.

The invention relates to the technical field of computer vision and robot positioning, and discloses a dynamic 3DGS-SLAM method and system based on Gaussian pyramid and adaptive densification. The method comprises the following steps: generating a semantic mask through real-time dynamic target detection so as to distinguish a static background from a dynamic object; constructing a 3D Gaussian map model of the scene; rendering images in parallel on a full-resolution view and a down-sampling resolution view by adopting a motion perception Gaussian pyramid rendering method, and fusing rendering results by utilizing a dynamic mask so as to eliminate motion edge artifacts; a self-adaptive densification strategy based on a monotonic attenuation function is adopted, the initialization radius of the Gaussian ellipsoid is dynamically adjusted based on the training progress, and high-density initialization and progressive redundancy pruning are executed to optimize the geometric fidelity; and finally, performing coarse-to-fine camera pose optimization, and outputting a precise pose and a high-fidelity static map. According to the method, the problems of tracking drift and rendering artifacts under dynamic interference are effectively solved, and the positioning precision and the rendering quality are improved.

The invention discloses a dynamic scene reconstruction method, computer equipment and a program product. According to the method, picture information and camera poses of the same scene are obtained, a standard field composed of three-dimensional Gaussian primitives is constructed in a three-dimensional space, a deformation field is constructed, the standard field and the deformation field at different times are subjected to joint rendering in a three-dimensional Gaussian splashing mode, and various parameters are trained and optimized according to the difference between a synthetic image and a real image. Performing motion perception according to the motion condition of the three-dimensional Gaussian primitives, taking the primitives with smaller motion as static primitives, and performing partition modeling on the primitives with larger motion in time to obtain a layered dynamic scene model; and calling the model for rendering according to the given time and the camera pose in the reasoning stage, and outputting a target image. According to the method, the reconstruction precision of the high-dynamic target can be improved while the rendering efficiency is ensured, and the continuity and visual quality of the dynamic scene in the time dimension are improved by introducing the time consistency constraint at the boundary of the time subinterval.

The invention relates to a handle positioning method and device for sparse features, and the method comprises the steps: constructing a non-rigid multi-sensor joint state estimation model, and building a non-rigid connection through a camera and an IMU (Inertial Measurement Unit); performing joint optimization on the visual re-projection error and the IMU pre-integration error in the sliding window to obtain initial attitude estimation of the handle; based on the number of visible feature points, a geometric degradation state is analyzed through observability, a self-adaptive marginalization strategy of motion perception is established, and a sliding window is dynamically adjusted; based on a self-adaptive marginalization strategy, a prior quality monitoring and multi-hypothesis maintenance mechanism is implemented, initial attitude estimation is optimized, and an optimal handle attitude is output; equipment is realized based on the method. According to the method, high pose estimation precision can still be maintained in a complex interaction scene, long-term drift caused by inaccurate models is remarkably reduced, the survivability under the severe conditions of shielding, rapid rotation and the like is greatly improved, limited visual information in each frame is efficiently and accurately utilized, and the real-time operation capability of a system is ensured.

The invention relates to the technical field of computer vision, in particular to a motion perception rockfall intelligent monitoring method and device based on deep learning. The method comprises the following steps: receiving a monitoring video from a monitoring area; constructing a rockfall detection and tracking framework based on a YOLOv8 target detection network and a ByteTrack target tracking algorithm; preprocessing the monitoring image, inputting the monitoring image into a YOLOv8 detection network to carry out moving target feature extraction, and outputting a rockfall detection result; and track matching is carried out by using a ByteTrack algorithm in combination with Euclidean distance and motion direction constraint, and a rockfall motion track is generated. According to the method, a traditional image algorithm and a deep learning algorithm are combined, moving target detection and multi-target tracking are constructed, and rockfall in a complex rock background is accurately captured from multiple dimensions; according to the method, the space coordinate position of the rockfall target can be accurately obtained, a complete movement track can be generated for each rockfall target, statistical information of rockfall events is effectively obtained, and a reliable basis is provided for disaster grade evaluation and early warning.

The invention relates to the technical field of image segmentation, and provides a gastrointestinal tract endoscope image segmentation method, which comprises the following steps: carrying out time sequence preprocessing on an obtained intestinal tract video, eliminating fuzzy and shielded invalid frames, constructing a rhythm signal based on a multi-scale time sequence visual feature, and estimating a creeping phase; outputting a phase time sequence and a phase label of the video; selecting a representative phase key frame, performing pixel-level focus segmentation on the key frame, and calculating a pixel-level uncertainty score to obtain a key frame segmentation result with high confidence; carrying out time sequence consistency recovery on the segmentation mask of the key frame based on phase consistency mapping and phase alignment, and recovering an uninterpretable section caused by shielding with the assistance of adaptive segmentation correction of motion perception; and carrying out time sequence aggregation on a cross-frame segmentation result, extracting fragment-level statistical representation and confidence, forming a fragment-level suspicious area report, and outputting structured data for clinical auditing and index retrieval.

The invention discloses a vital sign monitoring method and system based on multi-modal perception and space-time restoration, and belongs to the technical field of artificial intelligence, and the method comprises the steps: obtaining an original video frame sequence of a target object, and constructing a multi-modal input tensor containing physical prior information based on the original video frame sequence; inputting the multi-modal input tensor into a feature extraction network, extracting a spatial-temporal feature and a stability feature through parallel processing, and fusing the spatial-temporal feature and the stability feature to obtain a basic feature; generating an attention weight based on motion information in the multi-modal input tensor, and performing dynamic weighting correction on the basic features by using the attention weight to obtain anti-interference features; and carrying out time sequence long-range dependence modeling on the anti-interference features, and carrying out logic repair and smoothing processing on the feature fragments which are subjected to motion interference currently by using historical feature fragments which are not interfered, so as to predict and obtain rPPG signals. According to the invention, industrial light environment adaptation and physical motion perception are realized, and meanwhile, long-range time sequence restoration can be realized when signals are lost.

The invention discloses a real-time pair practice training system and method based on a multi-modal large model, and relates to the technical field of large model pair practice. A multi-source sensor array comprising a visual capturing module, an auditory processing module and an action sensing module is deployed, and space-time synchronization of sensor data is realized by using a multi-modal alignment engine; generating a visual space-time vector reflecting a facial dynamic state and a limb time sequence rule; generating a fundamental frequency of the voice signal and an auditory feature vector of formant dynamic change; and generating a pressure sensitive feature vector fusing the hand motion trail and the contact force data. And carrying out weighted fusion on the feature vectors, calculating a comprehensive behavior evaluation index, comparing the comprehensive behavior evaluation index with a preset positive threshold value and a preset negative threshold value, and judging a user behavior state. And calculating a prediction error according to the actual state feedback of the user, and adjusting a positive threshold and a negative threshold in real time by using a dynamic threshold learning updating mechanism to realize adaptive optimization of an evaluation standard.

The application provides a dynamic anti-aliasing rendering method and system based on edge detection and motion perception, and relates to the technical field of computer graphics rendering. The method comprises the following steps: performing 8-direction edge intensity detection on a pixel point in an original scene image by using a 3*3 sampling matrix and calculating an edge intensity value; calculating a motion factor and dynamically adjusting an edge detection threshold; if the edge intensity value of the pixel point is greater than the edge detection threshold, performing adaptive blur processing based on a 9-point weighted average algorithm to generate a blurred pixel value, calculating a mixing factor, mixing the original pixel value of the pixel point and the blurred pixel value to generate a final pixel value; if the edge intensity value of the pixel point is less than or equal to the edge detection threshold, keeping the original pixel value of the pixel point unchanged, and the final pixel value is the original pixel value, and finally outputting a final scene image by collecting the final pixel value, so that the anti-aliasing processing strategy can be adaptively adjusted according to the static and dynamic characteristics of the scene, the rendering performance is optimized while the image quality is ensured.

The application discloses an intelligent high-saturation snapshot method and system based on motion sensing and adaptive white balance delay, and belongs to the field of intelligent terminal image processing. The method is applied to an intelligent terminal comprising an inertial motion sensing module, an ambient light sensing module, a camera acquisition module and an end-side processing unit. Data is collected by the inertial motion sensing module, combined with dynamic threshold, trajectory continuity verification and false touch prevention condition identification effective swing. Based on ambient light data, a continuous mapping model is used to calculate the white balance delay time. Frame-level synchronous trigger shutter is adopted to collect images by using single camera continuous frames or double camera synchronous frames. After motion blur compensation and dynamic saturation enhancement, high-saturation clear images are output, and the output image metadata contains a special identification field. The application breaks through the industry technical prejudice, reduces the false trigger rate to below 0.5%, and improves the image forming rate to above 95%. Without adding new hardware, the application can be widely adapted to various intelligent terminals, solves the industry pain points of high false trigger, uncontrollable best imaging opportunity and swing blur of traditional snapshots, and reduces the difficulty of infringement and right protection evidence.

The invention discloses a lightweight inertial navigation positioning method and a lightweight inertial navigation positioning system based on a motion perception early-leaving mechanism, which are applied to mobile terminal positioning scenes such as pedestrian dead reckoning and the like. The method comprises the following steps: acquiring a three-axis acceleration and a three-axis angular velocity by using a built-in inertial measurement unit of a terminal, inputting preprocessed inertial data into an inertial navigation neural network, and sequentially outputting displacement estimation and covariance thereof at each layer by the network; and the early-backing judgment module takes the variance as a confidence index, compares the variance with a preset early-backing threshold value, terminates reasoning in advance at a current outlet and outputs a displacement result when the confidence meets the requirement, otherwise, continues to spread to a deeper layer, and realizes adaptive adjustment of the reasoning depth according to the motion complexity. According to the method, the reasoning efficiency is remarkably improved in low-dynamic scenes such as static scenes and slow scenes, the method has the advantages of being small in calculation overhead, low in energy consumption, suitable for real-time deployment on smart phones, wearable devices and augmented reality terminals and the like, and the endurance of a deep learning inertial navigation positioning system can be greatly improved.

This invention discloses an adaptive temperature control method based on motion state perception, comprising the following steps: Step 1, the motion sensing module senses the user's motion intensity and transmits the motion intensity data to the main control module; Step 2, the main control module determines the motion intensity level based on the motion intensity data, matches a target heating level based on the motion intensity level, and sends a target heating level command to the heating module; Step 3, the heating module adjusts the temperature to the target temperature range according to the target heating level command. An adaptive temperature control system based on motion state perception is also disclosed, comprising a motion sensing module, a main control module, and a heating module; the temperature control system of this invention can automatically match the heating level temperature according to the motion intensity, so that the user does not need to manually adjust the temperature during exercise.

The invention discloses a gait recognition method based on local appearance and micro-motion perception, and belongs to the technical field of computer vision and biological feature recognition. The method comprises the following steps: firstly, preprocessing and aligning an input gait contour sequence; extracting spatial features by using a convolutional neural network, and performing non-uniform part division according to human body structure priori knowledge so as to focus on leg equal-height discriminant regions; further, an independent micro-motion sensing module is designed for each component, and unique short-distance space-time dynamic characteristics of each component are captured through a double-branch time sequence attention mechanism; and finally fusing the enhanced features of all the components to complete identity recognition. By emphasizing the local appearance difference and the micro-motion mode, the robustness and stability of the gait recognition model in a real complex scene are remarkably improved, and the method has important security monitoring and identity authentication application value.

The invention discloses a field depth information sensing method based on a single light-operated memristor array. The method comprises the following steps: 1, constructing a light-operated memristor array; 2, projecting the optical pulse to the light-operated memristor array, and writing and storing optical information; step 3, erasing the stored optical information; 4, performing one-time conductivity reading on the light-operated memristor array to obtain a two-dimensional current mapping graph; and step 5, sending the obtained two-dimensional current mapping graph as input data into a pre-trained artificial neural network so as to realize rapid identification of the motion direction and depth-of-field information of the target object. According to the invention, plane motion and depth-of-field information are extracted in the same array at the same time, hardware configuration is simplified, through a single-array structure, dependence on multiple arrays or additional sensors is not needed any more, and system conciseness is ensured. By using the light-operated memory characteristic of the array element, the planar motion perception is successfully expanded into the depth-of-field perception, and the function expansion is realized.

The application discloses a kind of key frame adaptive screening and efficient modeling method for monocular unmanned aerial vehicle, comprising the following steps: S1 visual space information entropy model construction;S2 key frame screening based on information entropy;S3 motion perception and tracking quality joint evaluation;S4 key frame adaptive compensation of motion state;S5 the key frame obtained by screening is input into the rear end local mapping thread, executes map point update and local bundle adjustment optimization, and outputs monocular unmanned aerial vehicle motion trajectory and environment map.The application constructs information entropy gate mechanism and motion state compensation cooperative decision-making strategy, suppresses redundant key frame insertion at the same time, avoids key frame missing problem in high dynamic scene, solves the redundancy problem of key frame screening and guarantees trajectory continuity, so as to improve the overall stability and efficiency of visual SLAM system, suitable for power limited monocular unmanned aerial vehicle autonomous modeling scene.

The invention provides a dynamic anti-aliasing rendering method and system based on edge detection and motion perception, and relates to the technical field of computer graphic rendering, and the method comprises the steps: employing a 3 * 3 sampling matrix to carry out 8-direction edge intensity detection on pixel points in an original scene image, and calculating an edge intensity value; calculating a motion factor and dynamically adjusting an edge detection threshold value; if the edge intensity value of the pixel point is greater than the edge detection threshold value, performing adaptive fuzzy processing based on a nine-point weighted average algorithm to generate a fuzzy pixel value, calculating a mixing factor, and mixing the original pixel value of the pixel point and the fuzzy pixel value to generate a final pixel value; if the edge intensity value of the pixel point is smaller than or equal to the edge detection threshold value, the original pixel value of the pixel point is kept unchanged, the final pixel value is the original pixel value, the final pixel value is summarized, and a final scene image is output, so that the anti-aliasing processing strategy can be adaptively adjusted according to the static and dynamic characteristics of the scene, and the image quality is improved. And the rendering performance is optimized while the image quality is ensured.

The motion quality evaluation method based on motion perception and double-flow space-time graph convolution relates to the motion quality evaluation field, solves the problems that the existing rehabilitation motion quality evaluation method is difficult to accurately model the cooperation between joints, the fixed joint grouping strategy is not flexible enough, the model has insufficient recognition of the detail differences of complex rehabilitation motions, and the reliability of the evaluation result is affected, a dynamic joint grouping strategy based on motion amplitude driving is provided, a double-flow STGCN architecture is designed, the position and orientation information of the joints are processed respectively and then fused, a two-stage self-attention module SADG is developed, the inter-group time sequence mode and the intra-group spatial relationship are modeled in stages, and multi-granularity feature interaction fusion is realized. The attention mechanism introduces a constraint guided by the motion amplitude, so that the attention weight is more interpretable and has physical meaning.

The application discloses a cross-modal silent speech reconstruction method and system based on ear canal air pressure micro-motion sensing, and belongs to the technical field of human-computer interaction and wearable computing. The method uses a micro-pressure sensing unit placed in an in-ear earphone to collect a non-acoustic air pressure sequence caused by the movement of a sound-producing organ; through adaptive baseline drift suppression and rhythm perception data enhancement processing, a robust feature space is constructed; further, an end-to-end deep neural network containing domain adversarial adaptation, cross-modal semantic alignment, coarse-grained mel-spectrogram generation and residual detail correction is used to map the TPVS to a high-fidelity acoustic mel spectrum. The application effectively breaks through the technical bottleneck of the lack of high-frequency acoustic features in low-frequency mechanical signals, realizes high-precision silent speech command analysis in a mobile and noisy scene, and introduces a coupled quality evaluation gate and trigger-based start / stop control at the inference end to suppress invalid inference and reduce power consumption when wearing is poor or there is no trigger condition.

This invention discloses an edge-side adaptive image deblurring method and system based on motion perception, belonging to the field of image processing technology. The method includes: periodically acquiring the angular velocity of a camera collected by an inertial measurement unit (IMU), calculating the pixel-level motion blur amplitude based on the angular velocity, and calculating the region of interest (ROI) information based on the pixel-level motion blur amplitude; mapping the time window of the ROI information to the host time domain based on the timestamp, current clock offset, and validity window of the ROI information; matching the timestamp of the camera's image frame with the time window mapped to the host time domain to determine the target ROI information corresponding to the image frame, so as to map the pixel coordinates in the image frame to the target resolution to obtain the ROI; inputting the ROI into a deblurring network to obtain the deblurred ROI; and performing inverse mapping on the deblurred ROI to obtain the deblurred image frame. This method reduces image processing latency.

This invention discloses a method for identifying the edge of micro-motion damage in ancient building structures based on lightweight deep learning, belonging to the field of cultural heritage protection. The method includes: micro-motion perception fusion, employing a high-sensitivity accelerometer and micro-displacement laser measurement to achieve dual-mode perception of "dynamic mode + static deformation"; signal preprocessing; lightweight time-frequency analysis network identification, generating time-frequency spectra from acceleration samples through continuous wavelet transform, inputting them into the lightweight time-frequency analysis network LTFANet, and outputting modal parameters and variation characteristics; crack propagation trend analysis; structural damage index calculation, fusing multi-source information to calculate a physically interpretable structural damage index (SDI); graded early warning, pushing early warning information based on a four-level threshold system; and early warning verification. This invention achieves high-precision perception and real-time edge identification of micro-damage in ancient buildings, with high early warning accuracy, and is applicable to immovable cultural relics such as ancient pagodas and wooden structures.

The invention discloses a motor imagery coupling system and method based on emotion prediction. The method comprises the steps that dynamic emotion electroencephalogram data of an emotion-related brain area and motor imagery electroencephalogram data of a motion perception-related brain area are collected; extracting emotional features and motor imagery features to form an emotional feature vector and a motor feature vector; based on the historical emotion feature vector and the current emotion feature vector, obtaining a predicted emotion feature vector at a subsequent moment; fusing the current emotional feature vector and the predicted emotional feature vector to obtain an emotional context vector, thereby determining an emotional index vector, and determining an emotional index weight vector based on the emotional index vector; and performing weighted fusion on the emotion index weight vector and the motion feature vector to obtain a weighted enhanced motion imagination feature vector, decoding the motion imagination feature vector, and outputting a control signal to drive terminal equipment according to a decoding result. Decoding deviation caused by emotion interference is reduced, and decoding accuracy, adaptability and reliability are improved.

The invention discloses a cross-modal silent voice reconstruction method and system based on ear canal air pressure micro-motion sensing, and belongs to the technical field of man-machine interaction and wearable computing. The method comprises the following steps: acquiring a non-acoustic air pressure sequence caused by vocal organ movement by using a micro air pressure sensing unit arranged in the in-ear earphone; constructing a robust feature space through adaptive baseline drift suppression and rhythm perception data enhancement processing; and further mapping the TPVS into a high-fidelity acoustic Mel spectrum by using an end-to-end deep neural network including domain adversarial adaptation, cross-modal semantic alignment, coarse-grained Mel spectrum generation and residual detail correction. The technical bottleneck that low-frequency mechanical signals lack high-frequency acoustic features is effectively broken through, and high-precision silent voice instruction analysis in a mobile and noise scene is achieved; and coupling quality evaluation gating and trigger type start / stop control are introduced at a reasoning end, so that invalid reasoning is inhibited and power consumption is reduced under the condition of poor wearing or no trigger.

The application is suitable for the field of medical image processing and mixed reality technology, and provides a laparoscopic surgery mixed reality navigation method based on deep learning and dynamic point tracking, comprising: dynamically registering a three-dimensional model containing a kidney, a tumor and a vessel with a laparoscope initial frame through a mixed reality alignment technology, constructing a surgery forceps motion perception model based on a time series deep neural network, realizing real-time control and parameter locking of the three-dimensional model pose, dynamically updating a two-dimensional feature point set using a multi-feature point joint tracker, constructing a candidate feature combination through a cross-quadrant sampling strategy, generating a candidate feature combination through four-quadrant division and a cross-region sampling strategy, combining a re-projection error and a pose continuity constraint to generate optimal camera pose parameters, and dynamically covering a semi-transparent three-dimensional model to a surgery video. The method can significantly enhance the spatial perception ability of the kidney anatomical structure, reduce the registration error of the three-dimensional integrated kidney structure model and the kidney in the laparoscope video, and improve the navigation accuracy.

The invention relates to the technical field of visual perception, discloses a biologically inspired collision detection system and robot for atopic impending perception recognition, and constructs a brand-new collision detection model for atopic impending perception recognition. According to the technical scheme, through a six-layer structure (P-> E / I-> A-> S-> G-> LGMD), exponential attenuation, nonlinear modulation and a spatial clustering mechanism are combined, and accurate distinguishing and real-time collision early warning functions of approaching and leaving motion states are jointly achieved from the three aspects of structure, function and biological simulation; the method has the advantages of high direction selectivity, high robustness and low power consumption calculation efficiency, and is particularly suitable for rapid motion sensing tasks in embedded robots and low-calculation-power visual platforms.

The application relates to the field of intelligent vision and image fusion technology, and discloses a dual-light image fusion method of an intelligent camera and a dual-light integrated intelligent camera, which comprises the following steps: collecting original infrared images and original visible light images; preprocessing the original infrared images and the original visible light images to obtain dual-path preprocessed images; calculating an optical flow field by adopting an improved optical flow algorithm on the dual-path preprocessed images; generating an infrared band motion perception feature set and a visible light band motion perception feature set; generating a preliminary fusion image based on the infrared band motion perception feature set and the visible light band motion perception feature set; obtaining an evaluation result including evaluation indexes of clarity, motion consistency and detail richness by taking the preliminary fusion image as an input; dynamically adjusting a cross-modal attention weight and a gate value according to the evaluation result; and obtaining an optimized final fusion feature based on the dynamically adjusted cross-modal attention weight and the gate value. The application enables the fusion image to be free of dislocation when a target moves rapidly.

The application discloses a model training method, a video generation method, a system, an electronic device, a storage medium and a computer program product, and relates to the technical field of large models and video generation. The method comprises the following steps: acquiring training video data, wherein the training video data comprises scene images of a training scene; performing scene motion perception on the training video data to obtain scene motion representation, wherein the scene motion representation is used for describing motion features corresponding to the scene images from the dimensions of objects and perspectives; and adjusting model parameters of an initial video generation model based on the training video data and the scene motion representation to obtain a target video generation model, wherein the target video generation model is used for generating a target video according to reference images and motion control information of a target scene. The application solves the technical problem that, in the related art, a video generation tool only provides preset video effects, resulting in poor flexibility and controllability of generated videos.

The application relates to a skeleton action recognition method based on region perception motion contrast learning, and relates to the field of computer vision. The method solves the problems that existing contrast learning skeleton action recognition methods focus on global feature contrast learning, ignore the importance of local action patterns for semantic discrimination, and a large amount of redundancy exists in the time and space dimensions of a skeleton sequence, and the method comprises the following steps: skeleton data preprocessing and standardization; multi-stream data conversion, generating three kinds of representations of joint flow, motion flow and skeleton flow; motion perception time sequence enhancement, adaptively retaining key frames based on interframe motion intensity; spatial multi-level data enhancement, implementing progressive three-level data enhancement operation; spatiotemporal feature extraction and region perception mining, extracting features through an encoder and unsupervisedly identifying significant motion regions by using a contrast motion perception region mining method; contrast learning optimization, constructing a dynamic negative sample queue to learn features; and multi-stream model fusion, weightedly integrating recognition results of the data streams.

This application provides an immersive education system and method based on motion perception. It integrates the motion deviation characteristics between the learner's motion intention and standard teaching actions with the smooth transition amount when the learner switches between educational scenarios to obtain the educational logic difficulty for the interaction engine in rendering the next stage of teaching. This educational logic difficulty then sets the learner's interaction response threshold. When the learner's interaction success rate within a preset time window is lower than the interaction response threshold, a predefined resource scheduling strategy is used to determine the teaching anchor point for the next stage of teaching based on the rendering priority of each teaching information element in the immersive education scenario and the current educational logic progress. Based on this teaching anchor point, the content presentation flow of the immersive education scenario in the next stage of teaching is updated in real time. Based on the above scheme, personalized rendering updates of teaching difficulty and presented content in immersive education can be achieved.

This invention discloses a multimodal fusion method for human motion representation and quality assessment, belonging to the field of human motion perception and assessment technology. It addresses the problems of large modal differences, insufficient sensor characteristic modeling, lack of physical constraints, and insufficient motion quality assessment indicators in existing traditional human motion representation and quality assessment methods. This invention collects multimodal data, preprocesses the data, and inputs the preprocessed multimodal data into a teacher-student network framework, outputting high-precision kinematic features and spatiotemporal relationships. A physical heuristic constraint loss is introduced to obtain a trained teacher-student network framework. Motion phase curves are extracted, and coordination and rhythmic stability indicators are calculated based on the phase curves, outputting the motion quality assessment results. This invention effectively improves the accuracy of human motion assessment and can be applied to sports rehabilitation training, sports movement assessment, ergonomics analysis, and intelligent human-computer interaction.