1105 results about "Action recognition" patented technology

An action localization method, device, electronic equipment, and computer-readable storage medium are provided. The action localization method includes: identifying at least one target video segment containing a target object in a video; acquiring a first action recognition result of at least one image frame in the at least one target video segment and a second action recognition result of the target video segment; and acquiring an action localization result of the video based on the first action recognition result and the second action recognition result.

The invention discloses a video analysis-based multi-scene operator violation behavior identification method and system, and belongs to the technical field of intelligent operation safety monitoring and artificial intelligence identification, and the method comprises the steps: collecting a real-time video stream of a multi-scene operation site; recognizing a continuous action time sequence in the real-time video stream by using an action recognition depth model; constructing the continuous action time sequence into an action behavior sequence; the action behavior sequence is constructed into a directed behavior graph with time, space and action labels, the directed behavior graph is compared with a directed behavior graph corresponding to the standard action behavior sequence, and illegal behaviors are recognized; and carrying out multi-mode early warning on the identified illegal behaviors. According to the method, the bottleneck that the traditional image recognition technology is weak in action sequence semantic understanding and poor in environmental adaptability is broken through, and accurate recognition and real-time early warning of illegal behaviors in multi-scene operation are achieved.

The invention discloses a physical fitness evaluation method based on human skeleton trajectory tracking, which comprises the following steps: S1, acquiring whole-process video data of a physical fitness test through video acquisition equipment, and generating a video frame according to an acquisition frequency; s2, processing the video frame by using a skeleton dynamic analysis algorithm, and extracting human skeleton key points; s3, extracting identity features of the testee through a motion map identity recognition network, and tracking and confirming the identity features; s4, performing action recognition, fragment segmentation and compliance judgment on the skeleton key point time sequence track; s5, aiming at the abnormal skeleton key points, performing complementation and time sequence smoothing by adopting an inverse kinematics inference method; s6, inputting the complete skeleton key point time sequence data into the human body action evaluation model for processing, and quantizing and outputting a physical ability evaluation index; and S7, generating a physical ability evaluation report and giving action feedback and optimization suggestions. According to the invention, objectivity, accuracy and intelligent level of physical ability evaluation are effectively improved.

The invention relates to the technical field of action recognition, in particular to a somatosensory action interaction recognition method and system based on skeleton coordinate points. The method comprises the following steps of collecting real-time skeleton coordinate data of a human body and performing multi-modal feature extraction to obtain a real-time skeleton coordinate sequence; obtaining a standard skeleton posture corresponding to the target interaction action, performing pre-recording and feature coding, and generating a target posture skeleton feature template library; performing skeleton time sequence filtering and joint mapping and joint included angle calculation on the real-time skeleton coordinate sequence, performing similarity measurement and dynamic binding tracking at the same time, and starting a binding recovery mechanism when binding loss is detected so as to guide the user to execute a preset binding posture and re-establish a binding relationship; and mapping the joint included angle time sequence data to a corresponding joint of the virtual human shape interaction model in real time, outputting a somatosensory interaction instruction, and driving to repeat a human body action so as to trigger a somatosensory action interaction event. According to the invention, the stability of somatosensory action interaction recognition can be improved.

The invention belongs to the technical field of vehicle control, and particularly relates to a vehicle-mounted image recognition and target detection system based on deep learning, and the system comprises a distributed monitoring module which collects the operation, obstacle and traffic signal information of a target vehicle through multi-modal classification and scene matching, completes the marking of a shielding region and the matching of information through the combination of shared data, and achieves the recognition of the target vehicle. Forming an enhanced monitoring set; the label planning module constructs an enhanced topological space based on the enhanced monitoring set, and adjusts moving tracks in different scenes by combining with vehicle and pedestrian track probability distribution fed back by dynamic intention recognition; the action recognition module predicts trajectory parameters and collision probabilities of non-target vehicles and pedestrians by using Bayesian and multi-modal algorithms; the decision-making module generates a real-time control instruction through particle swarm optimization and fuzzy control, and optimal control parameters are fed back through simulation; according to the invention, intelligent track planning and real-time control in a complex scene are realized, and the detection precision and control robustness of the shielded and label-free area are improved.

The invention discloses a human motion posture recognition method and system based on multi-modal data fusion, and relates to the technical field of posture recognition, and the method comprises the steps: firstly, synchronously obtaining a human motion posture video frame and an IMU segment, then carrying out the feature extraction of the two kinds of heterogeneous data in a shunt parallel mode, and carrying out the feature extraction of the two kinds of heterogeneous data; and respectively capturing visual space attitude information and dynamic inertial characteristics of the IMU. Afterwards, specific features of the modals are uniformly expressed through a mixed Token and embedding mechanism, and a space-inertia collaborative attention fusion mechanism is further introduced to realize dynamic association and deep fusion of cross-modal information; and finally, classifying the multi-modal fusion representation vector obtained by fusion so as to realize accurate recognition of the human motion posture. In this way, the defect that traditional fusion is insufficient in capturing subtle action differences can be overcome, and the accuracy and stability of action recognition in a complex scene are improved.

The invention relates to the technical field of shot action recognition, and discloses a shot throwing action recognition method based on a single attitude sensor. The method comprises the following steps: acquiring a single attitude sensor original data stream in a shot throwing process, and extracting a three-axis acceleration sequence and a three-axis angular velocity sequence from the single attitude sensor original data stream; performing motion event segmentation processing on the two sequences, and generating a key action time period mark set comprising a preparation stage, a sliding step stage, a power generation stage and a release stage; based on the mark set, calling an attitude feature solution algorithm to perform spatial trajectory reconstruction on the original data stream to obtain a key attitude frame set containing a joint angle change curve and a centroid displacement trajectory; and inputting the key attitude frame set into an action recognition model for attitude mode matching, and outputting a shot throwing action classification result which comprises an action deviation parameter generated based on a standard action template library and a key frame correction identifier. According to the method, key attitude information is reconstructed, and support is provided for shot throwing action analysis and improvement.

The invention relates to the technical field of data processing, in particular to an integrated multi-mode culture resource intelligent data governance and management system. The system comprises a data acquisition module, an intelligent processing module, a data management module, a knowledge organization module and an interactive display module. Synchronous acquisition of videos, audios, images and three-dimensional point clouds is realized through camera equipment, audio acquisition equipment and a laser scanner, and cultural elements are extracted through action recognition, speech recognition model processing dialect transcription, image semantic segmentation and named entity recognition by adopting a convolutional neural network. The system realizes structured description and semantic mapping of data on the basis of non-abandoned domain ontology, and constructs a knowledge graph by means of a graph database and a graph neural network. And finally, realizing three-dimensional visual display of the cultural resources by combining a virtual reality technology. According to the method, the problems of non-abandoned multi-modal data processing splitting, insufficient semantic organization and weak display interactivity in the prior art are solved, and the digital governance level and the propagation capability of cultural resources are improved.

The invention discloses a sign language recognition method and device based on multi-modal deep learning, and the method comprises the steps: multi-modal data input: capturing hand motions, gesture tracks and facial expressions at the same time through a camera, a motion capture sensor and other devices, and forming multi-modal data input; sign language action recognition: precise recognition of sign language actions is realized through a combined model of a deep convolutional neural network and a long-short-term memory network; facial expression and gesture track combined recognition: realizing understanding and translation of complex sign language sentences by combining the captured facial expressions and gesture tracks; and context natural language processing: generating a target statement in combination with context semantic understanding, and outputting a translation result. Through hand motion capture, facial expression analysis, gesture trajectory tracking and context natural language processing, complex sign language motions can be recognized more accurately and translated into characters or voices in real time, and low delay and high accuracy are achieved.

The invention discloses a method, a device and equipment for identifying illegal actions in a video and a storage medium, and relates to the technical field of computers. The method comprises the following steps: carrying out joint point detection on a target video stream to determine a joint point coordinate of each video frame, and generating a joint point coordinate sequence based on the joint point coordinates; inputting the articulation point coordinate sequence and the target video stream into a double-flow feature modeling network, and extracting articulation point spatial-temporal features of the articulation point coordinate sequence and image semantic features of the target video stream; and performing cross attention processing on the joint point spatial-temporal features and the image semantic features, and performing illegal action recognition according to the obtained fused features. It can be seen that the double-flow feature modeling network is utilized to extract the joint point spatio-temporal features and the image semantic features at the same time, the fused features rich in the context relation are generated through cross attention processing, illegal action recognition is conducted according to the fused features, and the accuracy of illegal action recognition in the video is improved through the multi-modal features.

The invention discloses a motion recognition method and system based on a redox photoelectric memristor, a terminal and a storage medium, and the method comprises the steps: selecting classical motions based on a human body motion data set, extracting time sequence data, and coding the time sequence data into an optical pulse sequence; constructing a reservoir array composed of a plurality of photoelectric memristors, and expanding an optical pulse sequence signal into a high-dimensional state vector; constructing a supervised training model, solving a weight matrix, and constructing a memristive cross array; and outputting an action classification result through simulation domain operation based on the output current multi-path light current signals in combination with the memristor cross array. According to the method, the motion features can be directly fed into a rear-end classification network for action recognition without depending on a complex digital feature extraction algorithm, so that the transmission and processing overhead of redundant data is fundamentally eliminated; a high-efficiency, low-delay and high-robustness hardware solution is provided for real-time and anti-noise motion recognition in scenes such as intelligent monitoring and man-machine interaction.

The invention relates to the technical field of image segmentation, in particular to an intensive care unit video image processing method based on image semantic segmentation, which comprises the following steps of: acquiring a video image through monitoring camera shooting, dividing a semantic region, calculating a pixel displacement direction of each part, identifying an abnormal track, erasing interference, recombining a boundary contour, and comparing shape change. And constructing a trend and re-drawing a structure, evaluating stability in combination with directions, speeds and amplitudes, screening and correcting inconsistent labels, and outputting an index result. According to the method, track features are constructed by introducing pixel direction coding, a deviation region is identified and interference is eliminated in combination with a direction change trend, an edge structure is reconstructed by using a boundary connection sequence and curve fitting, contour division is optimized through deformation direction consistency, and labels are corrected by synthesizing direction change and boundary speed. Dynamic tracking and accurate labeling of the patient state are achieved, the abnormal action recognition efficiency and the label updating accuracy are improved, and the time sequence integrity and the space expression ability of image data are enhanced.

The invention discloses a dance movement recognition method, device and equipment based on dynamic time warping and a computer readable storage medium. The method comprises the steps of obtaining a user skeleton point sequence corresponding to a user dance movement and a standard skeleton point sequence corresponding to a standard dance movement; for each pair of frames in the user skeleton point sequence and the standard skeleton point sequence, multi-dimensional differences including joint point position differences, angle differences and motion amplitude differences are calculated; a double-layer dynamic weighting mechanism is applied to perform weighted fusion on the multi-dimensional difference values, and local cost between two frames is calculated; constructing a cost matrix based on local cost, and determining an optimal regular path and a dance motion similarity score by applying a path constraint dynamic time warping algorithm; and on the basis of the dance motion similarity score and the optimal regular path, generating an identification report including an overall score, a subitem score and visual correction information. The method has the advantages of being high in dance movement recognition precision, high in specialty and the like.

The invention relates to the technical field of action recognition, in particular to a pattern rope skipping action recognition system and method, and the system comprises a limb angle collection module, a stability recognition module, an action triggering and screening module, a rope body direction analysis module and a cross mapping judgment module. According to the method, by introducing three-dimensional coordinate extraction and included angle change trend analysis of limb key joints in the jumping period, the dynamic characteristics of limbs in the action process can be accurately reflected, and the action amplitude and stability change can be effectively expressed through an angle data sequence constructed by multi-dimensional indexes such as the maximum value, the minimum value and the average value of the included angle in the period; a stability identification mechanism is constructed to carry out local regression fitting and residual error statistics on a periodic angle trend, accurate labeling of a stable state of an action posture is realized, and then a key frame sequence with an actual jump intention and a continuous posture in the jump action is effectively identified through fusion screening of labeled frames and a jump-empty state.

The invention discloses a zero sample skeleton behavior recognition method based on cross-modal progressive interaction. The method comprises the following steps: S1, processing an input skeleton sequence by adopting a pre-trained graph convolutional network, and extracting skeleton features with space-time characterization capability; s2, constructing a shared set containing M semantic attributes, and forming a standardized attribute feature set; s3, inputting the skeleton features and the attribute feature set into a semantic attribute decoupling module to obtain an attribute feature sequence aggregated with the skeleton features and a correlation matrix used for feature aggregation; s4, inputting the attribute feature sequence output in the S3 and the skeleton feature with the space-time representation capability output in the S1 into an attribute perception enhancement module, and outputting a skeleton enhancement feature with the length consistent with that of the skeleton sequence; and S5, carrying out average pooling on the skeleton enhanced features for training and testing, and finally realizing classification of behavior data samples with unknown categories. According to the method, efficient zero sample recognition can be realized only by optimizing a small number of interaction parameters.

The invention relates to a sign language action recognition problem, and provides a sign language recognition method based on a millimeter wave radar 3D point cloud, sign language recognition is carried out in a sequence-to-sequence mode through a non-contact FMCW radar, and the sign language recognition method mainly comprises a first part, namely a continuous sign language action recognition method based on a millimeter wave radar; a second part: establishing an evaluation index STT Score of radar continuous sign language action recognition; according to the invention, a low-cost portable wearable acquisition device is adopted, so that a subject can use the device anytime and anywhere; meanwhile, millimeter-wave radar detection does not depend on light rays, the calculated amount is small, the privacy of a user is effectively protected due to the fact that signals collected by the millimeter-wave radar are radar radio frequency signals, and meanwhile the use feeling is good and the cost is low due to the non-contact collection mode of the millimeter-wave radar.

The invention relates to the field of gesture visual recognition, in particular to a touch state detection method and system based on hand key points. The method comprises the following steps: acquiring a hand image of a user, performing super-resolution reconstruction, and constructing a super-resolution image; performing hand feature point recognition and point cloud modeling on the super-resolution image, and constructing a real-time hand posture model; gesture action recognition is carried out on the real-time hand gesture model, and a plurality of complete gesture action results are recognized; performing gesture action virtual remodeling based on the plurality of complete gesture action results, and constructing gesture action virtual behavior data; and performing instant gesture state semantic analysis according to the gesture action virtual behavior data, and outputting an instant gesture semantic result. By efficiently and accurately recognizing the operation gesture of the user, the touch recognition response speed and the non-contact touch experience of the user are improved.

The invention provides an interaction method based on finger pinching action recognition, electronic equipment and a storage medium. The method comprises: acquiring an image in real time; determining the type of finger pinching gestures in a preset number of continuously acquired frame images; when same gesture images in the preset number of frame images meet a target condition, the finger pinching gesture is determined to be an effective finger pinching gesture, the same gesture images are images with finger pinching gestures of the same type, and the target condition includes that the number of the same gesture images is larger than or equal to a first number; determining a man-machine interaction instruction corresponding to the effective finger pinching gesture; and executing the man-machine interaction instruction. According to the method, the pinching gesture is taken as a gesture instruction in the man-machine interaction process, so that the pinching gesture is more difficult to trigger by mistake compared with a common gesture, and the safety of man-machine interaction is enhanced.

The invention discloses an action recognition method based on a virtual inertial measurement signal generation model, and relates to the technical field of action recognition, and the method comprises the steps: collecting a surface electromyogram signal and an inertial measurement signal, and carrying out the preprocessing of the collected signals; constructing and training a generator to convert the surface electromyogram signals into virtual inertial measurement signals; inputting the generated virtual inertial measurement signal and the original surface electromyogram signal into an action recognition model, and training to obtain a classification module; the performance of an action recognition model is evaluated by using a test data set, the accuracy rate, the recall rate and the F1 score index are calculated, the recognition effect of the model is measured, a classification model and a generator are optimized according to the recognition effect, and the advantages of two modal signals can be fully utilized by fusing surface electromyogram signals and virtual inertial measurement signals, so that the accuracy of the action recognition model is improved. And the motion information of the fingers, the wrist, the forearm and other parts is analyzed, so that the complex limb motion is identified more accurately, and the performance of the limb motion identification system in practical application is improved.

The invention relates to a method, a system and equipment for intelligently identifying unsafe behaviors of coal mine operating personnel and a medium. The method comprises the following steps: extracting individual trajectory data from an underground video stream through a multi-target tracking algorithm, decomposing continuous actions into an atomic behavior sequence with a space-time mark by using attitude estimation and a space-time diagram convolutional network, and capturing space-time relevance of a behavior chain through an attention mechanism enhanced long and short term memory network to generate a feature vector; risk reasoning is carried out in combination with the coal mine safety knowledge graph to predict the risk event type and probability, and finally early warning information is generated based on a multi-level early warning strategy. According to the scheme, the crossing from isolated action recognition to behavior chain risk prediction is realized, and the early warning capability of potential safety risks in coal mine operation is remarkably improved through fusion of space-time correlation analysis and domain knowledge of the behavior sequence.

The embodiment of the invention relates to the technical field of man-machine interaction, and discloses a joint data feedback method and device based on a flexible sensor, a medium and a product. The method comprises the following steps: determining a time sequence calibration feature set according to a multi-source synchronization signal of a flexible sensor array; according to the time sequence calibration feature set, determining a joint function evaluation result and an action classification result; in combination with the joint function evaluation result, according to the deviation between the action classification result and a preset biomechanical standard library, joint dynamic function deviation data are determined; determining a biomechanical feedback instruction of the wearable device according to the joint dynamic function deviation data; wherein the biomechanical feedback instruction is matched with the functional state corresponding to the joint function evaluation result, and the method and device can be at least used for solving the technical problems that in the related technology, joint functions and actions are not accurate in recognition, and precise personalized control is difficult to achieve.

The invention relates to the technical field of action recognition, in particular to a competitive sports-oriented skeleton trajectory deep learning compensation method and system, and the system comprises a data preprocessing module, a data integrity evaluation module, a core compensation network, a biomechanical rationality optimization module, a personalized habit encoder and an advanced compensation path. Compared with the prior art that a single trajectory compensation model is generally adopted, challenges of shielding scenes of different degrees are difficult to effectively deal with, and the problem of insufficient short-time shielding compensation precision or trajectory distortion under long-time shielding often occurs, an intelligent routing mechanism based on data integrity evaluation is adopted, and the method has the advantages that the complexity is reduced, and the reliability is improved. Through dynamic switching of a basic compensation path and an advanced compensation path fused with personalized prediction, adaptive processing of different shielding scenes is realized, reconstruction precision under short-time shielding is ensured, track rationality and continuity under a long-time shielding scene are remarkably improved through motion trend prediction, and the reconstruction precision is improved. And the practicability and the reliability of the system in an actual competitive environment are enhanced.

The invention provides a system for controlling a manipulator in real time through hand electromyographic signals, which comprises a data acquisition unit, a terminal equipment unit and a tail end execution unit which are in communication connection through serial ports. The data acquisition unit comprises a data acquisition device and a signal acquisition module, and is configured to acquire a multi-channel electromyographic signal of a hand of a user in real time and perform signal conditioning and digitization so as to obtain a digital signal sequence representing a muscle activity intention of the user; the terminal equipment unit comprises a data preprocessing module, a model training module, a gesture recognition module and an instruction generation module; the tail end execution unit comprises a manipulator control unit which is in communication connection with the control output module and is configured to be used for receiving the manipulator control instruction and driving a manipulator to execute corresponding actions; in combination with a control method, the problems of insufficient real-time performance, poor hand action recognition precision and robustness, unstable control instruction, poor user adaptability and the like in the prior art are solved, and the man-machine interaction performance is improved.

The invention relates to the technical field of image processing, and discloses a mine personnel behavior recognition method and system based on YOLOv5. The method comprises the following steps: acquiring a video frame sequence of a mine site, extracting an environment interference vector from the video frame sequence, and optimizing the video frame sequence based on the environment interference vector to obtain a clear frame sequence; human body key point coordinates and posture vectors are extracted from the clear frame sequence, and time sequence changes of the key point coordinates are analyzed to obtain preliminary action features; calculating a similarity score between the preliminary action feature and a preset standard template, and refining the preliminary action feature based on the similarity score to obtain a refined action component vector; and matching the refined action component vector from a preset action library to obtain an action recognition result, and generating corresponding action response output based on the action recognition result. According to the invention, the accuracy of personnel action recognition in a complex mine environment is improved.

The invention discloses a light-weight few-sample man-machine interaction action recognition method, system and equipment, and belongs to the technical field of video processing and computer vision. The problems that an existing method is large in parameter quantity, high in computing resource requirement and difficult to meet the real-time performance are solved, the spatial-temporal features of the action fragments are extracted through the lightweight deep neural network, dynamic and scene information is fused, the classification model is optimized by combining sliding collection and the cycle completion technology, low-time-delay and high-robustness action recognition is achieved, and the real-time performance is improved. The method is suitable for real-time monitoring of industrial processes, does not need a large amount of labeled data, and reduces the calculation complexity. By analyzing the difference between action transition and normal action and utilizing a Gaussian mixture model and Bayesian optimization to dynamically adjust a threshold value, the accuracy and robustness of action recognition are improved, high-quality data support is provided for model training and action recognition, and the automation level of action segmentation is remarkably improved.

The invention discloses an intelligent wheelchair control method based on multi-source information fusion, and belongs to the field of data recognition and wheelchair intelligent control. The implementation method comprises the following steps: constructing a corresponding relation between gesture actions and wheelchair control directions, and presetting an identification framework; processing the obtained multi-source information to obtain a basic probability distribution result of the evidence corresponding to each single-source information; determining the absolute importance of the evidence corresponding to each piece of single-source information in the fusion process; determining the relative importance of the evidence corresponding to each piece of single-source information in the fusion process; determining the comprehensive importance of the evidence according to the obtained relative importance and absolute importance; according to the comprehensive importance of the evidence corresponding to the single-source information, redistributing a fusion weight for the evidence corresponding to each piece of single-source information to obtain a weighted new evidence; dS evidence fusion is applied, a final fused gesture action recognition result is obtained, the behavior of the wheelchair is controlled according to the gesture recognition result, and intelligent wheelchair control based on multi-source information fusion is achieved.

The invention belongs to the technical field of intelligent sports and computer vision crossing. The adaptive training method based on Tai Chi action recognition is provided, and space action features are extracted according to three-dimensional human skeleton key point coordinates of a human body; extracting multi-scale motion features of the human body in a time domain according to the spatial motion features; global attention aggregation is carried out according to the multi-scale motion features of the human body in the time domain, and a human body motion recognition result is obtained; according to a human body action recognition result, matching a corresponding Tai Chi style drawing theory; superposing multi-dimensional visual prompts into a Tai Chi motion video according to the Tai Chi motion style theory so as to intuitively prompt a user about action key points; calculating an action problem based on priori knowledge or a preset template, prompting an error position and an error reason by adopting graphs and / or characters, and giving an adjustment suggestion; according to the invention, more personalized training experience can be provided, cognitive load can be reduced, and training efficiency, training effect and cognitive learning fluency can be improved.

The invention belongs to the technical field of computer vision and action recognition, and discloses a table tennis swing key frame recognition method and system, and the method comprises the steps: carrying out the posture estimation of a table tennis motion video frame by frame, extracting twelve skeleton key points including shoulders, elbows, wrists, hips, knees and ankles, and carrying out the recognition of the key frames; normalization is completed with the shoulder midpoint as a translation reference and the shoulder width as a scaling factor, and a standardized skeleton sequence is obtained; sequentially inputting the sequence into a three-layer one-dimensional convolutional network to extract local features, adding a learnable position code after convolution output, and then inputting a time sequence modeling network formed by six layers of Transformer encoders to capture a long-time dependency relationship; and finally, outputting five types of results of forehand shooting, forehand swing, backhand shooting, backhand swing and non-key frames through a two-layer full-connection network. The system is composed of a data processing module, a feature modeling module and a key frame recognition module, and stable recognition of table tennis swing key frames can be achieved.

The invention relates to the technical field of body posture recognition, and particularly discloses an automatic body posture recognition system based on video analysis, which comprises a video acquisition module, a preprocessing module, a key point detection module, a skeleton tracking module, an action recognition module and a cross-view fusion module. According to the scheme, a parallel detection and information fusion mode is adopted, and the detection precision of the key points of the human body is remarkably improved by utilizing the accurate position of the high-resolution feature and the rich semantics of the low-resolution feature; the target is tracked according to the topological structure and the key point position of the human skeleton, the method has extremely high robustness for illumination and dressing changes, and a more stable and continuous human skeleton movement track is generated; the time sequence convolution and the space-time diagram convolution are combined to carry out joint modeling on the space and time double-domain features, complete action representation is obtained, and the stability, the accuracy and the adaptability of body posture recognition are effectively improved.

The invention relates to the technical field of sports biomechanics, in particular to an automatic track and field action recognition and capture method based on image data, which comprises the following steps: synchronizing a visible infrared image, a polarization image and an event stream according to a mutual information criterion and generating a polarization phase diagram; inputting a polarized neural radiation field network and a skeleton graph neural network based on Lie group message passing, and alternately optimizing to obtain a continuous voxel light field and a joint rotation sequence; performing Vietoris-Rips persistent homology and third-order path signature on the sequence to extract topological path features, encoding the topological path features into a pulse sequence to drive a liquid state machine, and generating a semantic confidence flow by combining bone potential tensor and action text embedding; and constructing a saliency curve by using the voxel light field time derivative, the neuromorphic difference, the semantic complementary value and the skeleton difference, carrying out strategy gradient on-line adjustment on the weight, taking curve minimum mapping as a key frame index, and outputting an image frame. The track and field action key frames can still be accurately captured in real time under complex illumination and shielding conditions.