37 results about "Movement recognition" patented technology

The invention relates to a simulation digital human real-time intelligent voice interaction system and a simulation digital human real-time intelligent voice interaction method based on vision and a large model, and aims to solve the problems of inaccurate target speaker recognition, high response delay and the like in digital human voice interaction in a complex scene. The system circles an effective recognition range through a camera, triggers audio collection in combination with face detection, locks a target speaker and reduces noise by using lip movement recognition and sound image fusion technologies, converts the target speaker into a text through voice wake-up, generates an answer by means of a large language model (LLM) and knowledge retrieval enhancement (RAG) technologies, generates low-delay voice through a voice synthesis technology accelerated by the vLLM, and performs voice recognition on the target speaker. And driving the preloaded digital human image to synthesize a video stream and pushing the video stream to a front end for rendering in real time. Accurate pickup, low-delay interaction and rapid digital human image switching in a complex environment are realized, the accuracy and real-time performance of intelligent voice question answering are improved, and the method is suitable for government affair halls, exhibition halls and other scenes.

The invention discloses an automatic classroom director method and electronic equipment. Audio data and video data of a current classroom scene are acquired; based on the audio data, sound source direction positioning is carried out through a microphone array, and the current sound source direction is determined; face key points are extracted according to the video data, lip movement recognition is carried out according to the face key points, and a lip movement recognition result is obtained; extracting human body key points according to the video data, and performing action recognition according to the human body key points to obtain a target action recognition result; based on the sound source direction, the lip movement recognition result and the target action recognition result, a current shooting picture of the camera device is controlled, and the current shooting picture of the camera device is a director picture; and outputting and displaying the director picture. According to the application, audio and video multi-mode information is fused, high-precision identification and natural and accurate automatic picture switching of the speaker and the interaction object in the classroom scene are realized, and the automation level of director and the overall classroom recording and broadcasting effect are improved.

The invention discloses a personnel state analysis method and system based on limb movement recognition, relates to the technical field of limb movement recognition, and determines the movement state of a corresponding limb part based on recognition of each movement-expression combination. And the driving response state of the personnel is determined based on the action state of each limb part, the corresponding part priority and the driving state of the truck, so that the accuracy of the driving response state of the personnel is improved. Therefore, the change event of the facial expressions is determined according to the plurality of facial expressions at different time, and the driving fatigue state of the personnel is determined according to the change event of the facial expressions and the lane changing frequency of the truck; according to the method, the voice interaction event of the person in the driving process is collected, the multiple state key contents are determined according to recognition of the voice interaction event, the state analysis system of the person is determined according to the multiple state key contents, the driving response state of the person and the driving fatigue state, and the accuracy of the state analysis system of the person is improved.

The invention provides a backpack multi-sensor data fusion-based motion attitude anomaly real-time detection method, which comprises the following steps of: acquiring multi-channel original data through a backpack-type integrated triaxial accelerometer, a gyroscope, a magnetometer and a barometer, implementing synchronous sampling and time-space alignment, and constructing a standardized time sequence data stream through wavelet denoising and multi-dimensional normalization preprocessing; high-dimensional features representing motion differences are extracted, nonlinear dimensionality reduction is achieved through principal component and Laplacian feature mapping, and a motion mode prototype library is generated; according to the method, a new motion situation can be continuously and adaptively learned, and the motion recognition accuracy, the system robustness and the attitude anomaly detection capability in a complex scene are effectively improved.

Aspects relate to a virtual reality management technique for facilitating precise long-range and short-range movement in a virtual environment while enabling complex multi-task actions. The virtual reality management technique includes determining, by analyzing the set of video data using a movement recognition model, a short-range movement action for an avatar in the virtual reality environment in a case that a set of movement factors detected for both a first arm and a second arm of the user satisfy a predetermined movement threshold and executing the short-range movement action, determining, by analyzing the set of video data using the movement recognition model, a long-range movement action for the avatar in the virtual reality environment in a case that a first hand of the user corresponds to a pointing gesture and a confirmation action is received from the user, and executing the long-range movement action.

This invention relates to the interdisciplinary field of image recognition and intelligent medical technology, specifically disclosing a method for orthopedic rehabilitation movement recognition and assessment based on image recognition. This method acquires patient motion images using a non-invasive multi-view visual sensor, extracts the three-dimensional coordinates of key skeletal points, and constructs a parametric biomechanical digital twin model integrating joint range of motion, muscle force lines, and ligament tension mechanisms. The key point data is input into a graph neural network for posture recognition, and the digital twin model is simultaneously invoked for physical feasibility verification and posture correction. Finally, the spatiotemporal trajectory similarity is calculated using a standard rehabilitation movement template, generating a comprehensive assessment result including movement completion, safety level, and injury risk. This invention, through the above technical solution, improves the robustness, physiological rationality, and clinical safety of movement recognition.

This invention relates to a scoring system and method for aerobics competitions, belonging to the field of sports competition scoring. The system includes a data acquisition module, a first data processing module, a second data processing module, a core calculation module, and an output module. The data acquisition module acquires scoring data input by the art, execution, and difficulty judging groups and the chief judge. The first data processing module performs arithmetic mean calculations on the art, execution, and difficulty scores respectively to generate average scores for each item. The second data processing module sums the points deducted by the chief judge and the points deducted based on the line of sight to generate the total points deducted by the chief judge. The core calculation module sums the average scores for each item with the total points deducted to generate the total score for the routine. The output module displays the total score. Through automated data calculation, combined with difficulty suggestion scoring provided by the movement recognition module, data compliance ensured by the verification module, and full traceability achieved through the storage module, the system improves the accuracy, efficiency, and objectivity of scoring, and enhances the transparency and credibility of the scoring.

The present application takes the teacher's hand gesture and upper limb movement as the research object, and discloses a teacher teaching intention detection method and system based on staggered attention for the auxiliary role of computer vision technology in the teaching scene. The method comprises the following steps: 1) respectively acquiring the teacher's hand gesture and upper limb movement video stream under the RGB-D camera and 3D structured light camera in the teaching environment; 2) respectively pre-processing the image data stream. The collected video stream is converted into teacher gesture, teacher upper limb movement key point heat map data stream and teacher gesture, teacher upper limb movement structured light image data stream; 3) the pre-processed data stream is respectively input into the teacher hand gesture recognition model and the upper limb movement recognition model. 4) after model prediction, the teacher hand gesture recognition result and the teacher upper limb movement recognition result are respectively output. The teacher gesture and the upper limb movement recognition result are combined to judge the teaching gesture. The present application uses a teacher teaching gesture estimation model based on a staggered attention mechanism, so that the network can recognize 3D teaching gestures according to the key point heat map when predicting the teacher gesture. At the same time, by combining the teacher upper limb movement recognition result with the gesture recognition result, the detection of the teacher's teaching intention is realized, and reference support is provided for the non-verbal ability evaluation of the teacher's teaching process.

This application provides a method, system, and device for monitoring intelligent knee braces. The method is applied to an intelligent knee brace, which includes a motion data acquisition device comprising multiple sensors. The method includes: acquiring motion data of a target user (a user wearing the intelligent knee brace) collected by each sensor; based on the motion data, identifying basic movement types using a basic classification algorithm model to obtain basic movement recognition results; if the basic movement recognition results are not composite movements, using the basic movement recognition results as the target user's movement state; if the basic movement recognition results are composite movements, further identifying subdivided movement types using a subdivided movement algorithm model to obtain the target user's movement state.

The invention relates to a physical education teaching and training all-in-one machine integrating AI motion recognition and voice interaction, belongs to the field of physical education teaching and intelligent motion equipment, and solves the problems that existing campus motion equipment is single in function and lacks intelligent guidance. The device comprises a machine body shell, a core function module, an AI motion recognition module, an AI voice large model dialogue module, a data statistics module, a display module and a storage module. The core function module covers free training, classroom teaching and other scenes; the AI motion recognition module collects images and corrects postures; the AI voice large model dialogue module realizes intelligent dialogue; the data statistics module analyzes the score; the display module displays the information; and the storage module is used for storing data. The all-in-one machine integrates multiple functions, supports multi-person interaction, improves the standardization and interestingness of sports, and meets the requirements of campus physical education.

A method for lower limb movement recognition based on surface electromyography (EMG) decoding using S-transform energy concentration is proposed. First, the raw EMG signal is preprocessed. The preprocessed signal is then truncated to a useful time segment using endpoint detection. Next, the signal within this time segment undergoes segmented S-transform and energy concentration calculation. Segmented operations are used to extract signal features of a specified dimension. Movement pattern classification is performed using SVM, and multi-channel signal features are fused and analyzed for lower limb movement recognition. This invention performs segmented S-transform on the signal to preserve its temporal information, while optimizing the energy concentration calculation process to improve movement classification accuracy. An SVM multi-classifier is built, and the accuracy of lower limb movement pattern recognition is further improved through S-transform energy concentration and feature fusion.

The application discloses a lip movement recognition method and device based on three-dimensional lip reconstruction, and the method comprises the following steps: acquiring a lip reading video to be recognized; generating a lip image to be recognized; inputting the lip image to be recognized into a target three-dimensional lip reconstruction model to obtain three-dimensional lip data to be recognized; and inputting the three-dimensional lip data to be recognized into a target lip movement recognition model to obtain a lip movement recognition result. Wherein, the method comprises the following steps: acquiring a lip language video; generating a lip image to be trained and text information; constructing an initial three-dimensional lip reconstruction model; constructing an initial lip movement recognition model; training the initial three-dimensional lip reconstruction model to obtain a target three-dimensional lip reconstruction model; inputting the lip image to be trained into the target three-dimensional lip reconstruction model to obtain three-dimensional lip data to be trained; and training the initial lip movement recognition model to obtain a target lip movement recognition model. The application realizes lip movement recognition, improves the accuracy and robustness, and can be widely applied to the technical field of visual speech recognition.

The present application relates to a device and method for controlling vehicle motion. In the vehicle motion control device and its control method, it can provide vehicle motion synchronized with passenger action in the vehicle based on passenger action, the recognizer is configured to obtain passenger motion information from the image in response to the case that the image including the passenger in the vehicle is input, and determine whether there is passenger dance action; the determiner is configured to obtain vehicle state information in response to the case that there is passenger dance action in the image, to determine whether vehicle motion control can be carried out, based on the determination that vehicle motion control can be carried out, the characteristics of passenger dance action are extracted; the controller is configured to generate target vehicle motion based on the characteristics, and control the vehicle motion by driving the actuator corresponding to the target vehicle motion.

The application provides a kind of eight-sections palm-chi movement evaluation method and system based on dynamic programming and deep learning, including collecting eight-sections palm-chi movement video data;Through the eight-sections palm-chi movement recognition model based on MediaPipe and DTW, the matching path, DTW distance and single frame score are obtained, the feature sequence of pivot action output by the eight-sections palm-chi movement recognition model is uniformly frame extracted and processed and then input into the eight-sections palm-chi movement evaluation model based on LSTM to obtain the classification label of user pivot action, and the error of user in key action is reflected according to the classification label. The skeleton points are extracted through the Mediapipe framework, the total score is obtained according to the DTW distance, the similarity between the overall movement of user and the standard action is evaluated according to the rhythm score, and the rhythm stability of the movement can be reflected in detail according to the single frame score;The problem of unstable length of video sequence is solved by using the uniform frame extraction method, and the model effectively learns and recognizes the action features.

1. Name of the designed product: Livestock mandibular movement detection device. 2. Use of the designed product: Livestock mandibular movement identification and analysis application. 3. Design points of the designed product: In shape. 4. Picture or photo best indicating the design points: Perspective view.

The application discloses a kind of based on LSTM model identification human motion and electric stimulation's wearing training equipment, it is related to medical health technical field, including: intelligent insole sensor system is used to collect and upload the sensor time series data of foot when human activity;LSTM network model human motion recognition system is used to receive data and the intelligent insole sensor time series data is preprocessed, the data after processing is input to LSTM network model, and the motion mode of the motion of human motion and the behavior of kicking are identified;Electric stimulation system releases current pulse to foot bottom part according to the motion mode of the motion of human motion and the behavior of kicking identified.The application innovatively introduces LSTM neural network technology, gives the intelligent motion mode identification ability of electric stimulation equipment, can dynamically adjust electric stimulation parameter according to real-time motion state, can also predict kicking action in advance, through accurate electric stimulation auxiliary wearer optimizes kicking effect, improves training efficiency and motion performance.

The utility model discloses an eight-segment brocade motion identification auxiliary device based on multi-modal data fusion, which comprises a wearing bandage, an integrated module, a power supply module, a feedback module and an image analysis module, the wearing bandage is provided with a magic tape, a mounting cavity and a buffer layer are arranged in the wearing bandage, and the integrated module comprises a nine-axis attitude sensor, a data processing unit and a Bluetooth unit. The motion data can be collected, processed and sent to an external terminal, the image analysis module is composed of a camera and a processor and can capture motion images, position joint points and calculate the similarity with standard motion, the power supply module is a bendable lithium polymer battery, and the feedback module is provided with a vibration unit and a terminal graphical interface and achieves tactile and visual feedback. And the data processing unit and the image analysis module synchronize data through Bluetooth and fuse the data to generate a comprehensive evaluation result. In use, the device can assist a user in accurately identifying eight-segment brocade actions, provides comprehensive action evaluation through multi-modal data fusion, helps the user to correct the actions in time, and improves the exercise effect.

The application discloses a kind of operation information display method and electronic equipment.Method includes: in response to function trigger operation, show motion recognition page;Wherein, motion recognition page includes target object real-time image information;If the motion attribute information of target object in target object real-time image information is identified to meet preset condition, then show target object real-time image information and the motion attribute information of target object in motion recognition page.The application realizes the same screen partition display of motion image and motion information, and motion image information is used for target object to observe own action posture and calibrate in real time, and motion attribute information synchronously shows relevant information such as motion time, motion type, target object can obtain visual feedback and data feedback simultaneously without switching interface, improve the coherence of training process, realize the adaptive update of interface state, simplify operation path, improve man-machine interaction frequency and user experience.

A dynamic vision sensing system, which includes a dynamic vision sensor, a AI recognition module connected to the dynamic vision sensor, and a static mode triggering module coupled to the Ai recognition module. The static mode triggering module is adapted to trigger an intensity change in an environment captured by the dynamic vision sensor to observe a static object of interest, a property that conventional dynamic vision sensor failed to capture. The motion recognition module upon detecting a change of a motion in the environment is adapted to send a command to the static mode triggering module to trigger the intensity change. An AI controlled electro-optical system is therefore proposed to capture object of interest even if the object is static by triggering effective intensity change of the static object at sensing end, hence eliminate blind spot of existing DVS system.

The invention discloses continuous laser processing equipment for abrasive paper sheets and a continuous processing method of the continuous laser processing equipment, and belongs to the technical field of abrasive paper processing equipment.The continuous laser processing equipment comprises a continuous conveying device and a laser processing device, and the continuous conveying device comprises a supporting frame, a conveying mechanism arranged on the supporting frame and a motion recognition mechanism; and the motion recognition mechanism is used for acquiring position information of the abrasive paper sheet on the abrasive paper die holder and controlling laser processing based on the position information. According to the abrasive paper sheet feeding device, continuous and automatic feeding and circulation of abrasive paper sheets are achieved, and the machining efficiency is remarkably improved; meanwhile, the sheet is stably adsorbed in a machining area, so that displacement in the machining process is effectively prevented, and the hole site machining precision and consistency are ensured; and finally, a motion recognition mechanism obtains position information of the abrasive paper sheet in dynamic conveying in real time, synchronous flight machining is completed in combination with a position compensation algorithm, and the deviation between the image capturing position and the actual marking position is overcome.

The application relates to a voice recognition enhancement and multi-modal interaction method for a smart cockpit, which comprises the following steps: acquiring multi-modal data of a driver, performing multi-modal recognition on the multi-modal data, and acquiring voice recognition text and lip movement recognition text of the multi-modal data; taking a first candidate probability in an N-best candidate word list corresponding to the voice recognition text and the lip movement recognition text as global confidence, which is used to define a voice modal weight and a lip movement modal weight; performing basic probability distribution and combination by using the voice modal weight and the lip movement modal weight, acquiring a highest fusion trust degree as a result of voice recognition enhancement; and inputting the result of the voice recognition enhancement into a locally deployed artificial intelligence large model to generate corresponding control instructions or natural language replies. The application solves the problems of insufficient reliability of current smart cockpit voice recognition in a noisy environment and insufficient diversity of interactive functions.

The invention discloses an upper limb motion recognition method and system based on a time-frequency feature brain muscle fusion graph, EEG and EMG signals are synchronously collected through a multi-channel device, and after band-pass filtering, notch filtering and standardized preprocessing, a time-frequency feature matrix is extracted by adopting FSWT; the method comprises the following steps: designing a decision-level GCN fusion architecture, a feature-level GCN fusion architecture and a structure-level GCN fusion architecture, constructing an optimal graph topology and training a model by combining five edge weight calculation methods of cosine similarity and transfer entropy with Top-k screening, deploying a lightweight GCN, and realizing real-time reasoning with a window of 0.25 second and an overlapping rate of 50%. According to the method, complementary information of EEG and EMG is fully utilized, signal space topology characteristics are adapted, high precision, low complexity and scene flexibility are achieved, the method is applied to the fields of rehabilitation medicine and auxiliary robots, and the upper limb motion intention is recognized in real time.

To provide a motion recognition device, a method and an electronic device.SOLUTION: The method includes performing key-point recognition on an object in a video frame using a neural network, obtaining key-point information and a PAF score of the object, making key-point connections based on the key-point information and the PAF score, generating a plurality of key-point connection candidates based on the key-point connection results, determining whether one of at least two key-point connection candidates of the plurality of key-point connection candidates is valid to perform selection among the plurality of key-point connection candidates, as well as recognizing the motion of the object based on the selected key-point connection candidates.SELECTED DRAWING: Figure 1

The invention belongs to the technical field of floating crane equipment in the maritime work industry, and particularly relates to an offshore pile lifting swing restraining device with an automatic adjusting function and a swing restraining method, and the device comprises a pile extractor connecting flange plate which is installed on a pile extractor and is a circular plate; the four steel cable lifting and lowering units are arranged on the lower portion of the pile extractor connecting flange plate at equal intervals in the circumferential direction with the center of the pile extractor connecting flange plate as the circle center, the length of the steel cable lifting and lowering units can be adjusted, and the four steel cable lifting and lowering units jointly hoist the swing restraining ball; the oil supply unit is located in the center of the lower portion of the pile extractor connecting flange plate and used for injecting oil into the swing restraining ball. The motion recognition detection unit comprises an original point detection unit and a motion recognition monitoring point P, the original point detection unit is located in the center of the lower portion of the pile puller connecting flange plate, and the motion recognition monitoring point P is located at the top of the swing suppression ball. The method is suitable for solving the technical problem that in the offshore pile hoisting operation process affected by wind power, the construction progress is slow due to swinging during pile hoisting positioning and butt joint of a pile feeder and a foundation pile.

This invention discloses a method and system for assessing the degree of clinical injury based on medical imaging. The method includes: acquiring image sequences of clinical users; performing auxiliary injury assessment on the image sequences to obtain a first injury assessment result; based on the image sequences, performing user movement recognition and medical image acquisition quality analysis to obtain medical image quality influence parameters; configuring the number of medical image acquisitions and recognition resource coefficients according to the medical image quality influence parameters; acquiring medical images of the user according to the number of medical image acquisitions to obtain a medical image set; using the recognition resource coefficients to identify the medical image set, fusing and processing to obtain a second injury assessment result; and combining the first injury assessment result to obtain a final clinical injury assessment result. By analyzing patient movement characteristics, dynamically adjusting medical image acquisition parameters, and fusing multiple assessment results, the accuracy of clinical injury assessment is improved.

The application relates to a muscle movement recognition method and a surface electromyogram signal collection device, and the method comprises the following steps: collecting historical surface electromyogram signals by using the surface electromyogram signal collection device; processing the historical surface electromyogram signals to obtain a surface electromyogram signal feature subset; constructing a convolutional neural network model; training the convolutional neural network model by using the surface electromyogram signal feature subset to obtain a muscle movement recognition model; and recognizing muscle movement according to the muscle movement recognition model. The device comprises an electromyogram signal sensor, a low-pass filter, an operational amplifier, an analog-to-digital converter and a wireless transmission module. The surface electromyogram signal feature subset is obtained by processing historical surface electromyogram signals, and the muscle movement recognition model is obtained by training the surface electromyogram signal feature subset; therefore, the muscle movement can be recognized, and the recognition process is faster and more accurate.

The application relates to a visual and large model-based simulation digital human real-time intelligent voice interaction system and a method thereof, and aims to solve the problems of inaccurate target speaker recognition and high response delay in digital human voice interaction in a complex scene. The system effectively identifies the range through a camera ring, triggers audio collection in combination with face detection, locks the target speaker by using lip movement recognition and sound image fusion technology and reduces noise, generates an answer by means of a large language model (LLM) and knowledge retrieval enhancement (RAG) technology after the voice is converted into text, generates low-delay voice by using vLLM accelerated voice synthesis technology, drives a preloaded digital human image to synthesize a video stream and pushes the video stream to a front end for real-time rendering. The application realizes accurate sound pickup, low-delay interaction and fast switching of the digital human image in a complex environment, improves the accuracy and real-time performance of intelligent voice question and answer, and is suitable for scenes such as government offices and exhibition halls.

The application discloses an interactive system based on lower limb movement, comprising a flexible insole, a motion analysis end, a mapping end and a driving end; the motion analysis end receives lower limb movement data from the motion insole, and identifies lower limb movement information containing primary movement and secondary movement from the lower limb movement data by using a neural network based on machine learning, and sends the lower limb movement information to the mapping end; the mapping end receives the lower limb movement information, and generates mapping movement instructions suitable for the driving end according to the lower limb movement information; and the driving end receives the mapping movement instructions from the mapping end, and performs interactive operation on a control target object. By using the application, the flexible insole has a large activity range of lower limb movement and small binding requirement, and can easily meet the requirements of various sites; the accuracy of lower limb movement recognition is improved by using a machine algorithm, and immersive interaction is realized.

The invention discloses a spokesman tracking method based on audiovisual bimodal and related equipment, and relates to the technical field of data processing, and the method comprises the steps: obtaining audio information and video information in response to a spokesman tracking instruction, carrying out voiceprint comparison based on a historical database and the audio information, and obtaining a spokesman tracking result; the method comprises the following steps: acquiring a historical database, determining whether a voice ID corresponding to a spokesman exists in the historical database, the historical database comprising a plurality of voice IDs and face IDs corresponding to the spokesman, and determining the spokesman based on the voice ID corresponding to the spokesman if the voice ID corresponding to the spokesman exists in the historical database; and if the video information does not exist, performing lip movement identification based on the video information so as to determine a spokesman. According to the method, multi-dimensional information such as sound source comparison and lip movement detection is comprehensively utilized, and rapid confirmation and stable tracking of the identity of the spokesman in a complex environment are realized through cross-modal feature fusion.

The application discloses an immersive operation and maintenance tool calling method based on hand and whole body action fusion recognition, and belongs to the fields of human-computer interaction, virtual reality (VR) and pattern recognition. The method comprises six steps: (1) a mixed collection environment is constructed, multi-participant operation data is collected based on "intention driving", and time stamp alignment and action classification are performed; (2) a self-centered coordinate system is established, whole body data is mirror corrected and normalized, and is uniformly mapped to a first person perspective; (3) a differential sliding window is adopted to construct an offline data set and an online ring buffer, data enhancement and real-time action segmentation are realized; (4) four types of feature streams of joints, bones, speed are decoupled, input into an asymmetric double flow network, and hand fine features and body noise resistant features are respectively extracted; (5) a double granularity gated residual fusion mechanism is adopted to dynamically calculate complementary weights, four flow feature classifications are combined, and tool calling intentions are output; and (6) based on a PC-VR architecture, a sliding window voting and cooling strategy are used to confirm the intention, a tool model is generated and is adsorbed to the hand. Compared with traditional menus and static gestures, the method significantly reduces operation interruption and cognitive load, captures continuous dynamic semantics, fuses heterogeneous skeleton data, and improves the immersion, efficiency and robustness of maintenance training.