30 results about "Activity recognition" patented technology

This invention discloses a method and system for protecting privacy in wearable devices for human activity recognition, aiming to solve the problems of existing methods struggling to balance privacy and utility, and high resource consumption on the edge. The method first acquires multi-channel time-series signals from the wearable device and constructs fixed-length feature vectors. Based on a sensitive attribute inference model and the number of high-risk features, it divides the device into high-risk and low-risk feature sets. Only the high-risk feature sub-vectors are subjected to perturbation noise to generate perturbation latent representations, which are then uploaded to an edge server along with the low-risk feature set. The edge server generates the final feature vectors and assesses the risk of privacy leakage. An offline evaluation is used to construct a privacy-utility trade-off, dynamically adjusting the perturbation dimension and noise amplitude. This invention achieves targeted protection of high-risk features, maximizing the preservation of key recognition information, reducing edge computing power and communication overhead, adapting to the differentiated needs of different scenarios, and balancing privacy security and activity recognition performance.

The application discloses a kind of edge cloud cooperation human activity recognition modeling methods based on heterogeneous multi-modal data, and is divided into two stages of centralized pre-training and multi-modal semi-supervised fine-tuning.In the first stage, by dynamic mask contrast learning, pre-train the basic model based on converter using the joint dataset with super-class information, so that it has the ability to extract robust features from any modal combination.In the second stage, the local unlabeled data is used by the end-side client to generate pseudo-labels through weak mask view and calculate losses based on strong mask view for local update;Cloud side aggregates heterogeneous model parameters from different clients, and fine-tunes the global model using a small amount of labeled data.The application effectively alleviates the data heterogeneity problem by separating single-modal feature encoding and cross-modal information fusion, and can efficiently train a high-performance multi-modal human activity recognition model using a small amount of labeled data and unlabeled data while protecting data privacy.

Embodiments of the present disclosure relate to activity recognition methods utilizing automatic training based on inertial sensors. A technical advance is disclosed that utilizes inertial sensor data associated with a device to determine a new feature array and determine whether the new feature array is within an existing class within a state space associated with the inertial sensor data. In response to the new feature array being included in the existing class, the new feature array is added to the existing class and a representation of the existing class in the state space is updated based on the new feature array and an existing representation of the existing class. In response to the new feature array not being included in the existing class, a new class is created based on the new feature array.

An apparatus using a lightweight transformer model for human activity recognition in a portable device, includes: a data collector configured to collect data for human activity recognition based on an mmWave radar; a data processor configured to perform a data processing process to process mmWave data and convert the mmWave data into an input form for a model; a lightweight GST model part configured to generate a feature vector by combining a grouped attention mechanism, which splits an input sequence into several small groups and independently calculates attention within each group, and a sparse attention mechanism, which calculates attention only for selected location pairs rather than calculating attention for all location pairs, to classify an output class through a fully connected layer; and a human activity recognition result outputter configured to output human activity recognition results based on human activity types classified in the lightweight GST model part.

The present invention relates to systems and methods for detecting occupant illness symptoms. Systems and methods for detecting occupant illness symptoms are disclosed herein. In embodiments, a memory is configured to maintain a visualization application and data from one or more sources, such as an audio source, an image source, and / or a radar source. A processor is in communication with the memory and a user interface. The processor is programmed to receive data from the one or more sources, execute a human detection model based on the received data, execute an activity recognition model that identifies illness symptoms based on the data from the one or more sources, determine a location of the identified symptoms, and execute the visualization application to display information in the user interface. The visualization application can display a background image with an overlay image that includes an indicator for each location of the identified illness symptoms. Additionally, data from the audio source, the image source, and / or the radar source can be fused.

The present invention is a Multiple Intelligences Recognition and Activities System (MIRA System) that observes a student's activities inside the classroom, identifies a multiple intelligence that needs to be supplemented or is least detected for the student, and recommends activities directed to the least detected multiple intelligence(s) for each student in the class. A classroom may be in building, in a remote environment such as a home school, or outside of a building. The MIRA System comprises a digital camera or a mobile digital device and a computer program that processes captured pictures and videos from the camera, analyzes the quantity of each multiple intelligence activity for each student in the class and then recommends activities to enhance multiple intelligence skills related to the teacher's objectives.

The application relates to a human activity recognition method and device, equipment and a readable storage medium, and relates to the technical field of intelligent sensing. The application comprises the following steps: performing feature extraction on original skeleton joint features of a target action to obtain skeleton joint features; performing feature extraction on original acceleration features of the target action to obtain acceleration features; performing fusion processing on the skeleton joint features and the acceleration features to obtain multi-modal fusion features; and performing recognition based on the multi-modal fusion features to obtain a recognition result corresponding to the target action. The application realizes multi-modal recognition of human activity by fusing skeleton joint data and acceleration data, so as to solve the problem of poor single-modal recognition effect and effectively improve the accuracy of human activity recognition.

The application discloses a human activity recognition method and system based on attention mask and hierarchical contrast learning, comprising: standardizing and preprocessing multi-source sensor data; calculating time step importance scores through a self-attention mechanism, guiding mask area selection, reconstructing key timing segments to learn semantic features; extracting hierarchical timing features by using a multi-scale dilated convolution network, and optimizing the model by combining instance-level and time-level contrast losses; and finally realizing low-delay real-time recognition and visual output on a mobile terminal. The application solves the problems in the prior art that random masks lack semantic guidance, single-scale contrast learning is difficult to capture multi-level timing dependencies, and the model has poor cross-dataset generalization capability, and significantly improves the accuracy and robustness of mobile human activity recognition.

The present invention is a Multiple Intelligences Recognition and Activities System (MIRA System) that observes a student's activities inside the classroom, identifies a multiple intelligence that needs to be supplemented or is least detected for the student, and recommends activities directed to the least detected multiple intelligence(s) for each student in the class. A classroom may be in building, in a remote environment such as a home school, or outside of a building. The MIRA System comprises a digital camera or a mobile digital device and a computer program that processes captured pictures and videos from the camera, analyzes the quantity of each multiple intelligence activity for each student in the class and then recommends activities to enhance multiple intelligence skills related to the teacher's objectives.

The application discloses a cross-user wearable activity recognition method based on group-specific concept-aware representation learning, which comprises the following steps: collecting multi-user sensor data and preprocessing; measuring the concept drift degree from the time sequence perspective and the semantic perspective respectively, fusing the multi-perspective measurement results to perform user clustering, and generating group-specific concept labels; constructing a perception model comprising an activity encoder, a user encoder and multiple classifiers, and performing supervised learning by minimizing the joint loss of activity, user and group-specific concept classification; introducing a conditional discriminator to construct a representation pair of joint distribution and marginal distribution, minimizing the conditional mutual information of activity representation and user representation under the group-specific concept through adversarial training, and obtaining a trained model; and inputting test data into the trained model for activity recognition. The application explicitly models the group-specific concept and decouples the activity and user features, eliminates the cross-user concept drift, and significantly improves the activity recognition generalization ability of the model on new users.

This invention discloses a method and system for protecting privacy in wearable devices for human activity recognition, aiming to solve the problems of existing methods struggling to balance privacy and utility, and high resource consumption on the edge. The method first acquires multi-channel time-series signals from the wearable device and constructs fixed-length feature vectors. Based on a sensitive attribute inference model and the number of high-risk features, it divides the device into high-risk and low-risk feature sets. Only the high-risk feature sub-vectors are subjected to perturbation noise to generate perturbation latent representations, which are then uploaded to an edge server along with the low-risk feature set. The edge server generates the final feature vectors and assesses the risk of privacy leakage. An offline evaluation is used to construct a privacy-utility trade-off, dynamically adjusting the perturbation dimension and noise amplitude. This invention achieves targeted protection of high-risk features, maximizing the preservation of key recognition information, reducing edge computing power and communication overhead, adapting to the differentiated needs of different scenarios, and balancing privacy security and activity recognition performance.

Some embodiments relate to a system for automated gaming recognition, the system comprising: at least one image sensor configured to capture image frames of a field of view including a table game; at least one depth sensor configured to capture depth of field images of the field of view; and a computing device configured to receive the image frames and the depth of field images, and configured to process the received image frames and depth of field images in order to produce an automated recognition of at least one gaming state appearing in the field of view. Embodiments also relate to methods and computer-readable media for automated gaming recognition. Further embodiments relate to methods and systems for monitoring game play and / or gaming events on a gaming table.

This invention relates to the field of wearable computing and signal processing technology, specifically to a method for human activity recognition based on photoplethysmography (PPG) signals. It addresses the problem of recognizing single-channel PPG signals under complex motion conditions. The method includes the following steps: acquiring single-channel PPG signals of the human body in different activity states, the signals being derived from existing data sources; performing bandpass filtering on the signals to remove baseline drift and high-frequency noise; and dividing the signals into fixed-length time window samples. An adaptive joint denoising module and a multi-domain feature collaborative modeling structure are constructed. By decomposing, filtering, and reconstructing the PPG signals, and combining time-domain and time-frequency domain feature extraction, joint modeling of physiological rhythm information and motion dynamic information is achieved, thereby improving the accuracy and stability of human activity recognition.

A method including receiving a request to restart a workflow, wherein the workflow includes a plurality of activities, identifying a state of each activity of the plurality of activities of the workflow, determining a restart condition for each activity of the plurality of activities based on the state of each activity, and executing restart of the workflow based on the determined restart condition for each activity of the plurality of activities.

This invention relates to a target activity identification method for a high-reliability intelligent detector in the field of next-generation information technology. The method includes: acquiring multimodal data streams from the environment via sensors to obtain raw data containing thermal radiation signals and motion signals; preprocessing the raw data to suppress noise and extract features to obtain a preprocessed signal sequence; extracting thermal signal intensity variation components and motion velocity-related components from the preprocessed signal sequence to form an initial feature vector set; performing data fusion on the initial feature vector set to obtain a fused feature set; analyzing the correlation between the thermal signal intensity variation components and motion velocity-related components based on the fused feature set to obtain a comprehensive pattern index set; and determining the target activity pattern based on the comprehensive pattern index set to determine the risk identification result.

A method for cross-modal semantic understanding and generation of IMU signals for LLM (Limited Language Model) is proposed. In the offline multimodal alignment stage, a pre-trained visual language model is used to generate motion-related text descriptions from videos synchronized with the IMU signals. Multi-granular features are extracted through temporal fusion, and an IMU encoder is trained using coarse-grained inter-sample alignment and fine-grained intra-sample alignment strategies to achieve initial alignment of IMU signals and text in a shared semantic space. In the online retrieval and enhancement generation stage, features are extracted from newly input IMU signals using the pre-trained IMU encoder, and similar signal-text pairs are retrieved from a pre-constructed IMU-text pairing RAG database. A structural similarity graph between IMU signals is constructed, and a central node and its neighbors are selected to form an anchor point-cluster structure. This structure, along with the retrieved text descriptions, is input into a large language model to generate high-quality text descriptions consistent with the semantics of the IMU signals. This invention enables a large language model to generate high-quality text descriptions consistent with motion semantics, even under the constraint of imprecise alignment between the IMU and text, thus improving its perceptual performance in tasks such as activity recognition, behavior analysis, and human-computer interaction.

The application discloses a wall-penetrating human activity recognition method based on CSI, which comprises the following steps: collecting 5GHz frequency band signals emitted by a signal source and calculating channel state information (CSI); separating human dynamic components by using a reference channel; segmenting action segments and static segments, and obtaining signal subspaces thereof through eigenvalue decomposition; selecting two antennas to perform conjugate multiplication, and obtaining a time-frequency spectrum through Butterworth band-pass filtering, PCA and short-time Fourier transform; and inputting the signal subspaces and the time-frequency spectrum into an attention-enhanced neural network to recognize human activities. The application has the advantages of high signal collection efficiency, high analysis accuracy and strong reliability.

A method includes extracting, from each radar frame in a data stream, a set of features that a machine-learning (ML) model is configured to receive as input. The method includes detecting human presence for a current frame. The method includes in response detecting the human is present: inputting the set of features into the ML model that is configured to estimate a pose of the human based on learned spatial relationships among a set of different human body parts and a learned temporal variations of the different human body parts across multiple time instances. The method includes accumulating a queue of Nv consecutive frames; and selecting a set of activity frames corresponding to a single action, based on motion features extracted from the queue. The method includes inferring and labeling a user action based on a sequence of respective poses of the human corresponding to the set of activity frames.

This invention provides a method for recognizing human behavior activities based on a temporal decomposition attention network, belonging to the field of pattern recognition technology. It solves the technical problem of low efficiency in traditional similar activity recognition. The technical solution includes the following steps: S1: Acquire raw temporal signal data from a multi-axis inertial sensor; S2: Use a multi-scale temporal decomposition network, employing three kernels to decompose the input signal, and averaging the decomposition results to obtain a global trend component and a local periodic component; S3: Construct a temporal decomposition attention module, including trend and periodic sub-modules, and perform different attention calculations on the two components to obtain enhanced features; S4: Globally average pool the trend features, periodic features, and their corresponding attention-enhanced features, and then fuse and concatenate them; S5: Utilize a fully connected layer to perform output classification for human behavior activity recognition. This invention effectively improves the recognition accuracy of similar and easily confused activities.

An efficient smart home embedded human activity recognition (HAR) system is provided. The HAR system according to an embodiment of the present invention extracts a region of interest (RoI) video in advance in a video preprocessing process and inputs the extracted RoI video to a SlowFast network, thereby efficiently reducing the memory usage of the embedded system through separation of the RoI extraction and feature extraction processes and improving the processing efficiency of the SlowFast network. In addition, the HAR system according to an embodiment of the present invention groups people interacting in the video preprocessing process, integrates RoIs, and processes the RoIs as one in the SlowFast network, thereby more efficiently driving the SlowFast network with limited computing resources of the embedded system, improving behavior recognition performance for multiple objects, and enabling stable operation even in complex situations.

The application discloses a kind of based on hash space optimization wearable human behavior activity identification method, belong to human activity identification technical field, solve the problem of high behavior recognition calculation complexity, low processing efficiency in wearable device.Its technical scheme is as follows: including the following steps: S1, data preprocessing;S2, feature extraction and hash embedding;S3, MMD optimization;S4, center optimization;S5, loss calculation;S6, classification prediction.The beneficial effects of the application are that the method has the advantages of compact feature expression, high classification efficiency and excellent recognition accuracy, and is suitable for wearable devices such as smart bands and smart watches, and has a wide range of practical application prospects.

A robotic system is configured to automatically identify surgical instruments used during a bronchoscopy procedure. The robotic system can include a video capture device, a robotic manipulator, sensors configured to detect a configuration of the robotic manipulator, and control circuitry communicatively coupled to the robotic manipulator. The control circuitry is configured to perform, using a machine learning classifier, a first analysis of a bronchoscopy video of a patient site to track a medical instrument in the bronchoscopy video. The control circuitry can then identify a set of possible instrument identifications for the medical instrument in the bronchoscopy video based on the first analysis and an identified phase of the bronchoscopy procedure. The control circuitry can then track a motion of the medical instrument in the bronchoscopy video and select an identification from the set of possible instrument identification for the medical instrument based at least on the tracked motion.

A computer-implemented method for potential hazard identification, includes receiving, from one or more cameras, image data of a scene, identifying, based on the image data and as identified activities, one or more activities occurring at the scene, based on the identified activities, detecting a safety-relevant entity activity and zone at the scene, determining whether additional image data is required to detect whether a hazard occurred based on the detected safety-relevant entity activity and zone at the scene, accordingly adjust the camera control system autonomously, if additional image data is not required, then analyzing the image data to detect whether a hazard occurred, detecting that a hazard occurred, and communicating to a user device, a notification indicating that the hazard occurred and providing a mitigation action, where the additional image data is focused high-spatial resolution image data.

The application discloses a wearable activity recognition method based on multi-modal learnable blocks, and belongs to the technical field of wearable human activity recognition, and comprises the following steps: inputting activity data into a modal shared block network and a modal specific block network respectively to obtain modal shared block information and modal specific block information; resampling the activity data by using the modal shared block information to obtain modal shared block activity data, and resampling the modal shared block activity data by using the modal specific block information to obtain modal specific block activity data; extracting a first data representation from the modal specific block activity data, encoding the modal shared block information to guide the modal shared block activity data to extract a second data representation, fusing the two types of representations to obtain a block representation, and aggregating all the block representations to send into a classifier to output a recognition result. The application can adaptively extract complete local semantics and local subtle patterns in the activity data, and improve the recognition accuracy and robustness in a complex activity scene.

The application discloses a human activity recognition method and system for a multi-modal mobile sensing scene, and belongs to the technical field of human activity recognition. The application proposes a two-stage pre-training technical solution for gradually reducing motion interference in a mobile scene: the first stage is cross-modal feature reconstruction, after feature extraction is performed on multiple modal data, the features of static data are used to constrain the features of dynamic data, and motion interference within the modal is eliminated; the second stage is reliable modal guided multi-modal alignment, inertial sensing data which is strong in motion interference robustness is used as a reference, features of other modal reconstruction are aligned in a feature space based on a Gram matrix, and motion interference between the modal is eliminated; when a downstream task is applied, the subject parameter is retained and the classification head is fine-tuned, so that the new task can be quickly adapted. Experimental results show that the application has more accurate, stable and effective multi-modal human activity recognition ability in a mobile scene.

This invention provides a method and system for human activity recognition based on a lightweight Transformer encoder. The method obtains a training dataset by acquiring Channel State Information (CSI) for different human activities and assigning real activity labels; obtains a preprocessed CSI data matrix; forms a patch embedding sequence with position encoding and stacks it to obtain a three-dimensional tensor; constructs a dual-domain lightweight attention network; constructs a multi-scale inverse residual convolutional feedforward network; obtains human activity recognition results; constructs a human activity recognition model based on a lightweight Transformer encoder; obtains a trained lightweight human activity recognition model; and finally, obtains the recognition results. This invention achieves better recognition accuracy, effectively reduces the number of model parameters, effectively alleviates the overfitting problem that easily occurs on small sample datasets, and significantly enhances the model's recognition robustness.

This invention discloses a multi-index federated optimization method, system, and medium for human activity recognition. The method includes: a server distributing a global model; a client training on local data and uploading updates; the server constructing a similarity matrix for the client's update direction, a similarity matrix for weights, and a similarity matrix for update size; generating a fusion similarity matrix that dynamically evolves with training rounds; calculating the client's reputation score; adjusting the client model aggregation weights using the reputation score; correcting the similarity matrix and performing hierarchical clustering during mid-training using the reputation score; and training sub-clusters as independent aggregation units. This invention effectively addresses the challenge of data heterogeneity in human activity recognition scenarios and accurately identifies malicious clients, enhancing the system's robustness and effectiveness while ensuring model performance.

A method including receiving a request to restart a workflow, wherein the workflow includes a plurality of activities, identifying a state of each activity of the plurality of activities of the workflow, determining a restart condition for each activity of the plurality of activities based on the state of each activity, and executing restart of the workflow based on the determined restart condition for each activity of the plurality of activities.

This application discloses a method, apparatus, device, and storage medium for analyzing pedestrian density. The method includes: acquiring video images of the scene to be monitored; inputting the video images into a preset pedestrian detection model for inference to determine the pedestrian positions, number of pedestrians, pedestrian density distribution map, and confidence level in each frame; if the confidence level is less than a preset threshold, inputting the video images into a preset abnormal activity recognition model for identification to determine the type of abnormal crowd activity; the preset abnormal activity recognition model includes a 3D residual network, a region proposal network, and a region graph convolutional network; based on the pedestrian density distribution map, a Gaussian kernel density estimation algorithm is used to generate a pedestrian density heatmap; and the positions, numbers, heatmap, and types of abnormal crowd activity are visualized on a monitoring platform. Thus, the preset pedestrian detection model first outputs the pedestrian positions, numbers, density distribution, and confidence level, and then the fused abnormal activity recognition model is activated to accurately identify abnormal types such as pushing and crowding.