Human motion recognition method and device based on infrared event sensor

Abstract

The application discloses a human action recognition method and device based on an infrared event sensor, relates to the technical field of computer vision and artificial intelligence, and mainly aims to solve the problem of high resource consumption of existing human action recognition. Mainly includes acquiring the event stream infrared signal collected by the infrared event sensor in the event mode, and dividing the event stream infrared signal into a time sequence thermal event frame sequence according to a preset time window; constructing a dynamic space-time graph according to the time sequence thermal event frame sequence, and performing action recognition on the dynamic space-time graph to obtain an action recognition result and a recognition confidence; in the case that the recognition confidence meets a preset confidence condition, outputting the action recognition result; in the case that the recognition confidence does not meet the preset confidence condition, adjusting the preset time window according to the recognition confidence, and returning to the step of dividing the time sequence thermal event frame sequence according to the adjusted time window to reacquire the action recognition result and the recognition confidence. Mainly used for recognizing human actions.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to a method and apparatus for human motion recognition based on an infrared event sensor. Background Technology

[0002] Human motion recognition is an important branch of computer vision. Its core task is to enable computers to automatically analyze and understand human actions or activities in video sequences, thereby clarifying what the monitored object is currently doing. Current human motion recognition methods mainly include acquiring color video sequences at a fixed frame rate and using deep learning algorithms to extract temporal features to identify human actions, and using infrared sensors to detect the infrared radiation of objects to achieve imaging and detect human targets in dark environments. Infrared sensors are widely used because they can be used in all weather conditions under different lighting conditions and do not pose privacy risks.

[0003] Currently, infrared imaging technology based on traditional full-frame sampling has inherent technical bottlenecks when used for continuous monitoring and behavior analysis. Its operating mode requires the sensor to continuously capture and output a complete infrared image of the entire field of view at a fixed frame rate, regardless of whether the scene content changes. This "blind" sampling mechanism leads to significant redundancy in static background data. It requires the continuous acquisition, transmission, and processing of a large amount of unchanged and invalid pixel information in the scene (such as walls, floors, and stationary furniture), resulting in huge data processing overhead and wasted transmission bandwidth, thus leading to high resource consumption in the human motion recognition process. Summary of the Invention

[0004] In view of this, the present invention provides a human motion recognition method and device based on an infrared event sensor, the main purpose of which is to solve the problem of high resource consumption in the existing human motion recognition process.

[0005] According to one aspect of the present invention, a method for human motion recognition based on an infrared event sensor is provided, comprising: The infrared event stream signals collected by the infrared event sensors in the target monitoring area in event mode are acquired, and the event stream infrared signals are divided into a time-series thermal event frame sequence according to a preset time window. A dynamic spatiotemporal graph is constructed based on the time-series thermal event frame sequence, and action recognition is performed on the dynamic spatiotemporal graph to obtain the action recognition results and recognition confidence. If the recognition confidence level meets the preset confidence level conditions, the action recognition result is output. If the recognition confidence does not meet the preset confidence condition, the preset time window is adjusted according to the recognition confidence, so as to return to the step of dividing the time-series hot event frame sequence according to the adjusted time window to re-acquire the action recognition result and recognition confidence.

[0006] Further, adjusting the preset time window based on the identification confidence level includes: A feedback factor is calculated based on the identification confidence level, wherein the feedback factor is negatively correlated with the identification confidence level; Extract the real-time event rate of the event stream infrared signal; The preset time window is adjusted based on the feedback factor and the real-time event rate to obtain the adjusted time window.

[0007] Furthermore, after adjusting the preset time window based on the recognition confidence level, the method further includes: If the identification confidence level does not meet the preset confidence level conditions, update the number of consecutive abnormal confidence levels. If the number of times the updated confidence level remains abnormal exceeds a preset threshold, a first mode switching control command is generated and sent to the infrared event sensor to adjust the working mode of the infrared event sensor from event mode to frame mode. In frame mode, a static spatiotemporal map is constructed based on the acquired continuous frame infrared signals, and action recognition is performed on the static spatiotemporal map to obtain the action recognition result and recognition confidence level. If the obtained recognition confidence level meets the preset confidence level condition, a second mode switching control command is generated and sent to the infrared event sensor to restore the working mode of the infrared event sensor to the event mode.

[0008] Further, the step of constructing a static spatiotemporal map based on the acquired continuous frame infrared signals, and performing action recognition on the static spatiotemporal map to obtain action recognition results and recognition confidence levels includes: For each frame of infrared image in a continuous frame of infrared signal, key points in the infrared image are extracted by a pose estimation model. A static spatiotemporal graph is constructed with each key point as a node and the connection relationship of the human skeletal structure as a spatial edge. The spatial relationship between each node in the static spatiotemporal graph is extracted by a graph convolutional network to obtain the global spatial structure features between key points. The global spatial structure features of the global infrared image are constructed into a global spatial structure feature sequence according to the time sequence, and the continuous motion trajectory of each key point over time is captured by the temporal modeling network to obtain the global temporal evolution features of the action. By fusing the global spatial structure features and the global temporal evolution features, and classifying and predicting the global fused features, action recognition results and recognition confidence are obtained.

[0009] Furthermore, constructing a dynamic spatiotemporal graph based on the time-series thermal event frame sequence includes: Extract key points and associated infrared event rates within the receptive field of each key point from the time-series thermal event frame sequence, and identify key points whose associated infrared event rates are greater than a preset infrared event rate threshold as thermal key points. Based on the spatial positional relationship between the thermal key points, the spatial edges in the initial dynamic spatiotemporal graph are recalculated, and based on the evolution of the same thermal key point in consecutive frames, the temporal edges in the initial dynamic spatiotemporal graph are recalculated to obtain the dynamic spatiotemporal graph.

[0010] Further, action recognition is performed on the dynamic spatiotemporal graph to obtain action recognition results and recognition confidence levels, including: From the dynamic spatiotemporal graph, the local thermal subgraph where the hot node is located is determined, and the spatial relationship between each node in the local thermal subgraph is extracted by the graph convolutional network to obtain the local spatial structure relationship features between real-time motion key points. By capturing the motion trajectory of each node in the local heatmap over time using a two-stream asynchronous network, the local temporal evolution characteristics of the action are obtained. By fusing the local spatial structure relationship features and the local temporal evolution features, and using the classifier to classify and predict the local fused features, action recognition results and recognition confidence are obtained.

[0011] Furthermore, the process of constructing the initial dynamic spatiotemporal graph includes: Key points are extracted from the initial thermal event frame image using a pose estimation model; An initial dynamic spatiotemporal graph is constructed using each key point as a node, the connection relationship of the human skeletal structure as a spatial edge, and the continuous positions of the same node in time as a temporal edge.

[0012] According to another aspect of the present invention, a human motion recognition device based on an infrared event sensor is provided, comprising: The segmentation module is used to acquire the event stream infrared signals collected by the infrared event sensors in the target monitoring area in event mode, and to segment the event stream infrared signals into a time-series thermal event frame sequence according to a preset time window. The dynamic recognition module is used to construct a dynamic spatiotemporal graph based on the time-series thermal event frame sequence, and to perform action recognition on the dynamic spatiotemporal graph to obtain action recognition results and recognition confidence. The output module is used to output the action recognition result when the recognition confidence meets the preset confidence conditions; The adjustment module is used to adjust the preset time window according to the recognition confidence level when the recognition confidence level does not meet the preset confidence level conditions, so as to re-acquire the action recognition result and recognition confidence level according to the step of returning to the division of the time-series hot event frame sequence according to the adjusted time window.

[0013] Furthermore, the adjustment module includes: A calculation unit is configured to calculate a feedback factor based on the identification confidence level, wherein the feedback factor is negatively correlated with the identification confidence level; An extraction unit is used to extract the real-time event rate of the event stream infrared signal; An adjustment unit is used to adjust the preset time window based on the feedback factor and the real-time event rate to obtain the adjusted time window.

[0014] Furthermore, the device also includes: The update module is used to update the number of times the confidence level is continuously abnormal when the identification confidence level does not meet the preset confidence level conditions. The first generation module is used to generate and send a first mode switching control command to the infrared event sensor if the number of times the updated confidence level continues to be abnormal is greater than a preset number threshold, so as to adjust the working mode of the infrared event sensor from event mode to frame mode. The static recognition module is used to construct a static spatiotemporal map based on the acquired continuous frame infrared signals in frame mode, and to perform action recognition on the static spatiotemporal map to obtain action recognition results and recognition confidence. The second generation module is used to generate and send a second mode switching control command to the infrared event sensor when the obtained recognition confidence level meets the preset confidence level condition, so as to restore the working mode of the infrared event sensor to the event mode.

[0015] Furthermore, the static recognition module includes: The first extraction unit is used to extract key points in each frame of infrared image in the continuous frame infrared signal through a pose estimation model, construct a static spatiotemporal graph with each key point as a node and the connection relationship of human skeletal structure as a spatial edge, and extract the spatial relationship between each node in the static spatiotemporal graph through a graph convolutional network to obtain the global spatial structure features between key points. The second extraction unit is used to construct the global spatial structure features of the global infrared image into a global spatial structure feature sequence according to the time sequence, and capture the coherent motion trajectory of each key point over time through the temporal modeling network to obtain the global temporal evolution features of the action. The global feature fusion unit is used to fuse the global spatial structure features and the global temporal evolution features, and to classify and predict the global fused features to obtain the action recognition result and recognition confidence.

[0016] Furthermore, the dynamic recognition module includes: The filtering unit is used to extract key points and the associated infrared event rate within the receptive field of each key point from the time-series thermal event frame sequence, and to determine key points whose associated infrared event rate is greater than a preset infrared event rate threshold as thermal key points. The first construction unit is used to recalculate the spatial edges in the initial dynamic spatiotemporal graph based on the spatial positional relationship between the thermal key points, and to recalculate the temporal edges in the initial dynamic spatiotemporal graph based on the evolution of the same thermal key point in consecutive frames, so as to obtain the dynamic spatiotemporal graph.

[0017] Furthermore, the dynamic recognition module also includes: The third extraction unit is used to determine the local thermal subgraph where the hot node is located from the dynamic spatiotemporal graph, and extract the spatial relationship between each node in the local thermal subgraph through a graph convolutional network to obtain the local spatial structure relationship features between real-time motion key points. The fourth extraction unit is used to capture the motion trajectory of each node in the local heat map over time through a two-stream asynchronous network, and obtain the local temporal evolution characteristics of the action. The local feature fusion unit is used to fuse the local spatial structure relationship features and the local temporal evolution features, and to classify and predict the local fused features through the classifier to obtain the action recognition result and recognition confidence.

[0018] Furthermore, the dynamic recognition module also includes: The fifth extraction unit is used to extract key points from the initial thermal event frame image through the pose estimation model; The second building unit is used to construct an initial dynamic spatiotemporal graph with each key point as a node, the connection relationship of the human skeletal structure as a spatial edge, and the continuous position of the same node in time as a temporal edge.

[0019] According to another aspect of the present invention, a storage medium is provided, wherein at least one executable instruction is stored therein, the executable instruction causing a processor to perform operations corresponding to the above-described human motion recognition method based on an infrared event sensor.

[0020] According to another aspect of the present invention, a terminal is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus; The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the above-described human motion recognition method based on infrared event sensors.

[0021] By employing the above-described technical solutions, the technical solutions provided by the embodiments of the present invention have at least the following advantages: This invention provides a method and apparatus for human motion recognition based on infrared event sensors. In this embodiment, the invention acquires event stream infrared signals collected by infrared event sensors in an event mode within a target monitoring area, and divides these signals into a time-series thermal event frame sequence according to a preset time window. A dynamic spatiotemporal graph is constructed based on the time-series thermal event frame sequence, and motion recognition is performed on the dynamic spatiotemporal graph to obtain motion recognition results and recognition confidence levels. If the recognition confidence level meets a preset confidence level condition, the motion recognition result is output. If the recognition confidence level does not meet the preset confidence level condition, the preset time window is adjusted according to the recognition confidence level, so that the process returns to the step of dividing the time-series thermal event frame sequence according to the adjusted time window to re-acquire motion recognition results and recognition confidence levels. This significantly reduces data redundancy and computational burden caused by continuous image acquisition and processing in traditional video surveillance, lowering system power consumption and storage requirements. Simultaneously, it ensures the adaptability and robustness of the motion recognition process in complex or dynamic scenarios, thereby improving the efficiency of human motion recognition.

[0022] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention, it can be implemented according to the contents of the specification. Furthermore, in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0023] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart of a human motion recognition method based on an infrared event sensor provided by an embodiment of the present invention is shown; Figure 2 A flowchart of another human motion recognition method based on an infrared event sensor provided by an embodiment of the present invention is shown; Figure 3 This diagram illustrates a block diagram of a human motion recognition device based on an infrared event sensor, according to an embodiment of the present invention. Figure 4A schematic diagram of the structure of a terminal provided in an embodiment of the present invention is shown. Detailed Implementation

[0024] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0025] To address the high resource consumption issue in existing infrared image-based human motion recognition processes, this invention provides a human motion recognition method based on an infrared event sensor. This method applies the infrared event sensor to the data acquisition stage of motion recognition. This infrared event sensor can electrically switch its physical operating mode based on bias voltage, including event mode and frame mode. In event mode, under zero bias or reverse bias, the sensor physically operates in a capacitive mode. In this mode, the sensor does not respond to constant infrared radiation, only outputting an event stream in the form of transient pulses when the infrared radiation intensity changes (i.e., human movement). In frame mode, when the applied forward bias voltage exceeds a specific threshold (e.g., 0.6V), the device's pn junction is forward-biased, and the operating mode physically changes to a mode that continuously responds to infrared radiation intensity. In this mode, the sensor outputs signal frames similar to those in traditional thermal imaging. The infrared event sensor can be a wide / narrow bandgap semiconductor heterojunction sensor (such as ZnO / PbSe), and this invention does not impose specific limitations.

[0026] like Figure 1 As shown, the method includes: 101. Acquire the event stream infrared signals collected by the infrared event sensors in the target monitoring area in event mode, and divide the event stream infrared signals into a time-series thermal event frame sequence according to a preset time window.

[0027] In this embodiment of the invention, the event mode of the infrared event sensor is the primary operating mode. The infrared event sensor continuously collects infrared signals of the event stream from the target monitoring area. The current execution entity can be a local server or a cloud server of the monitoring system. This server is communicatively connected to the infrared event sensor in the target monitoring area to obtain the sensor's infrared signals and perform action recognition based on these signals. This server can also be connected to alarm devices and prevention and control execution mechanisms to control the alarm devices to sound an alarm or the prevention and control execution mechanism to perform prevention and control operations when the detected action type corresponds to a warning or prevention and control trigger condition.

[0028] Because images acquired in this mode only record changes in infrared radiation at each pixel, when a warm object (such as a person, vehicle, or animal) moves, the pixels it passes over will trigger events sequentially, forming an event stream composed of coordinates, timestamps, and polarity. For subsequent human motion recognition, the continuous event stream needs to be segmented into meaningful time segments, i.e., time-series thermal event frame sequences. Each time-series thermal event frame sequence describes all thermal changes occurring within a preset time window. The length of the preset time window can be assigned during initial device setup. After a human motion recognition is completed, the length of the preset time window will adaptively adjust based on the recognition confidence level to meet the needs of image recognition, extracting more effective event streams from the infrared signal of the event stream, thereby continuously improving the accuracy of subsequent motion recognition.

[0029] It should be noted that the target monitoring area can be a privacy-protected area (such as nursing home rooms, hospital wards, hotel corridors, etc.), an all-weather / harsh environment monitoring area (such as border patrols, nuclear power plant perimeters, chemical plant warehouses, forest fire prevention, etc.), a monitoring area with extremely low power consumption and bandwidth requirements (such as wireless sensor networks, battery-powered remote monitoring, IoT devices, etc.), and a smart home and building automation monitoring area (such as offices, homes). Using an infrared event sensor to acquire infrared images of the target monitoring area provides a motion recognition method that can operate stably in complex environments such as complete darkness, strong light, smoke, or rain and fog. Furthermore, it can achieve human motion recognition without recording or recovering visible light image information, thus achieving inherent privacy protection. This sensor, through an event-driven infrared signal acquisition mechanism, outputs effective information only when thermal radiation changes, avoiding the need for continuous acquisition of complete infrared image frames, which leads to a large amount of static background data redundancy and high power consumption, as required by traditional frame-based thermal imaging methods. This effectively reduces data redundancy and the power consumption of various devices.

[0030] 102. Construct a dynamic spatiotemporal graph based on the time-series thermal event frame sequence, and perform action recognition on the dynamic spatiotemporal graph to obtain action recognition results and recognition confidence.

[0031] In this embodiment of the invention, the dynamic spatiotemporal graph is a graphical structure capable of simultaneously representing temporal and spatial information. Each valid hot event point in a time-series hot event frame sequence, or its aggregated feature unit (such as a human joint), is defined as a node in the graph structure. Subsequently, based on the spatial proximity and temporal continuity of these units, edge relationships are established between them. Finally, the dynamic spatiotemporal graph is constructed based on the spatial distribution of nodes and their temporal evolution. To identify action types, the relationships between nodes in the dynamic spatiotemporal graph are extracted to capture motion trajectory features. These features are then used for classification and prediction to obtain different action types and the recognition confidence score for each action type. The recognition confidence score reflects the algorithm's judgment on the reliability of the recognition result, typically ranging from 0 to 1, with values ​​closer to 1 indicating a more reliable result. The action recognition result is the action type with the highest recognition confidence score, and the final output recognition confidence score is the recognition confidence score corresponding to the action type in the action recognition result.

[0032] The action recognition process can be based on spatiotemporal graph convolutional networks such as ST-GCN, simultaneously learning the spatial relationships (how key points connect at the same time) and temporal relationships (how the same key point moves over time) between nodes, ultimately completing action classification. Alternatively, temporal convolutional networks (TCNs) can capture long-term dependencies in the graph structure, combined with recurrent neural networks (LSTM / GRUs), which use their internal control mechanisms to memorize or forget information, learning the context and patterns in the time series to achieve action recognition. Another approach is to identify only the local subgraph composed of active nodes in the dynamic spatiotemporal graph to determine the recognition result. The above methods can be selected according to the specific application scenario requirements.

[0033] 103. If the recognition confidence level meets the preset confidence level conditions, output the action recognition result.

[0034] In this embodiment of the invention, the preset confidence condition is a pre-set threshold used to determine whether the recognition result is reliable. For example, the average confidence level within one second is greater than or equal to 70%. When the recognition confidence level obtained through action recognition is greater than or equal to this threshold, it indicates that the algorithm has a high degree of confidence in the recognition result and considers the result to be reliable. At this time, the action recognition result is output for further processing or application, such as triggering corresponding alarm prompts based on the corresponding action type, generating corresponding key event records, etc.

[0035] 104. If the recognition confidence does not meet the preset confidence condition, adjust the preset time window according to the recognition confidence, so as to return to the step of dividing the time-series hot event frame sequence according to the adjusted time window to re-acquire the action recognition result and recognition confidence.

[0036] In this embodiment of the invention, when the recognition confidence level consistently fails to meet the preset confidence level conditions, it indicates that the reliability of the current action recognition result is low. At this time, the preset time window is adjusted according to the specific value of the recognition confidence level. For example, if the confidence level is too low, the time window may be too small, resulting in insufficient information in the frame sequence; in this case, the time window can be appropriately increased. Conversely, if the confidence level, while not meeting the standard, is not particularly low, the time window may be too large, resulting in the frame sequence containing too much redundant information, interfering with recognition; in this case, the time window can be appropriately decreased. After adjusting the time window, the process returns to the step of dividing the temporal hot event frame sequence, regenerating the frame sequence according to the new time window, reconstructing the dynamic spatiotemporal graph, and performing action recognition to obtain new action recognition results and recognition confidence levels.

[0037] By adaptively adjusting the preset time window, the data acquisition and processing methods can be optimized according to the actual situation, improving the accuracy and reliability of action recognition, and better adapting to different monitoring scenarios and action characteristics. This ensures recognition accuracy even when performing low-power action recognition based on non-continuous frame infrared images.

[0038] In one embodiment of the present invention, for further illustration and limitation, such as Figure 2 As shown, the step of adjusting the preset time window based on the recognition confidence level includes: 201. Calculate the feedback factor based on the identification confidence level.

[0039] 202. Extract the real-time event rate of the event stream infrared signal.

[0040] 203. Adjust the preset time window based on the feedback factor and the real-time event rate to obtain the adjusted time window.

[0041] In this embodiment of the invention, when the recognition confidence level is consistently lower than a preset confidence threshold (e.g., average confidence level < 70% within one consecutive second), an adjustment coefficient is calculated using a feedback factor negatively correlated with the recognition confidence level, and then the preset time window is corrected using the adjustment coefficient. For example, the time window for event accumulation is increased to obtain input frames with more obvious features through software optimization. Here, the feedback factor β is a function mapping from the recognition confidence level c (usually ∈ [0,1]) to the window magnification factor. The larger β is, the greater the ΔT magnification (accumulating more events to increase information content). β is defined to satisfy the interval [β_min, β_max]. The relationship between the feedback factor and the recognition confidence level is still expressed as: β(c) = β_min + (β_max) β_min)×(1 c); Wherein, when c=1, β=β_min (close to 1 or less); when c=0, β=β_max (magnified to the maximum value). β_min ≈ 0.8–1 (less than 1 can be used if a slightly shorter window is desired at high confidence levels), or 1; β_max is chosen based on the maximum tolerable magnification factor (e.g., 2–4).

[0042] The adjusted time window ΔT_final is calculated using the following formula: ; Where clamp() represents the clamping function, M represents the real-time event rate (characterizing the speed of movement), and k represents the number of targets accumulated by the events. The value of k is based on the event rate of a typical scenario. Compared with the expected baseline cumulative time Calculated. Since the number of events within a time window is approximately equal to the product of the event rate and the window length, it is defined as follows: Preferably, it can be taken Typical event rate Therefore, k = 5. That is, in When the calculation result falls within the range of the maximum and minimum values ​​of the time window, the time window after the jump is equal to... ,exist When the result is less than the minimum time window, the adjusted time window takes the minimum time window. When the result exceeds the maximum time window, the adjusted time window takes the maximum time window. The real-time event rate is calculated using a sliding time window. Specifically, the current event rate M (unit: events / second) can be obtained by continuously counting the total number of events received within the most recent fixed time period (e.g., the past 100 milliseconds) and then dividing by the length of that time period.

[0043] In one embodiment of the present invention, for further explanation and limitation, after adjusting the preset time window based on the identification confidence level, the method further includes: If the identification confidence level does not meet the preset confidence level conditions, update the number of consecutive abnormal confidence levels. If the number of times the updated confidence level remains abnormal exceeds a preset threshold, a first mode switching control command is generated and sent to the infrared event sensor to adjust the working mode of the infrared event sensor from event mode to frame mode. In frame mode, a static spatiotemporal map is constructed based on the acquired continuous frame infrared signals, and action recognition is performed on the static spatiotemporal map to obtain the action recognition result and recognition confidence level. If the obtained recognition confidence level meets the preset confidence level condition, a second mode switching control command is generated and sent to the infrared event sensor to restore the working mode of the infrared event sensor to the event mode.

[0044] In this embodiment of the invention, in addition to adjusting the time window at the software level, a mode adjustment strategy is also configured at the hardware level. If the recognition confidence still cannot meet the preset confidence condition after multiple time window adjustments, a control signal is further sent to the infrared event sensor to adjust its bias voltage above the forward conduction threshold, causing the sensor to temporarily switch from the low-power event mode to the more information-rich frame mode. One or several frames of high-information-density infrared images are captured to assist the recognition algorithm (action recognition based on a static spatiotemporal map constructed from continuous frame infrared signals) in overcoming the low-confidence state. Once the recognition confidence returns to a normal level, the sensor is switched back to event mode to restore the low-power monitoring state.

[0045] It should be noted that when reliable judgments cannot be made continuously due to insufficient information in event mode, the system switches to the traditional frame mode to obtain richer and more certain scene information to ensure the continuity of recognition. Once the reliability of recognition is restored, it switches back to the energy-efficient event mode, realizing a degraded redundant perception and autonomous recovery mechanism. This ensures both monitoring adaptability and long-term low-power operation.

[0046] In one embodiment of the present invention, for further explanation and limitation, the step of constructing a static spatiotemporal map based on the acquired continuous frame infrared signals, and performing action recognition on the static spatiotemporal map to obtain action recognition results and recognition confidence levels includes: For each frame of infrared image in a continuous frame of infrared signal, key points in the infrared image are extracted by a pose estimation model. A static spatiotemporal graph is constructed with each key point as a node and the connection relationship of the human skeletal structure as a spatial edge. The spatial relationship between each node in the static spatiotemporal graph is extracted by a graph convolutional network to obtain the global spatial structure features between key points. The global spatial structure features of the global infrared image are constructed into a global spatial structure feature sequence according to the time sequence, and the continuous motion trajectory of each key point over time is captured by the temporal modeling network to obtain the global temporal evolution features of the action. By fusing the global spatial structure features and the global temporal evolution features, and classifying and predicting the global fused features, action recognition results and recognition confidence are obtained.

[0047] In this embodiment of the invention, in frame mode, for each acquired frame of infrared image, key points (such as elbows and knees) are extracted using a pose estimation model. Using these key points as nodes, edges are constructed based on the natural connections of the human skeleton to build a static spatiotemporal graph. This graph structure describes the global spatial structure of the human body within a single frame. Subsequently, a graph convolutional network is used to extract the complex spatial relationships between nodes in the graph, outputting the spatial structure features of each frame. Next, these spatial features from multiple consecutive frames are arranged in temporal order to form a feature sequence, which is then input into a temporal modeling network to capture the motion patterns of key points over time, obtaining global temporal evolution features describing the action. Finally, the system fuses the spatial and temporal features, and a classifier makes predictions, simultaneously outputting the action category and its corresponding confidence score. The pose estimation model can be OpenPose, HRNet (High-Resolution Net), or YOLO-Pose; this embodiment of the invention does not impose specific limitations.

[0048] It should be noted that by separating spatial and temporal modeling, both structural robustness and dynamic description capabilities are achieved. The static spatiotemporal map effectively resists background interference, focusing on the human body structure itself; while subsequent temporal analysis accurately captures the continuity of movements. This divide-and-conquer strategy enables richer and more deterministic visual information to be obtained even when faced with complex or subtle movements in frame mode, thereby predicting high-confidence recognition results and providing a reliable guarantee for the sensor to recover from a degraded state and resume normal operation.

[0049] In one embodiment of the present invention, for further explanation and limitation, the step of constructing a dynamic spatiotemporal graph based on the time-series thermal event frame sequence includes: Extract key points and associated infrared event rates within the receptive field of each key point from the time-series thermal event frame sequence, and identify key points whose associated infrared event rates are greater than a preset infrared event rate threshold as thermal key points. Based on the spatial positional relationship between the thermal key points, the spatial edges in the initial dynamic spatiotemporal graph are recalculated, and based on the evolution of the same thermal key point in consecutive frames, the temporal edges in the initial dynamic spatiotemporal graph are recalculated to obtain the dynamic spatiotemporal graph.

[0050] In this embodiment of the invention, a temporal thermal event frame sequence is transformed into a dynamically evolving spatiotemporal graph structure. In this process, instead of processing all pixels, the focus is on extracting key points that change over time. First, key points are extracted from the input temporal thermal event frame sequence to obtain each key point. For each key point, a receptive field (a local spatial neighborhood, such as a circular or square region) is defined around it, and the total number of infrared events occurring within this receptive field per unit time is calculated, i.e., the associated infrared event rate. This feature quantifies the local thermal change activity of the key point. Then, a preset infrared event rate threshold is used to filter key points, identifying hot key points from the global key points. Next, the initial dynamic spatiotemporal graph is updated based on the relationships between hot key points in the same frame image. Simultaneously, the temporal edges in the initial dynamic spatiotemporal graph are updated based on the evolution of the same hot key point in consecutive frames. The initial dynamic spatiotemporal graph describes the structural relationships between key points, providing a graph structure for improving the spatiotemporal framework of subsequent updates. It can be constructed based on the previous few frames of the current time-series thermal event frame sequence, or it can be a dynamic spatiotemporal graph constructed from earlier time-series thermal event frame sequences. This graph update method, which focuses on thermal key points, dynamically focuses computational resources on the moving body parts, significantly reducing the computational overhead and response latency when processing static or minor movements without sacrificing the ability to capture critical actions.

[0051] The specific process of spatial edge updating can be as follows: Spatial edges are recalculated by analyzing the inherent human anatomical structure and kinematic constraints between all thermal keypoints within the same frame. Here, the establishment and strengthening of spatial edges no longer rely solely on simple Euclidean distance, but prioritize whether the points belong to a reasonable human pose configuration. For example, the left wrist and left elbow keypoints, because they are connected through the arm, should have a strong spatial edge established between them regardless of their actual distance in the image. The weight of this edge can be determined by the inherent anatomical connection strength between them (such as a pre-defined prior weight based on the human skeleton) and can be fine-tuned by the synchronicity of their receptive field event rates. The specific process of temporal edge updating includes: The recalculation of temporal edges also relies on the foundation established by the initial dynamic spatiotemporal graph. The node identifiers and their corresponding keypoint types (such as "left knee," "right shoulder") determined in the initial dynamic spatiotemporal graph provide identity identifiers for cross-frame tracking. In the next frame, based on the human motion model, the new position of each node is searched within the node identity framework defined in the initial graph. This process relies not only on the similarity of location and event rate, but also on the prior identity of this hot node as the left knee. This makes tracking more robust; even if a node temporarily disappears, its reappearance location can be predicted based on its identity and spatial relationships with other nodes, and the temporal edge can be re-established.

[0052] In one embodiment of the present invention, for further explanation and limitation, the process of constructing the initial dynamic spatiotemporal graph includes: Key points are extracted from the initial thermal event frame image using a pose estimation model; An initial dynamic spatiotemporal graph is constructed using each key point as a node, the connection relationship of the human skeletal structure as a spatial edge, and the continuous positions of the same node in time as a temporal edge.

[0053] In this embodiment of the invention, an initial thermal event frame image is input into a pre-trained pose estimation model (such as OpenPose or HRNet optimized based on thermal imaging data). This model locates and extracts key points representing human posture (such as nose, shoulder, elbow, wrist, hip, knee, and ankle) from the thermal image. Each extracted key point is instantiated as a node in the spatiotemporal graph. Subsequently, based on prior knowledge of human skeletal structure connections, spatial edges are established between physically connected node pairs. For example, connecting the "left shoulder" and "left elbow" nodes, and the "left hip" and "left knee" nodes. These spatial edges do not simply connect all nodes but accurately simulate the skeletal topology of the human body, thus organizing discrete key points into a physiologically meaningful spatial configuration. Temporal edges can be constructed through simple tracking associations (such as location-based nearest neighbor matching), connecting different instances of the same node across consecutive time steps (frames). The initial thermal event frame image can be the first few frames of the current temporal thermal event frame sequence (corresponding to the cold start mode); or it can be the last frame or several frames corresponding to a dynamic spatiotemporal graph already constructed from a previous temporal thermal event frame sequence (corresponding to the warm start mode). This is equivalent to the construction of the dynamic spatiotemporal graph not starting from scratch, but inheriting an existing, continuously evolving state. The keypoint extraction process can also be based on a Transformer pose estimation network, utilizing a self-attention mechanism to capture the global dependencies between human keypoints, which may perform better in complex poses; or it can be based on a heatmap regression model, predicting the probability heatmap of each keypoint's location on the image to locate keypoints, which usually achieves high localization accuracy. This embodiment of the invention does not impose specific limitations.

[0054] In one embodiment of the present invention, for further explanation and limitation, the step of performing action recognition on the dynamic spatiotemporal graph and obtaining action recognition results and recognition confidence includes: From the dynamic spatiotemporal graph, the local thermal subgraph where the hot node is located is determined, and the spatial relationship between each node in the local thermal subgraph is extracted by the graph convolutional network to obtain the local spatial structure relationship features between real-time motion key points. By capturing the motion trajectory of each node in the local heatmap over time using a two-stream asynchronous network, the local temporal evolution characteristics of the action are obtained. By fusing the local spatial structure relationship features and the local temporal evolution features, and using the classifier to classify and predict the local fused features, action recognition results and recognition confidence are obtained.

[0055] In this embodiment of the invention, local heatmaps centered on hot nodes are located and extracted from the dynamic spatiotemporal graph, thereby focusing on the core region where the action occurs. A graph convolutional network is then used to deeply mine the connection patterns between nodes within this local heatmap, extracting the local spatial structural relationship features between key points of real-time motion, thus capturing the static posture composition of actions such as arm swinging or leg bending. Simultaneously, a dual-stream asynchronous network is used to track the displacement vector of each node in a short time and its long-term motion trajectory, effectively capturing the subtle initiation, rapid transition, and continuous state of the action, thereby obtaining local temporal evolution features capable of capturing the dynamic process of the action. Finally, the aforementioned spatial structure and temporal evolution features are deeply fused and input into a classifier for prediction, outputting the final action recognition result and its corresponding recognition confidence. By extracting local heatmaps from the dynamic spatiotemporal graph, the recognition process is mainly concentrated on these "hot" subgraphs activated by events, while nodes in static areas maintain their original state and do not participate in the current round of calculation, achieving precise allocation of computing resources and effective suppression of noise interference. Two-Stream Asynchronous Graph Convolutional Network (2SAGCN) can effectively capture the complex spatial and temporal dependencies between nodes in a spatiotemporal graph, thereby more accurately understanding the dynamic changes of human actions and improving the accuracy of action recognition.

[0056] This invention provides a human motion recognition method based on an infrared event sensor. In this embodiment, the method acquires event stream infrared signals collected by an infrared event sensor in an event mode within a target monitoring area, and divides these signals into a time-series thermal event frame sequence according to a preset time window. A dynamic spatiotemporal graph is constructed based on the time-series thermal event frame sequence, and motion recognition is performed on the dynamic spatiotemporal graph to obtain motion recognition results and recognition confidence levels. If the recognition confidence level meets a preset confidence level condition, the motion recognition result is output. If the recognition confidence level does not meet the preset confidence level condition, the preset time window is adjusted according to the recognition confidence level, and the method returns to the step of dividing the time-series thermal event frame sequence according to the adjusted time window to re-acquire motion recognition results and recognition confidence levels. This significantly reduces data redundancy and computational burden caused by continuous image acquisition and processing in traditional video surveillance, lowering system power consumption and storage requirements. Simultaneously, it ensures the adaptability and robustness of the motion recognition process in complex or dynamic scenarios, thereby improving the efficiency of human motion recognition.

[0057] Furthermore, as a response to the above Figure 1 The implementation of the method shown in this invention provides a human motion recognition device based on an infrared event sensor, such as... Figure 3 As shown, the device includes: The segmentation module 31 is used to acquire the event stream infrared signal collected by the infrared event sensor in the target monitoring area in event mode, and to divide the event stream infrared signal into a time-series thermal event frame sequence according to a preset time window. The dynamic recognition module 32 is used to construct a dynamic spatiotemporal graph based on the time-series thermal event frame sequence, and to perform action recognition on the dynamic spatiotemporal graph to obtain action recognition results and recognition confidence. Output module 33 is used to output the action recognition result when the recognition confidence meets the preset confidence conditions; The adjustment module 34 is used to adjust the preset time window according to the recognition confidence level when the recognition confidence level does not meet the preset confidence level conditions, so as to re-acquire the action recognition result and recognition confidence level according to the step of returning to the division of the time-series hot event frame sequence according to the adjusted time window.

[0058] Furthermore, the adjustment module 34 includes: A calculation unit is configured to calculate a feedback factor based on the identification confidence level, wherein the feedback factor is negatively correlated with the identification confidence level; An extraction unit is used to extract the real-time event rate of the event stream infrared signal; An adjustment unit is used to adjust the preset time window based on the feedback factor and the real-time event rate to obtain the adjusted time window.

[0059] Furthermore, the device also includes: The update module is used to update the number of times the confidence level is continuously abnormal when the identification confidence level does not meet the preset confidence level conditions. The first generation module is used to generate and send a first mode switching control command to the infrared event sensor if the number of times the updated confidence level continues to be abnormal is greater than a preset number threshold, so as to adjust the working mode of the infrared event sensor from event mode to frame mode. The static recognition module is used to construct a static spatiotemporal map based on the acquired continuous frame infrared signals in frame mode, and to perform action recognition on the static spatiotemporal map to obtain action recognition results and recognition confidence. The second generation module is used to generate and send a second mode switching control command to the infrared event sensor when the obtained recognition confidence level meets the preset confidence level condition, so as to restore the working mode of the infrared event sensor to the event mode.

[0060] Furthermore, the static recognition module includes: The first extraction unit is used to extract key points in each frame of infrared image in the continuous frame infrared signal through a pose estimation model, construct a static spatiotemporal graph with each key point as a node and the connection relationship of human skeletal structure as a spatial edge, and extract the spatial relationship between each node in the static spatiotemporal graph through a graph convolutional network to obtain the global spatial structure features between key points. The second extraction unit is used to construct the global spatial structure features of the global infrared image into a global spatial structure feature sequence according to the time sequence, and capture the coherent motion trajectory of each key point over time through the temporal modeling network to obtain the global temporal evolution features of the action. The global feature fusion unit is used to fuse the global spatial structure features and the global temporal evolution features, and to classify and predict the global fused features to obtain the action recognition result and recognition confidence.

[0061] Furthermore, the dynamic recognition module 32 includes: The filtering unit is used to extract key points and the associated infrared event rate within the receptive field of each key point from the time-series thermal event frame sequence, and to determine key points whose associated infrared event rate is greater than a preset infrared event rate threshold as thermal key points. The first construction unit is used to recalculate the spatial edges in the initial dynamic spatiotemporal graph based on the spatial positional relationship between the thermal key points, and to recalculate the temporal edges in the initial dynamic spatiotemporal graph based on the evolution of the same thermal key point in consecutive frames, so as to obtain the dynamic spatiotemporal graph.

[0062] Furthermore, the dynamic recognition module 32 also includes: The third extraction unit is used to determine the local thermal subgraph where the hot node is located from the dynamic spatiotemporal graph, and extract the spatial relationship between each node in the local thermal subgraph through a graph convolutional network to obtain the local spatial structure relationship features between real-time motion key points. The fourth extraction unit is used to capture the motion trajectory of each node in the local heat map over time through a two-stream asynchronous network, and obtain the local temporal evolution characteristics of the action. The local feature fusion unit is used to fuse the local spatial structure relationship features and the local temporal evolution features, and to classify and predict the local fused features through the classifier to obtain the action recognition result and recognition confidence.

[0063] Furthermore, the dynamic recognition module 32 also includes: The fifth extraction unit is used to extract key points from the initial thermal event frame image through the pose estimation model; The second building unit is used to construct an initial dynamic spatiotemporal graph with each key point as a node, the connection relationship of the human skeletal structure as a spatial edge, and the continuous position of the same node in time as a temporal edge.

[0064] This invention provides a human motion recognition device based on an infrared event sensor. In this embodiment, the device acquires event stream infrared signals collected by an infrared event sensor in an event mode within a target monitoring area. These event stream infrared signals are then divided into a time-series thermal event frame sequence according to a preset time window. A dynamic spatiotemporal graph is constructed based on the time-series thermal event frame sequence, and motion recognition is performed on the dynamic spatiotemporal graph to obtain motion recognition results and recognition confidence levels. If the recognition confidence level meets a preset confidence level condition, the motion recognition result is output. If the recognition confidence level does not meet the preset confidence level condition, the preset time window is adjusted according to the recognition confidence level, returning to the step of dividing the time-series thermal event frame sequence according to the adjusted time window to re-acquire motion recognition results and recognition confidence levels. This significantly reduces data redundancy and computational burden caused by continuous image acquisition and processing in traditional video surveillance, lowering system power consumption and storage requirements. Simultaneously, it ensures the adaptability and robustness of the motion recognition process in complex or dynamic scenarios, thereby improving the efficiency of human motion recognition.

[0065] According to one embodiment of the present invention, a storage medium is provided, the storage medium storing at least one executable instruction, the computer-executable instruction being able to execute the human motion recognition method based on an infrared event sensor in any of the above method embodiments.

[0066] Figure 4 The diagram shows a structural schematic of a terminal according to an embodiment of the present invention. The specific implementation of the present invention is not limited to the specific implementation of the terminal.

[0067] like Figure 4 As shown, the terminal may include: a processor 402, a communication interface 404, a memory 406, and a communication bus 408.

[0068] The processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408.

[0069] Communication interface 404 is used for network communication with other devices such as clients or other servers.

[0070] The processor 402 is used to execute program 410, which can specifically execute the relevant steps in the above embodiment of the human motion recognition method based on infrared event sensors.

[0071] Specifically, program 410 may include program code that includes computer operation instructions.

[0072] Processor 402 may be a central processing unit (CPU), a specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The terminal includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0073] Memory 406 is used to store program 410. Memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0074] Specifically, program 410 can be used to cause processor 402 to perform the following operations: The infrared event stream signals collected by the infrared event sensors in the target monitoring area in event mode are acquired, and the event stream infrared signals are divided into a time-series thermal event frame sequence according to a preset time window. A dynamic spatiotemporal graph is constructed based on the time-series thermal event frame sequence, and action recognition is performed on the dynamic spatiotemporal graph to obtain the action recognition results and recognition confidence. If the recognition confidence level meets the preset confidence level conditions, the action recognition result is output. If the recognition confidence does not meet the preset confidence condition, the preset time window is adjusted according to the recognition confidence, so as to return to the step of dividing the time-series hot event frame sequence according to the adjusted time window to re-acquire the action recognition result and recognition confidence.

[0075] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0076] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for human motion recognition based on an infrared event sensor, characterized in that, include: The infrared event stream signals collected by the infrared event sensors in the target monitoring area in event mode are acquired, and the event stream infrared signals are divided into a time-series thermal event frame sequence according to a preset time window. A dynamic spatiotemporal graph is constructed based on the time-series thermal event frame sequence, and action recognition is performed on the dynamic spatiotemporal graph to obtain the action recognition results and recognition confidence. If the recognition confidence level meets the preset confidence level conditions, the action recognition result is output. If the recognition confidence does not meet the preset confidence condition, the preset time window is adjusted according to the recognition confidence, so as to return to the step of dividing the time-series hot event frame sequence according to the adjusted time window to re-acquire the action recognition result and recognition confidence. After adjusting the preset time window based on the recognition confidence level, the method further includes: If the identification confidence level does not meet the preset confidence level conditions, update the number of times the confidence level remains abnormal; If the number of times the updated confidence level remains abnormal exceeds a preset threshold, a first mode switching control command is generated and sent to the infrared event sensor to adjust the working mode of the infrared event sensor from event mode to frame mode. In frame mode, a static spatiotemporal map is constructed based on the acquired continuous frame infrared signals, and action recognition is performed on the static spatiotemporal map to obtain the action recognition result and recognition confidence level. If the obtained recognition confidence level meets the preset confidence level condition, a second mode switching control command is generated and sent to the infrared event sensor to restore the working mode of the infrared event sensor to the event mode. A static spatiotemporal map is constructed based on the acquired continuous frame infrared signals, and action recognition is performed on the static spatiotemporal map to obtain action recognition results and recognition confidence levels, including: For each frame of infrared image in a continuous frame of infrared signal, key points in the infrared image are extracted by a pose estimation model. A static spatiotemporal graph is constructed with each key point as a node and the connection relationship of the human skeletal structure as a spatial edge. The spatial relationship between each node in the static spatiotemporal graph is extracted by a graph convolutional network to obtain the global spatial structure features between key points. The global spatial structure features of the global infrared image are constructed into a global spatial structure feature sequence according to the time sequence, and the continuous motion trajectory of each key point over time is captured by the temporal modeling network to obtain the global temporal evolution features of the action. By fusing the global spatial structure features and the global temporal evolution features, and classifying and predicting the global fused features, action recognition results and recognition confidence are obtained; Perform action recognition on the dynamic spatiotemporal graph to obtain action recognition results and recognition confidence, including: From the dynamic spatiotemporal graph, the local thermal subgraph where the hot nodes are located is determined, and the spatial relationship between each node in the local thermal subgraph is extracted by the graph convolutional network to obtain the local spatial structure relationship features between real-time motion key points. By capturing the motion trajectory of each node in the local heatmap over time using a two-stream asynchronous network, the local temporal evolution characteristics of the action are obtained. By fusing the local spatial structure relationship features and the local temporal evolution features, and using a classifier to classify and predict the local fused features, action recognition results and recognition confidence are obtained.

2. The method according to claim 1, characterized in that, Adjusting the preset time window based on the recognition confidence level includes: A feedback factor is calculated based on the identification confidence level, wherein the feedback factor is negatively correlated with the identification confidence level; Extract the real-time event rate of the event stream infrared signal; The preset time window is adjusted based on the feedback factor and the real-time event rate to obtain the adjusted time window.

3. The method according to claim 1, characterized in that, Constructing a dynamic spatiotemporal graph based on the time-series thermal event frame sequence includes: Extract key points and associated infrared event rates within the receptive field of each key point from the time-series thermal event frame sequence, and identify key points whose associated infrared event rates are greater than a preset infrared event rate threshold as thermal key points. Based on the spatial positional relationship between the thermal key points, the spatial edges in the initial dynamic spatiotemporal graph are recalculated, and based on the evolution of the same thermal key point in consecutive frames, the temporal edges in the initial dynamic spatiotemporal graph are recalculated to obtain the dynamic spatiotemporal graph.

4. The method according to claim 3, characterized in that, The process of constructing the initial dynamic spatiotemporal graph includes: Key points are extracted from the initial thermal event frame image using a pose estimation model; An initial dynamic spatiotemporal graph is constructed using each key point as a node, the connection relationship of the human skeletal structure as a spatial edge, and the continuous positions of the same node in time as a temporal edge.

5. A human motion recognition device based on an infrared event sensor, characterized in that, The device is used to perform the operations corresponding to the human motion recognition method based on an infrared event sensor as described in any one of claims 1-4, including: The segmentation module is used to acquire the event stream infrared signals collected by the infrared event sensors in the target monitoring area in event mode, and to segment the event stream infrared signals into a time-series thermal event frame sequence according to a preset time window. The dynamic recognition module is used to construct a dynamic spatiotemporal graph based on the time-series thermal event frame sequence, and to perform action recognition on the dynamic spatiotemporal graph to obtain action recognition results and recognition confidence. The output module is used to output the action recognition result when the recognition confidence meets the preset confidence conditions; The adjustment module is used to adjust the preset time window according to the recognition confidence level when the recognition confidence level does not meet the preset confidence level conditions, so as to re-acquire the action recognition result and recognition confidence level according to the step of returning to the division of the time-series hot event frame sequence according to the adjusted time window.

6. A storage medium, characterized in that, The storage medium stores at least one executable instruction, which causes the processor to perform the operation corresponding to the human motion recognition method based on an infrared event sensor as described in any one of claims 1-4.

7. A terminal, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the human motion recognition method based on an infrared event sensor as described in any one of claims 1-4.

Images