Activity monitoring method, electronic device and computer program product

Abstract

The invention provides an activity monitoring method, electronic equipment and a computer program product. The activity monitoring method comprises the following steps: acquiring first link data of a first monitoring area and second link data of a second monitoring area within a preset time length; generating activity monitoring data based on the first link data and the second link data; and according to the activity monitoring data, determining the activity state of the first monitoring area and / or the second monitoring area within the preset time duration. According to the activity monitoring method, the two links collected at different positions are adopted for monitoring, the monitoring range can be effectively increased, and the monitoring success rate can be effectively increased.

Description

Technical Field

[0001] This application relates to the field of behavior recognition technology, and in particular to an activity monitoring method, electronic device, and computer program product. Background Technology

[0002] WiFi-based wireless local area networks (WLANs) have been widely used indoors, and their services extend far beyond information communication. Rapidly emerging applications, such as human detection, indoor positioning, and through-wall tomography activity recognition, are constantly revolutionizing the application areas of WLANs. In indoor environments, transmitted wireless signals typically do not reach the receiver along a direct path. In fact, the received signal is usually a superposition of multiple signals formed by reflection, diffraction, and scattering from furniture, people, and other obstacles—a phenomenon known as multipath propagation. While physical space limits the propagation of wireless signals, correspondingly, wireless signals can also be used to sense the physical environment they pass through. Both environmental objects (such as walls and furniture) and human bodies (such as position and movement) can "modulate" wireless signals, forming periodic or time-varying signals. By analyzing these signals, environmental perception can be inferred.

[0003] However, current WiFi environmental sensing solutions are mostly designed for indoor environments, with limited monitoring range and relatively large errors in the monitoring results. Summary of the Invention

[0004] To at least address the aforementioned technical problems, this application proposes an activity monitoring method, an electronic device, and a computer program product.

[0005] To at least address the aforementioned technical problems, this application proposes an activity monitoring method, which includes:

[0006] Within a preset time period, acquire the first link data of the first monitoring area and the second link data of the second monitoring area;

[0007] Based on the first link data and the second link data, activity monitoring data is generated;

[0008] Based on the activity monitoring data, confirm the activity status of the first monitoring area and / or the second monitoring area within the preset time period.

[0009] To address the aforementioned technical problems, this application also proposes an electronic device, which includes a memory and a processor coupled to the memory; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the activity monitoring method described above.

[0010] To address the aforementioned technical problems, this application also proposes a computer program product, comprising a computer program that, when executed by a processor, implements the aforementioned activity monitoring method.

[0011] Compared with existing technologies, the beneficial effects of this application are: the electronic device acquires first link data of a first monitoring area and second link data of a second monitoring area within a preset time period; based on the first link data and the second link data, it generates activity monitoring data; and according to the activity monitoring data, it confirms the activity status of the first monitoring area and / or the second monitoring area within the preset time period. By using the above activity monitoring method, which employs two links collected from different locations for monitoring, the monitoring range and success rate can be effectively increased. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] in:

[0014] Figure 1 This is a schematic diagram of an embodiment of the activity monitoring scenario provided in this application;

[0015] Figure 2 This is a flowchart illustrating an embodiment of the activity monitoring method provided in this application;

[0016] Figure 3 This is a schematic diagram of the overall framework of the activity monitoring method provided in this application;

[0017] Figure 4 yes Figure 2 The diagram shows the specific flow chart of step S12 in the behavior detection method.

[0018] Figure 5 yes Figure 4 The diagram shows the detailed process flow of step S121 in the behavior detection method.

[0019] Figure 6 This is a schematic diagram of the training process of the lightweight machine learning model provided in this application;

[0020] Figure 7 This is a framework flowchart of an embodiment of the indoor intrusion detection method provided in this application.

[0021] intention;

[0022] Figure 8This is a flowchart illustrating an embodiment of the indoor intrusion detection method provided in this application.

[0023] intention;

[0024] Figure 9 This is the framework flow of another embodiment of the indoor intrusion detection method provided in this application.

[0025] Schematic diagram;

[0026] Figure 10 This is the overall flow of another embodiment of the indoor intrusion detection method provided in this application.

[0027] Schematic diagram;

[0028] Figure 11 This is the overall flow of another embodiment of the indoor intrusion detection method provided in this application.

[0029] Schematic diagram;

[0030] Figure 12 This is a schematic diagram of an embodiment of the activity monitoring device provided in this application;

[0031] Figure 13 This is a schematic diagram of the structure of an embodiment of the computer program product provided in this application. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0033] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] The main method used in this application for environmental perception using WiFi signals is to acquire and analyze CSI (Channel State Information) or RSSI (Received Signal Strength Indication) data. CSI is an estimate of the wireless network card's response to the wireless channel frequency. According to the IEEE 802.11n standard, wireless signals can be modulated into 56 or 114 subcarriers using orthogonal frequency division multiplexing. CSI can be seen as an estimate of the channel gain of each subcarrier by the wireless network card. CSI contains amplitude and phase information. Compared with RSSI (Signal Strength Information), CSI is a fine-grained RSSI value, which can accurately characterize the change of each channel state and has higher accuracy.

[0035] Therefore, the link data in this application can be either CSI data or RSSI data. RSSI is a single-dimensional metric that represents the strength of the received signal. It is typically expressed in dBm (decibels per milliwatt). CSI is a multi-dimensional metric that provides more detailed information about the channel state. It includes multiple parameters such as phase and amplitude on the subcarriers. Both RSSI and CSI are indicators used to measure wireless signals, but RSSI provides overall information about signal strength, while CSI provides more detailed and multi-dimensional channel state information.

[0036] In general wireless communication, RSSI might be used as a simple connection quality indicator. However, in applications with higher requirements for signal characteristics and requiring more in-depth analysis, such as indoor positioning and gesture recognition, CSI is more useful. Therefore, the following explanation uses CSI data as an example; the specific process is equally applicable to RSSI data.

[0037] In one exemplary embodiment, this application proposes an indoor intrusion detection device and method based on WiFi sensing. It uses a WiFi wireless sensing device to detect continuous actions of people entering a room through a door, determining whether an intrusion has occurred. This can at least solve the problems of false alarms and missed alarms caused by detecting a single action. Continuous actions can include outdoor activities of people or other target objects, opening and closing doors, opening and closing windows, and indoor activities. By establishing a one-to-two WiFi detection link, the indoor and outdoor detection range is covered.

[0038] In one exemplary implementation, an established machine learning model, such as a lightweight detection learning model, can be used to detect indoor and outdoor human activity in real time. When an intrusion is suspected, the system combines the detection of door opening and closing actions within a specific timeframe to determine if such actions have occurred. If so, it identifies an indoor intrusion or abnormal behavior and notifies the user or alerts personnel at the scene via an app or other means. In another implementation, the system can also quickly confirm actions or activities such as entering or leaving the room through doors / windows by comparing CSI-related information from both the amplitude, frequency, and phase changes with preset action values.

[0039] In one embodiment, the WiFi one-to-two sensing and detection device may include a pair of transmitters Tx (signal transmitting device) and receivers Rx1 (first signal receiving device) arranged indoors, and a receiver Rx2 (second signal receiving device) arranged outside the door. The transmitters Tx may be WiFi routers, CPEs (Customer Premise Equipment), etc.; the receivers may be specially configured WiFi sensing receivers. As a smart home application, the indoor receiver Rx1 may be a smart home device with WiFi connectivity, and the outdoor receiver Rx2 may be a smart doorbell, smart light, etc.

[0040] In another exemplary embodiment, the first link data of the first monitoring area and the second link data of the second monitoring area of ​​this application can be composed of two independent transceiver devices. Unlike the above-mentioned WiFi one-to-two sensing and detection device, this application chooses to deploy one set of sensing and detection transceiver devices in the first monitoring area and another set of sensing and detection transceiver devices in the second monitoring area, that is, to use two WiFi networks as signal transmission devices for different monitoring areas.

[0041] Further, please refer to Figure 1 , Figure 1 This is a schematic diagram of an embodiment of the activity monitoring scenario provided in this application.

[0042] like Figure 1 As shown, a one-to-many detection system is deployed inside and outside the room (or inside and outside the window, or any other applicable scenario). The indoor system includes one transmitter (Tx) and one receiver (Rx1), while the outdoor system includes one receiver (Rx2), forming a first link and a second link. The first link transmits data indoors, detecting activity near the indoor door; the second link passes through the doorway and wall, detecting activity near the outdoor door. Both links together cover the indoor and outdoor detection range. Rx1 must be greater than the vertical distance from Rx2 to the door to enhance the detection range and activity recognition accuracy. Different types of doors are used in the room, including hinged doors and sliding doors. Hinged doors include inward-opening, outward-opening, left-opening, and right-opening types.

[0043] like Figure 1 In the scenario shown, the first monitoring area is an indoor area, and the second monitoring area is an outdoor area. In other embodiments, the first and second monitoring areas are different monitoring areas, and there is at least a partial separation between them, such as... Figure 1 The walls and doors.

[0044] See Figure 2 and Figure 3 , Figure 2 This is a flowchart illustrating an embodiment of the activity monitoring method provided in this application. Figure 3 This is a schematic diagram of the overall framework of the activity monitoring method provided in this application.

[0045] The activity monitoring method of this application can be applied to an activity monitoring device, which can be a server, a terminal device, an electronic device, or a system in which the server and the terminal device cooperate with each other. Accordingly, the various parts of the activity monitoring device, such as each unit, subunit, module, and submodule, can all be set in the server, all in the terminal device, or separately in the server and the terminal device.

[0046] Furthermore, the aforementioned server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules, such as software or software modules used to provide distributed server functionality, or as a single software program or software module; no specific limitations are made here.

[0047] In one exemplary embodiment, the activity monitoring system containing the activity monitoring device of this application further includes a signal transmitting device, a first signal receiving device, and a second signal receiving device. The signal transmitting device and the first signal receiving device are located in a first monitoring area, and the second signal receiving device is located in a second monitoring area.

[0048] like Figure 2 As shown, its activity monitoring methods may include the following operations.

[0049] Step S11: Within a preset time period, acquire the first link data of the first monitoring area and the second link data of the second monitoring area.

[0050] In this embodiment of the application, the activity monitoring device collects CSI data of two links in real time or periodically within a set preset time length or a set window time, namely the first link data between the signal transmitting device and the first signal receiving device, and the second link data between the signal transmitting device and the second signal receiving device.

[0051] In one exemplary implementation, a WiFi detection device can be deployed before activity monitoring. A WiFi router is placed indoors, and an AP (Access Point) hotspot is activated as a transmitter (Tx). Two IoT (Internet of Things) devices, Rx1 and Rx2, are placed indoors and outdoors respectively, connected to the AP hotspot. The first link is located indoors, and the second link passes through doors and walls, together covering the indoor and outdoor detection range.

[0052] The vertical distance from device Rx1 to the door is greater than that from device Rx2, enhancing the accuracy of indoor and outdoor activity recognition. Door opening and closing actions originate outside the door, classifying this action as an outdoor activity. The WiFi devices operate in the 5GHz band, with both links operating on the same channel and a bandwidth of 80MHz. Each transceiver has two antennas, and each detection link has four data streams, each capable of collecting CSI data from 256 subcarriers.

[0053] Step S12: Generate activity monitoring data based on the first link data and the second link data.

[0054] In this embodiment, the activity monitoring device obtains activity monitoring data about the monitored area by analyzing and comparing first link data and second link data. This activity monitoring data is used for subsequent analysis of specific personnel activity behavior information. For example, the activity monitoring data can be used as feature input to a lightweight machine learning model to predict the specific type of personnel activity behavior. Alternatively, the activity monitoring data can be used by the activity monitoring device for data analysis of the first and second link data, i.e., analyzing information such as the amplitude and phase of the link data to determine whether there are environmental changes within the monitored area.

[0055] In addition, the activity monitoring device can also preset calculation logic or conversion table, which can calculate the specific type of personnel activity behavior by using the preset calculation logic to calculate the activity monitoring data; or it can input the numerical representation of the activity monitoring data into the conversion table to directly search for the specific type of corresponding personnel activity behavior.

[0056] In one exemplary embodiment, this application provides a method embodiment for generating activity monitoring data; please refer to [link to embodiment]. Figure 4 , Figure 4 yes Figure 2 The flowchart of step S12 of the behavior detection method is shown.

[0057] like Figure 4 As shown, it may include the following steps:

[0058] Step S121: Obtain the first subcarrier amplitude of the first link data and the second subcarrier amplitude of the second link data.

[0059] In this embodiment, the activity monitoring device extracts the first subcarrier amplitude of the first link data and the second subcarrier amplitude of the second link data. The first and second link data are represented as waveform data, which consists of amplitude and phase information. The activity monitoring method of this application primarily analyzes the amplitude information of the link data.

[0060] Furthermore, after acquiring the first link data and the second link data, the activity monitoring device can extract the CSI amplitude information within the interval, remove abnormal data, and then perform normalization processing.

[0061] In one exemplary embodiment, the time interval used in this application is 3 seconds, and the sampling frequency is 50Hz. Other values ​​may be used in other embodiments, which will not be elaborated here.

[0062] Due to hardware and environmental noise, the CSI data collected by the activity monitoring device contains outliers. The Hampel algorithm can be used to remove the outlier CSI data from the two links and normalize the data.

[0063] It should be noted that when analyzing the amplitude information of link data, this application can analyze the amplitude of all subcarriers of the link data, or it can analyze the amplitude of some subcarriers of the link data.

[0064] When analyzing the amplitude of some subcarriers in the link data, this application also involves the selection of high-quality subcarriers in the link data. Furthermore, since the interference and noise received by the first link data are less than those received by the second link, the activity monitoring device of this application selects the optimal subcarriers based on the first link data. Then, based on the carrier sequence number of the optimal subcarriers selected from the first link data, it confirms the subcarriers to be analyzed in the second link data. For example, if the first link signal transmission is mainly a LOS path (direct path), and the second link passes through doors and walls with more noise, the optimal 20 subcarriers are calculated based on the CSI amplitude data of the first link, and the 20 subcarriers selected for the second link CSI amplitude data are confirmed based on the carrier sequence number.

[0065] Please refer to details. Figure 5 , Figure 5 yes Figure 4 The diagram shows the specific flow chart of step S121 of the behavior detection method.

[0066] like Figure 5 As shown, its activity monitoring methods may include the following operations.

[0067] Step S1211: Calculate the ratio of the amplitudes of all effective subcarriers in several data streams of the first link data, where the ratio is the ratio of the amplitude mean to the amplitude variance.

[0068] In this embodiment, the activity monitoring device calculates the ratio of the mean to the variance of the CSI sampled data amplitudes of all valid subcarriers in the four data streams of the link. In another exemplary embodiment, the link may also include other numbers of data streams, which are not specifically limited here.

[0069] Step S1212: Select the data stream with the smallest sum of ratios in the first link data as the optimal data stream.

[0070] In this embodiment, the activity monitoring device calculates the sum of all carrier ratios for each data stream, and the data stream with the smallest sum is the optimal data stream.

[0071] Step S1213: Select a preset number of subcarriers in the optimal data stream as the optimal subcarriers based on the ratio.

[0072] In this embodiment, the activity monitoring device selects the top 20 subcarriers by ratio from the optimal data stream as the optimal subcarriers. The number of optimal subcarriers can be set according to actual needs.

[0073] Step S1214: Obtain the first subcarrier amplitude of the first link data and the second subcarrier amplitude of the second link data based on the optimal subcarrier.

[0074] In this embodiment, the activity monitoring device obtains the carrier sequence number of the optimal subcarrier, and extracts the first subcarrier amplitude of the first link data and the second subcarrier amplitude of the second link data based on the carrier sequence number.

[0075] In one exemplary embodiment, the activity monitoring device can also denoise the link data, i.e., remove high-frequency noise from the CSI data of the two links using different algorithms. For example, to reduce computational load, the first link data can be filtered using a Butterworth low-pass filter, with the bandwidth threshold obtained based on experimental experience. The second link passes through a doorway, where noise increases, and can be filtered using a Discrete Wavelet Transform (DWT) to provide higher frequency resolution and perform high-frequency subband filtering.

[0076] Step S122: Obtain the deviation value between the amplitude of the first subcarrier and the amplitude of the second subcarrier, wherein the deviation value includes the amplitude variance and / or the absolute deviation of the amplitude median.

[0077] In this embodiment, the activity monitoring device acquires the deviation values ​​of the first link data and the second link data, respectively. The deviation values ​​of the link data include, but are not limited to, the variance and median absolute deviation of the subcarrier amplitude.

[0078] Step S123: Use the deviation value to generate activity monitoring data.

[0079] In this embodiment, the activity monitoring device encodes the deviation value of the link data in step S122 to obtain the feature input of the machine learning model, i.e., the activity monitoring data. Additionally, since the second link experiences attenuation, the activity monitoring device can also add the subcarrier correlation coefficient of the second link data as a feature input to the machine learning model.

[0080] Step S13: Based on the activity monitoring data, confirm the activity status of the first monitoring area and / or the second monitoring area within the preset time period.

[0081] In this embodiment, the activity monitoring device can use a lightweight machine learning model to classify human behavior in the monitored area, such as indoor and outdoor activities, based on the input feature data, i.e., the feature input data generated in steps S121 to S123 above. The lightweight machine learning model used in this application can be, but is not limited to, SVM (Support Vector Machine) and LightGBM (Light Gradient Boosting Machine).

[0082] Please see Figure 3 , Figure 3 The example diagram for online device detection illustrates the entire process of implementing online detection of link data using a lightweight machine learning model, as described above. Additionally, Figure 3 Offline training in this context refers to the entire process of training the lightweight machine learning model used in online detection.

[0083] Please refer to details. Figure 6 , Figure 6 This is a schematic diagram of the training process for the lightweight machine learning model provided in this application.

[0084] like Figure 6 As shown, its training process may include the following operations.

[0085] Step S21: Obtain the real activity status information, third link data and fourth link data within the preset time length.

[0086] The real activity status information, third-link data, and fourth-link data are all training sample data. After training, the third-link and fourth-link data can be manually labeled to generate real activity status information. Real activity status information includes personnel activity events occurring in the monitored area during the collection of third-link and fourth-link data, serving as real values ​​for training the activity status detection model.

[0087] In this embodiment of the application, within the experimental environment and within a set window time, the activity monitoring device collects CSI data from two links under four different activity scenarios of the test personnel to obtain training sample data. Specifically, the indoor and outdoor personnel activities in this application are categorized into four scenarios: outdoor activity only, outdoor activity only, activity both indoors and outdoors, and no activity indoors or outdoors.

[0088] In the offline training phase, the steps for data preprocessing, optimal subcarrier acquisition, data denoising, and feature extraction of link data are the same as the data processing process during online device detection, and will not be repeated here.

[0089] Furthermore, because the completion times of actions by different testers are inconsistent, the corresponding CSI data dimensions for each action are inconsistent. Therefore, before using the link data for training, the activity monitoring device needs to extract action segment data, that is, extract action data from the collected data to obtain the start and end times of each action segment. The action segment data extraction method is as follows: within the action time period, a sliding window is set, the root mean square error of the waveform amplitude within the sliding window is calculated to form a new sequence waveform, and the mean of the new waveform is calculated. The minimum and maximum times of the intersection of the weighted mean line and the waveform are the start and end times of the action.

[0090] In one exemplary embodiment, the activity monitoring device acquires one optimal subcarrier from each of the two link data points: the first optimal subcarrier of the third link data and the second optimal subcarrier of the fourth link data. The activity monitoring device selects a sliding window and calculates the variance of the CSI amplitude data of the optimal subcarrier of the third link data within the sliding window, forming a variance sequence waveform. The mean M1 of this variance sequence is then calculated. Next, the activity monitoring device selects the coefficient k1 of the third link data and calculates the intersection point of the variance waveform and the weighted mean line k1M1. The minimum time t at the intersection point is calculated. 11 and the maximum time t 12 These are the start and end times of the third link data action, i.e., the first candidate time point.

[0091] Similarly, the activity monitoring device selects a sliding window and calculates the variance of the optimal subcarrier CSI amplitude data of the fourth link within the active window, forming a variance sequence waveform. The mean M2 of this variance sequence is then calculated. Next, the activity monitoring device selects the coefficient k2 of the fourth link data and calculates the intersection point of the variance waveform and the weighted mean line k2M2. The minimum time t at the intersection point is determined. 21 and the maximum time t 22 These are the start and end times of the fourth link data action, i.e., the second candidate time point.

[0092] Finally, the activity monitoring device calculates [min(t)] 11 ,t 21 ),max(t 12 ,t 22 [)] represents the start and end times of the action, i.e., the time points of the action segment. The activity monitoring device can extract action segment data from the third and fourth link data for model training.

[0093] After extracting motion segment data, the activity monitoring device also needs to perform motion data normalization, resampling and scaling of the motion data to unify the dimensions of the motion data.

[0094] In one exemplary implementation, the duration of door opening and closing actions varies between different people or different instances, and the corresponding CSI values ​​for these action segments also differ. To train a machine learning model, the action data needs to be regularized and its dimensions unified. The algorithm is as follows:

[0095] (1) Interpolate the CSI data segments corresponding to the actions.

[0096] (2) Specify the standard duration of the door opening and closing action, and stretch or scale the action segment data to make its duration the standard duration.

[0097] (3) Resample the action segment data to obtain action segment data of the same dimension.

[0098] The activity monitoring device uses the algorithm described above to acquire CSI action segment data corresponding to the 20 optimal subcarriers of the third link data and the fourth link data, respectively.

[0099] Step S22: Generate training data based on the third link data and the fourth link data.

[0100] In this embodiment, the activity monitoring device extracts the data features of the third-link data and the fourth-link data to generate the feature input of the personnel activity detection model.

[0101] Step S23: Input the training data into the activity state detection model to obtain the predicted activity state information.

[0102] In this embodiment, the activity monitoring device generates training sample data according to the above algorithm, wherein the training sample data is represented as the amplitude sequence of CSI. The personnel activity detection model, i.e., the activity state detection model, in this embodiment can use a Residual Network (ResNet) as a deep learning model.

[0103] Step S24: Train the activity state detection model based on the difference between the real activity state information and the predicted activity state information.

[0104] In this embodiment, the activity monitoring device uses the feature values ​​of the actual activity state information as the target value and the feature values ​​of the predicted activity state information as the predicted value. Then, it calculates the loss value for this training using the difference between the target value and the predicted value, i.e., the difference information. If the loss value is less than a preset threshold, it indicates that the activity state detection model has achieved the desired training effect, and the training phase ends. If the loss value is greater than the preset threshold, it indicates that the activity state detection model has not achieved the desired training effect, and the loss value is further used to guide parameter optimization of the activity state detection model, continuing the iterative process.

[0105] In this embodiment, the electronic device acquires first link data of a first monitoring area and second link data of a second monitoring area within a preset time period; based on the first link data and the second link data, it generates activity monitoring data; and according to the activity monitoring data, it confirms the activity status of the first monitoring area and / or the second monitoring area within the preset time period. By using the above activity monitoring method, which employs two links collected from different locations for monitoring, the monitoring range and success rate can be effectively increased.

[0106] The personnel activity detection model trained and used based on the above embodiments can distinguish between four situations: outdoor activities only, outdoor activities only, activities both indoors and outdoors, and no activities both indoors and outdoors. Using this function, this application can also realize indoor intrusion detection.

[0107] The indoor intrusion detection method provided in this application is as follows:

[0108] S1: When the user leaves home, the router's security mode is activated to perform indoor intrusion detection. This can be manually set by the user or automatically activated based on the user's absence indicator.

[0109] S2: The two links of the WiFi sensing device acquire CSI data in real time.

[0110] S3: Process CSI data at predetermined time intervals, input the data into the indoor activity detection model, and obtain the activity status of people indoors and outdoors.

[0111] S4: Based on the changes in indoor and outdoor activities, determine whether to detect whether a door opening or closing action occurs within the change interval.

[0112] S5: Determine if an intrusion has occurred based on the continuous movement of indoor and outdoor activities and door opening / closing actions. If the door opening / closing is initiated by the homeowner, no intrusion message is reported. The determination method is the user's arrival / departure status.

[0113] S6: When the user arrives home, turn off the router's security mode and stop indoor intrusion detection.

[0114] The indicators for a user arriving home include, but are not limited to: the user connecting to a Wi-Fi hotspot, the smart lock closing, and the user's mobile phone entering the electronic fence. Conversely, the indicators for a user leaving home include, but are not limited to: the user disconnecting from the Wi-Fi hotspot, the smart lock opening, and the user's mobile phone leaving the electronic fence.

[0115] Furthermore, the app includes a whitelist function, allowing users to add people to the whitelist. When a whitelisted member is detected entering the home, no indoor intrusion alarm will be triggered. When someone is detected entering, users can also individually verify their legitimacy; if authorized, the alarm will be temporarily deactivated.

[0116] Current theoretical research suggests using CSI data to detect habitual movements and identify individuals, such as gait, heart rate, arm swing frequency, duration of habitual movements, and corresponding CSI waveforms. However, in practical commercial applications, the accuracy may be low due to the diversity of individuals. If future technology matures, it may also be possible to identify users based on CSI information, confirm their legitimacy, and store the corresponding CSI characteristics of legitimate individuals in a whitelist.

[0117] Please refer to details. Figure 7 and Figure 8 , Figure 7 This is a schematic flowchart of an embodiment of the indoor intrusion detection method provided in this application. Figure 8 This is a schematic diagram of the overall process of an embodiment of the indoor intrusion detection method provided in this application.

[0118] like Figure 7 and Figure 8 As shown, its indoor intrusion detection method may include the following operations.

[0119] Step S31: Based on the activity status of the first monitoring area and / or the second monitoring area, obtain the time interval between the repeated occurrence of the same type of activity status in different time windows, wherein the monitoring area where the same type of activity status occurs is the same.

[0120] In this embodiment, the activity monitoring device learns the type and duration of personnel activity based on personnel activity behavior information. Therefore, the activity monitoring device can analyze the time intervals between repeated occurrences of the same type of personnel activity behavior within different preset time windows. For example, when the type of personnel activity behavior of someone being outside a door is repeated, the activity monitoring device can record the time interval between these recurrences.

[0121] In one exemplary implementation, please refer to Figure 8 The logic for the activity monitoring device to record the time intervals between repeated occurrences of someone being active outside the door is as follows:

[0122] Step S311: The activity monitoring device detects personnel activity within a set time window, i.e. Figure 2 The embodiment detects the activity status within the monitored area. Additionally, the activity monitoring device creates time points T1 and T2 in the register.

[0123] Step S312: When the activity monitoring device detects that there is no activity inside or outside the door, it clears the time T1 in the register.

[0124] Step S313: When the activity monitoring device detects that there is only activity outside the door, proceed to step S314.

[0125] Step S314: The activity monitoring device determines whether the time T1 value in the register is empty. If yes, proceed to step S315; otherwise, proceed to step S316.

[0126] Step S315: The activity monitoring device updates the time T1 in the register to the latest time, which represents the first monitoring time when the activity state of only people moving outside the door is recorded.

[0127] Step S316: The activity monitoring device updates the time T2 in the register to the latest time, indicating the latest monitoring time of the continuously occurring activity state where only people are active outside the door. At this time, the time interval in step S31 is determined by T2-T1.

[0128] Step S32: If the confirmed time interval is greater than the preset threshold, confirm that the activity status of the first monitoring area and / or the second monitoring area is abnormal.

[0129] In this embodiment, the activity monitoring device determines whether the time interval determined in step S31 is greater than a set threshold. If so, it confirms that the activity status of the first monitoring area and / or the second monitoring area is abnormal. If not, no prompt is given, and the detection process continues.

[0130] In one exemplary implementation, please refer to Figure 8The logic for the activity monitoring device to confirm whether an abnormal activity state has occurred is as follows:

[0131] Step S317: The activity monitoring device determines whether the time interval between T2 and T1 is greater than a set threshold. If so, proceed to step S318.

[0132] Step S318: The activity monitoring device alerts the user to an abnormal activity status indicating that someone outside the door is behaving abnormally for an extended period of time.

[0133] In this embodiment, the activity monitoring device can confirm the presence of repeated human activity within a given time interval, which can be identified as abnormal activity. For example, the activity monitoring device can alert the user to unusual activity outside the door for an extended period.

[0134] In addition, when users arrive home and turn off the security functions, they can also turn on the outdoor activity detection function separately. If the algorithm detects that someone is active outdoors for a long time, it will send a message to the user through the APP to remind them to pay attention.

[0135] Please refer to details. Figure 9 , Figure 10 and Figure 11 , Figure 9 This is a schematic flowchart illustrating another embodiment of the indoor intrusion detection method provided in this application. Figure 10 This is a schematic flowchart of another embodiment of the indoor intrusion detection method provided in this application. Figure 11 This is a schematic diagram of the overall process of another embodiment of the indoor intrusion detection method provided in this application.

[0136] like Figure 9 As shown, its indoor intrusion detection method may include the following operations.

[0137] Step S41: Based on the activity status of the first monitoring area and / or the second monitoring area, obtain the time intervals in which different types of activity statuses occur in different time windows, wherein the monitoring areas where different types of activity statuses occur are different.

[0138] In this embodiment, the activity monitoring device learns the type and duration of personnel activity based on personnel activity behavior information. Therefore, the activity monitoring device can analyze the time intervals between different types of personnel activity behavior occurring within different preset time windows. For example, when there is a personnel activity type outside a door, a personnel activity type inside a door, or a personnel activity type occurring both inside and outside a door, the activity monitoring device can record the time interval between their occurrences.

[0139] In one exemplary implementation, please refer to Figure 10 The logic for the activity monitoring device to record the time intervals between different types of activity states is as follows:

[0140] Step S411: The activity monitoring device detects personnel activity within a set time window, i.e. Figure 2 The example demonstrates the activity status detected within the monitored area.

[0141] Step S412: When the activity monitoring device detects activity with only someone outside the door within the set time window, it updates the time T1 in the register. The logic of step S4112 includes: when the activity monitoring device first detects an activity state with only someone outside the door, it creates a new time T1 in the register, and then continuously monitors the activity state with only someone outside the door, using the time of continuously detected activity state with only someone outside the door to continuously update the time T1 in the register, so that the time T1 in the register records the latest time of the activity state with only someone outside the door.

[0142] Step S413: The activity monitoring device determines whether an activity state of only indoor human activity is detected in the adjacent time window after time T1. If not, time T1 is cleared from the register; if so, time T2 is created in the register. At this time, the time interval in step S41 is determined by T2-T1.

[0143] In one exemplary implementation, please refer to Figure 11 The logic for the activity monitoring device to record the time intervals between different types of activity states is as follows:

[0144] Step S421: The activity monitoring device detects personnel activity within a set time window, i.e. Figure 2 The example demonstrates the activity status detected within the monitored area.

[0145] Step S422: When the activity monitoring device detects activity with only someone outside the door within the set time window, it updates the time T1 in the register. The logic of step S4122 includes: when the activity monitoring device first detects activity with only someone outside the door, it creates a new time T1 in the register; then it continuously monitors the activity with only someone outside the door and uses the times of continuously detected activity with only someone outside the door to continuously update the time T1 in the register, so that the time T1 in the register records the latest time of the activity with only someone outside the door.

[0146] Step S423: The activity monitoring device determines whether the adjacent time window after time T1 detects activity inside or outside the door. If not, time T1 is cleared from the register; if so, time T2 is created in the register. At this time, the time interval in step S41 is determined by T2-T1.

[0147] Step S42: If an activity state of the first preset type occurs within the confirmation time interval, confirm that the activity state of the first monitoring area and / or the second monitoring area is an abnormal state.

[0148] In this embodiment, the activity monitoring device determines whether a preset type of personnel activity behavior, such as opening or closing a door, occurs within the time interval determined in step S41. If such behavior exists, the activity status of the first monitoring area and / or the second monitoring area is confirmed to be abnormal. If not, interference or intrusion behavior is suspected, and the CSI data within the interval is saved for analysis.

[0149] In one exemplary implementation, please refer to Figure 10 The logic for the activity monitoring device to confirm whether an abnormal activity state has occurred is as follows:

[0150] Step S414: The activity monitoring device determines whether a door opening or closing action occurs during the time interval T2-T1. If yes, proceed to step S415; otherwise, proceed to step S416.

[0151] Step S415: The activity monitoring device alerts the user to an abnormal activity status indicating an indoor intrusion.

[0152] Step S416: Confirm suspected interference and intrusion behavior, and analyze the CSI data within the save interval.

[0153] In one exemplary implementation, please refer to Figure 11 The logic for the activity monitoring device to confirm whether an abnormal activity state has occurred is as follows:

[0154] Step S424: The activity monitoring device determines whether a door opening or closing action occurs during the time interval T2-T1. If yes, proceed to step S425; otherwise, proceed to step S426.

[0155] Step S425: The activity monitoring device alerts the user to an abnormal activity status indicating multiple intrusion attempts indoors.

[0156] Step S426: Confirm suspected interference and intrusion behavior, and analyze the CSI data within the save interval.

[0157] In this embodiment, the activity monitoring device alerts the user to an indoor intrusion, or an intrusion by multiple people.

[0158] In addition, when an intrusion occurs, users can also verify whether the intruder is a family member or a legitimate person. If so, they can choose to add the member to the whitelist or temporarily verify the user's legitimacy and temporarily lift the alert.

[0159] The activity monitoring method of this application uses WiFi sensing devices for intrusion detection, protecting privacy and eliminating the need to deploy dedicated sensor devices; it performs intrusion detection based on continuous human actions, improving the success rate; it uses two links for detection, increasing the detection range and also increasing the success rate; it uses a lightweight learning model for human activity detection and a deep learning model for action detection, which are only activated when necessary, reducing the consumption of system computing power.

[0160] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0161] To implement the above-mentioned activity monitoring method, this application also proposes an electronic device, for details please refer to [link / reference needed]. Figure 12 , Figure 12 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application.

[0162] The activity monitoring device 400 in this embodiment includes a processor 41, a memory 42, an input / output device 43, and a bus 44.

[0163] The processor 41, memory 42, and input / output device 43 are respectively connected to the bus 44. The memory 42 stores program data, and the processor 41 is used to execute the program data to implement the activity monitoring method described in the above embodiments.

[0164] In this embodiment, processor 41 can also be referred to as a CPU (Central Processing Unit). Processor 41 may be an integrated circuit chip with signal processing capabilities. Processor 41 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor, or processor 41 can be any conventional processor.

[0165] This application also provides a computer program product; please refer to further details. Figure 13 , Figure 13This is a schematic diagram of the structure of an embodiment of the computer program product provided in this application. The computer program product 600 stores a computer program 61, which, when executed by a processor, is used to implement the activity monitoring method of the above embodiment.

[0166] When the embodiments of this application are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0167] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method of activity monitoring, characterized by, The method comprises: acquiring first link data of a first monitoring area and second link data of a second monitoring area within a preset time length; generating activity monitoring data based on the first link data and the second link data; confirming an activity state of the first monitoring area and / or the second monitoring area within the preset time length according to the activity monitoring data.

2. The method according to claim 1, wherein the confirming an activity state of the first monitoring area and / or the second monitoring area within the preset time length according to the activity monitoring data comprises: inputting the activity monitoring data into an activity state detection model to acquire an activity state of the first monitoring area and / or the second monitoring area within the preset time length output by the activity state detection model; the method further comprises: acquiring real activity state information, third link data and fourth link data within the preset time length; generating training data based on the third link data and the fourth link data; inputting the training data into the activity state detection model to acquire predicted activity state information; training the activity state detection model based on difference information of the real activity state information and the predicted activity state information.

3. The method according to claim 2, wherein after the acquiring real activity state information, third link data and fourth link data within the preset time length, the method further comprises: acquiring a first optimal subcarrier in the third link data and a second optimal subcarrier in the fourth link data; acquiring a first variance sequence waveform within a sliding window based on the first optimal subcarrier; determining a first candidate time point of the third link data based on an amplitude mean value of the first variance sequence waveform; acquiring a second variance sequence waveform within a sliding window based on the second optimal subcarrier; determining a second candidate time point of the fourth link data based on an amplitude mean value of the second variance sequence waveform; determining an action segment time point based on the first candidate time point and the second candidate time point; extracting action segment data from the third link data and the fourth link data respectively according to the action segment time point, wherein the action segment data is used for training the activity state detection model.

4. The method according to claim 1, further comprising: acquiring a time interval in which the same type of activity state repeatedly appears in different time windows based on the activity state of the first monitoring area and / or the second monitoring area, wherein the same type of activity state appears in the same monitoring area; confirming that the activity state of the first monitoring area and / or the second monitoring area is an abnormal state if it is confirmed that the time interval is greater than a preset threshold.

5. The method according to claim 4, wherein ​ ​ ​ The time interval in which the same type of activity state repeatedly occurs in different time windows is obtained based on the activity state of the first monitoring area and / or the second monitoring area, including: a first monitoring time when the first monitoring area or the second monitoring area first appears in a preset type of activity state is obtained; a second monitoring time when the last time the preset type of activity state appears in the same monitoring area after the first monitoring time is obtained; the time interval is determined based on the first monitoring time and the second monitoring time.

6. The activity monitoring method of claim 1, further comprising: obtaining a time interval in which different types of activity states appear in different time windows based on the activity state of the first monitoring area and / or the second monitoring area, wherein the monitoring areas in which the different types of activity states appear are different; in a case where it is confirmed that the first preset type of activity state appears in the time interval, confirming that the activity state of the first monitoring area and / or the second monitoring area is an abnormal state.

7. The activity monitoring method of claim 6, wherein the time interval in which different types of activity states appear in different time windows is obtained based on the activity state of the first monitoring area and / or the second monitoring area, including: a third monitoring time when the first monitoring area last appears in a second preset type of activity state is obtained; a fourth monitoring time when the second monitoring area first appears in a third preset type of activity state is obtained; the time interval is determined based on the third monitoring time and the fourth monitoring time.

8. The activity monitoring method of claim 7, wherein the third monitoring time when the first monitoring area last appears in the second preset type of activity state is obtained, including: when the first monitoring area first appears in the second preset type of activity state, the third monitoring time is recorded; when the second preset type of activity state appears again in the next time window, the third monitoring time is updated using the monitoring time of the subsequent time window.

9. The activity monitoring method of claim 8, wherein the first preset type of activity state is a door opening and closing action, the second preset type of activity state is a person outside the door activity, and the third preset type of activity state is a person inside the door activity or a person inside and outside the door activity. The electronic device includes a memory and a processor coupled to the memory; 10. An electronic device, comprising: wherein the memory is configured to store program data, and the processor is configured to execute the program data to implement the activity monitoring method of any one of claims 1 to 9. A computer program is included, which, when executed by a processor, implements the activity monitoring method of any one of claims 1 to 9.

11. A computer program product, characterised in that, ​

Images