Action recognition method and apparatus, and device and storage medium

By using the signal feature modeling and identification results calibration of multiple WiFi transmitters in a wireless mesh network, the problem of low recognition accuracy of single-point WiFi signals in complex scenarios is solved, and more accurate recognition of human behavior is achieved.

WO2024244649A1PCT designated stage expired Publication Date: 2024-12-05ZTE CORP
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2024/083819
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-31
Filing Date
2024-03-26
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

In the prior art, in human behavior recognition, it is difficult to provide accurate identification results in complex scenarios based on a single WiFi signal, and the lack of calibration objects leads to difficulty in improving the recognition accuracy.

Method used

Using a wireless mesh (Mesh) network, the signal feature modeling and identification results calibration of multiple WiFi transmitters is adopted, combined with channel status information and received signal strength indication, and multiple calibrations are performed using a behavior recognition model to determine the behavior of the object to be identified.

Benefits of technology

It improves the accuracy and accuracy of human behavior recognition, reduces the uncertainty of recognition results, and is suitable for behavior monitoring in complex scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024083819_05122024_PF_FP_ABST
    Figure CN2024083819_05122024_PF_FP_ABST
Patent Text Reader

Abstract

Provided in the present disclosure are an action recognition method and apparatus, and a device and a storage medium. The method comprises: acquiring signal parameter information of each node in a wireless mesh network, wherein the wireless mesh network comprises at least two nodes; determining a plurality of action recognition results on the basis of the signal parameter information of each node; and according to the plurality of action recognition results, determining an action of an object to be recognized, wherein the action of said object is determined on the basis of target action recognition results, with a number thereof greater than or equal to a preset number, among the plurality of action recognition results.
Need to check novelty before this filing date? Find Prior Art

Description

Behavior recognition method, device, equipment and storage medium

[0001] This disclosure claims priority to Chinese patent application No. 202310647985.6, filed on May 31, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present disclosure relates to the field of wireless communication technologies, and in particular to a behavior recognition method, apparatus, device, and storage medium. Background Art

[0003] With the popularization of the Internet of Things, passive behavior recognition has been widely studied due to its ease of use in human-computer interaction, smart elderly care and other fields. For example, based on the perception of WiFi signals, people's behavior can be identified and their physical condition can be understood in a timely manner.

[0004] Summary of the Invention

[0005] In a first aspect, a behavior recognition method is provided. The behavior recognition method includes: obtaining signal parameter information of each node in a wireless mesh network, the wireless mesh network including at least two nodes; determining multiple behavior recognition results based on the signal parameter information of each node; and determining the behavior of an object to be recognized based on the multiple behavior recognition results, wherein the behavior of the object to be recognized is determined based on a number of target behavior recognition results greater than or equal to a preset number among the multiple behavior recognition results.

[0006] In a second aspect, a behavior recognition device is provided. The behavior recognition device includes an acquisition unit and a determination unit. The acquisition unit is configured to acquire signal parameter information of each node in a wireless mesh network, the wireless mesh network including at least two nodes; the determination unit is configured to determine multiple behavior recognition results based on the signal parameter information of each node; and the determination unit is further configured to determine the behavior of an object to be recognized based on the multiple behavior recognition results, wherein the behavior of the object to be recognized is determined based on a number of target behavior recognition results greater than or equal to a preset number among the multiple behavior recognition results.

[0007] In a third aspect, an electronic device is provided, comprising: a processor and a memory for storing processor-executable instructions; wherein the processor is configured to execute instructions so that the electronic device performs the behavior recognition method as described in the first aspect above.

[0008] In a fourth aspect, a computer-readable storage medium is provided, on which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the behavior recognition method as described in the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] To more clearly illustrate the technical solutions of the present disclosure, the following briefly introduces the drawings required for use in some embodiments of the present disclosure. Obviously, the drawings described below are only drawings of some embodiments of the present disclosure, and those skilled in the art can also derive other drawings based on these drawings.

[0010] FIG1 is a schematic diagram of a Mesh network scenario according to some embodiments.

[0011] FIG2 is a schematic diagram of Mesh network coverage according to some embodiments.

[0012] FIG3 is a flowchart of a behavior recognition method according to some embodiments.

[0013] FIG4 is a schematic diagram of a WiFi data message structure according to some embodiments.

[0014] FIG5 is a flowchart of a behavior recognition method according to some embodiments.

[0015] FIG6 is a schematic diagram of packet request information according to some embodiments.

[0016] FIG7 is a schematic structural diagram of a behavior recognition device according to some embodiments.

[0017] FIG8 is a schematic structural diagram of an electronic device according to some embodiments. DETAILED DESCRIPTION

[0018] The following will clearly and completely describe the technical solutions of this disclosure in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this disclosure, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of this disclosure without making any creative efforts shall fall within the scope of protection of this disclosure.

[0019] It should be noted that in this disclosure, words such as "exemplarily" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this disclosure as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplarily" or "for example" is intended to present related concepts in some way.

[0020] In the following, the terms "first," "second," etc. are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the quantity of the technical features indicated. Therefore, a feature specified as "first," "second," etc. may explicitly or implicitly include one or more of the features.

[0021] In the description of this disclosure, unless otherwise specified, " / " means "or." For example, A / B can mean A or B. "And / or" herein is merely a description of an association relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: only A, only B, and both A and B. Furthermore, "at least one" means one or more, and "a plurality" means two or more.

[0022] Behavior recognition technology is a technology that automatically detects, analyzes, and understands the behavioral actions of the identified object (such as speaking / gesturing / falling / breathing). It is widely used in emerging fields such as smart homes, security monitoring, medical rehabilitation, and human-computer interaction.

[0023] Behavior recognition is generally categorized as contact-based and contactless. Wearable devices are key enablers of contact-based human behavior recognition technology, but they face limitations such as high cost, inconvenience, and distraction. In contrast, contactless behavior recognition offers device-free sensing services and user-friendly interactions, and holds a promising future. Related contactless behavior recognition methods primarily rely on cameras and radio signals. The former is limited by lighting conditions, line-of-sight interference, blind spots, and privacy concerns, while the latter typically requires the deployment of specialized experimental equipment, making it difficult to scale up.

[0024] Currently, as part of urban infrastructure, commercial wireless network (WiFi) devices provide wireless signals with advantages such as ubiquity, ease of use, high cost-effectiveness, and long transmission distance. In recent years, these signals have garnered widespread attention in the field of human behavior recognition. Related technologies that use WiFi signals to detect human activity typically employ the Received Signal Strength Indicator (RSSI) technique to detect human activity. However, this technique has a relatively coarse granularity and cannot accurately identify human behavior.

[0025] Furthermore, related technologies typically rely on a single WiFi signal transmission point, analyzing the signal from that transmission point to determine human behavior. However, in complex real-world scenarios, such as those with numerous obstacles or walls, WiFi signals can be easily interfered with, leading to inaccurate recognition results from analyzing the signal from that transmission point. Furthermore, the amount of data from a single transmission point is limited, and there is no reference or calibration object to determine the accuracy of the recognition results, making it difficult to improve recognition accuracy.

[0026] In view of this, the present disclosure provides a behavior recognition method, device, equipment and storage medium, which, in combination with a wireless mesh network, provides multiple WiFi transmitters (for example, at least two or more). Each transmission point in the Mesh network can be interconnected. The present disclosure models the signal characteristics of each transmission point separately and calibrates the recognition results to obtain more accurate human activity perception and recognition results.

[0027] The following describes in detail the behavior recognition method provided by the embodiments of the present disclosure in conjunction with the accompanying drawings. It should be noted that the various embodiments of the present disclosure can refer to each other, for example, the same or similar steps, method embodiments, system embodiments, and device embodiments can all refer to each other without limitation.

[0028] The behavior recognition method provided by the embodiment of the present disclosure can be applied to a Mesh network scenario. FIG1 shows a schematic diagram of the Mesh network scenario.

[0029] As shown in FIG1 , the Mesh network scenario may include a behavior recognition device 11 and multiple nodes 12 . The behavior recognition device 11 may communicate with each node 12 .

[0030] Node 12 can be any device capable of transmitting and receiving wireless signals (e.g., a WiFi device). As shown in Figure 2, a mesh network is deployed indoors, comprising WiFi devices A, B, ..., G. The object to be identified is located indoors (i.e., within the coverage area of ​​the mesh network). The behavior of the object to be identified can affect the wireless signals of each WiFi device.

[0031] In an exemplary embodiment, the connection mode of the Mesh network may be at least that the Mesh network sub-node is connected to the wireless access point (AP) of the Mesh network master node through its own station, and transmits data to and from the AP through its own station.

[0032] The behavior recognition device 11 can obtain signal parameter information from each node 12, determine a behavior recognition result based on the signal parameter information, obtain multiple behavior recognition results, and determine the behavior of the object to be recognized based on the multiple behavior recognition results. The example behavior recognition process can refer to the behavior recognition method described in the following method embodiment and will not be repeated here.

[0033] The behavior recognition device 11 can implement the behavior recognition method of the present embodiment in various electronic devices. For example, the electronic device can be a mobile phone, a tablet computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a desktop computer, a cloud server, etc. The present embodiment does not limit the specific type of electronic device.

[0034] In different application scenarios, the behavior recognition device 11 and the electronic device can be independent devices or integrated into the same device, which is not specifically limited in the embodiments of the present disclosure.

[0035] When the behavior recognition device 11 and the electronic device are integrated into the same device, the data transmission between the behavior recognition device 11 and the electronic device is carried out as data transmission between the internal modules of the device. In this case, the data transmission process between the two is the same as the data transmission process between the behavior recognition device 11 and the electronic device when the two are independent of each other.

[0036] The application scenarios of the embodiments of the present disclosure are not limited. The system architecture and business scenarios described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Those skilled in the art will appreciate that with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present disclosure are equally applicable to similar technical problems.

[0037] In the following embodiments provided in the present disclosure, the behavior recognition device 11 and the electronic device are independently configured as an example for description.

[0038] Figure 3 is a flow chart of a behavior recognition method according to some exemplary embodiments. In some embodiments, the behavior recognition method described below can be applied to the aforementioned behavior recognition device and electronic device, as well as other similar devices.

[0039] As shown in FIG3 , the behavior recognition method provided by the embodiment of the present disclosure may include the following steps:

[0040] S201: Acquire signal parameter information of each node in the wireless mesh network.

[0041] Signal parameter information is used to reflect the wireless signal characteristics of a node. For example, the signal parameter information can reflect the waveform, duration, amplitude of each frequency component, frequency, phase, etc. of the WiFi signal transmitted by the node.

[0042] As a possible implementation method, the behavior recognition device obtains WiFi message information of each node in the wireless mesh network, and parses the signal parameter information of each node from the WiFi message.

[0043] For example, as shown in FIG4 , the behavior recognition device obtains a WiFi data message from WiFi transmitter A and adds a flag to the WiFi data message, identified as A. The behavior recognition device can parse the WiFi data message to obtain channel state information (CSI) data, received signal strength indication (RSSI) data, and the like.

[0044] It should be noted that, in general, the receiving end evaluates the CSI and quantifies it and feeds it back to the transmitting end (in a time-division duplex system, reverse evaluation is required). Therefore, the WiFi message information of each node usually contains CSI data, which at least includes the amplitude and phase of the carrier signal.

[0045] In some embodiments, the behavior recognition device may further perform noise removal processing on the signal parameter information to obtain valid data.

[0046] For example, for the CSI and RSSI data from WiFi transmitter A, the behavior recognition device adds a flag bit to the corresponding WiFi data packet, identifying it as A. The behavior recognition device then uses the denoising module to remove noise, obtains valid data, and stores the valid data. Similarly, for the CSI and RSSI data from WiFi transmitter B, the behavior recognition device adds a flag bit to the corresponding WiFi data packet, identifying it as B. The behavior recognition device then uses the denoising module to remove noise, obtains valid data, and stores the valid data.

[0047] In practical applications, nonlinear denoising methods can be used to denoise carriers transmitted by different transmitting ends through antennas. For example, the denoising methods include wavelet transform and other methods.

[0048] S202: Determine multiple behavior recognition results based on signal parameter information of each node.

[0049] As a possible implementation method, the behavior recognition device determines the channel state information CSI data in each signal parameter information to obtain multiple CSI data. Further, the behavior recognition device determines each behavior recognition result based on each CSI data to obtain multiple behavior recognition results.

[0050] It can be understood that the behavior recognition device extracts physical-layer channel state information (CSI) from Wi-Fi signals and determines the behavior recognition result based on CSI. CSI can characterize the multipath link changes caused by the perceived target, thereby utilizing subcarrier-level amplitude and phase information to achieve more complex and sophisticated behavior recognition, such as indoor activity monitoring, fall detection, gait authentication, or gesture recognition.

[0051] It should be noted that the behavior recognition device may determine the same or different behavior recognition results based on the CSI data of different nodes. Therefore, the accuracy of the behavior recognition results obtained based on only the CSI data of a single transmitter is difficult to guarantee.

[0052] For example, a wireless mesh network includes nodes 1, 2, and 3. Node 1's CSI is CSI-1, node 2's CSI is CSI-2, and node 3's CSI is CSI-3. Based on CSI-1, the behavior recognition device determines that the behavior recognition result is A. Based on CSI-2, the behavior recognition device determines that the behavior recognition result is B. Based on CSI-3, the behavior recognition device determines that the behavior recognition result is B.

[0053] In some embodiments, the behavior recognition device may determine the behavior recognition result corresponding to each CSI data according to the characteristics of each CSI data.

[0054] In some embodiments, the behavior recognition device may further input each CSI data into a behavior recognition model to determine a behavior recognition result corresponding to each CSI data.

[0055] The behavior recognition model can be a pre-trained model for the behavior recognition device or a model pre-stored in the behavior recognition device by operations and maintenance personnel. The behavior recognition model extracts features from CSI data and converts them into quantifiable data such as frequency (for example, converting CSI data into a four-dimensional feature vector based on device name, time value, and subcarrier). Based on this converted data, the model outputs the behavior recognition results.

[0056] For example, the behavior recognition device can input the CSI data of node A, which is closest to a person in a mesh network, at a certain moment into a classification algorithm or hidden Markov model learning algorithm to identify the type of human activity and determine whether the person is currently performing the activity. Similarly, the behavior recognition device can also input the CSI data of other transmitters in the mesh network, such as nodes B...G, into the classification algorithm or hidden Markov model learning algorithm to identify the type of human activity.

[0057] Different behavior recognition models are used to identify different behaviors. For example, within CSI signal data, the behavior recognition device labels human breathing data to build a breathing recognition model. Within CSI signal data, the behavior recognition device labels human posture data to build a behavior recognition model. In other words, the behavior recognition device can train the model based on the different characteristics of the CSI signal and the targeted behavior, resulting in a behavior recognition model that can recognize the target action.

[0058] For example, a wireless mesh network includes nodes 1, 2, and 3. Node 1's CSI is CSI-1, node 2's CSI is CSI-2, and node 3's CSI is CSI-3. The behavior recognition device inputs CSI-1 into a gesture recognition model and determines that the behavior recognition result is hand-raised. The behavior recognition device inputs CSI-2 into the gesture recognition model and determines that the behavior recognition result is hand-raised. The behavior recognition device inputs CSI-3 into the gesture recognition model and determines that the behavior recognition result is hand-not-raised.

[0059] S203: Determine the behavior of the object to be identified based on the multiple behavior recognition results.

[0060] The behavior of the object to be identified is determined based on target behavior identification results whose number is greater than or equal to a preset number among the multiple behavior identification results. One behavior identification result can correspond to one behavior.

[0061] It should be noted that the objects to be identified in the embodiments of the present disclosure can be any active object within the coverage area of ​​the wireless mesh network, including but not limited to people and pets. For example, the behavior recognition method of the embodiments of the present disclosure can successfully identify human behavior within the coverage area of ​​the wireless mesh network.

[0062] As a possible implementation, the behavior recognition device obtains the number of each behavior in the plurality of behavior recognition results and determines behaviors whose number is greater than or equal to a preset number as behaviors of the object to be recognized.

[0063] In actual applications, the preset number can be set in advance by the operation and maintenance personnel in the behavior recognition device. For example, if more than half of the results of multiple behavior recognition are breathing or a certain gesture, the behavior recognition device will determine the result as the behavior of the object to be recognized.

[0064] In some embodiments, the behavior recognition device may obtain the distance between each node in the wireless mesh network and the object to be recognized, and based on the distance between each node and the object to be recognized, assign a weight to the behavior recognition result corresponding to each node. For example, the node with a smaller distance to the object to be recognized may have a larger weight value. Furthermore, the behavior recognition device may determine the behavior of the object to be recognized based on multiple behavior recognition results and their corresponding weights. For example, the behavior recognition device may determine the behavior of the object to be recognized based on one or more nodes with higher weights.

[0065] In some embodiments, to enhance the reference value of the determined behavior recognition results, the behavior recognition device determines a target node in the wireless mesh network. The distance between the target node and the object to be recognized is greater than or equal to a preset distance. The behavior recognition device discards the behavior recognition results corresponding to the target node. Furthermore, the behavior recognition device determines the behavior of the object to be recognized from the remaining behavior recognition results. An example of this determination method can be found in the above embodiments and will not be further described here.

[0066] In an embodiment of the present disclosure, signal parameter information of each node in a wireless mesh network is obtained to obtain multiple signal parameter information, thereby understanding the wireless signal characteristics of each node. Furthermore, based on each signal parameter information, a behavior recognition result is determined to obtain multiple behavior recognition results. The present disclosure determines the behavior of the object to be identified based on multiple behavior recognition results. The behavior of the object to be identified is the behavior of which the number in the multiple behavior recognition results is greater than or equal to a preset number. Compared with the related art that performs behavior recognition based on the signal of a single WiFi transmitting point, the present disclosure combines a WiFi Mesh network to provide multiple WiFi transmitting terminals, and performs behavior recognition based on the signal of each WiFi transmitting terminal to obtain multiple behavior recognition results. Furthermore, the present disclosure refers to multiple behavior recognition results and determines the behavior of the object to be identified that is greater than or equal to the preset number in the multiple behavior recognition results. The amount of data obtained by a transmitting terminal is limited, and there is no reference or calibration object for whether the recognition result is accurate. The present disclosure refers to multiple transmitting terminals to obtain a more accurate behavior of the object to be identified.

[0067] In one design, to improve the accuracy of the behavior recognition model, as shown in FIG5 , the behavior recognition method provided by the embodiment of the present disclosure further includes:

[0068] S301: Obtain a training sample set and a prediction sample set.

[0069] The training sample set includes a plurality of first sample CSI data, where one first sample CSI data corresponds to one first sample behavior; and the prediction sample set includes a plurality of second sample CSI data.

[0070] As a possible implementation, the training sample set and the prediction sample set can be pre-stored in the behavior recognition device by the operation and maintenance personnel, or the behavior recognition device can obtain them from the Internet. To improve the accuracy of model training, the first sample CSI data is different from the second sample CSI data.

[0071] S302: Based on the training sample set, obtain a behavior recognition model through training.

[0072] As a possible implementation method, the behavior recognition device can use the first sample CSI data as input and the first sample behavior as a label to train the behavior recognition model to be trained, thereby obtaining a trained behavior recognition model.

[0073] S303 : Determine, based on the behavior recognition model, an initial behavior recognition result corresponding to each second sample CSI data in the prediction sample set, to obtain multiple initial behavior recognition results.

[0074] As a possible implementation manner, the behavior recognition device inputs each second sample CSI data into the behavior recognition model respectively to obtain multiple initial behavior recognition results.

[0075] It should be noted that the second sample CSI data may be historical CSI data of each node in the Mesh network, and the first sample CSI data may be CSI data pre-collected by operation and maintenance personnel and stored in the behavior recognition device.

[0076] For example, the first sample behavior corresponding to each first sample CSI data is raising a hand or not raising a hand. When training the behavior recognition model, the behavior recognition device uses the first sample CSI data as input and the first sample behavior of the first sample CSI data as a label to train the behavior recognition model.

[0077] In practical applications, the behavior recognition device can obtain CSI data from each node at a certain historical moment and use this CSI data as the second sample CSI data. The behavior recognition device inputs each second sample CSI data into the aforementioned behavior recognition model to determine whether each second sample CSI data corresponds to a hand-raising or no hand-raising (corresponding to the first and second behaviors, respectively), thereby obtaining multiple initial behavior recognition results.

[0078] It is understood that in the disclosed embodiments, the behavior recognition device can initially train a behavior recognition model using multiple first sample CSI data and multiple first sample behaviors. However, this behavior recognition model can only be trained for a single CSI and output an initial behavior recognition result. The accuracy of this initial behavior recognition result is not verified by any reference or calibration object, making it difficult to improve recognition accuracy.

[0079] S304: Adjust the behavior recognition model based on the multiple initial behavior recognition results to obtain an adjusted behavior recognition model.

[0080] As a possible implementation method, the behavior recognition device determines the confidence of the first behavior and the confidence of the second behavior. When the confidence of the first behavior is greater than the confidence of the second behavior, the behavior recognition device determines the initial behavior recognition result corresponding to the second behavior as the result to be verified, and adjusts the label of the CSI data corresponding to the result to be verified to the first behavior, and continues to train the behavior recognition model to obtain an adjusted behavior recognition model. When the confidence of the first behavior is less than the confidence of the second behavior, the behavior recognition device determines the initial behavior recognition result corresponding to the first behavior as the result to be verified, and adjusts the label of the CSI data corresponding to the result to be verified to the second behavior, and continues to train the behavior recognition model to obtain an adjusted behavior recognition model.

[0081] It should be noted that the behavior recognition device can determine the confidence level of the first behavior and the confidence level of the second behavior based on the initial behavior recognition result of each second sample CSI data according to the behavior recognition model.

[0082] For example, the behavior recognition device can determine the distance between the object to be identified and each node in the mesh network based on the RSSI value and assign different weights to nodes at different distances, where the closer the distance, the heavier the weight. Furthermore, the behavior recognition device uses the sum of the weights of nodes whose initial behavior recognition result is the first behavior as the confidence level of the first behavior, and the behavior recognition device uses the sum of the weights of nodes whose initial behavior recognition result is the second behavior as the confidence level of the second behavior.

[0083] In some embodiments, the behavior recognition device can determine the distance between each node and the object to be recognized. The behavior recognition device determines the weight of each node based on the distance between each node and the object to be recognized. The weight is negatively correlated with the distance. The behavior recognition device determines the sum of the weights of each first node as the confidence level of the first behavior. The initial behavior recognition result corresponding to the first node is the first behavior. The behavior recognition device determines the sum of the weights of each second node as the confidence level of the second behavior. The initial behavior recognition result corresponding to the second node is the second behavior.

[0084] In a first example, assume there are three node devices a, b, and c in a Mesh network. Node a is closest to the human body, so its weight can be set to 0.5. Node b is the next closest, so its weight can be set to 0.4. Node c is the furthest from the human body, so its weight can be set to 0.2. For a specific action, such as breathing, if the initial recognition result corresponding to node a is yes, while the initial recognition results corresponding to nodes b and c are no, but the combined weights of nodes b and c are 0.6, which is greater than the weight of node a of 0.5, the initial recognition result corresponding to node a (yes breathing) is determined as the result to be verified, and the label of node a's CSI data is adjusted to not breathing. The behavior recognition model continues to be trained to obtain the adjusted behavior recognition model.

[0085] In the second example, the initial recognition result corresponding to node A in the Mesh network is action a, while the initial recognition results corresponding to other nodes are action b. Node A needs to calibrate the recognition result. The behavior recognition device classifies the CSI data of node A as action b, and continues to train the behavior recognition model to obtain an adjusted behavior recognition model.

[0086] Understandably, in related technologies, the amount of data available to a single transmitter is limited, and there's no reference or calibration object to determine the accuracy of the recognition results, making it difficult to improve recognition accuracy. The disclosed embodiments improve the accuracy of the behavior recognition model by individually modeling and calibrating the CSI data of WiFi nodes.

[0087] Accordingly, the behavior recognition device can input each CSI data point into the adjusted behavior recognition model to obtain multiple behavior recognition results. Compared to the pre-adjusted behavior recognition model, the adjusted behavior recognition model improves its recognition accuracy by cross-calibrating the recognition results of different CSI data points, making the adjusted behavior recognition model more suitable for mesh network scenarios containing multiple nodes. Therefore, the disclosed embodiments can utilize the adjusted behavior recognition model to obtain more accurate human activity perception and recognition results.

[0088] In one design, in order to determine the distance between each node and the object to be identified, the behavior recognition device obtains the received signal strength information of each node, and determines the distance between each node and the object to be identified based on the received signal strength information of each node.

[0089] In practical applications, the behavior recognition device can obtain RSSI from the signal parameter information of each node and determine the distance between each node and the object to be identified based on the angle-of-arrival (AOA) technology.

[0090] It should be noted that the positioning algorithm based on signal arrival angle is a typical ranging-based positioning algorithm. It uses certain hardware devices to sense the arrival direction of the transmitting node signal, calculates the relative direction or angle between the receiving node and the anchor node, and then uses triangulation or other methods to calculate the position of the unknown node.

[0091] For example, a node in a mesh network initiates an LL_CTE_REQ PDU packet, as shown in Figure 6. This packet requests information including the minimum fixed frequency extension signal (CTE) length, the CTE type, and radio frequency unit (RFU) information (1 bit reserved). The CTE data in the CTE type is appended to the data packet. The CTE data can be a data string containing a series of switching time slots and sampling time slots. For example, these time slots can be defined as 1us or 2us. The mesh node controls the switching of multiple different antennas based on the angle in the data packet containing the CTE and obtains the receive time slot information of different antennas.

[0092] The behavior recognition device uses the AOA method to calculate the time difference between the broadcast message reaching different antennas and obtain the position information of the human body relative to the node with orientation. This position is the position of the antenna with the shortest arrival time.

[0093] In some embodiments, the behavior recognition device obtains the CSI data of node A in the Mesh network at a certain time t, which is recorded as T a The distance between node A and the human body is determined by using the RSSI and AOA technology of node A and recorded as D a , D a Contains distance and direction information. The weight value is based on D a Quantize: f*D a , f is a pre-determined weight factor. Similarly, the behavior recognition device obtains the CSI data T of each node in the Mesh network at time t n and direction distance D n , and obtain the data shown in Table 1.

[0094] Table 1

[0095] In one design, to ensure that the signal from each node can cover the object to be detected, the behavior recognition method provided in an embodiment of the present disclosure further includes: upon determining that the position of the object to be identified has changed, the behavior recognition device sends an indication message to the master node. The indication message is used to instruct the master node to adjust its transmit power based on the changed position of the object to be identified, and to instruct the master node to broadcast the changed position of the object to be identified to each slave node, instructing each slave node to adjust its transmit power based on the changed position of the object to be identified.

[0096] In practical applications, the behavior recognition device may obtain signal parameter information of the master node at a first moment and signal parameter information at a second moment. If the signal parameter information at the first moment is different from the signal parameter information at the second moment, the behavior recognition device determines that the position of the object to be recognized has changed.

[0097] It is understandable that when a person moves around, the CSI data of each node in the Mesh network and the distance and direction between each node and the person will change. Therefore, the behavior recognition device can determine the position change of the object to be recognized based on the changes in signal parameter information at different times.

[0098] In some embodiments, upon determining a change in the position of an object to be identified, the behavior recognition device sends an indication to the master node. Accordingly, the master node can increase or decrease the weighting factor f based on the distance and direction from the human body. The weighting factor f can be updated in real time or only when the distance from the object to be detected is greater than or equal to a threshold. The master node can also broadcast the location of the object to be identified to subordinate slave nodes via a mesh channel. In response, the slave nodes can increase or decrease the RF power in real time based on this broadcast message to cover the location of the object to be identified.

[0099] Exemplarily, node B increases or decreases the radio frequency power according to the broadcast message of node A, and recalculates the direction distance to the human body, and automatically increases or decreases the weight factor f.

[0100] In some embodiments, for a certain action, a node in the Mesh network can also broadcast its own CSI data (including information such as amplitude, frequency, and channel) and recognition results through the Mesh network broadcast message Beacon. Other nodes receive the message and adjust their own transmission frequency band and channel to maintain consistency. Accordingly, the behavior recognition device can correct the recognition results based on the weight of each node. The calculation formula is as follows:

[0101] A*α1+B*α2+C*α3...+G*αn>1 (or threshold), where AG represent different nodes, and α1-αn represent the weights corresponding to each node.

[0102] The above embodiments mainly introduce the solutions provided by the embodiments of the present disclosure from the perspective of the device (equipment). It is understandable that in order to implement the above methods, the device or equipment includes hardware structures and / or software modules corresponding to the execution of each method flow, and these hardware structures and / or software modules corresponding to the execution of each method flow can constitute an electronic device. It should be easy for those skilled in the art to appreciate that, in combination with the algorithm steps of each example described in the embodiments applied for herein, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present disclosure.

[0103] The embodiments of the present disclosure can divide the functional modules of the device or equipment according to the above method examples. For example, the device or equipment can divide each functional module corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiments of the present disclosure is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0104] FIG7 shows a schematic diagram of a functional module behavior recognition device 40, wherein each functional module is divided into functional modules corresponding to each function. The functional module behavior recognition device 40 can be used to perform the behavior recognition method provided in the above embodiment. As shown in FIG7 , the behavior recognition device 40 includes an acquisition unit 401 and a determination unit 402.

[0105] An acquisition unit 401 is configured to acquire signal parameter information of each node in a wireless mesh network, the wireless mesh network including at least two nodes; a determination unit 402 is configured to determine a plurality of behavior recognition results based on the signal parameter information of each node; and the determination unit 402 is further configured to determine the behavior of an object to be recognized based on the plurality of behavior recognition results, wherein the behavior of the object to be recognized is determined based on a number of target behavior recognition results greater than or equal to a preset number among the plurality of behavior recognition results.

[0106] In an exemplary embodiment, the signal parameter information includes channel state information CSI data; the determination unit 402 can be used to: determine the channel state information CSI data in each signal parameter information to obtain multiple CSI data; determine each behavior recognition result based on each CSI data to obtain the multiple behavior recognition results.

[0107] In an exemplary embodiment, the determining unit 402 may be configured to: obtain the number of each behavior in the plurality of behavior recognition results; and determine behaviors whose number is greater than or equal to the preset number as behaviors of the object to be recognized.

[0108] In an exemplary embodiment, the determination unit 402 is further used to: determine the initial behavior recognition result corresponding to each second sample CSI data based on the behavior recognition model, and obtain multiple initial behavior recognition results; the behavior recognition model is obtained based on multiple first sample CSI data and multiple first sample behaviors trained; one first sample CSI data corresponds to one first sample behavior; based on the multiple initial behavior recognition results, adjust the behavior recognition model to obtain an adjusted behavior recognition model.

[0109] In an exemplary embodiment, the determining unit 402 may be configured to: input each CSI data into the adjusted behavior recognition model respectively to obtain the multiple behavior recognition results.

[0110] In an exemplary embodiment, the multiple initial behavior recognition results include a first behavior and a second behavior; the determination unit 402: determines the confidence of the first behavior and the confidence of the second behavior; when the confidence of the first behavior is greater than the confidence of the second behavior, the initial behavior recognition result corresponding to the second behavior is determined as the result to be verified, and the label of the CSI data corresponding to the result to be verified is adjusted to the first behavior, and the behavior recognition model is continued to be trained to obtain the adjusted behavior recognition model; when the confidence of the first behavior is less than the confidence of the second behavior, the initial behavior recognition result corresponding to the first behavior is determined as the result to be verified, and the label of the CSI data corresponding to the result to be verified is adjusted to the second behavior, and the behavior recognition model is continued to be trained to obtain the adjusted behavior recognition model.

[0111] In an exemplary embodiment, the determination unit 402 can be used to: determine the distance between each node and the object to be identified; determine the weight of each node based on the distance between each node and the object to be identified; the weight is negatively correlated with the distance; determine the sum of the weights of each first node as the confidence of the first behavior; the initial behavior recognition result corresponding to the first node is the first behavior; determine the sum of the weights of each second node as the confidence of the second behavior; the initial behavior recognition result corresponding to the second node is the second behavior.

[0112] In an exemplary embodiment, the determining unit 402 may be configured to: obtain received signal strength information of each node; and determine the distance between each node and the object to be identified based on the received signal strength information of each node.

[0113] In an exemplary embodiment, the wireless mesh network includes a master node and at least one slave node; the determination unit 402 is further used to: when it is determined that the position of the object to be identified has changed, send indication information to the master node; the indication information is used to instruct the master node to adjust the transmission power according to the changed position of the object to be identified, and to instruct the master node to broadcast the changed position of the object to be identified to each slave node, and to instruct each slave node to adjust the transmission power according to the changed position of the object to be identified.

[0114] In an exemplary embodiment, the determination unit 402 can be used to: obtain signal parameter information of the master node at a first moment and signal parameter information at a second moment; the first moment is different from the second moment; when the signal parameter information at the first moment is different from the signal parameter information at the second moment, determine that the position of the object to be identified has changed.

[0115] In the case of implementing the functions of the above-mentioned integrated modules in hardware, the embodiments of the present disclosure provide a possible structure of the electronic device involved in the above-mentioned embodiments. As shown in Figure 8, the electronic device 50 includes: a processor 502 and a bus 504. In an exemplary embodiment, the electronic device 50 may also include a memory 501; in some embodiments of the present disclosure, the electronic device 50 may also include a communication interface 503.

[0116] The processor 502 can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the embodiments of the present disclosure. The processor 502 can be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof, and can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the embodiments of the present disclosure. The processor 502 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0117] The communication interface 503 is used to connect to other devices via a communication network, such as Ethernet, wireless access network, or wireless local area network (WLAN).

[0118] The memory 501 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0119] As a possible implementation, memory 501 can exist independently of processor 502. Memory 501 can be connected to processor 502 via bus 504 to store instructions or program code. When processor 502 calls and executes the instructions or program code stored in memory 501, the behavior recognition method provided in the embodiments of the present disclosure can be implemented.

[0120] In another possible implementation, the memory 501 may also be integrated with the processor 502 .

[0121] Bus 504 can be an Extended Industry Standard Architecture (EISA) bus, etc. Bus 504 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, FIG8 shows only one thick line, but this does not mean that there is only one bus or only one type of bus.

[0122] Some embodiments of the present disclosure provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium), which stores computer program instructions. When the computer program instructions are executed on a computer, the computer executes the behavior recognition method described in any of the above embodiments.

[0123] Exemplarily, the above-mentioned computer-readable storage media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes, etc.), optical disks (e.g., compact disks (CDs), digital versatile disks (DVDs), etc.), smart cards, and flash memory devices (e.g., erasable programmable read-only memories (EPROMs), cards, sticks, or key drives, etc.). The various computer-readable storage media described in the present disclosure may represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0124] An embodiment of the present disclosure provides a computer program product comprising instructions. When the computer program product is run on a computer, the computer is enabled to execute the behavior recognition method described in any one of the above embodiments.

[0125] The above is only a specific embodiment of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or replacements within the technical scope disclosed in the present disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A behavior recognition method, comprising: Acquire signal parameter information of each node in a wireless mesh network, wherein the wireless mesh network includes at least two nodes; Determine multiple behavior recognition results based on signal parameter information of each node; The behavior of the object to be identified is determined according to the multiple behavior recognition results, wherein the behavior of the object to be identified is determined based on target behavior recognition results whose number is greater than or equal to a preset number among the multiple behavior recognition results.

2. The method according to claim 1, wherein: In the case where the signal parameter information includes channel state information CSI data, determining multiple behavior recognition results based on the signal parameter information of each node includes: Determine the channel state information CSI data in each signal parameter information; According to the CSI data corresponding to each node, the behavior recognition result corresponding to each node is determined to obtain the multiple behavior recognition results.

3. The method according to claim 1, wherein: The step of determining the behavior of the object to be identified according to the multiple behavior identification results includes: Count the number of each behavior recognition results; The target behavior recognition results whose number is greater than or equal to the preset number are determined as the behaviors of the objects to be recognized.

4. The method according to claim 2, further comprising: Acquire a training sample set and a prediction sample set; the training sample set includes a plurality of first sample CSI data, one first sample CSI data corresponds to one first sample behavior; the prediction sample set includes a plurality of second sample CSI data; Based on the training sample set, a behavior recognition model is trained; Based on the behavior recognition model, determining an initial behavior recognition result corresponding to each second sample CSI data in the prediction sample set to obtain a plurality of initial behavior recognition results; Based on the multiple initial behavior recognition results, the behavior recognition model is adjusted to obtain an adjusted behavior recognition model.

5. The method according to claim 4, wherein: The determining, according to the CSI data corresponding to each node, the behavior recognition result corresponding to each node, and obtaining the multiple behavior recognition results, includes: Each CSI data is input into the adjusted behavior recognition model to obtain the multiple behavior recognition results.

6. The method according to claim 4, wherein: The multiple initial behavior recognition results include a first behavior and a second behavior; and adjusting the behavior recognition model based on the multiple initial behavior recognition results to obtain an adjusted behavior recognition model includes: determining a confidence level of the first behavior and a confidence level of the second behavior; In a case where the confidence of the first behavior is greater than the confidence of the second behavior, determining the initial behavior recognition result corresponding to the second behavior as the result to be verified, adjusting the label of the CSI data corresponding to the result to be verified to the first behavior, and continuing to train the behavior recognition model to obtain the adjusted behavior recognition model; When the confidence of the first behavior is less than the confidence of the second behavior, the initial behavior recognition result corresponding to the first behavior is determined as the result to be verified, and the label of the CSI data corresponding to the result to be verified is adjusted to the second behavior, and the behavior recognition model is continued to be trained to obtain the adjusted behavior recognition model.

7. The method according to claim 6, wherein: The determining of the confidence level of the first behavior and the confidence level of the second behavior includes: Determine the distance between each node and the object to be identified; Determine the weight of each node according to the distance between each node and the object to be identified; the weight is negatively correlated with the distance; The sum of the weights of the first nodes is determined as the confidence of the first behavior; the initial behavior recognition result corresponding to the first node is the first behavior; The sum of the weights of the second nodes is determined as the confidence of the second behavior; and the initial behavior recognition result corresponding to the second node is the second behavior.

8. The method according to claim 7, wherein: The determining the distance between each node and the object to be identified includes: Obtain the received signal strength information of each node; The distance between each node and the object to be identified is determined according to the received signal strength information of each node.

9. The method according to claim 1, wherein: The wireless mesh network includes a master node and at least one slave node; the method further includes: When it is determined that the position of the object to be identified has changed, an indication message is sent to the master node; the indication message is used to instruct the master node to adjust the transmission power according to the changed position of the object to be identified, and to instruct the master node to broadcast the changed position of the object to be identified to each slave node, and to instruct each slave node to adjust the transmission power according to the changed position of the object to be identified.

10. The method according to claim 9, further comprising: Obtain signal parameter information of the master node at a first moment and signal parameter information at a second moment; The first moment is different from the second moment; When the signal parameter information at the first moment is different from the signal parameter information at the second moment, it is determined that the position of the object to be identified has changed.

11. The method according to claim 1, further comprising: Obtaining the distance between each node in the wireless mesh network and the object to be identified; Based on the distance between each node and the object to be identified, a weight is set for the behavior recognition result corresponding to each node; The behavior of the object to be identified is determined according to the multiple behavior identification results and their corresponding weights.

12. The method according to claim 1, further comprising: determining a target node in the wireless mesh network; The distance between the target node and the object to be identified is greater than or equal to a preset distance; The behavior recognition result corresponding to the target node is discarded.

13. A gateway device, comprising: a processor and a memory for storing instructions executable by the processor; The processor is configured to execute the instruction so that the gateway device executes the behavior recognition method according to any one of claims 1-12.

14. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer is enabled to execute the behavior recognition method according to any one of claims 1 to 12.

Citation Information

Patent Citations

  • WiFi-based indoor human body action detection method and WiFi-based indoor human body action detection system

    CN107331136A

  • K-rank criterion multi-link data fusion algorithm based on credibility weighting

    CN111510943A

  • Indoor personnel action recognition method and system, medium, equipment and terminal

    CN114423034A

  • Device-free activity identification using fine-grained WIFI signatures

    US20160277529A1