Behavior recognition method and device, equipment and storage medium

By using signal characteristics modeling and calibration of multiple WiFi transmitters in wireless mesh networks, the problem of inaccurate recognition caused by WiFi signal interference in complex scenarios is solved, and the accuracy and accuracy of behavior recognition are improved.

CN119071839APending Publication Date: 2024-12-03ZTE CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310647985.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-31
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing behavior recognition methods based on WiFi signals are susceptible to obstacle interference in complex scenarios, resulting in inaccurate identification results and limited data reliance on a single transmission point, making it difficult to improve recognition accuracy.

Method used

Using a wireless mesh (Mesh) network, the behavior of the object to be identified is determined through the signal feature modeling and identification results of multiple WiFi transmitters, combined with channel status information and received signal strength indication.

Benefits of technology

It improves the accuracy and accuracy of behavior recognition, and can more accurately identify human activities in complex scenarios, avoiding the problem of insufficient single point of data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119071839A_ABST
    Figure CN119071839A_ABST
Patent Text Reader

Abstract

The invention provides a behavior recognition method and device, equipment and a storage medium, relates to the technical field of wireless communication, and is used for improving the accuracy of behavior recognition. The method comprises the following steps: acquiring signal parameter information of each node in a wireless mesh network, wherein the wireless mesh network comprises at least two nodes; determining a plurality of behavior recognition results based on the signal parameter information of each node; according to the multiple behavior recognition results, the behavior of the to-be-recognized object is determined, and the behavior of the to-be-recognized object is determined based on the target behavior recognition results with the number larger than or equal to the preset number in the multiple behavior recognition results.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of wireless communication technology, and in particular to a behavior recognition method, device, equipment and storage medium. Background Art

[0002] With the popularization of the Internet of Things, passive behavior recognition has been widely studied due to its ease of use in human-computer interaction, smart elderly care and other fields. For example, based on the perception of WiFi signals, people's behavior can be identified and their physical condition can be understood in a timely manner.

[0003] Existing behavior recognition methods based on WiFi signals are usually based on a WiFi signal emission point, and human behavior is judged by analyzing the signal of the emission point. However, in the case of complex real-life scenarios, such as when there are many obstacles or walls in the recognition scene, WiFi signals are easily interfered, so the recognition results obtained by analyzing the signal of the emission point will also be inaccurate. Summary of the invention

[0004] The present application provides a behavior recognition method, apparatus, device and storage medium for improving the accuracy of behavior recognition.

[0005] In order to at least achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0006] In a first aspect, a behavior recognition method is provided, the method comprising: obtaining signal parameter information of each node in a wireless mesh network, the wireless mesh network comprising at least two nodes; determining a plurality of behavior recognition results based on the signal parameter information of each node; determining a behavior of an object to be recognized based on the plurality of behavior recognition results, wherein the behavior of the object to be recognized is determined based on a number of target behavior recognition results greater than or equal to a preset number among the plurality of behavior recognition results.

[0007] In another aspect, a behavior recognition device is provided, the device comprising an acquisition unit and a determination unit. The acquisition unit is used to acquire signal parameter information of each node in a wireless mesh network, the wireless mesh network comprising at least two nodes; the determination unit is used to determine multiple behavior recognition results based on the signal parameter information of each node; the determination unit is also used to determine the behavior of an object to be recognized based on the multiple behavior recognition results, wherein the behavior of the object to be recognized is determined based on target behavior recognition results whose number is greater than or equal to a preset number among the multiple behavior recognition results.

[0008] On the other hand, an electronic device is provided, comprising: a processor and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions so that the electronic device performs the behavior recognition method as described in the first aspect above.

[0009] On the other hand, a computer-readable storage medium is provided, on which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the behavior recognition method of the first aspect described above.

[0010] In an embodiment of the present application, the signal parameter information of each node in the wireless mesh network is obtained to obtain multiple signal parameter information, so as to understand the wireless signal characteristics of each node. Further, based on each signal parameter information, a behavior recognition result is determined to obtain multiple behavior recognition results. The present application determines the behavior of the object to be recognized based on multiple behavior recognition results. Among them, the behavior of the object to be recognized is the behavior whose number is greater than or equal to the preset number in the multiple behavior recognition results. Compared with the related art that performs behavior recognition based on the signal of a WiFi transmitting point, the present application combines a WiFi Mesh network to provide multiple WiFi transmitting terminals, and performs behavior recognition based on the signal of each WiFi transmitting terminal to obtain multiple behavior recognition results. Further, the present application refers to multiple behavior recognition results, and determines the behaviors whose number is greater than or equal to the preset number in the multiple behavior recognition results as the behavior of the object to be recognized. The amount of data obtained by a transmitting terminal is limited, and there is no reference or calibration object for whether the recognition result is accurate. The present application refers to multiple transmitting terminals to obtain more accurate behaviors of the object to be recognized. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solution in the present application, the drawings required for use in some embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only drawings of some embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0012] Figure 1 A schematic diagram of a Mesh network scenario provided for some embodiments of the present application;

[0013] Figure 2 A schematic diagram of Mesh network coverage provided for some embodiments of the present application;

[0014] Figure 3 A flowchart of a behavior recognition method provided in some embodiments of the present application;

[0015] Figure 4 A schematic diagram of the WiFi data message structure provided for some embodiments of the present application;

[0016] Figure 5 A flowchart of a behavior recognition method provided in some other embodiments of the present application;

[0017] Figure 6 A schematic diagram of packet request information provided by some embodiments of the present application;

[0018] Figure 7 A schematic diagram of the structure of a behavior recognition device provided in some embodiments of the present application;

[0019] Figure 8 A schematic diagram of the structure of an electronic device provided for some embodiments of the present application. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in this application to clearly and completely describe the technical solutions in this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] It should be noted that, in this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0022] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0023] In the description of this application, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, "at least one" means one or more, and "multiple" means two or more.

[0024] Behavior recognition technology is a technology that automatically detects, analyzes and understands the behavioral actions of the identified object (such as speaking / gesturing / falling / breathing). It is widely used in emerging fields such as smart homes, security monitoring, medical rehabilitation, and human-computer interaction.

[0025] Behavior recognition is generally divided into two categories: contact and non-contact. Wearable devices are the key carriers of contact human behavior recognition technology, but wearable devices have limitations such as high cost, inconvenience for users to wear, and attention intrusion. In comparison, non-contact behavior recognition can provide device-free perception services and friendly user interaction, and has a broad prospect. Related non-contact behavior recognition mainly relies on cameras and radio signals. The former is limited by lighting conditions, line-of-sight path interference, monitoring blind spots and privacy issues, while the latter usually requires the deployment of dedicated experimental equipment and is difficult to promote on a large scale.

[0026] At present, as an urban infrastructure, the wireless signals provided by commercial wireless network (WiFi) equipment have the advantages of being universal, easy to use, cost-effective, and having a long transmission distance. In recent years, they have attracted extensive attention in the field of human behavior recognition. In related technologies, the use of WiFi signals to detect human activities usually uses the received signal strength indicator (RSSI) technology to sense whether the human body is active. The sensing granularity is relatively coarse and it is impossible to accurately identify human behavior.

[0027] In addition, related technologies are usually based on a WiFi signal emission point, and human behavior is judged by analyzing the signal of the emission point. However, in the case of complex real-life scenarios, such as when there are many obstacles or walls in the recognition scene, the WiFi signal is easily interfered, so the recognition result obtained by analyzing the signal of the emission point will also be inaccurate. In addition, the amount of data obtained from a single emission point is limited, and there is no reference or calibration object to determine whether the recognition result is accurate, so it is difficult to improve the recognition accuracy.

[0028] In view of this, the present application provides a behavior recognition method, device, equipment and storage medium, which are combined with a wireless mesh network to provide multiple WiFi transmitters (for example, at least two or more). Each transmission point in the Mesh network can be interconnected. The present application models the signal characteristics of each transmission point separately and calibrates the recognition results with each other to obtain more accurate human activity perception and recognition results.

[0029] The behavior recognition method provided by the embodiment of the present application is described in detail below in conjunction with the accompanying drawings. It should be noted that the embodiments of the present application can refer to each other, for example, the same or similar steps, method embodiments, system embodiments and device embodiments can refer to each other without limitation.

[0030] The behavior recognition method provided in the embodiment of the present application can be applied to Mesh network scenarios. Figure 1 A schematic diagram of the Mesh network scenario is shown.

[0031] like Figure 1As shown, the Mesh network scenario may include a behavior recognition device 11 and multiple nodes 12. The behavior recognition device 11 may communicate with each node 12.

[0032] The node 12 can be any device with wireless signal receiving and sending function (such as a WiFi device). Figure 2 As shown, a Mesh network is deployed indoors, including WiFi devices A, B...G. The object to be identified is located indoors (i.e., within the coverage of the Mesh network), and the behavior of the object to be identified can affect the wireless signals of each WiFi device.

[0033] In an exemplary embodiment, the connection mode of the Mesh network may be at least that the Mesh network sub-node is connected to the wireless access point (Access Point, AP) of the Mesh network main node through its own station, and sends data to and from the AP through its own station.

[0034] The behavior recognition device 11 can obtain the signal parameter information of each node 12, determine the behavior recognition result based on each signal parameter information, obtain multiple behavior recognition results, and determine the behavior of the object to be recognized based on the multiple behavior recognition results. The specific behavior recognition process can refer to the behavior recognition method introduced in the following method embodiment, which will not be repeated here.

[0035] The behavior recognition device 11 can implement the behavior recognition method of the embodiment of the present application in various electronic devices. For example, the electronic device can be a mobile phone, a tablet computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a desktop computer, a cloud server, etc. The embodiment of the present application does not limit the specific type of the electronic device.

[0036] In different application scenarios, the behavior recognition device 11 and the electronic device may be independent devices or integrated into the same device, which is not specifically limited in the embodiments of the present application.

[0037] When the behavior recognition device 11 and the electronic device are integrated into the same device, the data transmission method between the behavior recognition device 11 and the electronic device is the data transmission between the internal modules of the device. In this case, the data transmission process between the two is the same as the "data transmission process between the two when the behavior recognition device 11 and the electronic device are independent of each other".

[0038] The application scenarios of the embodiments of the present application are not limited. The system architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. It is known to those skilled in the art that with the evolution of the network architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0039] In the following embodiments provided in the embodiments of the present application, the behavior recognition device 11 and the electronic device are set up independently of each other as an example for explanation.

[0040] Figure 3 The following is a flow chart of a behavior recognition method according to some exemplary embodiments. In some embodiments, the following behavior recognition method can be applied to the above-mentioned behavior recognition device, electronic device, and other similar devices.

[0041] like Figure 3 As shown, the behavior recognition method provided in the embodiment of the present application may include the following steps:

[0042] S201. Obtain signal parameter information of each node in the wireless mesh network.

[0043] Among them, a signal parameter information is used to reflect the wireless signal characteristics of a node. For example, the signal parameter information can reflect the waveform, duration, amplitude of each frequency component, frequency, phase, etc. of the WiFi signal transmitted by the node.

[0044] As a possible implementation method, the behavior recognition device obtains WiFi message information of each node in the wireless mesh network, and parses the signal parameter information of each node from the WiFi message.

[0045] For example, Figure 4 As shown, the behavior recognition device obtains the WiFi data message of the WiFi transmitter A, and adds a flag Flag to the WiFi data message, which is marked as A. The behavior recognition device can parse the channel state information (CSI) data, received signal strength indication (RSSI) data, etc. from the WiFi data message.

[0046] It should be noted that, usually, the receiving end evaluates the CSI and feeds it back to the transmitting end in a quantitative manner (in a time division duplex system, reverse evaluation is required). Therefore, the WiFi message information of each node usually contains CSI data, and the CSI data at least includes the amplitude and phase of the carrier signal.

[0047] In some embodiments, the behavior recognition device may also perform noise removal processing on the signal parameter information to obtain valid data.

[0048] For example, for the CSI data and RSSI data of WiFi transmitter A, the behavior recognition device adds a flag bit to the corresponding WiFi data message, marked as A, and further the behavior recognition device removes noise through the denoising module to obtain valid data and stores the valid data. Similarly, for the CSI data and RSSI data of WiFi transmitter B, the behavior recognition device adds a flag bit to the corresponding WiFi data message, marked as B, and further the behavior recognition device removes noise through the denoising module to obtain valid data and stores the valid data.

[0049] In practical applications, nonlinear denoising methods may be used to denoise carrier waves transmitted by different transmitting ends through antennas. For example, the denoising methods include wavelet transform and other methods.

[0050] S202: Determine multiple behavior recognition results based on signal parameter information of each node.

[0051] As a possible implementation method, the behavior recognition device determines the channel state CSI data in each signal parameter information to obtain multiple CSI data. Further, the behavior recognition device determines each behavior recognition result based on each CSI data to obtain multiple behavior recognition results.

[0052] It can be understood that the behavior recognition device extracts the physical layer channel state information (CSI) from the WiFi signal and determines the behavior recognition result based on the CSI. CSI can characterize the multipath link changes caused by the perceived target, thereby using the amplitude and phase information at the subcarrier level to achieve more complex and sophisticated behavior recognition, such as indoor activity monitoring, fall detection, gait authentication or gesture recognition.

[0053] It should be noted that the behavior recognition results determined by the behavior recognition device based on the CSI data of different nodes may be the same or different. Therefore, the accuracy of the behavior recognition results obtained based on only the CSI data of one transmitting end is difficult to guarantee.

[0054] Exemplarily, the wireless mesh network includes node 1, node 2, and node 3. The CSI of node 1 is CSI-1, the CSI of node 2 is CSI-2, and the CSI of node 3 is CSI-3. The behavior recognition device determines that the behavior recognition result is A based on CSI-1. The behavior recognition device determines that the behavior recognition result is B based on CSI-2. The behavior recognition device determines that the behavior recognition result is B based on CSI-3.

[0055] In some embodiments, the behavior recognition device may determine the behavior recognition results corresponding to each CSI data according to the characteristics of each CSI data.

[0056] In some embodiments, the behavior recognition device may further input each CSI data into a behavior recognition model to determine a behavior recognition result corresponding to each CSI data.

[0057] The behavior recognition model can be a model pre-trained by the behavior recognition device, or a model pre-stored in the behavior recognition device by the operation and maintenance personnel. The behavior recognition model can extract features from the CSI data and convert it into quantifiable data such as frequency (for example, CSI data is converted into a four-dimensional feature vector based on the device name, time value and subcarrier), and output the behavior recognition result based on the converted data.

[0058] Exemplarily, the behavior recognition device can input the CSI data of the node A at a certain moment in the Mesh network that is closest to the human body into the classification algorithm hidden Markov model learning algorithm to identify the type of human activity and determine whether the human body is currently active. Similarly, the behavior recognition device can also input the CSI data of other transmitting terminals such as B...G in the Mesh network into the classification algorithm hidden Markov model learning algorithm to identify the type of human activity.

[0059] Different behavior recognition models are used to recognize different behaviors. For example, in the CSI signal data, the behavior recognition device marks the human breathing action data for establishing a breathing recognition model. In the CSI signal data, the behavior recognition device marks the human body posture data for establishing a behavior recognition model. That is, the behavior recognition device can perform targeted training on the model based on the different characteristics of the CSI signal and the targeted behavior, and obtain a behavior recognition model that can recognize the target action.

[0060] Exemplarily, the wireless mesh network includes node 1, node 2, and node 3. The CSI of node 1 is CSI-1, the CSI of node 2 is CSI-2, and the CSI of node 3 is CSI-3. The behavior recognition device inputs CSI-1 into the gesture recognition model, and the behavior recognition result determined is hand raising. The behavior recognition device inputs CSI-2 into the gesture recognition model, and the behavior recognition result determined is hand raising. The behavior recognition device inputs CSI-3 into the gesture recognition model, and the behavior recognition result determined is hand not raised.

[0061] S203: Determine the behavior of the object to be identified according to the multiple behavior identification results.

[0062] The behavior of the object to be identified is determined based on target behavior identification results whose number is greater than or equal to a preset number among the multiple behavior identification results. One behavior identification result may correspond to one behavior.

[0063] It should be noted that the object to be identified in the embodiment of the present application can be any active object within the coverage of the wireless mesh network, including but not limited to people and pets. For example, the behavior recognition method of the embodiment of the present application can successfully identify human behavior within the coverage of the wireless mesh network.

[0064] As a possible implementation, the behavior recognition device obtains the number of each behavior in the multiple behavior recognition results, and determines the behaviors whose number is greater than or equal to a preset number as the behaviors of the object to be recognized.

[0065] In practical applications, the preset number can be set in advance in the behavior recognition device by the operation and maintenance personnel. For example, if more than half of the results of multiple behavior recognition are breathing or a certain gesture, the behavior recognition device determines the result as the behavior of the object to be recognized.

[0066] In some embodiments, the behavior recognition device can obtain the distance between each node in the wireless mesh network and the object to be recognized, and set a weight for the behavior recognition result corresponding to each node based on the distance between each node and the object to be recognized. For example, the smaller the distance to the object to be recognized, the greater the weight value of the node. Further, the behavior recognition device determines the behavior of the object to be recognized based on multiple behavior recognition results and their corresponding weights. For example, the behavior recognition device can determine the behavior of the object to be recognized based on one or more nodes with higher weights.

[0067] In some embodiments, in order to improve the reference value of the determined behavior recognition result, the behavior recognition device determines the target node in the wireless mesh network. The distance between the target node and the object to be recognized is greater than or equal to the preset distance. The behavior recognition device discards the behavior recognition result corresponding to the target node. Further, the behavior recognition device determines the behavior of the object to be recognized from the remaining behavior recognition results. The specific determination method can refer to the above embodiment and will not be repeated here.

[0068] In an embodiment of the present application, the signal parameter information of each node in the wireless mesh network is obtained to obtain multiple signal parameter information, so as to understand the wireless signal characteristics of each node. Further, based on each signal parameter information, a behavior recognition result is determined to obtain multiple behavior recognition results. The present application determines the behavior of the object to be recognized based on multiple behavior recognition results. Among them, the behavior of the object to be recognized is the behavior whose number is greater than or equal to the preset number in the multiple behavior recognition results. Compared with the related art that performs behavior recognition based on the signal of a WiFi transmitting point, the present application combines a WiFi Mesh network to provide multiple WiFi transmitting terminals, and performs behavior recognition based on the signal of each WiFi transmitting terminal to obtain multiple behavior recognition results. Further, the present application refers to multiple behavior recognition results, and determines the behaviors whose number is greater than or equal to the preset number in the multiple behavior recognition results as the behavior of the object to be recognized. The amount of data obtained by a transmitting terminal is limited, and there is no reference or calibration object for whether the recognition result is accurate. The present application refers to multiple transmitting terminals to obtain more accurate behaviors of the object to be recognized.

[0069] In one design, in order to improve the accuracy of the behavior recognition model, Figure 5 As shown, the behavior recognition method provided in the embodiment of the present application also includes:

[0070] S301: Obtain a training sample set and a prediction sample set.

[0071] The training sample set includes a plurality of first sample CSI data, one first sample CSI data corresponds to one first sample behavior; and the prediction sample set includes a plurality of second sample CSI data.

[0072] As a possible implementation, the training sample set and the prediction sample set may be pre-stored in the behavior recognition device by the operation and maintenance personnel, or may be obtained from the Internet by the behavior recognition device. In order to improve the accuracy of model training, the first sample CSI data is different from the second sample CSI data.

[0073] S302: Based on the training sample set, a behavior recognition model is trained.

[0074] As a possible implementation manner, the behavior recognition device may take the first sample CSI data as input and the first sample row as a label to train the behavior recognition model to be trained, so as to obtain a trained behavior recognition model.

[0075] S303: Based on the behavior recognition model, determine the initial behavior recognition results corresponding to each second sample CSI data in the prediction sample set to obtain multiple initial behavior recognition results.

[0076] As a possible implementation manner, the behavior recognition device inputs each second sample CSI data into the behavior recognition model respectively to obtain multiple initial behavior recognition results.

[0077] It should be noted that the second sample CSI data may be historical CSI data of each node in the Mesh network. The first sample CSI data may be CSI data pre-collected by the operation and maintenance personnel and stored in the behavior recognition device.

[0078] For example, the first sample behavior corresponding to each first sample CSI data is raising a hand or not raising a hand. When training the behavior recognition model, the behavior recognition device uses the first sample CSI data as output and the first sample row of the first sample CSI data as a label to train the behavior recognition model.

[0079] In practical applications, the behavior recognition device can obtain the CSI data of each node at a certain historical moment and use these CSI data as the second sample CSI data. The behavior recognition device inputs each second sample CSI data into the above-mentioned behavior recognition model to determine whether each second sample CSI data corresponds to raising the hand or not raising the hand (corresponding to the first behavior and the second behavior, respectively), that is, to obtain multiple initial behavior recognition results.

[0080] It is understandable that in the embodiment of the present application, the behavior recognition device can initially train a behavior recognition model through multiple first sample CSI data and multiple first sample behaviors. However, the behavior recognition model can only output an initial behavior recognition result for one CSI. There is no reference or calibration object to determine whether the initial behavior recognition result is accurate, so it is difficult to improve the recognition accuracy.

[0081] S304: Based on the multiple initial behavior recognition results, the behavior recognition model is adjusted to obtain an adjusted behavior recognition model.

[0082] As a possible implementation method, the behavior recognition device determines the confidence of the first behavior and the confidence of the second behavior. When the confidence of the first behavior is greater than the confidence of the second behavior, the behavior recognition device determines the initial behavior recognition result corresponding to the second behavior as the result to be verified, and adjusts the label of the CSI data corresponding to the result to be verified to the first behavior, and continues to train the behavior recognition model to obtain the adjusted behavior recognition model. When the confidence of the first behavior is less than the confidence of the second behavior, the behavior recognition device determines the initial behavior recognition result corresponding to the first behavior as the result to be verified, and adjusts the label of the CSI data corresponding to the result to be verified to the second behavior, and continues to train the behavior recognition model to obtain the adjusted behavior recognition model.

[0083] It should be noted that the behavior recognition device can determine the confidence of the first behavior and the confidence of the second behavior according to the initial behavior recognition result of the behavior recognition model for each second sample CSI data.

[0084] For example, the behavior recognition device can determine the distance between the object to be recognized and each node in the mesh network based on the RSSI value, and assign different weights to nodes at different distances, wherein the closer the distance, the heavier the weight. Furthermore, the behavior recognition device uses the sum of the weights of the nodes whose initial behavior recognition result is the first behavior as the confidence of the first behavior, and the behavior recognition device uses the sum of the weights of the nodes whose initial behavior recognition result is the second behavior as the confidence of the second behavior.

[0085] In some embodiments, the behavior recognition device can determine the distance between each node and the object to be recognized. The behavior recognition device determines the weight of each node based on the distance between each node and the object to be recognized. The weight is negatively correlated with the distance. The behavior recognition device determines the sum of the weights of each first node as the confidence of the first behavior. The initial behavior recognition result corresponding to the first node is the first behavior. The behavior recognition device determines the sum of the weights of each second node as the confidence of the second behavior. The initial behavior recognition result corresponding to the second node is the second behavior.

[0086] In example 1, assume that there are three node devices a, b, and c in the Mesh network, among which node a is closest to the human body, so the weight of node a can be set to 0.5, node b is the second closest to the human body, and the weight can be set to 0.4, and node c is the farthest from the human body, and the weight can be set to 0.2. For a certain action such as breathing, when the initial recognition result corresponding to node a is yes, and the initial recognition results corresponding to nodes b and c are no, but the sum of the weights of nodes b and c is 0.6, which is greater than the weight of node a of 0.5, the initial recognition result corresponding to node a (breathing) is determined as the result to be verified, and the label of the CSI data of node a is adjusted to not breathing, and the behavior recognition model continues to be trained to obtain the adjusted behavior recognition model.

[0087] Example 2: In the Mesh network, the initial recognition result corresponding to node A is action a, while the initial recognition results corresponding to other nodes are action b. Node A needs to calibrate the recognition result. The behavior recognition device classifies the CSI data of node A as action b, and continues to train the behavior recognition model to obtain an adjusted behavior recognition model.

[0088] It is understandable that in the related art, the amount of data obtained by a transmitter is limited, and there is no reference or calibration object for whether the recognition result is accurate, so it is difficult to improve the recognition accuracy. In the embodiment of the present application, the CSI data of the WiFi nodes are modeled and calibrated with each other to improve the accuracy of the behavior recognition model.

[0089] Correspondingly, the behavior recognition device can input each CSI data into the adjusted behavior recognition model to obtain multiple behavior recognition results. Compared with the behavior recognition model before adjustment, the adjusted behavior recognition model improves its own recognition accuracy by mutual calibration of the recognition results of different CSI data, making the adjusted behavior recognition model more suitable for Mesh network scenarios containing multiple nodes. Therefore, the embodiment of the present application can obtain more accurate human activity perception recognition results using the adjusted behavior recognition model.

[0090] In one design, in order to determine the distance between each node and the object to be identified, the behavior recognition device obtains the received signal strength information of each node, and determines the distance between each node and the object to be identified based on the received signal strength information of each node.

[0091] In practical applications, the behavior recognition device can obtain RSSI from the signal parameter information of each node, and determine the distance between each node and the object to be recognized based on the angle-of-arrival (AOA) technology.

[0092] It should be noted that the positioning algorithm based on signal arrival angle is a typical ranging-based positioning algorithm. It senses the arrival direction of the transmitting node signal through certain hardware devices, calculates the relative direction or angle between the receiving node and the anchor node, and then uses triangulation or other methods to calculate the position of the unknown node.

[0093] Exemplarily, a node in a Mesh network initiates a LL_CTE_REQ PDU packet, such as Figure 6 The information requested by the packet includes the minimum fixed frequency extension signal (CTE) length, CTE type, and radio frequency unit (RFU) information (1 bit is reserved). The node receives the message containing the CTE field. The CTE data is attached to the back of the data packet and contains a series of switching time slots and sampling time slots. The data string is defined as 1us or 2us. The Mesh node controls the switching of several different antennas according to the angle in the data message containing the CTE and obtains the receiving time slot information of different antennas.

[0094] The behavior recognition device uses the AOA method to calculate the time difference of the broadcast message reaching different antennas, and obtains the position information of the human body relative to the node with orientation. The position is the position of the antenna with the shortest arrival time.

[0095] In some embodiments, the behavior recognition device obtains the CSI data of node A in the Mesh network at a certain time t, denoted as T a The distance between node A and the human body is determined by using the RSSI and AOA technology of node A and recorded as Da , D a Contains distance and direction information. The weight value is based on D a Quantize: f*D a , f is a pre-determined weight factor. Similarly, the behavior recognition device obtains the CSI data T of each node in the Mesh network at time t n and direction distance D n , and obtain the data shown in Table 1.

[0096] Table 1

[0097] A t <![CDATA[T a ]]> <![CDATA[f*D a ]]> B t <![CDATA[T b ]]> <![CDATA[f*D b ]]> ... ... ... ... N t <![CDATA[T n ]]> <![CDATA[f*D n ]]>

[0098] In one design, in order to ensure that the signal of each node can cover the object to be detected, the behavior recognition method provided in the embodiment of the present application further includes: when determining that the position of the object to be identified has changed, the behavior recognition device sends an indication message to the master node. The indication message is used to instruct the master node to adjust the transmission power according to the changed position of the object to be identified, and to instruct the master node to broadcast the changed position of the object to be identified to each slave node, and to instruct each slave node to adjust the transmission power according to the changed position of the object to be identified.

[0099] In practical applications, the behavior recognition device can obtain the signal parameter information of the master node at the first moment and the signal parameter information at the second moment. The first moment is different from the second moment. When the signal parameter information at the first moment is different from the signal parameter information at the second moment, the behavior recognition device determines that the position of the object to be recognized has changed.

[0100] It is understandable that when a person moves back and forth, the CSI data of each node in the Mesh network and the distance direction between each node and the person will change. Therefore, the behavior recognition device can determine that the position of the object to be recognized has changed according to the changes in signal parameter information at different times.

[0101] In some embodiments, when determining that the position of the object to be identified has changed, the behavior recognition device sends an indication message to the master node. Accordingly, the master node can increase or decrease the weight factor f according to the distance and direction from the human body. Among them, the weight factor f can be updated in real time, or it can be updated when the distance to the object to be detected is greater than or equal to a threshold. The master node can also broadcast the position of the object to be identified to the subordinate slave nodes through the Mesh channel. Accordingly, the slave node can increase or decrease the RF power in real time according to the broadcast message to cover the location of the object to be identified.

[0102] Exemplarily, node B increases or decreases the radio frequency power according to the broadcast message of node A, and recalculates the direction distance to the human body, and automatically increases or decreases the weight factor f.

[0103] In some embodiments, for a certain action, the nodes in the Mesh network can also broadcast their own CSI data (including information such as amplitude, frequency and channel) and recognition results through the Mesh network broadcast message Beacon. Other nodes receive the message and adjust their own transmission frequency band and channel to maintain consistency. Correspondingly, the behavior recognition device can correct the recognition result according to the weight of each node. The calculation formula is as follows:

[0104] A*α1+B*α2+C*α3...+G*αn>1 (or threshold), where AG represent different nodes, and α1-αn represent the weights corresponding to each node.

[0105] The above-mentioned embodiment mainly introduces the scheme provided by the embodiment of the present application from the perspective of the device (equipment). It is understandable that, in order to implement the above-mentioned method, the device or equipment includes a hardware structure and / or software module corresponding to the execution of each method flow, and these hardware structures and / or software modules corresponding to the execution of each method flow can constitute an electronic device. It should be easy for those skilled in the art to realize that, in conjunction with the algorithm steps of each example described in the embodiment applied for herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but this implementation should not be considered to exceed the scope of the present application.

[0106] The embodiment of the present application can divide the functional modules of the device or equipment according to the above method example. For example, the device or equipment can divide each functional module corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0107] In the case of dividing each functional module into corresponding functional modules, Figure 7 FIG. 4 is a schematic diagram showing the structure of a behavior recognition device 40 of a functional module, which can be used to execute the behavior recognition method provided in the above embodiment. Figure 7 As shown, the behavior recognition device 40 includes an acquisition unit 401 and a determination unit 402 .

[0108] The acquisition unit 401 is used to acquire signal parameter information of each node in the wireless mesh network, and the wireless mesh network includes at least two nodes; the determination unit 402 is used to determine multiple behavior recognition results based on the signal parameter information of each node; the determination unit 402 is also used to determine the behavior of the object to be recognized based on the multiple behavior recognition results, wherein the behavior of the object to be recognized is determined based on the target behavior recognition results whose number is greater than or equal to a preset number among the multiple behavior recognition results.

[0109] In an exemplary embodiment, the signal parameter information includes channel state CSI data; the determination unit 402 is specifically used to: determine the channel state CSI data in each signal parameter information to obtain multiple CSI data; determine each behavior identification result based on each CSI data to obtain the multiple behavior identification results.

[0110] In an exemplary implementation, the determination unit 402 is specifically configured to: obtain the number of each behavior in the multiple behavior recognition results; and determine behaviors whose number is greater than or equal to the preset number as behaviors of the object to be recognized.

[0111] In an exemplary embodiment, the determination unit 402 is also used to: determine the initial behavior recognition result corresponding to each second sample CSI data based on the behavior recognition model, and obtain multiple initial behavior recognition results; the behavior recognition model is obtained based on multiple first sample CSI data and multiple first sample behaviors; one first sample CSI data corresponds to one first sample behavior; based on the multiple initial behavior recognition results, adjust the behavior recognition model to obtain an adjusted behavior recognition model.

[0112] In an exemplary implementation, the determination unit 402 is specifically configured to: input each CSI data into the adjusted behavior recognition model respectively to obtain the multiple behavior recognition results.

[0113] In an exemplary embodiment, the multiple initial behavior recognition results include a first behavior and a second behavior; the determination unit 402: determines the confidence of the first behavior and the confidence of the second behavior; when the confidence of the first behavior is greater than the confidence of the second behavior, the initial behavior recognition result corresponding to the second behavior is determined as the result to be verified, and the label of the CSI data corresponding to the result to be verified is adjusted to the first behavior, and the behavior recognition model is continued to be trained to obtain the adjusted behavior recognition model; when the confidence of the first behavior is less than the confidence of the second behavior, the initial behavior recognition result corresponding to the first behavior is determined as the result to be verified, and the label of the CSI data corresponding to the result to be verified is adjusted to the second behavior, and the behavior recognition model is continued to be trained to obtain the adjusted behavior recognition model.

[0114] In an exemplary embodiment, the determination unit 402 is specifically used to: determine the distance between each node and the object to be identified; determine the weight of each node according to the distance between each node and the object to be identified; the weight is negatively correlated with the distance; determine the sum of the weights of each first node as the confidence of the first behavior; the initial behavior recognition result corresponding to the first node is the first behavior; determine the sum of the weights of each second node as the confidence of the second behavior; the initial behavior recognition result corresponding to the second node is the second behavior.

[0115] In an exemplary implementation, the determination unit 402 is specifically configured to: obtain received signal strength information of each node; and determine the distance between each node and the object to be identified according to the received signal strength information of each node.

[0116] In an exemplary embodiment, the wireless mesh network includes a master node and at least one slave node; the determination unit 402 is also used to: when it is determined that the position of the object to be identified has changed, send indication information to the master node; the indication information is used to instruct the master node to adjust the transmission power according to the changed position of the object to be identified, and instruct the master node to broadcast the changed position of the object to be identified to each slave node, and instruct each slave node to adjust the transmission power according to the changed position of the object to be identified.

[0117] In an exemplary embodiment, the determination unit 402 is specifically used to: obtain signal parameter information of the main node at a first moment and signal parameter information at a second moment; the first moment is different from the second moment; when the signal parameter information at the first moment is different from the signal parameter information at the second moment, determine that the position of the object to be identified has changed.

[0118] In the case of implementing the functions of the above-mentioned integrated modules in the form of hardware, the embodiments of the present application provide a possible structure of the electronic device involved in the above-mentioned embodiments. Figure 8 As shown, the electronic device 50 includes: a processor 502 and a bus 504. In an exemplary embodiment, the electronic device 50 may further include a memory 501; optionally, the electronic device 50 may further include a communication interface 503.

[0119] The processor 502 may be a processor that implements or executes various exemplary logic blocks, modules, and circuits described in conjunction with the embodiments of the present application. The processor 502 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the embodiments of the present application. The processor 502 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0120] The communication interface 503 is used to connect with other devices via a communication network, such as Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0121] The memory 501 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program codes in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0122] As a possible implementation, the memory 501 may exist independently of the processor 502, and the memory 501 may be connected to the processor 502 via a bus 504 to store instructions or program codes. When the processor 502 calls and executes the instructions or program codes stored in the memory 501, the behavior recognition method provided in the embodiment of the present application can be implemented.

[0123] In another possible implementation, the memory 501 may also be integrated with the processor 502 .

[0124] The bus 504 may be an extended industry standard architecture (EISA) bus, etc. The bus 504 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0125] Some embodiments of the present application provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium), which stores computer program instructions. When the computer program instructions are executed on a computer, the computer executes the behavior recognition method described in any of the above embodiments.

[0126] Exemplarily, the above-mentioned computer-readable storage media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks or magnetic tapes, etc.), optical disks (e.g., compact disks (CD), digital versatile disks (DVD), etc.), smart cards and flash memory devices (e.g., erasable programmable read-only memory (EPROM), cards, sticks or key drives, etc.). The various computer-readable storage media described in this application may represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing and / or carrying instructions and / or data.

[0127] An embodiment of the present application provides a computer program product including instructions. When the computer program product is run on a computer, the computer is enabled to execute the behavior recognition method described in any of the above embodiments.

[0128] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A behavior recognition method, characterized in that: The method comprises: Acquiring signal parameter information of each node in the wireless mesh network, wherein the wireless mesh network includes at least two nodes; Determine multiple behavior recognition results based on signal parameter information of each node; The behavior of the object to be identified is determined according to the multiple behavior recognition results, wherein the behavior of the object to be identified is determined based on target behavior recognition results whose number is greater than or equal to a preset number among the multiple behavior recognition results.

2. The method according to claim 1, characterized in that In the case where the signal parameter information includes channel state CSI data, the determining of multiple behavior recognition results based on the signal parameter information of each node includes: Determine the channel state CSI data in each signal parameter information; According to the CSI data corresponding to each node, the behavior recognition result corresponding to each node is determined to obtain the multiple behavior recognition results.

3. The method according to claim 1, characterized in that Determining the behavior of the object to be identified according to the multiple behavior identification results includes: Count the number of each behavior recognition results; The target behavior recognition results whose number is greater than or equal to the preset number are determined as the behaviors of the objects to be recognized.

4. The method according to claim 2, characterized in that: The method further comprises: Acquire a training sample set and a prediction sample set; the training sample set includes a plurality of first sample CSI data, one first sample CSI data corresponds to one first sample behavior; the prediction sample set includes a plurality of second sample CSI data; Based on the training sample set, a behavior recognition model is trained; Based on the behavior recognition model, determining an initial behavior recognition result corresponding to each second sample CSI data in the prediction sample set to obtain a plurality of initial behavior recognition results; Based on the multiple initial behavior recognition results, the behavior recognition model is adjusted to obtain an adjusted behavior recognition model.

5. The method according to claim 4, characterized in that Determining each behavior recognition result according to each CSI data to obtain the multiple behavior recognition results includes: Each CSI data is input into the adjusted behavior recognition model to obtain the multiple behavior recognition results.

6. The method according to claim 4, characterized in that The multiple initial behavior recognition results include a first behavior and a second behavior; and adjusting the behavior recognition model based on the multiple initial behavior recognition results to obtain an adjusted behavior recognition model includes: determining a confidence level of the first behavior and a confidence level of the second behavior; In a case where the confidence of the first behavior is greater than the confidence of the second behavior, determining the initial behavior recognition result corresponding to the second behavior as the result to be verified, adjusting the label of the CSI data corresponding to the result to be verified to the first behavior, and continuing to train the behavior recognition model to obtain the adjusted behavior recognition model; When the confidence of the first behavior is less than the confidence of the second behavior, the initial behavior recognition result corresponding to the first behavior is determined as the result to be verified, and the label of the CSI data corresponding to the result to be verified is adjusted to the second behavior, and the behavior recognition model is continued to be trained to obtain the adjusted behavior recognition model.

7. The method according to claim 6, characterized in that The determining of the confidence level of the first behavior and the confidence level of the second behavior includes: Determine the distance between each node and the object to be identified; Determine the weight of each node according to the distance between each node and the object to be identified; the weight is negatively correlated with the distance; The sum of the weights of the first nodes is determined as the confidence of the first behavior; the initial behavior recognition result corresponding to the first node is the first behavior; The sum of the weights of the second nodes is determined as the confidence of the second behavior; and the initial behavior recognition result corresponding to the second node is the second behavior.

8. The method according to claim 7, characterized in that The determining the distance between each node and the object to be identified includes: Obtain the received signal strength information of each node; The distance between each node and the object to be identified is determined according to the received signal strength information of each node.

9. The method according to claim 1, characterized in that: The wireless mesh network includes a master node and at least one slave node; the method further includes: When it is determined that the position of the object to be identified has changed, an indication message is sent to the master node; the indication message is used to instruct the master node to adjust the transmission power according to the changed position of the object to be identified, and to instruct the master node to broadcast the changed position of the object to be identified to each slave node, and to instruct each slave node to adjust the transmission power according to the changed position of the object to be identified.

10. The method according to claim 9, characterized in that The method further comprises: Acquire signal parameter information of the master node at a first moment and signal parameter information of the master node at a second moment; the first moment is different from the second moment; When the signal parameter information at the first moment is different from the signal parameter information at the second moment, it is determined that the position of the object to be identified has changed.

11. The method according to claim 1, characterized in that: The method further comprises: Obtaining the distance between each node in the wireless mesh network and the object to be identified; Based on the distance between each node and the object to be identified, a weight is set for the behavior recognition result corresponding to each node; The behavior of the object to be identified is determined according to the multiple behavior identification results and their corresponding weights.

12. The method according to claim 1, characterized in that The method further comprises: Determine a target node in the wireless mesh network; the distance between the target node and the object to be identified is greater than or equal to a preset distance; The behavior recognition result corresponding to the target node is discarded.

13. A gateway device, characterized in that: include: a processor and a memory for storing instructions executable by the processor; The processor is configured to execute the instruction so that the gateway device executes the behavior recognition method as described in any one of claims 1-12.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer is enabled to execute the behavior recognition method according to any one of claims 1 to 12.