Activity sensing method and apparatus, device, storage medium

Abstract

The application provides an activity sensing method and device, equipment and a storage medium. The method comprises the following steps: in the case that a sensor is separated from a sensing target, sensing the activity of the sensing target by using at least one wireless communication module, wherein the sensor is used for sensing the activity of the sensing target.

Description

Technical Field

[0001] This application relates to electronic technology, including but not limited to methods, devices, equipment, and storage media for sensing activity. Background Technology

[0002] Human activity recognition has significant application value in many fields such as order, security, health, and smart homes. To sense human activity, relevant sensing solutions are mainly divided into two categories: non-contact sensing and contact sensing. Non-contact sensing methods primarily utilize video or image data of human activity collected by cameras to identify human activity; contact sensing methods mainly use data collected by sensors in wearable devices to sense human activity.

[0003] However, both the non-contact sensing methods and the contact sensing methods mentioned above have certain limitations. The former requires sufficient lighting in the sensing environment and no obstructions between the sensing target and the camera; the latter requires the sensing target to wear / carry an electronic device equipped with sensors. Therefore, the shortcomings of these two sensing methods limit their application in practical daily activities. Summary of the Invention

[0004] In view of this, the activity sensing method, apparatus, device, and storage medium provided in this application activate the wireless sensing function when the sensor is detached from the sensing target, thereby achieving seamless connection of activity sensing of the sensing target and making up for the problem that activity cannot be sensed by the sensor when the sensor is detached from the sensing target.

[0005] According to one aspect of the embodiments of this application, an activity sensing method is provided, comprising: sensing the activity of the sensing target using at least one wireless communication module when a sensor is detached from the sensing target; wherein the sensor is used to sense the activity of the sensing target.

[0006] According to one aspect of the embodiments of this application, an activity sensing device is provided, comprising: a sensing module configured to sense the activity of the sensing target using at least one wireless communication module when the sensor is detached from the sensing target; wherein the sensor is used to sense the activity of the sensing target.

[0007] According to one aspect of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor executes the program to implement the method described in the embodiments of the present application.

[0008] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the methods provided in the embodiments of this application.

[0009] In this embodiment of the application, when the sensor is detached from the sensing target, at least one wireless communication module is used to sense the activity of the sensing target, thereby achieving seamless connection in sensing the activity of the sensing target and making up for the problem that the sensor cannot sense the activity when it is detached from the sensing target.

[0010] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0011] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application. Obviously, the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0012] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0013] Figure 1 A schematic diagram illustrating the implementation flow of the activity sensing method provided in the embodiments of this application;

[0014] Figure 2 A schematic diagram illustrating the principle of wireless sensing provided in an embodiment of this application;

[0015] Figure 3 A schematic diagram illustrating the implementation process of the activity sensing method provided in this application embodiment;

[0016] Figure 4 A schematic diagram illustrating a scenario of multi-device collaboration for wireless sensing provided in an embodiment of this application;

[0017] Figure 5 A schematic diagram illustrating the implementation flow of the activity sensing method provided in the embodiments of this application;

[0018] Figure 6 This is a schematic diagram of the structure of an activity sensing device provided in an embodiment of this application;

[0019] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of this application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0022] In the following description, references to "some embodiments," "this embodiment," "this application embodiment," and examples, etc., describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subset of all possible embodiments and may be combined with each other without conflict.

[0023] It should be noted that the terms "first, second, third, fourth, fifth," etc., used in the embodiments of this application are for distinguishing similar or different objects and do not represent a specific order of objects. It is understood that "first, second, third, fourth, fifth" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0024] This application provides an activity sensing method, which includes: sensing the activity of the sensing target using at least one wireless communication module when the sensor is detached from the sensing target, wherein the sensor is used to sense the activity of the sensing target.

[0025] This application provides an activity sensing method. Figure 1 This is a schematic diagram illustrating the implementation flow of the activity sensing method provided in the embodiments of this application, such as... Figure 1 As shown, the method may include the following steps 101 to 103:

[0026] Step 101: Determine that the sensor has moved away from the target; wherein the sensor is used to sense the activity of the target.

[0027] In some embodiments, it can be determined whether the sensor has moved away from the target based on the motion data of the target collected by the sensor. For example, the motion speed or distance traveled by the sensor can be determined based on the motion data; if the motion speed or distance traveled is less than a certain threshold, it is determined that the sensor has moved away from the target. Of course, it is also possible to determine whether the sensor has moved away from the target by combining long-term learning of user habits.

[0028] In other embodiments, if the electronic device equipped with the sensor is also equipped with a light sensor, the sensor can also determine whether it has moved away from the target based on the intensity of light sensed by the light sensor.

[0029] The sensor may include at least one of the following: accelerometer, gyroscope, inertial measurement unit (IMU), image sensor, etc.

[0030] The sensing target can be a moving object such as a human body, animal, or robot.

[0031] Step 102: Determine that the current mode of at least one wireless communication module is the working mode.

[0032] In step 102, when it is determined that the sensor has moved away from the sensing target, if the at least one wireless communication module is in sleep mode, the current mode of the wireless communication module is set to working mode, and then the process proceeds to step 103.

[0033] Step 103: Sensing the activity of the target based at least on the signal characteristics of the wireless signals received by the at least one wireless communication module.

[0034] In some embodiments, when the activity of the target is sensed, activity information of the target (e.g., movement speed, location information, and / or posture) can be displayed. Alternatively, based on the activity information, corresponding control operations can be performed or prompts can be output. For example, when an abnormal heart rate is determined based on the sensed activity information, a prompt can be output; or, when it is determined based on the sensed activity information that the target has entered sleep, a control command can be sent to a television to turn it off, and / or a control command can be sent to a light to turn it off, etc.

[0035] In summary, the purpose of sensing the activities of the target can be varied and can be applied to a wide range of fields that serve the target. Taking the human body as an example, the activity information of the target can be used to monitor and provide early warnings for the vital signs, daily behaviors, and living conditions of the elderly; it can also be used for gesture recognition, real-time location tracking, intrusion detection, and smart home control.

[0036] In this application embodiment, the number of wireless communication modules participating in activity sensing is not limited; it can be one, two, or more. For embodiments where two or more wireless communication modules participate in activity sensing, the final activity sensing result can be determined by a voting method. For example, suppose three wireless communication modules (hereinafter referred to as Module 1, Module 2, and Module 3) participate in activity sensing. Based on the wireless signals received by Module 1, Module 2, and Module 3 respectively, the location coordinates including the activity information of the sensed target are determined to be P1(x1,y1), P2(x2,y2), and P1(x1,y1). Then, based on the voting method, the location coordinates included in the final activity sensing result are determined to be P1(x1,y1).

[0037] It should be noted that when there are two or more wireless communication modules involved in the sensing of the target, these wireless communication modules can be modules in the same electronic device or modules in different electronic devices. Furthermore, it is not limited whether the electronic device having the wireless communication module is detached from the sensing target; it can be detached from the sensing target or carried on the sensing target.

[0038] In this embodiment, the type of wireless communication module participating in activity sensing is not limited. For example, it may include: a wireless communication module that supports receiving Wi-Fi physical layer Channel State Information (CSI), a wireless communication module that supports receiving 4G or 5G signals, a wireless communication module that supports receiving ultrasonic signals, and / or a wireless communication module that supports receiving Ultra Wide Band (UWB) signals, etc. Correspondingly, the type of wireless signal is also not limited, and it may be various types of radio electromagnetic wave signals, sound signals, or optical signals with multipath effects. For example, the wireless signal may be Wi-Fi physical layer CSI, 4G signals, 5G signals, ultrasonic signals, and / or UWB signals, etc. The CSI-based activity sensing method can perceive the behavior of human bodies or other moving objects with finer granularity. By extracting, analyzing, and interpreting the amplitude and / or phase changes of each subcarrier in the CSI, the activity information of the sensed target can be determined.

[0039] It should be noted that the device implementing the activity sensing method provided in this application embodiment can be the electronic device configured with the sensor, or other devices, and there is no limitation thereto. For example, the electronic device implementing the activity sensing method provided in this application embodiment is a mobile phone, and the electronic device configured with the sensor is a wearable device such as a watch or bracelet. Alternatively, the electronic device implementing the activity sensing method provided in this application embodiment and the electronic device configured with the sensor may be the same device, such as a mobile phone.

[0040] Understandably, if the sensor is still relied upon to monitor the activity of the target when it is detached from the target, the result will be invalid. Therefore, in this embodiment, when the sensor is detached from the target, wireless sensing function is used (i.e., steps 102 and 103 are executed) to achieve seamless connection of the perception of the target's activity, thus overcoming the problem that the activity cannot be perceived by the sensor when it is detached from the target.

[0041] The perception of a target's activity based on the signal characteristics of wireless signals is called wireless sensing. Wireless sensing, for example... Figure 2 As shown, the basic idea is as follows: In physical space, such as an indoor environment, radio waves generated by a signal transmitting device propagate through multiple paths, including direct transmission, reflection, and scattering, forming a multipath superposition signal at the signal receiving device. The multipath superposition signal is affected by the physical space in which it propagates and carries information reflecting environmental characteristics. Therefore, the activity of a target (such as a human body) can be inferred from the multipath superposition signal received by the signal receiving device. For example, it can detect human activity information such as breathing, heartbeat, gestures, body position, movement speed, and / or whether a person has fallen.

[0042] This application provides another activity sensing method. Figure 3 This is a schematic diagram illustrating the implementation flow of the activity sensing method provided in the embodiments of this application, such as... Figure 3 As shown, the method includes the following steps 301 to 311:

[0043] Step 301: Determine whether the sensor has moved away from the target; if so, proceed to step 302; otherwise, proceed to step 307; wherein, the data collected by the sensor is used to sense the activity of the target.

[0044] Step 302: Determine whether the current mode of the sensor is a low-power mode or a working mode; if it is a working mode, proceed to step 303; if it is a low-power mode, proceed to step 304.

[0045] In some embodiments, the difference between the sensor's low-power mode and operating mode is that the frequency at which the sensor collects data differs between the two modes; the frequency of data collection in low-power mode is lower than that in operating mode.

[0046] Step 303: Control the sensor to enter a low-power mode, and then proceed to step 304;

[0047] Step 304: Determine whether the current mode of at least one wireless communication module is sleep mode or working mode; if it is working mode, proceed to step 306; if it is sleep mode, proceed to step 305.

[0048] In some embodiments, the difference between the sleep mode and the working mode of the wireless communication module is that in sleep mode, the wireless communication module is in a power-off state, that is, it does not receive wireless signals, while in working mode, the wireless communication module is in a power-on state, that is, it can receive wireless signals.

[0049] It should be noted that, in the embodiments of this application, the execution order of steps 302 and 304 is not limited. Step 304 can be executed before step 302, after step 302, or in parallel with step 302.

[0050] Step 305: Control the wireless communication module to enter the working mode, and then proceed to step 306;

[0051] Step 306: Sensing the activity of the target based at least on the signal characteristics of the wireless signals received by the at least one wireless communication module.

[0052] Understandably, in the embodiments of this application, if the sensor is in a working mode when it is detached from the sensing target, it is set to a low-power mode. In this way, while performing wireless sensing, it is ensured that the sensor used to monitor the activity of the sensing target is in a low-power mode, thereby saving the power consumption of the sensor while solving the problem of the sensor being unable to sense activity due to detachment from the sensing target.

[0053] Step 307: Determine whether the current mode of the wireless communication module is working mode or sleep mode; if it is working mode, proceed to step 308; if it is sleep mode, proceed to step 309.

[0054] Step 308: Control the wireless communication module to enter sleep mode, and then proceed to step 309.

[0055] Understandably, when the sensor is carried on the target being sensed, the wireless communication module can be controlled to enter sleep mode to save power.

[0056] Step 309: Determine whether the current mode of the sensor is a working mode or a low-power mode; if it is a working mode, proceed to step 311; if it is a low-power mode, proceed to step 310.

[0057] It should be noted that, in the embodiments of this application, the execution order of steps 307 and 309 is not limited. Step 309 can be executed before step 307, after step 307, or in parallel with step 307.

[0058] Step 310: Control the sensor to enter the working mode, and then proceed to step 311;

[0059] Step 311: Using the data collected by the sensor, perceive the activity of the target.

[0060] In this embodiment, when the sensor is detached from the target, wireless sensing is used to sense the target's activity; when the sensor is attached to the target, contact sensing is used to sense the target's activity. Thus, regardless of whether the sensor is attached to the target, accurate sensing of the target's activity can be achieved, thereby enabling all-weather monitoring of the target and expanding the application scope and value of activity sensing.

[0061] Understandably, when a target begins to move rapidly or is in a state of rapid movement, the signal characteristics of wireless signals in the physical space change drastically; conversely, when the target moves slowly or is dormant, the degree of change in the signal characteristics of wireless signals in the physical space slows down. In view of this, in some embodiments, the method further includes: determining statistical values ​​of the signal characteristics of the wireless signal based on the wireless signal received by the wireless communication module; the statistical values ​​reflecting the degree of change of the signal characteristics of the wireless signal in the propagation environment; and adjusting the transmission frequency of the device transmitting the wireless signal based on the statistical values; thus, by adaptively adjusting the transmission frequency of the device transmitting the wireless signal according to the degree of change of the signal characteristics of the wireless signal in the propagation environment, the overall power consumption of the device transmitting the wireless signal can be reduced.

[0062] For example, in some embodiments, adjusting the transmission frequency of the device transmitting the wireless signal based on the statistical value includes: adjusting the transmission frequency to a first frequency based on determining that the statistical value is less than or equal to a first threshold; wherein the statistical value is used to measure the dispersion of the signal characteristics of the received wireless signal; and adjusting the transmission frequency to a frequency greater than the first frequency based on determining that the statistical value is greater than the first threshold.

[0063] Thus, on the one hand, when the signal characteristics of the wireless signal corresponding to the statistical value change little, reducing the transmission frequency of the wireless signal transmitter can save the transmission power of the wireless signal transmitter without affecting the accuracy of the wireless sensing results; on the other hand, when the signal characteristics of the wireless signal corresponding to the statistical value change much, increasing the transmission frequency of the transmitter can improve the accuracy of the wireless sensing results.

[0064] In this embodiment, when the statistical value is greater than the first threshold, the adjustment strategy for the transmission frequency is not limited; in short, the adjusted transmission frequency only needs to be greater than the first frequency. For example, as shown in Table 1, when the statistical value is less than or equal to the first threshold, the transmission frequency corresponding to the transmitting device is adjusted to the first frequency; when the statistical value is greater than the first threshold and less than or equal to the second threshold, the transmission frequency corresponding to the transmitting device is adjusted to the second frequency; when the statistical value is greater than the second threshold and less than or equal to the third threshold, the transmission frequency corresponding to the transmitting device is adjusted to the third frequency; when the statistical value is greater than the third threshold, the transmission frequency corresponding to the transmitting device is adjusted to the fourth frequency; wherein, the fourth frequency > the third frequency > the second frequency > the first frequency. Of course, this is not limited to this example; the adjustment level and corresponding threshold can be set according to the specific application.

[0065] Table 1

[0066] sta≤first threshold First frequency The first threshold < sta ≤ the second threshold Second frequency The second threshold < sta ≤ the third threshold Third frequency Third threshold <sta Fourth frequency

[0067] It should be noted that, in the embodiments of this application, the electronic device having the wireless communication module and the device having the transmitting device are not the same device. The former may be, for example, a mobile phone, a watch or a bracelet, while the latter may be, for example, a router, a base station or other device with the function of transmitting wireless signals.

[0068] In some embodiments, a device implementing the activity sensing method may send an adjustment command to a device transmitting the wireless signal; the adjustment command instructs the device to adjust the transmission frequency to a specified frequency.

[0069] In some embodiments, the activity sensing method further includes: determining statistical values ​​of signal characteristics of the wireless signal based on the wireless signal received by the wireless communication module; the statistical values ​​are used to reflect the degree of change of the signal characteristics of the wireless signal in the propagation environment; and adjusting the number of devices participating in the activity sensing of the sensing target based on the statistical values.

[0070] Furthermore, in some embodiments, the number of wireless communication modules and / or sensors involved in the activity sensing of the target can be adjusted based on the statistical values.

[0071] It should be noted that the statistical value can be a value calculated based on the signal characteristics of the wireless signal received by any wireless communication module, or it can be a value calculated based on the signal characteristics of the wireless signals received by multiple wireless communication modules. For example, the statistical value is the average of the statistical values ​​corresponding to each wireless communication module.

[0072] Understandably, in this embodiment, the decision to add or remove devices participating in activity sensing is based on the statistical values ​​of the signal characteristics of the wireless signal. Theoretically, wireless communication between multiple devices can sense human activity over a larger area and improve sensing accuracy. If the statistical values ​​indicate drastic changes in the wireless signal, suggesting the target is moving rapidly, then more smart devices can be added to improve sensing accuracy. Conversely, if the statistical values ​​indicate slow changes in the signal characteristics, suggesting the user is nearly stationary, then fewer smart devices can be added to reduce power consumption while ensuring sensing accuracy.

[0073] Taking CSI (Communication Signaling System) with Wi-Fi as an example, many home appliances have CSI modules (an example of a wireless communication module) built in, for example... Figure 4 The illustrated devices include a mobile phone 401, a smartwatch 402, a speaker 403, and a television 404. Assuming the device executing the activity sensing method is the mobile phone 401, then the mobile phone 401 can determine whether to add or remove smart devices participating in activity sensing based on the statistical values ​​of the signal characteristics of the wireless signal. Assuming the smartwatch 402 and television 404 also participate in activity sensing, then the smartwatch 402 and television 404 can send the signal characteristics or statistical values ​​of the received wireless signals to the mobile phone 401. The mobile phone 401, based on the signal characteristics of the wireless signals received by its own wireless communication module and the signal characteristics or statistical values ​​of the signal characteristics sent by the smartwatch 402 and television 404, senses the activity of the target. Figure 4 The Wi-Fi router 405 shown is a device that transmits wireless signals.

[0074] In this embodiment, the type of signal feature is not limited, and can be the amplitude, phase, and / or phase difference of the wireless signal, etc. The type of parameter for the statistical value is also not limited; in short, anything capable of characterizing the degree of change in the signal features of the wireless signal is acceptable. For example, the parameter type of the statistical value could be the variance, standard deviation, and / or covariance of the signal feature, etc.

[0075] It should be noted that the subject executing the activity sensing method provided in this application embodiment and the sensor or the wireless communication module can be in the same electronic device or in different electronic devices. For example, the sensor can be located in a watch, and the wireless communication module can be located in a mobile phone.

[0076] The electronic device equipped with the sensor can be of various types, such as a mobile phone, a wearable device (e.g., a smart bracelet, a smartwatch, and headphones), a tablet computer, etc. The electronic device equipped with the wireless communication module can also be of various types, such as a mobile phone, a wearable device (e.g., a smart bracelet, a smartwatch, and headphones), a tablet computer, etc.

[0077] To sense human activity, sensing methods can be divided into two categories: contact-based and non-contact-based. Non-contact sensing methods are mainly based on computer vision, using cameras to collect video and / or image data of human activity and leveraging neural networks to classify and identify the types of human activity. Contact-based sensing methods mainly use sensors on wearable devices to collect human activity information, then extract the effective features and use traditional machine learning methods for analysis and recognition; wearable devices include smartphones, watches, and wristbands; sensors on wearable devices include accelerometers and gyroscopes.

[0078] Non-contact sensing methods can also utilize electromagnetic waves to detect human activity. Existing wireless signals in the environment (such as sound, light, Wi-Fi, 4G, and 5G signals) can be used to sense the environment while fulfilling their primary functions (such as lighting or communication). Radio waves generated by a signal transmitter undergo physical phenomena such as direct propagation, reflection, diffraction, and scattering during propagation, thus forming multiple propagation paths. Therefore, the multipath superposition signal formed at the signal receiver carries information reflecting the space in which the signal propagates.

[0079] Wireless sensing refers to the use of ubiquitous wireless signals to detect objects in the environment, which cause reflection, diffraction, and scattering due to the presence of a target. Influenced by the target, the signal received by the receiver undergoes corresponding changes in amplitude and / or phase. By detecting and analyzing this signal, non-contact detection of the target can be achieved. Wireless sensing targets include the environment, objects, and / or people, with a wide range of potential applications.

[0080] Wireless sensing using Wi-Fi signals is called Wi-Fi sensing. Its basic principle is to use the CSI signal between at least one Wi-Fi transmitter and one Wi-Fi receiver to sense user behavior, and the transmitter and receiver are fixed.

[0081] However, the limitations of computer vision-based perception schemes mainly include: (1) sufficient light in the perception environment; (2) no obstructions between the object being detected and the camera; (3) high requirements for real-time processing and storage capabilities; and (4) user privacy.

[0082] The limitations of sensor-based sensing solutions mainly include: (1) users need to wear smartphones or watches with sensors installed at all times, and user activities cannot be sensed when the user takes off the device; (2) the device can only sense the behavior of a part of the body; and (3) additional costs are required.

[0083] The shortcomings of these two sensing schemes make them difficult to apply to daily activities that require real-time sensing.

[0084] Based on this, the following will describe an exemplary application of the embodiments of this application in a practical application scenario.

[0085] This application provides a low-power activity sensing method that utilizes a Wi-Fi chip and sensors on a smart device to sense user behavior and posture. When a user wears and / or carries a smart device, the built-in sensors on the smart device are used to sense user behavior; and when the user does not wear or carry the smart device (e.g., a watch charging at night or a phone placed on a bedside table), the wireless signal received by the smart device (e.g., a Wi-Fi CSI signal) is used to sense user behavior.

[0086] In everyday use cases, users wear smart devices to perform activities. The smart devices collect data through sensors, and then the processor calculates and analyzes the sensor data to complete the sensing task.

[0087] If a user is not wearing a smart device while active, such as when the smart device is placed on a table, the sensor output data can determine that the smart device is stationary, but the user may still be active. In this case, the sensor is set to low-power mode, and the CSI module (an example of a wireless communication module) is activated. The CSI signal received on the Wi-Fi antenna is used to sense human activity. Additionally, the degree of change in the current environment can be determined based on the statistical results of CSI amplitude and phase (such as the standard deviation of amplitude, phase, or phase difference, characterizing the drasticness of CSI changes). The transmission frequency of the CSI signal can be adjusted according to the degree of CSI change. For example, in a mobile phone device, the accelerometer can be configured so that when the accelerometer data exceeds a certain threshold, the sensor is activated, thus exiting low-power mode. Watches or wristbands can use optical proximity sensors to determine whether the user is wearing the wearable device.

[0088] When sensor data (such as ACC data) exceeds a certain threshold, it is determined that the user is wearing a smart device and is in motion. At this time, the CSI module is set to sleep mode, and the sensors are set to working mode. If the sensor data determines that the user is in motion, the sensors will continue to be used for sensing until the sensor data determines that the user is not wearing a smart device, i.e., the smart device is in a stationary state.

[0089] Figure 5 This is a schematic diagram illustrating the implementation flow of the activity sensing method provided in the embodiments of this application, such as... Figure 5 As shown, the method includes the following steps 501 to 514:

[0090] Step 501: Collect and analyze sensor data to obtain analysis results;

[0091] Step 502: Based on the analysis results, determine whether the smart device is being worn; if so, proceed to step 510; otherwise, proceed to step 503.

[0092] Step 503: Determine whether the sensor is in low-power mode; if yes, proceed to step 505; otherwise, proceed to step 504.

[0093] Step 504: Set the sensor to low power mode; then proceed to step 505.

[0094] Step 505: Determine whether the CSI module is in working mode; if yes, proceed to step 507; otherwise, proceed to step 506.

[0095] Step 506: Set the CSI module to working mode; then proceed to step 507;

[0096] Step 507: Utilize CSI signals for intelligent environmental monitoring and perception;

[0097] Step 508: Determine the statistical values ​​of the amplitude and / or phase of the CSI signal based on the received CSI signal;

[0098] Step 509: Adjust the CSI transmission frequency based on the relationship between the statistical results of the CSI signal amplitude and / or phase obtained by the CSI module and the threshold.

[0099] The statistical value results of the CSI signal amplitude and phase characterize the severity of the CSI signal change and reflect the severity of the current environment change. According to the severity of the CSI change, adjust the transmission frequency of the CSI signal. If the statistical value is less than the threshold TH1, adjust the transmission frequency to F1; if the statistical value is between the thresholds TH1 and TH2, adjust the transmission frequency to F2; if the statistical value is greater than the threshold TH3, then adjust the transmission frequency to F3. Among them, TH1, TH2, and TH3 satisfy TH1 < TH2 < TH3, and F1, F2, and F3 satisfy F1 < F2 < F3.

[0100] For example, if the user is in a sleeping state, the statistical value is less than the threshold TH1, and set the transmission frequency F1 to 30 Hz; if the user is in a walking / running state, the statistical value is between the thresholds TH1 and TH2, and set the transmission frequency F2 to 60 Hz; if the user is about to fall, the statistical value will be greater than the threshold TH3, and set the transmission frequency F3 to 120 Hz.

[0101] Step 510, determine whether the CSI module is in the sleep mode; if so, execute step 512; otherwise, execute step 511;

[0102] Step 511, set the CSI module to the sleep mode; then enter step 512;

[0103] Step 512, determine whether the sensor is in the working mode; if so, execute step 514; otherwise, execute step 513;

[0104] Step 513, set the sensor to the working mode; then enter step 514;

[0105] Step 514, use the sensor to perform intelligent monitoring and perception of the environment.

[0106] In the embodiment of the present application, without increasing the hardware cost, the existing sensors and Wi-Fi chips on the smart device cooperate to perceive the behavior posture of the user. Determine the wearing state of the smart device through the sensor, so as to select a suitable perception scheme; when the user does not wear the sensor, the CSI module can perform intelligent monitoring and perception of the user's health, activities, etc.; when the user has worn the sensor, perform intelligent monitoring and perception through the sensor.

[0107] In addition, by adjusting the signal frequency of the CSI module according to the magnitude relationship between the statistical value of the CSI signal amplitude or phase and the preset threshold, the effect of reducing power consumption can be achieved.

[0108] In the embodiment of the present application, without increasing the hardware cost, the existing sensors and Wi-Fi chips on the smart device cooperate to perceive the behavior posture of the user.

[0109] On the one hand, CSI sensing technology can overcome the difficulty of sensing through sensors when humans and machines are separated; on the other hand, by using real-time CSI signals to calculate the relationship between the statistical value of amplitude or phase and a preset threshold, the signal frequency of the CSI module can be adjusted according to this relationship, thereby reducing power consumption.

[0110] In some embodiments, the number of smart devices participating in the wireless sensing process can be adjusted based on the statistical values ​​of the amplitude or phase of the real-time CSI signal. Theoretically, CSI signal communication between multiple devices can detect human activity over a larger area and improve sensing accuracy. If the statistical values ​​indicate drastic changes in the CSI signal and the user is moving rapidly, increasing the number of smart devices participating in the Wi-Fi sensing process improves accuracy. Conversely, if the statistical values ​​indicate slow changes in the CSI signal and the user is nearly stationary, reducing the number of smart devices participating in the Wi-Fi sensing process, while ensuring sensing accuracy, can reduce power consumption.

[0111] In some embodiments, when a user wears a sensor, data from multiple sensors (such as an accelerometer, gyroscope, and / or heart rate sensor) can be selected simultaneously, or the CSI module can still be enabled, with both sensors sensing simultaneously, and the sensing accuracy can be improved through methods such as voting.

[0112] In some embodiments, the Wi-Fi signal may be replaced by UWB or ultrasound, etc.

[0113] It is understood that the embodiments of this application involve user-related data. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0114] It should be noted that although the steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps; or steps from different embodiments may be combined into a new technical solution.

[0115] Based on the foregoing embodiments, this application provides an activity sensing device, which includes various modules and units included in each module, and can be implemented by a processor; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be an AI acceleration engine (such as NPU), GPU, central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP) or field programmable gate array (FPGA), etc.

[0116] Figure 6 This is a schematic diagram of the structure of the activity sensing device provided in the embodiments of this application, such as... Figure 6 As shown, the activity sensing device 60 includes:

[0117] The sensing module 601 is configured to sense the activity of the sensing target at least via at least one wireless communication module when the sensor is separated from the sensing target; wherein the sensor is used to sense the activity of the sensing target.

[0118] In some embodiments, the sensing module 601 is configured to sense the activity of the sensing target based on the signal characteristics of the wireless signals received by the at least one wireless communication module when the sensor is separated from the sensing target.

[0119] In some embodiments, the activity sensing device 60 further includes a control module configured to control the sensor to enter a low-power mode if the sensor's current mode is an operating mode when the sensor is removed from the sensing target.

[0120] In some embodiments, the activity sensing device 60 further includes an adjustment module configured to determine statistical values ​​of signal characteristics of the wireless signal based on the wireless signal received by the wireless communication module; the statistical values ​​are used to reflect the degree of change of the signal characteristics of the wireless signal in the propagation environment; and adjust the transmission frequency of the device transmitting the wireless signal based on the statistical values.

[0121] Furthermore, in some embodiments, the adjustment module is configured to: adjust the transmission frequency to a first frequency based on determining that the statistical value is less than or equal to a first threshold; wherein the statistical value is used to measure the dispersion of the signal characteristics of the received wireless signal; and adjust the transmission frequency to a frequency greater than the first frequency based on determining that the statistical value is greater than the first threshold.

[0122] In some embodiments, the sensing module 601 is further configured to, when the sensor is carried on the sensing target, use the sensor to sense the activity of the sensing target if the current mode of the sensor is an operating mode.

[0123] In some embodiments, the activity sensing device 60 further includes a control module configured to control the wireless communication module to enter a sleep mode if the current mode of the wireless communication module is an operating mode when the sensor is carried on the sensing target.

[0124] In some embodiments, the activity sensing device 60 further includes a control module, which: determines statistical values ​​of signal characteristics of the wireless signal based on the wireless signal received by the wireless communication module; the statistical values ​​are used to reflect the degree of change of the signal characteristics of the wireless signal in the propagation environment; and adjusts the number of devices participating in the activity sensing of the sensing target based on the statistical values.

[0125] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0126] It should be noted that the module division of the activity sensing device described in this application embodiment is illustrative and only represents a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, exist as separate physical units, or have two or more units integrated into one unit. The integrated units can be implemented in hardware, as software functional units, or a combination of software and hardware.

[0127] It should be noted that, in the embodiments of this application, if the above-described methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0128] This application provides an electronic device. Figure 7 This is a schematic diagram of the hardware entity of the electronic device according to an embodiment of this application, such as... Figure 7As shown, the electronic device 70 includes a memory 701 and a processor 702. The memory 701 stores a computer program that can run on the processor 702. When the processor 702 executes the program, it implements the steps in the method provided in the above embodiments.

[0129] It should be noted that the memory 701 is configured to store instructions and applications executable by the processor 702, and can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data and video communication data) in the processor 702 and various modules in the electronic device 70. It can be implemented by flash memory or random access memory (RAM).

[0130] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method provided in the above embodiments.

[0131] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps in the method provided in the above-described method embodiments.

[0132] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium, storage medium, and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0133] It should be understood that the phrases "one embodiment," "an embodiment," or "some embodiments" mentioned throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment," "in one embodiment," or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments; their similarities or commonalities can be referred to mutually, and for the sake of brevity, they will not be repeated here.

[0134] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three kinds of relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist simultaneously, and object B exists alone.

[0135] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0136] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or modules can be electrical, mechanical, or other forms.

[0137] The modules described above as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0138] In addition, each functional module in the various embodiments of this application can be integrated into one processing unit, or each module can be a separate unit, or two or more modules can be integrated into one unit; the integrated modules can be implemented in hardware or in the form of hardware plus software functional units.

[0139] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0140] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0141] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0142] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0143] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0144] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An activity sensing method, characterized in that, The method includes: When the sensor is detached from the target, at least one wireless communication module is used to sense the activity of the target, wherein the sensor is used to sense the activity of the target; the wireless communication module includes a sleep mode; the sleep mode is when the wireless communication module is in a power-off state; When the sensor is carried on the target being sensed, the activity of the target being sensed is sensed using the sensor.

2. The method according to claim 1, characterized in that, The method further includes: The activity of the target is perceived based on the signal characteristics of the wireless signals received by the at least one wireless communication module.

3. The method according to claim 1, characterized in that, The method further includes: If the sensor is detached from the sensing target, and if the sensor's current mode is the operating mode, the sensor is controlled to enter a low-power mode.

4. The method according to claim 1, characterized in that, The method further includes: Based on the wireless signal received by the wireless communication module, statistical values ​​of the signal characteristics of the wireless signal are determined; the statistical values ​​are used to reflect the degree of change of the signal characteristics of the wireless signal in the propagation environment. Based on the statistical values, the transmission frequency of the device transmitting the wireless signal is adjusted.

5. The method according to claim 4, characterized in that, Adjusting the transmission frequency of the device transmitting the wireless signal based on the statistical value includes: Based on the determination that the statistical value is less than or equal to a first threshold, the transmission frequency is adjusted to a first frequency; wherein, the statistical value is used to measure the dispersion of the signal characteristics of the received wireless signal; Based on the determination that the statistical value is greater than the first threshold, the transmission frequency is adjusted to be greater than the first frequency.

6. The method according to claim 1, characterized in that, The method further includes: When the sensor is carried on the sensing target, if the current mode of the wireless communication module is the working mode, the wireless communication module is controlled to enter the sleep mode.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Based on the wireless signal received by the wireless communication module, statistical values ​​of the signal characteristics of the wireless signal are determined; the statistical values ​​are used to reflect the degree of change of the signal characteristics of the wireless signal in the propagation environment. Based on the statistical values, adjust the number of devices participating in the active sensing of the sensing target.

8. An activity sensing device, characterized in that, include: A sensing module is configured to sense the activity of the sensing target using at least one wireless communication module when the sensor is detached from the sensing target; wherein the sensor is used to sense the activity of the sensing target; the wireless communication module includes a sleep mode; the sleep mode is in a power-off state; The sensing module is further configured to, when the sensor is carried on the sensing target, use the sensor to sense the activity of the sensing target.

9. An electronic device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images