Human body sensing method and device for intelligent household electrical appliance, intelligent household electrical appliance system and computer readable storage medium

Abstract

The invention relates to the technical field of intelligent household electrical appliances, and discloses a human body sensing method for intelligent household electrical appliances, a plurality of intelligent household electrical appliances comprise ultrasonic modules, and each ultrasonic module has an independent frequency band; the method comprises the following steps: receiving reflected signals of each ultrasonic module, and processing the received reflected signals to extract information of each reflected signal; the extracted information comprises time information and phase difference; fusing the extraction information of each reflection signal based on a time difference of arrival algorithm and an angle of arrival algorithm to obtain a dynamic trajectory of the user; and analyzing the dynamic trajectory of the user to identify the behavior state of the user. According to the method, the user behavior state is accurately recognized based on the dynamic trajectory. Therefore, the intelligent degree of the intelligent household electrical appliance can be improved, and the control accuracy of the intelligent household electrical appliance can be improved. The invention further discloses a human body sensing device for the intelligent household electrical appliance, an intelligent household electrical appliance system and a computer readable storage medium.

Description

Technical Field

[0001] This application relates to the field of smart home appliance technology, such as a human body sensing method and device for smart home appliances, a smart home appliance system, and a computer-readable storage medium. Background Technology

[0002] With the development of air conditioning technology, consumer demand for air conditioning is gradually increasing. Smart air conditioners use human body sensing technology to detect user information in the current home environment in order to adjust the operating parameters of the air conditioner. However, existing human body sensing technology has the following bottlenecks: infrared sensors are easily interfered with by heat sources; the detection range of a single ultrasonic sensor is limited by furniture and other obstructions in the home environment; multiple ultrasonic sensors suffer from mutual interference of signals on the same frequency; and there are problems such as the lack of human body orientation recognition.

[0003] The related technology discloses an ultrasonic positioning method, comprising: sending a first ultrasonic signal from a reference point to a point to be measured and recording the signal transmission time; receiving a second ultrasonic signal at the reference point and recording the signal reception time, wherein the second ultrasonic signal is a signal in response of the point to be measured after receiving the first ultrasonic signal, and the first ultrasonic signal and the second ultrasonic signal have different frequencies; and locating the point to be measured based on the signal transmission time, the signal reception time, and the position of the reference point.

[0004] In the process of implementing the embodiments of this disclosure, at least the following problems were found in the related art:

[0005] The relevant technology sets the frequency of the ultrasonic signal transmitted from the reference point to be different from the frequency of the ultrasonic signal fed back by the point under test, thus avoiding the multipath effect during ultrasonic wave propagation. However, this technology can only locate the user and cannot identify the user's behavior.

[0006] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0007] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0008] This disclosure provides a human body sensing method and apparatus for smart home appliances, a smart home appliance system, and a computer-readable storage medium to sense user behavior information and improve the control accuracy of smart home appliances, especially air conditioners.

[0009] In some embodiments, multiple smart home appliances include ultrasonic modules, and each ultrasonic module has an independent frequency band; the method includes: receiving reflected signals from each ultrasonic module, and processing the received reflected signals to extract information from each reflected signal; the extracted information includes time information and phase difference; fusing the extracted information from each reflected signal based on a time difference of arrival algorithm and an angle of arrival algorithm to obtain the user's dynamic trajectory; and analyzing the user's dynamic trajectory to identify the user's behavioral state.

[0010] In some embodiments, the apparatus includes a processor and a memory storing program instructions, the processor being configured to, when executing the program instructions, perform the aforementioned human body sensing method for smart home appliances.

[0011] In some embodiments, the smart home appliance system includes: a plurality of smart home appliance bodies, each smart home appliance including an ultrasonic module, and each ultrasonic module having an independent frequency band; and a human body sensing device for smart home appliances as described above, installed on any one of the smart home appliance bodies.

[0012] In some embodiments, the computer-readable storage medium stores program instructions that, when executed, cause a computer to perform the aforementioned human body sensing method for smart home appliances.

[0013] The human body sensing method and apparatus for smart home appliances, smart home appliance system, and computer-readable storage medium provided in this disclosure can achieve the following technical effects:

[0014] This embodiment extracts information from the processed reflected signal, and then fuses it with the extracted information based on the time of arrival algorithm, the angle of arrival algorithm, and the extracted information to obtain a dynamic trajectory that can characterize the user's position and direction. Based on the dynamic trajectory, accurate identification of the user's behavior state is achieved. This helps to improve the intelligence level of smart home appliances and enhance the accuracy of smart home appliance control.

[0015] The above general description and the description below are exemplary and illustrative only and are not intended to limit this application. Attached Figure Description

[0016] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are shown as similar elements. The drawings are not to be scaled. And wherein:

[0017] Figure 1 This is a schematic diagram of a human body sensing method for smart home appliances provided in an embodiment of this disclosure;

[0018] Figure 2 This is a schematic diagram of another human body sensing method for smart home appliances provided in this embodiment of the disclosure;

[0019] Figure 3 This is a schematic diagram of another human body sensing method for smart home appliances provided in this embodiment of the disclosure;

[0020] Figure 4 This is another application illustration of an embodiment of the present disclosure;

[0021] Figure 5 This is a schematic diagram of a human body sensing device for smart home appliances provided in an embodiment of this disclosure;

[0022] Figure 6 This is a schematic diagram of a smart home appliance system provided in an embodiment of this disclosure. Detailed Implementation

[0023] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0024] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0025] Unless otherwise stated, the term "multiple" means two or more.

[0026] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0027] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0028] The term "correspondence" can refer to an association or binding relationship. The correspondence between A and B means that there is an association or binding relationship between A and B.

[0029] In this disclosure, smart home appliances refer to home appliances formed by incorporating microprocessors, sensor technology, and network communication technology. They possess characteristics of intelligent control, intelligent sensing, and intelligent applications. The operation of smart home appliances often relies on the application and processing of modern technologies such as the Internet of Things (IoT), the Internet, and electronic chips. For example, smart home appliances can be connected to electronic devices, enabling users to remotely control and manage them. Exemplary examples of smart home appliances include air conditioners, televisions, refrigerators, smart speakers, and robotic vacuum cleaners.

[0030] Furthermore, multiple smart home appliances include ultrasonic modules, each operating on an independent frequency band. Each smart home appliance with an ultrasonic module functions as both an ultrasonic sensing device and an ultrasonic relay device. When a smart home appliance acts as an ultrasonic relay device, ultrasonic relay forwarding can achieve cross-regional coverage, thereby breaking the limitations of ultrasonic propagation distance.

[0031] In this process, the ultrasonic frequency band for each smart home appliance is dynamically allocated by the master control node device (such as an air conditioner) before joint sensing. If the master control node is the node that was previously triggered by the user (e.g., the user says "blow air into me" to the air conditioner), then the ultrasonic frequency band starts with the master control node. The other nodes allocate frequency bands to devices in the network state according to their historical network connection order, with a step range of 1kHz to 2kHz.

[0032] While dynamically allocating ultrasonic frequency bands, the clocks of various smart home appliances are synchronized. For example, the ultrasonic frequency band of an air conditioner is 18kHz to 19kHz, the ultrasonic frequency band of a robot vacuum cleaner is 20kHz to 21kHz, and the ultrasonic frequency band of a television is 22kHz to 23kHz.

[0033] Combination Figure 1 As shown, this disclosure provides a human body sensing method for smart home appliances, including:

[0034] S101, the smart home appliance receives the reflected signals from each ultrasonic module and processes the received reflected signals to extract information from each reflected signal; the extracted information includes time and phase difference.

[0035] S102, the smart home appliance device uses the time difference of arrival algorithm and the angle of arrival algorithm to fuse the extracted information of various reflected signals in order to obtain the user's dynamic trajectory.

[0036] S103, smart home appliances analyze the user's dynamic trajectory to identify the user's behavioral state.

[0037] Here, during human perception, multiple smart home appliances sense the same event, meaning multiple smart home appliances collaboratively perceive the same event. The master control node smart home appliance (which can act as a base station) receives all reflected signals, while ordinary node smart home appliances only receive their own reflected signals and transmit them to the master control node for analysis and processing. Specifically, the master control node smart home appliance receives reflected signals from each ultrasonic module and processes them to extract information from each reflected signal. Understandably, the locations and frequency bands of the various smart home appliances with ultrasonic modules differ; therefore, the time it takes for the emitted ultrasonic signal to reach the user's location, be reflected by the human body, and be received by the corresponding smart home appliance may vary. Therefore, the master control node smart home appliance processes the received reflected signals, decomposing them according to frequency bands and extracting the reflected components from each smart home appliance. Further information from each reflected component, such as time of flight (or time difference of arrival) and phase difference, is extracted; that is, relevant information characterizing the user is extracted.

[0038] After obtaining the extracted information, the time-of-arrival (TOA) and angle-of-arrival (AOA) algorithms are used to fuse the extracted information, thereby obtaining the user's dynamic trajectory. Specifically, the TOA algorithm can be used to calculate the user's target location. For example, based on the time difference of arrival of ultrasonic signals to different reference points (smart home appliances of both the master control node and ordinary nodes can be used as reference points) and the reference point information, the user's target location can be calculated. The AOA algorithm can be used to calculate the user's angle of incidence. For example, based on the phase difference and angle equation of ultrasonic information, the angle of incidence of the signal can be calculated; based on the reference point and the angle of incidence, the user's direction information can be determined. In this way, the user's location information obtained separately is fused to obtain the user's dynamic trajectory. Among them, ultrasonic signals can be continuously emitted, and based on the fused information and the continuous signal (i.e., the time-series signal), the dynamic trajectory can be obtained. Thus, the user's location and direction information are obtained, improving the accuracy of user information.

[0039] Analyzing user dynamic trajectories helps identify user behavior states. This involves extracting information from the user's dynamic trajectory to obtain behavioral information over time, thereby accurately identifying the user's state. Compared to static information, dynamic trajectory information can more precisely identify user behavior, such as sudden changes in user speed or direction / angle; thus, it helps in identifying user actions.

[0040] Furthermore, in some embodiments, each smart home appliance can process the received reflected signals to extract information. The extracted information is then reported to the smart home appliances at the master control node, where they perform information fusion and analysis.

[0041] The human body perception method for smart home appliances provided in this disclosure extracts information from the processed reflected signals, and then fuses this information with the extracted information based on the time of arrival algorithm, the angle of arrival algorithm, and the extracted information to obtain a dynamic trajectory that can characterize the user's position and direction. Based on the dynamic trajectory, accurate identification of the user's behavior state is achieved. This helps to improve the intelligence level of smart home appliances and enhance the accuracy of smart home appliance control.

[0042] Optionally, in step S101, the smart home appliance processes the received reflected signal, including:

[0043] Smart home appliances use Fourier transform to separate reflected signals of different frequency bands.

[0044] Smart home appliances use adaptive filtering algorithms to filter out noise from separated reflected signals.

[0045] Here, the smart home appliance at the master control node processes the received reflected information, specifically including signal separation and noise filtering. Signal separation refers to the separation of signals from different frequency bands. Fourier transform is used to convert the time-domain signal into a frequency-domain signal, and each frequency band is identified based on factors such as the energy peak value of the frequency-domain signal. Then, the target audio signal is extracted using digital filters or frequency-domain masking techniques. Fast Fourier Transform can be used to separate signals from each frequency band to meet the real-time control and computational efficiency requirements of smart home appliances. After separating the reflected signals from each frequency band, broadband in-band noise may still exist in the signal, such as airflow noise and fan noise that are the same as the power frequency in the same frequency band. Therefore, an adaptive filtering algorithm is used to perform fine denoising on the separated reflected signals to dynamically eliminate residual in-band noise.

[0046] Furthermore, the formula for the adaptive filtering algorithm is:

[0047]

[0048] Among them, s k (t) represents the received signal at the k-th node, w k τ is the weighting coefficient. k For time delay compensation, e(t) is the residual error.

[0049] Optionally, before separating the reflected signals of different frequency bands using Fourier transform, the method further includes: the smart home appliance preprocessing the received reflected signals to eliminate bandwidth noise.

[0050] Here, out-of-band noise mainly refers to noise outside the ultrasonic frequency band, which can be filtered out using a fixed bandpass filter. Before separating signals in each frequency band, noise preprocessing is performed to avoid contamination of the signals. This also prevents the loss of some frequency signal components due to noise interference during frequency separation. Furthermore, noise preprocessing reduces the workload of subsequent processing stages and improves computational efficiency. Thus, coarse filtering before signal separation provides initial suppression of out-of-band noise, while fine noise filtering after signal separation eliminates interference from noise signals within the frequency band.

[0051] Combination Figure 2 As shown, this disclosure provides another method for human body sensing in smart home appliances, including:

[0052] S101, the smart home appliance receives the reflected signals from each ultrasonic module and processes the received reflected signals to extract information from each reflected signal; the extracted information includes time information and phase difference.

[0053] S121, the smart home appliance calculates the user's first location information based on the hyperbolic equation of the time difference of arrival algorithm and time information.

[0054] S122, the smart home appliance calculates the user's second positioning information based on the incident angle equation and phase difference of the arrival angle algorithm.

[0055] S123, smart home appliances integrate first and second location information to obtain the user's dynamic trajectory.

[0056] S103, smart home appliances analyze the user's dynamic trajectory to identify the user's behavioral state.

[0057] Here, the fusion processing of the extracted information mainly includes the calculation of different user information and the fusion of the calculated user information. The calculation of user information includes the calculation of user location (i.e., first positioning information) and user direction (second positioning information). Regarding the calculation of the first positioning information, it is calculated using the distance difference between the user and the base station, which is equal to the product of the time difference and the signal propagation speed. In the case of multiple base stations, the distance between the user and any two base stations forms a hyperbola, and the user is located at the intersection of the hyperbola. Thus, when the user's location coordinates are two-dimensional coordinates, the user's location can be determined through three pairs of base stations. The hyperbola equation is:

[0058]

[0059] Where c is the signal propagation speed, Δt ij Let x be the relative time difference between the ultrasonic signal received by base station i and the ultrasonic signal received by base station j. i ,yi Let (x) be the coordinates of base station i, and (x) be the coordinates of base station i. j ,y j Let (x, y) be the coordinates of base station j, and (x, y) be the coordinates of the user.

[0060] Regarding the calculation of the second positioning information, the incident angle is the angle between the ultrasonic signal reaching the receiver and the reference direction. The incident angle can be calculated using the phase difference of different received signals and the incident angle equation.

[0061] The azimuth equation is:

[0062] Among them, (x i ,y i Let (x, y) be the coordinates of base station i, (x, y) be the coordinates of the user, and θ be the coordinates of the user. i Let e ​​be the azimuth angle of the user relative to base station i. θ The measurement error typically follows a zero-mean Gaussian distribution.

[0063] Furthermore, the incident angle θ is calculated using the formula θ = arcsin(Δφ / π), where Δφ is the phase difference. The physical incident angle θ is then converted into the geometric azimuth angle θi of the target relative to the base station, i.e.: θi = θ + θ offset θ offset The angle between the ultrasonic sensor's installation direction and the coordinate system axis is a known value. Thus, the user's second positioning information can also be calculated using two base stations, i.e., the user's coordinates are obtained by calculating the angle.

[0064] Furthermore, the two location information sets are fused to obtain the user's dynamic trajectory. Specifically, a weighted least squares optimization is performed on the first and second location information sets, as shown in the following formula:

[0065]

[0066] in, f is the weighting coefficient. i (x,y) represents the TDoA value theoretically calculated based on the current user location (x,y), g i (x,y) is the theoretically calculated AoA value based on the current user location (x,y), θ j The angle of arrival (actual observation value, i.e., the geometric azimuth mentioned earlier) measured for the j-th base station, Δt i Let be the measurement time difference (actual observation value, i.e., the relative time difference mentioned earlier) for the i-th base station pair. Thus, by fusing these measurements, the user location that best meets all constraints is found, thereby ensuring the reliability of user information. User information based on time series data constitutes the user's dynamic trajectory.

[0067] Optionally, in step S123, the smart home appliance integrates the first positioning information and the second positioning information to obtain the user's dynamic trajectory, including:

[0068] Smart home appliances use the weighted least squares method to sum the first and second positioning information.

[0069] Smart home appliances dynamically adjust the corresponding weighting coefficients based on the signal-to-noise ratio and angular variance of the reflected signal.

[0070] Here, the information obtained by weighted least squares fusion summation is as described in the formula above and will not be repeated. The adjustment of the weight coefficients for the Time Difference of Arrival (TDOA) and Arrival Angle (AGA) algorithms is explained. Specifically, the weights of the TDO algorithm are positively correlated with the signal-to-noise ratio (SNR), while the weights of the AGA algorithm are negatively correlated with the angle variance. The accuracy of TDoA directly depends on the accuracy of the time difference measurement. A higher SNR results in a clearer signal waveform and a smaller time difference estimation error. Therefore, the two are positively correlated. Angle variance... This reflects the stability of AoA measurements. A larger variance indicates a more severe impact from multipath interference, noise, or array calibration errors, resulting in lower data reliability. Therefore, the two are negatively correlated. This allows TDoA / AoA hybrid positioning to maintain high accuracy and reliability even in complex environments.

[0071] Optionally, for every unit increase in angular variance, the corresponding weighting coefficient decreases by half. For every 10dB increase in signal-to-noise ratio (SNR), the corresponding weighting coefficient doubles. If the SNR increases from 10dB to 20dB, the weighting coefficients change from... Become

[0072] Furthermore, as mentioned earlier, the user's initial location information is determined using three pairs of base stations. The corresponding weights... It includes three sub-weights, the proportions of which are determined based on the signal-to-noise ratio (SNR) of the corresponding signals. For example, the SNRs are 10dB, 20dB, and 30dB; the sub-weights are allocated as w1:w2:w3 = 1:2:4 (doubling every 10dB). Similarly, the weights... It includes two sub-weights, the ratio of which is determined based on the angular variance. For example, if the angular variances are 4 and 25, the corresponding sub-weight allocation is w1:w2 = 25:4.

[0073] Combination Figure 3 As shown, this disclosure provides another method for human body sensing in smart home appliances, including:

[0074] S101, the smart home appliance receives the reflected signals from each ultrasonic module and processes the received reflected signals to extract information from each reflected signal; the extracted information includes time and phase difference.

[0075] S102, the smart home appliance device uses the time difference of arrival algorithm and the angle of arrival algorithm to fuse the extracted information of various reflected signals in order to obtain the user's dynamic trajectory.

[0076] S131, Smart home appliances extract motion feature parameters of the time series of the user's dynamic trajectory.

[0077] S132, Smart home appliances use dynamic time warping algorithms to calculate motion feature parameters of time series in order to match predefined behavioral states in the trajectory template library; the trajectory template library is built based on datasets and predefined behavioral states.

[0078] S133, smart home appliances will use the predefined behavioral state matched as the user's behavioral state.

[0079] Here, motion feature parameters of the user's dynamic time series are extracted, such as changes in speed, direction, and height. Then, a dynamic time warping algorithm is used to calculate the time series' operational feature parameters, such as sudden speed changes and the rate of change of direction angle. The calculated motion feature parameters of the time series are matched with predefined behavioral states in the trajectory module library to determine the user's behavioral state. For example, when a user is in a fall state, the sudden speed change is >1.5 m / s and the height decrease is >0.5 m. When a user is walking normally, the speed fluctuates periodically and is within the range of 0.8–1.2 m / s.

[0080] Dynamic Time Warping (DTW) is an algorithm for aligning two time series, suitable for handling sequences of different lengths or speeds. It uses dynamic programming to find the optimal path that minimizes the cumulative distance between the two sequences. In user behavior state recognition, a user's dynamic trajectory may be executed with different operating parameters; for example, the trajectory lengths of walking and running are different. The DTW algorithm can effectively handle this type of data. By aligning time series data with predefined behavior templates, DTW achieves high-precision recognition of complex actions (such as falls, turning, etc.). In other words, it matches the template to determine the most similar user behavior state. This solves problems such as speed differences in user behavior recognition and improves recognition accuracy.

[0081] Optionally, in S132, the trajectory template library is constructed in the following way:

[0082] S1321, Smart home appliances extract the user's historical motion characteristic parameters; historical motion characteristic parameters include speed and rate of change of direction angle.

[0083] S1322, Smart home appliances construct a distance matrix based on the extracted historical motion feature parameter time series and reference sequence to obtain a cumulative distance matrix. The reference sequence is a sequence of running feature parameters corresponding to predefined behavioral states.

[0084] S1323, the smart home appliance aligns the historical motion feature parameters time series according to the cumulative distance matrix to update the trajectory template. The trajectory templates constitute a trajectory template library.

[0085] Here, a trajectory template library is constructed using the user's historical running feature parameters and predefined behavioral states. The sequence of running feature parameters corresponding to the predefined behavioral states serves as the reference sequence (i.e., the initial trajectory template), and the user's historical running feature parameters serve as the target sequence. The trajectory template is updated using the target sequence and the reference sequence. Specifically, for the target sequence X = (x1, x2, ..., x... m ) and the reference sequence Y = (y1, y2, ..., y n Construct an m×n distance matrix D. Each element D(i,j) represents the distance between the i-th point in the target sequence and the j-th point in the reference sequence. The distance matrix can be constructed based on distance metrics such as Euclidean distance, Manhattan distance, or cosine similarity. For example, the distance matrix D(i,j) = ||x|| i -y j || 2 Furthermore, the cumulative distance matrix DTW(i,j) = D(i,j) + min(DTW(i-1,j),DTW(i,j-1),DTW(i-1,j-1)) is calculated using the distance matrix. DTW(0,0) = 0 is initialized, and the edge values ​​are set to infinity. The edges are gradually filled from (1,1) to (m,n), and DTW(m,n) is the final similarity.

[0086] Then, path backtracking is performed based on the cumulative distance matrix, tracing back from the endpoint (m,n) to (0,0), and the optimal path is recorded. The target sequence is aligned using the optimal path, and then the average value of the aligned target sequence is calculated. This average trajectory is the latest reference sequence, i.e., the updated trajectory template. In this way, a trajectory template for a predefined behavioral state can be obtained. Repeating the above steps yields multiple trajectory templates for predefined behavioral states, thus forming a trajectory template library. In this way, the trajectory template can be continuously updated and optimized based on new historical operational feature parameters, improving the matching degree between the trajectory template and the current scene; thereby improving the accuracy of user behavior state judgment.

[0087] Optionally, in S132, the predefined behavioral state is obtained in the following way:

[0088] Smart home appliances acquire users' historical location information and inertial measurement unit data.

[0089] Smart home appliances input historical positioning information and inertial measurement unit data into a decision tree model to output predefined behavioral states.

[0090] Here, the system utilizes the user's terminal devices (such as mobile phones, wearable devices, smart mobile devices, virtual reality devices, etc.) to acquire the user's historical location information (such as mean / variance of velocity, displacement entropy value) and inertial measurement unit (IMU) data (such as acceleration FFT main frequency, angular energy spectral density); simultaneously, it uses tags to identify and match user information and data to determine behavioral states. The acquired historical location information and IMU data are then input into a trained decision tree model to output predefined behavioral states. These predefined behavioral states include sitting, walking, running, falling, etc.

[0091] The parameters of the trained decision tree model can be fine-tuned (e.g., speed thresholds) based on trajectory templates from the trajectory template library. Simultaneously, the sequences corresponding to the behavioral states output by the trained decision tree model can be input into the trajectory template library to update and optimize the trajectory templates. This ensures the accuracy of both the decision tree model and the trajectory template library.

[0092] Combination Figure 5 As shown, this disclosure provides a human body sensing device 100 for smart home appliances, including a processor 101 and a memory 102. Optionally, the device may further include a communication interface 103 and a bus 104. The processor 101, communication interface 103, and memory 102 can communicate with each other via the bus 104. The communication interface 103 can be used for information transmission. The processor 101 can call logical instructions in the memory 102 to execute the human body sensing method for smart home appliances described in the above embodiment.

[0093] Furthermore, the logical instructions in the aforementioned memory 102 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0094] The memory 102, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor 101 executes functional applications and data processing by running the program instructions / modules stored in the memory 102, thereby implementing the human body sensing method for smart home appliances in the above embodiments.

[0095] The memory 102 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 102 may include high-speed random access memory and may also include non-volatile memory.

[0096] Combination Figure 6 As shown, this disclosure provides a smart home appliance system, including: multiple smart home appliance bodies 200, each smart home appliance including an ultrasonic module, and each ultrasonic module having an independent frequency band; and the aforementioned human body sensing device 100 for the smart home appliance. The human body sensing device 100 for the smart home appliance is installed in any one of the smart home appliance bodies. The installation relationship described herein is not limited to placement inside the smart home appliance body, but also includes installation and connection with other components of the smart home appliance, including but not limited to physical connection, electrical connection, or signal transmission connection. Those skilled in the art will understand that the human body sensing device 100 for the smart home appliance can be adapted to feasible product bodies to achieve other feasible embodiments.

[0097] This disclosure provides a computer-readable storage medium storing computer-executable instructions configured to perform the above-described human body sensing method for smart home appliances.

[0098] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this disclosure. The aforementioned storage medium can be a non-transitory storage medium, such as a USB flash drive, external hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., and other media capable of storing program code.

[0099] The foregoing description and accompanying drawings fully illustrate embodiments of this disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or” as used in this application means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0100] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0101] The methods and products (including but not limited to devices and equipment) disclosed in the embodiments herein can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed units may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to implement this embodiment according to actual needs. Furthermore, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0102] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. A human body sensing method for smart home appliances, characterized in that, Multiple smart home appliances include ultrasonic modules, and each ultrasonic module has an independent frequency band; the method includes: The system receives reflected signals from each ultrasonic module and processes the received reflected signals to extract information from each reflected signal; the extracted information includes time information and phase difference. The user's dynamic trajectory is obtained by fusing the extracted information from various reflected signals based on the time difference of arrival algorithm and the angle of arrival algorithm. Analyze users' dynamic trajectories to identify their behavioral states.

2. The method according to claim 1, characterized in that, Processing the received reflected signal includes: Fourier transform is used to separate reflected signals of different frequency bands; An adaptive filtering algorithm is used to filter out noise from the separated reflected signals.

3. The method according to claim 1, characterized in that, The user's dynamic trajectory is obtained by fusing extracted information from various reflected signals based on the time difference of arrival (TDOA) and angle of arrival (AHE) algorithms, including: The user's first location information is calculated based on the hyperbolic equation and time information of the Time Difference of Arrival algorithm; The user's second location information is calculated based on the azimuth equation and phase difference of the arrival angle algorithm; By fusing the first and second location information, the user's dynamic trajectory can be obtained.

4. The method according to claim 3, characterized in that, By fusing the first and second location information to obtain the user's dynamic trajectory, the following steps are taken: The weighted least squares method is used to sum the first and second positioning information in a weighted manner. The corresponding weighting coefficients are dynamically adjusted based on the signal-to-noise ratio and angular variance of the reflected signal.

5. The method according to claim 1, characterized in that, Analyze the user's dynamic trajectory to identify the user's behavioral state, including: Extract motion feature parameters from the time series of the user's dynamic trajectory; The dynamic time warping algorithm is used to calculate the running characteristic parameters of the time series in order to match the predefined behavioral states in the trajectory template library; the trajectory template library is built based on the dataset and the predefined behavioral states. Use the matched predefined behavioral state as the user's behavioral state.

6. The method according to claim 5, characterized in that, The trajectory template library is built in the following way: Extract the user's historical motion feature parameters; the historical motion feature parameters include velocity and rate of change of direction angle. Based on the extracted historical motion feature parameter time series and reference sequence, a distance matrix is ​​constructed to obtain the cumulative distance matrix; where the reference sequence is the historical running feature parameter sequence corresponding to a predefined behavior state; Based on the cumulative distance matrix, the historical motion feature parameters are aligned in time series to create an updated trajectory template; the trajectory templates constitute a trajectory template library.

7. The method according to claim 5, characterized in that, Predefined behavioral states are obtained in the following ways: Acquire historical location information and inertial measurement unit data of tagged users; The acquired historical positioning information and inertial measurement unit data are input into the trained decision tree model to output a predefined behavioral state.

8. A human body sensing device for smart home appliances, comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to execute, when running the program instructions, the human body sensing method for smart home appliances as described in any one of claims 1 to 7.

9. A smart home appliance system, characterized in that, include: Multiple smart home appliance units, each of which includes an ultrasonic module, and each ultrasonic module has an independent frequency band; The human body sensing device for smart home appliances as described in claim 8 is installed on any one of the smart home appliance bodies.

10. A computer-readable storage medium storing program instructions, characterized in that, When the program instructions are executed, they cause the computer to perform the human body sensing method for smart home appliances as described in any one of claims 1 to 7.

Images