Positioning information determination method and electronic device

Abstract

The invention discloses a positioning information determination method and an electronic device, and relates to the technical field of smart home, and the method comprises the steps: transmitting an ultrasonic detection signal to a target space through a smart home device, and obtaining an echo sequence; extracting a sampling sequence corresponding to the echo sequence based on the starting moments corresponding to the plurality of emission periods, and filtering the sampling sequence to obtain a first receiving signal used for indicating the dynamic change of the moving target in the target space; performing cross-correlation operation on the first receiving signal and the ultrasonic detection signal to obtain a channel impact response sequence corresponding to the target space; and performing clustering analysis on the channel impact response sequence to determine the positioning information of the moving target. The problem that a human body sensing technology cannot accurately position the moving target in a complex home scene under the condition that special hardware is not additionally installed is solved, and the effect that the moving target can be positioned without additional special hardware is achieved.

Description

Technical Field

[0001] This application relates to the field of smart home technology, and more specifically, to a method and electronic device for determining location information. Background Technology

[0002] Among related technologies, human body sensing technology often suffers from drawbacks such as poor adaptability, high cost, and weak anti-interference ability when applied to complex home scenarios. It is impossible to achieve efficient detection and continuous tracking of human bodies in a room without adding new hardware. For example, traditional infrared or millimeter-wave sensors often require additional dedicated hardware and perform poorly in environments with complex room layouts and cluttered furniture.

[0003] There is still no effective solution to the problem that human body sensing technology cannot accurately locate moving targets in complex home scenes without the installation of additional dedicated hardware.

[0004] Therefore, it is necessary to improve the relevant technology to overcome the aforementioned defects. Summary of the Invention

[0005] This application provides a method and electronic device for determining location information, so as to at least solve the problem in the related art that human perception technology cannot accurately locate moving targets in complex home scenes without the installation of additional dedicated hardware.

[0006] According to one aspect of the embodiments of this application, a method for determining location information is provided, comprising: emitting an ultrasonic detection signal to a target space via a smart home device to obtain echo sequences corresponding to the ultrasonic detection signals in multiple transmission cycles; extracting sampling sequences corresponding to the echo sequences based on the start times corresponding to the multiple transmission cycles, and filtering the sampling sequences to obtain a first received signal for indicating the dynamic changes of an active target in the target space; performing cross-correlation operation on the first received signal and the ultrasonic detection signal to obtain a channel impulse response sequence corresponding to the target space; and determining the location information of the active target by performing cluster analysis on the channel impulse response sequence, wherein the location information includes: distance information and / or azimuth indication information between the smart home device and the active target.

[0007] According to another aspect of the embodiments of this application, a device for determining location information is also provided, comprising: a signal transmitting module, configured to transmit an ultrasonic detection signal to a target space via a smart home device to obtain echo sequences corresponding to the ultrasonic detection signals in multiple transmission cycles; an extraction module, configured to extract sampling sequences corresponding to the echo sequences based on the start times corresponding to the multiple transmission cycles, and filter the sampling sequences to obtain a first received signal for indicating the dynamic changes of an active target in the target space; a cross-correlation module, configured to perform cross-correlation operations on the first received signal and the ultrasonic detection signal to obtain a channel impulse response sequence corresponding to the target space; and a determination module, configured to determine the location information of the active target by performing cluster analysis on the channel impulse response sequence, wherein the location information includes: distance information and / or azimuth indication information between the smart home device and the active target.

[0008] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program is configured to execute the above-described method for determining location information when it is run.

[0009] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method for determining the location information through the computer program.

[0010] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the above-described method for determining location information.

[0011] This application utilizes a smart home device to emit ultrasonic detection signals into a target space, obtaining echo sequences corresponding to the ultrasonic detection signals within multiple transmission cycles. Based on the start times corresponding to the multiple transmission cycles, sampling sequences corresponding to the echo sequences are extracted and filtered to obtain a first received signal indicating the dynamic changes of an active target within the target space. The first received signal and the ultrasonic detection signals are cross-correlated to obtain a channel impulse response sequence corresponding to the target space. Cluster analysis of the channel impulse response sequence determines the location information of the active target, where the location information includes distance information and / or azimuth indication information between the smart home device and the active target. Therefore, this technical solution solves the problem in related technologies where human perception technology cannot accurately locate active targets in complex home scenes without additional dedicated hardware; it improves the flexibility of video data transmission and achieves the technical effect of locating active targets without additional dedicated hardware. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram of the hardware environment for an optional method for determining location information according to an embodiment of this application;

[0015] Figure 2 This is a flowchart of an optional method for determining location information according to an embodiment of this application;

[0016] Figure 3 This is another flowchart of an optional method for determining location information according to an embodiment of this application;

[0017] Figure 4 This is yet another flowchart of an optional method for determining location information according to an embodiment of this application;

[0018] Figure 5 This is a structural block diagram of an optional positioning information determination device according to an embodiment of this application. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0020] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0021] According to one aspect of the embodiments of this application, a method for determining location information is provided. This method for determining location information is widely applicable to whole-house intelligent digital control application scenarios such as smart homes, smart home ecosystems, and intelligencehouse ecosystems. Optionally, in this embodiment, the above-mentioned method for determining location information can be applied to, for example... Figure 1 The hardware environment shown consists of multiple terminal devices 102 and a server 104. For example... Figure 1 As shown, server 104 is connected to multiple terminal devices 102 via a network and can be used to provide services (such as application services) to terminals or clients installed on terminals. A database can be set up on the server or independently of the server to provide data storage services for server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data processing services for server 104.

[0022] The aforementioned network may include, but is not limited to, at least one of the following: wired network, wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network, metropolitan area network, local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity), Bluetooth. The terminal device 102 may not be limited to PC, mobile phone, tablet computer, smart air conditioner, smart range hood, smart refrigerator, smart oven, smart stove, smart washing machine, smart water heater, smart washing equipment, smart dishwasher, smart projector, smart TV, smart clothes rack, smart curtains, smart audio-visual equipment, smart socket, smart speaker, smart speaker box, smart fresh air equipment, smart kitchen and bathroom equipment, smart bathroom equipment, smart robot vacuum cleaner, smart window cleaning robot, smart mopping robot, smart air purifier, smart steam oven, smart microwave oven, smart water heater, smart air purifier, smart water dispenser, smart door lock, etc.

[0023] This embodiment provides a method for determining location information, including but not limited to applications in smart home devices. Figure 2 This is a flowchart of a method for determining location information according to an embodiment of this application, which includes the following steps:

[0024] Step S202: Send an ultrasonic detection signal to the target space through a smart home device to obtain the echo sequence corresponding to the ultrasonic detection signal in multiple transmission cycles;

[0025] The target space includes the smart home devices.

[0026] Step S204: Extract the sampling sequence corresponding to the echo sequence based on the start time corresponding to the multiple transmission cycles, and filter the sampling sequence to obtain a first received signal used to indicate the dynamic changes of the active target in the target space;

[0027] Step S206: Perform cross-correlation operation on the first received signal and the ultrasonic detection signal to obtain the channel impulse response sequence corresponding to the target space;

[0028] Step S208: Determine the location information of the active target by performing cluster analysis on the channel impulse response sequence, wherein the location information includes: distance information and / or orientation indication information between the smart home device and the active target.

[0029] Through the above steps, an ultrasonic detection signal is emitted from a smart home device to a target space to obtain echo sequences corresponding to the ultrasonic detection signal in multiple transmission cycles. Based on the start time corresponding to each of the multiple transmission cycles, a sampling sequence corresponding to the echo sequence is extracted and filtered to obtain a first received signal indicating the dynamic changes of an active target within the target space. The first received signal and the ultrasonic detection signal are cross-correlated to obtain a channel impulse response sequence corresponding to the target space. Cluster analysis is performed on the channel impulse response sequence to determine the location information of the active target, wherein the location information includes distance information and / or azimuth indication information between the smart home device and the active target. Therefore, the above technical solution solves the problem in related technologies where human perception technology cannot accurately locate active targets in complex home scenes without additional dedicated hardware; it improves the flexibility of video data transmission and achieves the technical effect of locating active targets without additional dedicated hardware.

[0030] Regarding step S202 above, the process of emitting ultrasonic detection signals to the target space via a smart home device to obtain echo sequences corresponding to the ultrasonic detection signals within multiple transmission cycles includes: emitting the ultrasonic detection signals to the target space at a preset frequency via a sound-emitting component on the smart home device; acquiring the echo signals corresponding to the ultrasonic detection signals via a sound-receiving component on the smart home device; compensating the echo signals using the echo channel impulse response of the smart home device, wherein the echo channel impulse response is determined by the location and orientation information of the smart home device; and digitally processing the compensated echo signals to obtain the echo sequences.

[0031] The smart home device used to obtain location information in this application should have a sound-emitting component and a sound-receiving component; wherein, the sound-emitting component is, for example, a speaker, and the sound-receiving component is, for example, a microphone array. The moving target is, for example, a person, a pet, etc. Therefore, this application can be based entirely on the speakers and microphone arrays of existing smart home devices, without the need for additional hardware, to achieve efficient human detection and continuous tracking in complex home scenarios.

[0032] Specifically, a smart home device (such as a smart speaker) continuously emits ultrasonic detection signals into a target space (i.e., the room where the device is located). A microphone array receives echo signals reflected from various objects (including moving targets, such as human bodies). After compensating and digitally processing the received echo signals, an echo sequence is formed over multiple transmission cycles. The echo sequence captures the entire change of the ultrasonic detection signal from transmission to reception, including direct waves and multipath reflections. Optionally, the echo sequence can also be called a fast time series. Digital processing involves converting the echo signals into digital signals using an analog-to-digital converter. The echo channel impulse response of the smart home device is determined using the following formula:

[0033]

[0034] Where h(k) represents the echo channel impulse response at the k-th sampling point, A i The attenuation factor for the i-th path is determined by the room material's reflectivity and the angle of incidence. (Equivalent to the orientation information mentioned above) determines; r i The virtual reflection source is represented by r0 = (x0, y0, z0) which is the location of the device center (equivalent to the above location information), and c represents the speed of sound.

[0035] In an optional embodiment, extracting the sampling sequence corresponding to the echo sequence based on the start times corresponding to the plurality of transmission cycles includes: extracting target sampling points in the echo sequence based on the start times, wherein the time interval between the target time where the target sampling point is located and the start time is a target value; and arranging the plurality of target sampling points in chronological order between the plurality of start times to obtain the sampling sequence.

[0036] Understandably, for the echo sequences corresponding to multiple transmission cycles, the k-th sampling point in each echo sequence is extracted based on the start time of each echo sequence to form a sampling sequence, where k takes values ​​of 1, 2, 3, and so on. In other words, if there are K sampling points in the echo sequence, K sampling sequences will be extracted, where k and K are both positive integers. Therefore, the target values ​​corresponding to the sampling points in different sampling sequences are different. The sampling sequence reflects the signal strength variation of the same target value (time delay) sampling point within multiple transmission cycles, emphasizing the signal's trend over a long time scale, especially for moving targets, such as changes caused by human activity. Optionally, the sampling sequence can also be called a slow time series.

[0037] Further, filtering the sampling sequence to obtain a first received signal for indicating the dynamic changes of the moving target within the target space includes: filtering multiple sampling sequences respectively using a high-pass filter; wherein, the echo sequence corresponding to the ultrasonic detection signal of the nth transmission cycle includes: the nth sampling point in each sampling sequence, where n is a naturally increasing number; reconstructing the filtered multiple sampling sequences into a new echo sequence, and determining the new echo sequence as the first received signal.

[0038] In other words, the sampled sequence is high-pass filtered (e.g., a second-order Butterworth filter with a cutoff frequency of 0.5 Hz) to remove low-frequency and DC components. Low-frequency components mainly consist of slowly changing or unchanging static environmental signals, while signal changes caused by dynamic moving targets are reflected as high-frequency components. High-pass filters can preserve the high-frequency components in the sampled sequence.

[0039] The process of reconstructing the filtered sample sequences into a new echo sequence to obtain the first received signal includes: repositioning the nth sample point in each filtered sample sequence to its original position in the echo sequence corresponding to the nth transmission period, thus obtaining a new echo sequence and subsequently the first received signal; where n increases sequentially from 1. In this way, each sample point in the new echo sequence contains filtered dynamic change information. The first received signal, after eliminating static background clutter, accurately reflects the dynamic change characteristics of human activity and is used for subsequent human detection and localization.

[0040] Alternatively, high-pass filters can also be such as Chebyshev and Bessel. The parameters of the high-pass filter (such as cutoff frequency and order) should be selected according to the characteristics of environmental noise and the frequency characteristics of human activity signals in order to achieve the best noise reduction effect.

[0041] By extracting and high-pass filtering the sampled sequence, the system effectively suppresses background noise from static environments in the echo signal, highlighting and capturing signal changes caused by human activity, thereby achieving high-precision detection and positioning of the human body. This not only reduces the requirements for adaptability to complex environments but also avoids the need for additional hardware sensors, thus reducing costs and simplifying the deployment of smart home devices. Furthermore, due to the efficient processing of filtering and signal reconstruction, the system maintains good real-time performance, enabling rapid response to changes in human activity, making it suitable for applications such as smart lighting, security monitoring, and health monitoring.

[0042] For example, a high-pass filtering scheme for eliminating static clutter (slow-time processing) includes: an ultrasonic human detection system detects and tracks the position of a person indoors by continuously emitting multi-cycle ultrasonic signals. The received signal within each transmission cycle is called the echo sequence, with a sampling rate of 48 kHz. Each detection lasts for several transmission cycles, such as 10. For a fixed fast-time sampling point k, the received signal values ​​taken from it over N consecutive transmission cycles are represented as the sampling sequence:

[0043] x k (n)=r n (k), n=1,2,…,N.

[0044] Where, r n (k) represents the amplitude of the received signal at time k in the nth transmission cycle. Since the reflection from a static object is essentially constant in each cycle, x k The low-frequency components (or DC components) of n, n = 1, 2, ..., N, correspond to static background clutter. Furthermore, assuming the filter is a high-pass filter h(n), the high-pass filtering of the sampled sequence is expressed as:

[0045] y k (n)=x k (n)*h(n), n=1,2,…,N.

[0046] Where * denotes discrete convolution operation, y k (n) represents the filtered output in the slow time dimension. All fast time points k are processed sequentially to form a new echo sequence. The new echo sequence contains only samples of the changing parts of the dynamic target (e.g., the human body).

[0047] In an exemplary embodiment, performing a cross-correlation operation on the first received signal and the ultrasonic detection signal to obtain the channel impulse response sequence corresponding to the target space includes: determining cross-correlation values ​​corresponding to multiple time delays through a cross-correlation function; wherein the multiple time delays are multiple time delays between the first received signal and the ultrasonic detection signal; and determining the channel impulse response sequence through the cross-correlation values.

[0048] In digital signal processing, cross-correlation is defined as:

[0049]

[0050] Where s(k) is the reference transmitted signal (equivalent to the ultrasonic detection signal), r(k+τ) is the high-pass filtered received signal (equivalent to the first received signal), and τ represents the time delay. The result C sr(τ) is a signal delay function, with its maximum value corresponding to the point of best-matched echo delay. This function is the estimate of the dynamic CIR (Channel Impulse Response) sequence.

[0051] In an exemplary embodiment, determining the location information of the active target by performing cluster analysis on the channel impulse response sequence includes: extracting echo peak information from the channel impulse response sequence through peak detection, and summarizing the echo peak information to obtain a two-dimensional feature point set; wherein, the echo peak information includes: peak position information and amplitude information; constructing a two-dimensional data plane corresponding to the two-dimensional feature point set based on the echo peak information; clustering the data points in the two-dimensional data plane into at least one cluster using a preset clustering algorithm; calculating the representative distance corresponding to the at least one cluster to determine the location information through the representative distance.

[0052] Echo peaks indicate the presence of a moving target. Echo peak information includes: peak position information τ. ij Where i represents the number of frames in the channel impulse response sequence, corresponding to the number of transmission cycles, j indicates that the echo peak is the j-th peak in the channel impulse response sequence; and amplitude information A ij .

[0053] The peaks of all frames are aggregated into a two-dimensional feature point set. Using the peak position information and frame number as coordinates, a slow-time–fast-time two-dimensional data plane is constructed. Each peak can be represented as: p ij =(τ ij ,i). Where, τ ij This indicates the position of the j-th peak in the i-th frame (corresponding to fast time), where i represents the frame index (corresponding to slow time).

[0054] Next, a distance metric for clustering is defined, taking into account both location differences and inter-frame distance:

[0055]

[0056] Where p kl Is it waiting for the peak point p? ij Calculate other peaks in the distance metric, w τ It is the weight of the fast time dimension, i.e., the peak position index dimension, w T It's the weight of the slow time dimension, i.e., the frame index dimension, which is adjusted by w. τ and w T This can balance the weights of target distance continuity and detection time continuity. Where p kl =(τ kl ,k). Where, τ klThis indicates the position of the l-th peak in the k-th frame (corresponding to fast time), where k represents the frame index (corresponding to slow time).

[0057] Then, the DBSCAN density clustering algorithm (equivalent to the aforementioned preset clustering algorithm) is used to assign peaks with a distance D less than the threshold ε to the same cluster:

[0058] D(p ij ,p kl )≤ε→p ij ,p kl They belong to the same cluster.

[0059] Each cluster can be viewed as corresponding to an activity objective. For each cluster C l l = 1, 2, ..., which includes multipath echoes of the same dynamic target and peaks in adjacent frames. The representative distance of a cluster (i.e., the cluster peak location) can be estimated by intra-cluster weighted averaging:

[0060]

[0061] Where A ij This represents the peak amplitude. The distance information in the location data (representing the estimated one-way distance between the active target and the smart home device) can be further... The conversion yields (c is the speed of sound, approximately 343 m / s).

[0062] Using the methods described above, the system can effectively aggregate multipath echo peaks of the same target and peaks of the same target in adjacent frames, forming a trajectory cluster. This preserves the dynamic continuity of the target while enhancing multi-target separation capabilities. Through slow-time–fast-time two-dimensional clustering analysis, the system can output distance estimates for one or more targets, providing more robust input data for subsequent angle estimation and trajectory tracking, further avoiding the misidentification of isolated noise peaks as targets.

[0063] Optionally, the location information of the active target is determined by clustering analysis of the channel impulse response sequence, including: obtaining the distance between any two receiving elements in the receiving components of the smart home device, and determining the arrival delay between the second received signals corresponding to the two receiving elements respectively; wherein the first received signal includes the second received signal; calculating the target azimuth angle of the first received signal relative to the two receiving elements using the distance and the arrival delay; fusing multiple target azimuth angles using the least squares method to obtain the azimuth indication information, so as to determine the location information using the azimuth indication information.

[0064] When two microphones are deployed on a smart home device, if the distance between the two microphones is d, and the time difference between the arrival of the signal generated by the target at the two microphones is Δt, then the orientation information (azimuth angle) θ of the moving target relative to the smart home device can be approximately expressed by geometric acoustics as follows: In a practical system, Δt can be estimated by the position of the cross-correlation peak of the dynamic CIR of the two microphones.

[0065] Furthermore, calculating the target azimuth angle of the first received signal relative to any two receiving elements using the distance and the arrival delay includes: calculating the target azimuth angle θ using the following formula. mn : Where, Δt mn Indicates the arrival delay, d mn The distance is represented by , and c represents the speed of sound. If a microphone in the microphone array is damaged, the damaged microphone can be removed.

[0066] In other words, for a multi-microphone array, the distance between any two microphones m and n is set to d. mn Calculate the arrival time delay difference Δt between them. mn Therefore, the azimuth angle can be estimated using the following formula:

[0067]

[0068] Where θ mn Let Δt be the azimuth angle estimated by microphones m and n. mn Let d be the arrival delay difference between microphones m and n. mn Let m be the geometric distance between microphones m and n. Repeat the above calculation for all microphone pairs (m, n), and then fuse multiple estimates using the least squares method to obtain a more robust target azimuth.

[0069] In an exemplary embodiment, determining the location information of the active target by performing cluster analysis on the channel impulse response sequence includes: determining the predicted location information of the active target at the current time using the output result of the Kalman filter at the previous time step; in the event of anomalies, outputting the predicted location information as the location information using the Kalman filter; wherein, the anomalies include one of the following: measurement jitter, or the active target being obstructed; in the event of no anomalies, updating the predicted location information using the location information through the Kalman filter, and outputting the updated predicted location information as the location information.

[0070] A Kalman filter is a recursive filter used to predict the state of a dynamic system and correct it with measurement results to obtain an optimal estimate of the system state. In this application, the Kalman filter can effectively handle anomalies such as measurement jitter and target occlusion, thereby providing more continuous and reliable positioning data.

[0071] Specifically, at each time point, the Kalman filter predicts the target state at the current time, i.e., predicts the location information, based on the filter output of the previous time step (i.e., the optimal state estimate of the moving target, including the positioning information output at the previous time step) and the target motion model.

[0072] In certain situations, such as measurement jitter (fluctuations in measurement values ​​caused by signal noise) or target obstruction (e.g., loss of echo signal due to furniture blocking the view), the Kalman filter outputs the predicted positioning information generated based on the prediction model as the positioning information for the current moment, thus maintaining the continuity of the positioning information. When measurement data (i.e., positioning information obtained through channel impulse response sequence analysis) is available and no anomalies occur, the Kalman filter uses this real-time measurement data to correct its predicted state, obtaining more accurate positioning information. Specifically, the filter calculates the measurement residual, i.e., the difference between the measurement data and the predicted state, and then combines the measurement residual and the Kalman gain to update the target's state estimate.

[0073] Optionally, the Kalman filter in this application is further described below: In this application, during target tracking, a Kalman filter is used to dynamically estimate the distance information and orientation indication information (such as azimuth angle) between the smart home device and the moving target, so as to enhance the robustness and continuity of the system when the moving target is temporarily occluded or the signal jitter occurs. When occlusion and jitter cause peak drift or disappearance, the predicted value is used to replace the observed value to maintain target continuity; when the detected value exists, the predicted value is updated with the observed value.

[0074] The state vector of the Kalman filter is as follows: This application considers a single human target, whose two-dimensional polar coordinate state vector u t ,t=1,2,…, represents the dynamic information of the moving target at the current time t, including the translation and rate of change of range and azimuth:

[0075]

[0076] In the state vector, d t and θ t The target distance (i.e., distance information) and azimuth angle at the current moment (an estimate can be obtained once per launch cycle) are given. and

[0077] These are the rate of change of target distance (i.e., radial velocity) and the rate of change of azimuth angle (i.e., angular velocity), respectively. This allows the filter to consider both position and velocity information simultaneously, thus better handling the smooth estimation of target motion.

[0078] The state transition model of the Kalman filter is as follows: Since indoor human activities are basically slow movements, the movement of the target in a short period of time can generally be considered to satisfy uniform linear motion (or uniform circular motion approximation within a small range). Therefore, the state transition model is defined as follows:

[0079] u t =F·u t-1 +w t ;

[0080] Where w t For process noise, it is used to model disturbances caused by the motion of the active target itself or system uncertainties. It is generally assumed to have a mean of 0 and a covariance of Q; F is the state transition matrix.

[0081] in, T represents the launch period.

[0082] This model represents the period T during which:

[0083] Target distance update = distance at the previous moment + distance change rate * time;

[0084] Azimuth update = previous angle + angular velocity * time.

[0085] The observation model of the Kalman filter involves processing the dynamic channel impulse response sequence to extract target observations, including range and azimuth estimates, and constructing an observation vector z. t for:

[0086]

[0087] in The target distance estimate is obtained through CIR peak clustering. This is the azimuth estimate obtained by calculating the time delay difference between multiple microphone array elements.

[0088] The observation model can be expressed as a linear relationship: z t =H·u t +v t ;

[0089] Where v t Observation noise, used to describe the uncertainty of echo measurements, is generally assumed to have a mean of 0 and a covariance of R; H is the observation matrix.

[0090] Furthermore, such as Figure 3 As shown, the main steps for performing Kalman filtering in this application include:

[0091] Step S301: Prediction Step:

[0092] Predicted state vector: This indicates that the current state is predicted based on the previous filtering result and the motion model.

[0093] Predicted state covariance matrix: P t|t-1 =F·P t-1|t-1 ·F T +Q; where P t|t-1 It indicates the uncertainty of the state.

[0094] Step S302: Update steps:

[0095] Calculate the Kalman gain matrix: K t =P t|t-1 ·H T ·(H·P t|t-1 ·H T +R) -1 Used to balance the weights between predicted and observed values.

[0096] Update the state vector: in, This represents the observation residual. When the peak detection is lost, the predicted value is used to replace the observed value; when the detection value exists, the observed value is used to update the predicted value.

[0097] Update covariance matrix: P t|t =(IK t ·H)·P t|t-1 ; where I is the identity matrix.

[0098] By using a Kalman filter, the distance and angle estimation of the target can be smoothed, effectively reducing noise caused by multipath interference, echo blocking, measurement jitter, etc. (Process noise w) t It can be used to model situations where a target suddenly accelerates, stops, or changes direction. (Observation noise v) t This takes into account uncertainties in ultrasonic measurements (such as room reflections and furniture obstruction). The filtering model can both continuously track the target and make dynamic predictions as needed, providing smooth estimation during short-term target obstruction.

[0099] Obviously, the embodiments described above are only some embodiments of this application, and not all embodiments. To better understand the method for determining the above location information, the following description, in conjunction with embodiments, illustrates the process, but is not intended to limit the technical solutions of the embodiments of this application. Specifically:

[0100] Among related technologies, there are two main technical approaches to achieving indoor human presence detection and positioning: traditional sensor solutions and ultrasonic acoustic solutions.

[0101] 1) Traditional Solutions: Common human detection technologies include infrared sensing, laser ranging / imaging, and millimeter-wave radar. PIR (Passive Infrared Sensor) detects movement by detecting changes in the infrared thermal radiation of the human body. It is low-cost and consumes little power, but it can only sense motion and cannot obtain the target's distance or location. It is also sensitive to environmental heat sources and is prone to false triggering under temperature changes or direct sunlight. Laser ranging or depth cameras (such as LiDAR and Kinect infrared depth cameras) can obtain accurate distance and imaging information, but these devices are expensive, bulky, require specific installation and field of view, only work effectively when unobstructed, and are greatly affected by light. Infrared imaging solutions are even more expensive in dark environments. Millimeter-wave radar sensors (such as 24GHz or 60GHz radar) can actively emit electromagnetic waves to detect the human body. They have some penetration capability, can work in darkness, and provide distance and speed information. However, millimeter-wave radar requires dedicated high-frequency hardware and antenna arrays, resulting in higher costs and relatively higher power consumption, making integration into ordinary home appliances uneconomical. In addition, most of these traditional solutions require additional hardware modules, resulting in high deployment costs and a lack of compatibility with the existing smart home ecosystem.

[0102] 2) Ultrasonic Solutions: Due to its advantages such as low hardware cost and ease of short-range distance measurement, ultrasound has been used for indoor human detection in recent years. Common ultrasonic human detection methods include TOA (Time of Arrival) ranging. TOA ranging estimates the target distance by emitting ultrasonic pulses and measuring their round-trip arrival time. This method is simple in principle and can obtain target distance information in open environments. However, in complex indoor environments, ultrasonic signals are reflected by walls and furniture, creating multipath propagation. A single sensor struggles to distinguish which echo originates from a human body, easily mistaking reflections from stationary furniture for targets. Furthermore, TOA methods typically require precise synchronization calibration of the transmitter and receiver, especially in multi-sensor systems, making deployment and maintenance complex. In addition, there are Doppler frequency shift-based ultrasonic detection methods: continuously emitting ultrasonic waves of a fixed frequency or frequency modulation, and determining the presence of movement by detecting changes in the spectrum of the reflected signal. The Doppler method is simple to implement and sensitive to motion, but it can only provide speed or motion presence information, not target distance and specific direction, making subsequent trajectory localization difficult. In summary, existing ultrasonic solutions either lack reliability in complex environments or can only detect motion but not locate, thus their application in real-world home scenarios is subject to many limitations.

[0103] As mentioned above, existing technologies for indoor human detection using ultrasound have significant drawbacks: TOA ranging requires specialized calibration and multipath propagation, making it complex and costly to implement; while Doppler detection avoids the ranging problem, it cannot acquire location information and can only determine presence or absence, limiting its functionality. Ultimately, existing human detection solutions are poorly adaptable to complex home environments, performing poorly in situations with changes in room structure, cluttered reflections, and various interferences. Furthermore, they typically require additional sensor hardware, leading to complex deployment and high costs. These shortcomings make it difficult for existing solutions to meet the demands of smart homes for low-cost, highly reliable human monitoring.

[0104] The purpose of this application is to overcome the aforementioned shortcomings and provide an ultrasonic human activity sensing technology utilizing readily available smart home devices, significantly improving detection accuracy and robustness, simplifying deployment, and reducing costs. Specifically, this application aims to: utilize the built-in speakers and microphones of commercially available devices to achieve the detection of human presence, distance and orientation estimation, and motion trajectory tracking within a room without adding any hardware. By improving signal processing algorithms, this application can extract the echo signals of human targets with high accuracy, calculate their distance and direction in real time, and smoothly track their movement paths, thereby reliably sensing and recording human activity in complex home environments. In short, the purpose of this application is to provide a low-cost, easily deployable, and highly adaptable indoor human activity sensing solution for complex environments.

[0105] Specifically, this application embodiment utilizes the hardware structure of at least one speaker and two microphones found in typical smart home devices, combined with signal processing algorithms, to achieve human body detection and localization. The following provides a detailed description of the solution from both system structure and signal processing flow perspectives.

[0106] (1) System structure design.

[0107] The system hardware consists of a speaker-microphone array: the device's built-in speaker acts as the ultrasonic transmitter, and the microphone array (at least two microphones with a fixed, known spacing) serves as the receiving sensor. The speaker and microphone are typically positioned on the same terminal device (i.e., the same smart home device, such as a smart speaker with multiple microphones around its top and the speaker in the center), allowing them to emit and pick up echoes in the same spatial area. The speaker emits high-frequency sound signals (e.g., ultrasonic waves approaching or exceeding the upper limit of human hearing to avoid disturbing the user), while the microphone continuously listens for echoes in the environment. A processing unit (an embedded processor within the device or a connected host) is responsible for generating the emitted signals and acquiring microphone data for further processing. Because it uses built-in components of existing devices, no additional hardware modules are needed; a complete detection system can be constructed simply by controlling the speaker to emit specific ultrasonic signals and digitally processing the microphone input via software.

[0108] The location (x0, y0, z0) and orientation of the equipment within the room This directly affects the propagation and reception of multipath echo signals. The relative arrangement of the speakers and microphone arrays inside the equipment also affects the amplitude of the direct wave and multipath signals. By modeling the room using the mirror method, the echo channel impulse response h(k) can be characterized by the following formula:

[0109]

[0110] Among them, A i The attenuation factor for the i-th path is determined by the room material's reflectivity and the angle of incidence. Decision; r i Let r0 represent the position of the virtual reflection source equivalent to each reflection path; r0 = (x0, y0, z0) is the center position of the device. By analyzing A... i r i Modeling can quantify the impact of equipment location and orientation on echoes. Through simulation or calibration, the systematic deviation patterns of equipment location and orientation from measurements can be obtained, allowing for compensation.

[0111] (2) Signal processing flow. For example... Figure 4 As shown in the embodiments of this application, the system's software algorithm performs signal processing according to the following process to achieve the detection and localization of human targets, including:

[0112] Step S401: Ultrasonic Signal Transmission and Reception. The system periodically transmits preset ultrasonic detection signals (e.g., pseudo-random sequences or linear frequency modulated signals) into the room via a speaker. These signals propagate through space and are reflected by obstacles (such as walls, furniture, and people). Multiple microphones on the device simultaneously receive complete echo data, including direct signals and multipath reflection signals. The signals within each transmission cycle are called a "fast time" sequence. To ensure continuous detection, the transmission-reception process is cyclical at a certain frequency (e.g., tens of times per second), enabling the system to continuously sense rapidly occurring changes.

[0113] Step S402: High-pass filtering to eliminate static clutter. The acquired microphone signal is first processed by high-pass filtering to eliminate static background signals in the environment, i.e., reflections from stationary objects such as walls and furniture. Since the speaker's position relative to the microphone remains fixed, the speaker's emitted signal can also be considered a static background signal. Specifically, the received signal in each transmission cycle is extracted at a fixed time relative to its cycle start point (e.g., the k-th (k = 1, 2, 3...K) sampling point in each cycle), and arranged chronologically between different transmission cycles to obtain a slow time series. In the slow time dimension, the reflection signals generated by fixed reflectors (and the speaker's emitted signal) remain almost unchanged, exhibiting low-frequency or quasi-static components; while the reflection signals caused by human activity change over time, exhibiting high-frequency components in the slow time dimension. Therefore, applying a high-pass filter (such as a second-order Butterworth filter with a cutoff frequency of 0.5Hz) to each slow time series can filter out static reflection components, retaining only the dynamic changes caused by moving targets. This method does not require pre-stored background channels and does not rely on CIR alignment, exhibiting good stability and real-time performance.

[0114] Step S403: Cross-correlation extraction of dynamic CIR. Next, the system performs cross-correlation calculations between the filtered received signal and the known transmitted probe signal to obtain the channel impulse response (CIR) sequence of the current environment. Cross-correlation processing is equivalent to matching probe features in the received signal, which can highlight the arrival delay and intensity of echoes from each path. Since the static component has been filtered out in the previous step, the cross-correlation result mainly corresponds to the dynamic CIR, that is, the echo channel response caused by moving targets such as humans. CIR is usually manifested as several impulse peaks, the position of each peak corresponds to the propagation delay of a certain reflection path, and the peak size reflects the reflection intensity of that path. By extracting the dynamic CIR, the system obtains the difference echo information between the current instant and the static environment, where significant peaks often indicate the presence of reflections from moving targets.

[0115] Based on steps S401 to S403 above, this application provides a scheme that uses high-pass filtering to remove static clutter and combines cross-correlation operations to obtain the dynamic channel impulse response. Compared to the simple background difference method in related technologies, this application reduces the stringent requirements for signal alignment, preserves energy more completely, and can more reliably extract weak echo signals caused by human movement, which is the basis for realizing human detection without adding new hardware.

[0116] Step S404: Clustering Analysis for Distance Estimation: In the obtained dynamic CIR sequence (hereinafter, CIR refers to the dynamic CIR sequence), there may be one or more peaks, representing the reflections of one or more human bodies in the environment. To robustly estimate the distance to the target, this embodiment uses a clustering algorithm to process the detected echo peaks. Unlike clustering only on a single frame CIR sequence (one-dimensional fast time domain), this embodiment fully utilizes the periodic emission characteristics of the system, jointly constructing two-dimensional data of slow time–fast time from multiple frames of CIR sequences, significantly improving the multi-target separation capability and robustness. Specifically, firstly, peak detection is performed on each frame of the CIR sequence, and echo peak information in each frame is extracted, including the peak position (τ). ij Where i represents the frame number, corresponding to the number of transmission cycles, and j represents the j-th peak in that frame) and its amplitude (A). ij Then, the peaks of all frames are aggregated into a two-dimensional feature point set, using the peak position (distance information) and frame number (time information) as coordinates to construct a slow-time–fast-time two-dimensional data plane. Each peak can be represented as: p ij =(τ ij ,i), where τ ij Let represent the position of the j-th peak in the i-th frame (fast time), and i represent the frame index (slow time). Next, a clustering distance metric is defined, taking into account both positional differences and inter-frame distances:

[0117]

[0118] Where w τ It is the weight of the fast time dimension, i.e., the peak position index dimension, w T It is the weight of the slow time dimension, i.e., the frame index dimension, which is adjusted by w. τ and w T This approach balances the weights of target distance continuity and detection time continuity. Then, the DBSCAN density clustering algorithm is used to assign peaks with a distance D less than a threshold ε to the same cluster: D(p ij ,p kl )≤ε→p ij ,p kl They belong to the same cluster.

[0119] Each cluster can be considered as a target. For each cluster C l l = 1, 2, ..., which includes multipath echoes of the same target and peaks in adjacent frames. The representative distance estimate of a cluster can be calculated using a weighted average within the cluster:

[0120]

[0121] Among them, A ij This represents the peak amplitude. The distance estimate can be further processed... The conversion yields (c is the speed of sound).

[0122] Using the methods described above, the system can effectively aggregate multipath echo peaks of the same target and peaks of the same target in adjacent frames, forming a trajectory cluster. This preserves the dynamic continuity of the target while enhancing multi-target separation capabilities. Through slow-time–fast-time two-dimensional clustering analysis, the system can output distance estimates for one or more targets, providing more robust input data for subsequent angle estimation and trajectory tracking, further avoiding the misidentification of isolated noise peaks as targets.

[0123] In step S404, this embodiment of the application addresses the problem of cluttered echo signals in complex environments by employing a clustering algorithm to group multipath echoes and temporally continuous detections for processing, thereby identifying reflections from the same target. This strategy effectively improves the robustness of distance measurement, avoids misjudgments caused by noise in single measurements, ensures the accuracy and continuity of target distance estimation, and provides high-quality data for subsequent positioning.

[0124] Step S405: Phase Difference Azimuth Estimation. To obtain the target's orientation, this embodiment utilizes the phase difference of a dual-microphone array to estimate the target's azimuth. The principle is as follows: When an ultrasonic echo travels from a human target to the two microphones of the device, the path length of the sound wave reaching the two microphones will differ slightly due to the distance between the microphones, resulting in a phase difference or time difference in the received signal. For a target at a specific distance, the propagation time difference Δt can be calculated by comparing the phase of the signals from the two microphones near the peak of the target's echo. Then, the incident angle θ of the target relative to the device can be calculated based on the microphone distance and the speed of sound. For example, if the microphones are d apart and the measured echo arrival time difference Δt, the target's azimuth θ can be calculated using the formula θ = arcsin(Δt·c / d) (where c is the speed of sound). In actual implementation, the system performs cross-correlation on the dynamic CIR of the two microphones to calculate the phase difference between them, thereby obtaining a more accurate angle estimate. This method does not require additional dedicated array hardware; the target's orientation information can be obtained solely using the device's built-in dual microphones. When there are more microphones, array signal processing technology can be used to further improve angular resolution or achieve three-dimensional positioning. For example, for an array with more microphones, the distance between any two microphones m and n can be set to d. mn Calculate the arrival time delay difference Δt between them. mn Thus, the target azimuth angle is obtained:

[0125]

[0126] Repeat the above calculation for all microphone pairs (m,n), and then fuse multiple estimates using the least squares method to obtain a more robust target azimuth angle. If a microphone in a multi-microphone array is damaged, simply remove the relevant microphone pair.

[0127] In step S405, this embodiment of the application uses a phase difference angle positioning method to analyze the phase difference of the dynamic CIR using at least two microphones within the device, and accurately calculates the azimuth angle of the target. This method processes the dynamic CIR, rather than receiving signals through microphones, which can effectively avoid environmental interference and improve the accuracy of azimuth estimation. In this embodiment, phase difference positioning is combined with distance detection, enabling the system to output the two-dimensional position of the target (equivalent to the positioning information in the above embodiment), providing complete information support for human activity trajectory mapping.

[0128] Step S406: Kalman Filter Trajectory Tracking. With the distance and position observations of the target at each moment, the system introduces a Kalman filter to track and predict the trajectory of the human target. The Kalman filter uses a physical model of the moving target's motion (e.g., a uniform linear motion model) to smooth the measurement data, reducing noise and providing short-term predictions of the target's position. Specifically, each time a new distance and angle are measured, the Kalman filter treats it as an observation input, combines it with the state prediction from the previous moment, and updates the target's current optimal position estimate. Simultaneously, the filter calculates the target's velocity vector to predict its position at that moment. This feedback prediction mechanism maintains continuous target tracking even when the target is briefly obscured (e.g., someone's echo is partially blocked by furniture) or when there is measurement jitter. After Kalman filtering, the output trajectory is smoother and more reliable, accurately depicting the human's movement path within the room over time. Finally, the system combines the continuous target positions to form a motion trajectory record, which can be further used for behavior analysis or provided to upper-level applications (such as alarm triggering or home appliance linkage control). For example, determining that a target enters a specific area (specified distance and direction range) can trigger an alarm; tracking the target's movement speed and direction can be used to enable smart lights to turn on when a person moves or air conditioners to blow air when a person moves or avoid a person; and analyzing the target's trajectory patterns can be used to analyze abnormal behaviors such as falls.

[0129] Step S406, which introduces Kalman filtering to perform real-time trajectory tracking and prediction of the detected human target, is a crucial step in ensuring the reliability and practicality of the system in this application embodiment. By filtering and fusing measurement data from multiple moments, the system can provide a smooth and continuous target motion path, enhancing its tolerance to transient occlusion and measurement noise. This enables this application embodiment not only to detect instantaneous position but also to stably track human movement and comprehensively grasp the activity trajectory.

[0130] In summary, this embodiment utilizes a signal processing flow of "high-pass filtering + cross-correlation extraction (CIR) + clustering distance estimation + phase difference angle measurement + Kalman filtering tracking," fully leveraging the existing speaker and microphone hardware to achieve indoor human activity detection and positioning. The steps work together: high-pass filtering and cross-correlation extract clear dynamic echo signals; clustering distance estimation and phase difference calculation yield the polar coordinates (distance and angle) of the target position; and Kalman filtering ensures the continuity and anti-interference capability of the target trajectory. Therefore, this embodiment requires no additional sensors, achieving accurate human perception and tracking in complex home environments solely through algorithmic improvements.

[0131] Furthermore, the embodiments of this application also have the following advantages:

[0132] 1) No new hardware required, low cost: The solution fully utilizes the built-in speakers and microphones of existing commercial equipment, eliminating the need for dedicated sensors or hardware modules, significantly reducing system costs. Manufacturers can deploy this technology on existing products through software upgrades, saving on hardware development and deployment costs.

[0133] 2) Easy Deployment: Since no new hardware installation is involved, users can use the device without modifying their home environment. The method of this invention is implemented in software and can operate directly on terminals such as smart speakers. No professional installation or debugging is required, nor is a complex calibration process necessary; it provides human body sensing functionality plug-and-play, making its widespread application more convenient.

[0134] 3) High detection accuracy: By combining distance measurement and angle positioning, along with advanced signal processing (cross-correlation extraction, high-pass filtering for noise reduction) and tracking algorithms, this solution can achieve highly sensitive detection of human presence and high-precision estimation of position. Compared to traditional PIR or Doppler, which can only provide vague presence or absence information, the embodiments of this application can output quantitative distance and direction, and the accuracy is guaranteed by multiple algorithms, achieving sub-meter level position resolution in indoor environments.

[0135] 4) Strong Robustness: This scheme exhibits excellent suppression capabilities against environmental interference and background clutter. High-pass filtering removes static environmental clutter and reduces the impact of fixed reflections and slowly changing factors; Kalman filtering improves tolerance to noise and short-term signal loss. Therefore, the system can still operate stably in realistic home environments (such as cluttered furniture, multiple people moving around, and background noise), with a low false positive and false negative rate. In contrast, many alternative technologies experience significant performance degradation in complex environments, while this scheme demonstrates stronger environmental adaptability and anti-interference capabilities.

[0136] 5) Wide Applicability: This solution is applicable to homes of various layouts and sizes. From open living rooms to fully furnished bedrooms, the method of this application embodiment can operate adaptively. Ultrasonic signals are unaffected by light and are equally reliable in darkness; moreover, ultrasonic waves do not penetrate walls and will not falsely detect activity in neighboring rooms. Therefore, regardless of room lighting or the presence of walls, it can accurately detect human activity within the room. This universal applicability to different home layouts and conditions gives it broad practical value.

[0137] In summary, the embodiments of this application provide a high-precision, low-cost, robust, and easily deployable indoor human activity sensing solution, which has significant advantages over existing technologies in smart home environments. These embodiments are applicable to various COTS (Commercial Off-The-Shelf) smart home terminal devices, including but not limited to: smart speakers / voice assistant devices equipped with speakers and microphone arrays, home security monitoring devices (devices with sound alarms and radio units), smart TVs and set-top boxes (with audio playback and sound pickup functions), service robots, and advanced smart home appliances (with built-in voice interaction modules). Through software upgrades, these devices can acquire human presence detection and trajectory tracking capabilities, enabling a wide range of application functions such as automatic light turning on when someone is in the room, automatic light turning off when no one is in the room, security monitoring (alarm for strangers entering), fall / sedentary monitoring reminders for the elderly, and user activity statistics. In conclusion, utilizing the solutions of these embodiments, existing smart home products can seamlessly expand to include new environmental sensing features, bringing a higher level of intelligence and humanization in areas such as home security, energy management, and intelligent interaction.

[0138] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0139] This embodiment also provides a device for determining location information, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0140] Figure 5 This is a structural block diagram of an optional positioning information determining device according to an embodiment of this application. The device includes:

[0141] The signal transmitting module 52 is used to transmit ultrasonic detection signals to the target space through smart home devices to obtain echo sequences corresponding to the ultrasonic detection signals in multiple transmission cycles.

[0142] Extraction module 54 is used to extract the sampling sequence corresponding to the echo sequence based on the start time corresponding to the multiple transmission cycles, and filter the sampling sequence to obtain a first received signal used to indicate the dynamic changes of the active target in the target space;

[0143] The cross-correlation module 56 is used to perform cross-correlation operation on the first received signal and the ultrasonic detection signal to obtain the channel impulse response sequence corresponding to the target space;

[0144] The determination module 58 is used to determine the location information of the active target by performing cluster analysis on the channel impulse response sequence, wherein the location information includes: distance information and / or orientation indication information between the smart home device and the active target.

[0145] Using the aforementioned device, an ultrasonic detection signal is emitted from a smart home device into a target space to obtain echo sequences corresponding to the ultrasonic detection signal within multiple transmission cycles. Based on the start times corresponding to the multiple transmission cycles, a sampling sequence corresponding to the echo sequence is extracted and filtered to obtain a first received signal indicating the dynamic changes of an active target within the target space. The first received signal and the ultrasonic detection signal are cross-correlated to obtain a channel impulse response sequence corresponding to the target space. Cluster analysis of the channel impulse response sequence is performed to determine the location information of the active target, wherein the location information includes distance information and / or azimuth indication information between the smart home device and the active target. Therefore, this technical solution solves the problem in related technologies where human perception technology cannot accurately locate active targets in complex home scenes without additional dedicated hardware; it improves the flexibility of video data transmission and achieves the technical effect of locating active targets without additional dedicated hardware.

[0146] In an exemplary embodiment, the signal emitting module 52 is further configured to: emit the ultrasonic detection signal to the target space at a preset frequency through the sound-emitting component on the smart home device; acquire the echo signal corresponding to the ultrasonic detection signal through the sound-receiving component on the smart home device; compensate the echo signal through the echo channel impulse response of the smart home device, wherein the echo channel impulse response is determined through the location information and orientation information of the smart home device; and perform digital processing on the compensated echo signal to obtain the echo sequence.

[0147] In an exemplary embodiment, the extraction module 54 is further configured to extract target sampling points in the echo sequence based on the start time, wherein the time interval between the target time where the target sampling point is located and the start time is a target value; and to obtain the sampling sequence by arranging the multiple target sampling points in chronological order between the multiple start times.

[0148] In an exemplary embodiment, the extraction module 54 is further configured to filter the plurality of sampling sequences respectively using a high-pass filter; wherein, the echo sequence corresponding to the ultrasonic detection signal of the nth transmission cycle includes: the nth sampling point in each of the sampling sequences, where n is a naturally increasing number; the plurality of filtered sampling sequences are reconstructed into a new echo sequence, and the new echo sequence is determined as the first received signal.

[0149] In an exemplary embodiment, the cross-correlation module 56 is further configured to determine cross-correlation values ​​corresponding to multiple time delays through a cross-correlation function; wherein the multiple time delays are multiple time delays between the first received signal and the ultrasonic detection signal; and to determine the channel impulse response sequence through the cross-correlation values.

[0150] In an exemplary embodiment, the determining module 58 is further configured to extract echo peak information from the channel impulse response sequence through peak detection, and summarize the echo peak information to obtain a two-dimensional feature point set; wherein, the echo peak information includes: peak position information and amplitude information; construct a two-dimensional data plane corresponding to the two-dimensional feature point set based on the echo peak information; cluster the data points in the two-dimensional data plane into at least one cluster using a preset clustering algorithm; calculate the representative distance corresponding to the at least one cluster to determine the positioning information through the representative distance.

[0151] In an exemplary embodiment, the determining module 58 is further configured to obtain the distance between any two receiving elements in the receiving assembly of the smart home device, and determine the arrival delay between the second received signals corresponding to the two receiving elements respectively; wherein, the first received signal includes the second received signal; calculate the target azimuth angle of the first received signal relative to the two receiving elements using the distance and the arrival delay; fuse multiple target azimuth angles using the least squares method to obtain the azimuth indication information, so as to determine the positioning information using the azimuth indication information.

[0152] In an exemplary embodiment, the determining module 58 is further configured to calculate the target azimuth angle θ using the following formula. mn : Where, Δt mn Indicates the arrival delay, dmn The distance is represented by , and c represents the speed of sound.

[0153] In an exemplary embodiment, the determining module 58 is further configured to determine the predicted positioning information of the active target at the current time through the output result of the Kalman filter at the previous time; in the event of an abnormal situation, the predicted positioning information is output as the positioning information through the Kalman filter; wherein, the abnormal situation includes one of the following: measurement jitter, the active target being occluded; in the event of no abnormal situation, the predicted positioning information is updated through the positioning information by the Kalman filter, and the updated predicted positioning information is output as the positioning information.

[0154] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when run.

[0155] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0156] S1, an ultrasonic detection signal is sent to the target space through a smart home device to obtain the echo sequence corresponding to the ultrasonic detection signal in multiple transmission cycles;

[0157] S2, based on the start time corresponding to the multiple transmission cycles, extract the sampling sequence corresponding to the echo sequence, and filter the sampling sequence to obtain a first received signal used to indicate the dynamic changes of the active target in the target space;

[0158] S3, perform cross-correlation operation on the first received signal and the ultrasonic detection signal to obtain the channel impulse response sequence corresponding to the target space;

[0159] S4, determine the location information of the active target by performing cluster analysis on the channel impulse response sequence, wherein the location information includes: distance information and / or orientation indication information between the smart home device and the active target.

[0160] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0161] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0162] Embodiments of this application also provide an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0163] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0164] S1, an ultrasonic detection signal is sent to the target space through a smart home device to obtain the echo sequence corresponding to the ultrasonic detection signal in multiple transmission cycles;

[0165] S2, based on the start time corresponding to the multiple transmission cycles, extract the sampling sequence corresponding to the echo sequence, and filter the sampling sequence to obtain a first received signal used to indicate the dynamic changes of the active target in the target space;

[0166] S3, perform cross-correlation operation on the first received signal and the ultrasonic detection signal to obtain the channel impulse response sequence corresponding to the target space;

[0167] S4, determine the location information of the active target by performing cluster analysis on the channel impulse response sequence, wherein the location information includes: distance information and / or orientation indication information between the smart home device and the active target.

[0168] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0169] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0170] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0171] Embodiments of this application also provide a computer program that includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps in any of the above method embodiments.

[0172] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0173] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0174] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method of determining positioning information, characterized by The method comprises the following steps: An ultrasonic detection signal is sent to a target space by a smart home device to obtain a plurality of echo sequences corresponding to the ultrasonic detection signal in a plurality of transmission periods; A sampling sequence corresponding to the echo sequence is extracted based on a starting time corresponding to each of the plurality of transmission periods, and the sampling sequence is filtered to obtain a first received signal for indicating dynamic changes of a moving target in the target space; A channel impulse response sequence corresponding to the target space is obtained by cross-correlation operation of the first received signal and the ultrasonic detection signal; The positioning information of the moving target is determined by clustering analysis of the channel impulse response sequence, wherein the positioning information comprises distance information and / or orientation indication information between the smart home device and the moving target.

2. The method of determining positioning information according to claim 1, characterized in that, The method comprises the following steps: The ultrasonic detection signal is sent to the target space by a sound emitting component on the smart home device according to a preset frequency; The echo signal corresponding to the ultrasonic detection signal is obtained by a sound receiving component on the smart home device; The echo signal is compensated by an echo channel impulse response determined by position information and orientation information of the smart home device; The compensated echo signal is digitally processed to obtain the echo sequence.

3. The method of claim 1, wherein The sampling sequence corresponding to the echo sequence is extracted based on the starting time corresponding to each of the plurality of transmission periods, comprising: Target sampling points in the echo sequence are extracted based on the starting time, wherein the time interval between the target time when the target sampling point is located and the starting time is a target value; The target sampling points are arranged in the order of time according to the starting time to obtain the sampling sequence.

4. The method of claim 1, wherein The sampling sequence is filtered to obtain a first received signal for indicating dynamic changes of a moving target in the target space, comprising: Each of the sampling sequences is filtered by a high-pass filter; wherein the echo sequence corresponding to the ultrasonic detection signal of the nth transmission period comprises the nth sampling point in each of the sampling sequences, and n is a natural number that increases sequentially; The filtered sampling sequences are reconstructed into a new echo sequence, and the new echo sequence is determined as the first received signal.

5. The method of claim 1, wherein The first received signal and the ultrasonic detection signal are cross-correlated to obtain a channel impulse response sequence corresponding to the target space, comprising: A plurality of time delays are determined by a cross-correlation function, wherein the plurality of time delays are a plurality of time delays between the first received signal and the ultrasonic detection signal; The channel impulse response sequence is determined by the cross-correlation value.

6. The method of claim 1, wherein The positioning information of the moving target is determined by clustering analysis of the channel impulse response sequence, comprising: Peak information of an echo peak in the channel impulse response sequence is extracted by peak detection, and the peak information is summarized to obtain a two-dimensional feature point set; wherein the peak information includes peak position information and amplitude information; A two-dimensional data plane corresponding to the two-dimensional feature point set is constructed according to the peak information; Data points in the two-dimensional data plane are clustered into at least one cluster by a preset clustering algorithm; A representative distance corresponding to the at least one cluster is calculated to determine the positioning information through the representative distance.

7. The method of claim 1, wherein The positioning information of the moving target is determined by clustering analysis on the channel impulse response sequence, including: A distance between any two sound receiving elements in a sound receiving assembly on the smart home device is obtained, and a time delay between second received signals corresponding to the any two sound receiving elements is determined; wherein the first received signal includes the second received signal; A target azimuth angle of the first received signal relative to the any two sound receiving elements is calculated through the distance and the time delay; The orientation indication information is obtained by fusing a plurality of target azimuth angles through a least square method, so as to determine the positioning information through the orientation indication information.

8. The method of determining positioning information according to claim 7, characterized in that, A target azimuth angle of the first received signal relative to the any two sound receiving elements is calculated through the distance and the time delay, including: The target azimuth angle θ is calculated by the following equation mn : Where, Δt mn Indicates the arrival delay, d mn The distance is represented by , and c represents the speed of sound.

9. The method of claim 1, wherein, The positioning information of the moving target is determined by clustering analysis on the channel impulse response sequence, including: A predicted positioning information of the moving target at a current time is determined through an output result of a Kalman filter at a previous time; In the case of an abnormal condition, the predicted positioning information is output as the positioning information through the Kalman filter; wherein the abnormal condition includes one of the following: measurement jitter, the moving target being blocked; In the case of no abnormal condition, the predicted positioning information is updated through the positioning information through the Kalman filter, and the updated predicted positioning information is output as the positioning information. 10.An electronic device comprising a memory and a processor, the electronic device characterized by, The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 9 through the computer program.

Images