Ultrasonic transaction detection method, device, equipment, medium and program product

Abstract

The disclosure provides an ultrasonic transaction detection method and device, electronic equipment, computer readable storage medium and computer program product, relating to the technical field of computers, specifically to the technical field of artificial intelligence such as large model, neural network model and deep learning. The method comprises: emitting ultrasonic waves through a loudspeaker on a target terminal, and collecting ultrasonic echo signals formed by human motion through a microphone on the target terminal; pre-processing the ultrasonic echo signals to obtain frequency spectrum features corresponding to the ultrasonic echo signals in multiple frequency bands, and extracting Doppler spectrum corresponding to human motion from the frequency spectrum features; inputting the Doppler spectrum into a detection model to output the occurrence probability of human motion; wherein the detection model is used to predict the occurrence probability of human motion based on the frequency domain amplitude change characteristics of the Doppler spectrum. The method can improve the detection efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, specifically to the fields of artificial intelligence technology such as large models, neural network models, and deep learning, and particularly to an ultrasonic anomaly detection method, device, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] In recent years, utilizing the existing speakers and microphones of smart speakers for human motion detection has gradually become a research hotspot. This approach typically involves the speaker emitting high-frequency or ultrasonic signals, while the microphone collects the echoes from the environment to determine whether a person is moving. Currently, most active acoustic detection methods employ traditional digital signal processing techniques to extract human motion features for detection. Summary of the Invention

[0003] This disclosure provides an ultrasonic anomaly detection method, apparatus, electronic device, computer-readable storage medium, and computer program product.

[0004] In a first aspect, embodiments of this disclosure propose an ultrasonic motion detection method, comprising: emitting ultrasonic waves through a speaker on a target terminal and acquiring ultrasonic echo signals generated by human motion through a microphone on the target terminal; preprocessing the ultrasonic echo signals to obtain spectral features of the ultrasonic echo signals in multiple frequency bands, and extracting the Doppler spectrum corresponding to human motion from the spectral features; inputting the Doppler spectrum into a detection model and outputting the probability of human motion occurring; wherein, the detection model is used to predict the probability of human motion occurring based on the frequency domain amplitude variation characteristics of the Doppler spectrum.

[0005] Secondly, this disclosure also provides an ultrasonic anomaly detection device, comprising: a signal acquisition module for emitting ultrasonic waves through a speaker on a target terminal and acquiring ultrasonic echo signals generated by human movement through a microphone on the target terminal; a preprocessing module for preprocessing the ultrasonic echo signals to obtain spectral characteristics of the ultrasonic echo signals in multiple frequency bands, and extracting the Doppler spectrum corresponding to human movement from the spectral characteristics; and a detection module for inputting the Doppler spectrum into a detection model and outputting the probability of human movement occurring; wherein the detection model is used to predict the probability of human movement occurring based on the frequency domain amplitude variation characteristics of the Doppler spectrum.

[0006] Thirdly, embodiments of this disclosure provide an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the ultrasonic anomaly detection method as described in any implementation of the first aspect.

[0007] Fourthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions that enable a computer to implement the ultrasonic anomaly detection method as described in any implementation of the first aspect.

[0008] Fifthly, embodiments of this disclosure provide a computer program product including a computer program that, when executed by a processor, can implement the ultrasonic anomaly detection method as described in any implementation of the first aspect.

[0009] In this embodiment, ultrasonic waves are emitted by a speaker, and ultrasonic echo signals generated by human movement are collected by a microphone. After processing the ultrasonic echo signals generated by human movement, the Doppler spectrum is used as the sole input data for the detection model. By mining the frequency domain amplitude variation information of the spectrum itself, human movement is predicted. This disclosure can fully utilize the existing speaker and microphone resources of the target terminal, and automatically extract human movement features by combining deep learning, achieving high-precision and low-cost human movement detection in complex environments.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is an exemplary system architecture to which this disclosure can be applied; Figure 2 A flowchart of an ultrasonic anomaly detection method provided in this embodiment of the disclosure; Figure 3a A flowchart of another ultrasonic anomaly detection method provided in this disclosure embodiment; Figure 3b A schematic diagram of an ultrasonic anomaly detection method provided in an embodiment of this disclosure; Figure 3c This is a schematic diagram illustrating ultrasonic anomaly detection based on a neural network model, as provided in an embodiment of this disclosure. Figure 4 This is a structural block diagram of an ultrasonic anomaly detection device provided in an embodiment of the present disclosure; Figure 5 This is a schematic diagram of the structure of an electronic device suitable for performing an ultrasonic anomaly detection method, provided as an embodiment of the present disclosure. Detailed Implementation

[0012] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding; these should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0013] The collection, storage, use, processing, transmission, provision, and disclosure of any type of information, such as user personal information, in this technical solution comply with relevant laws and regulations and do not violate public order and good morals.

[0014] The following are several solutions for human motion detection based on smart speakers.

[0015] In some embodiments, human motion detection can be achieved based on a passive infrared (PIR) sensor.

[0016] In this embodiment, human motion is detected by the change in infrared energy radiated by the human body. When the human body enters the detection area, the PIR sensor outputs a corresponding detection signal to realize human motion detection.

[0017] In some embodiments, human motion detection can also be achieved based on camera vision.

[0018] In this embodiment, images are captured using a camera, and visual algorithms such as human detection and target recognition are used to determine whether a human body exists and whether it is moving.

[0019] In some embodiments, human motion detection can also be achieved based on millimeter-wave radar.

[0020] In this embodiment, the millimeter-wave radar emits electromagnetic waves and receives reflected signals from the human body, and detects the presence and movement of the human body based on information such as distance and Doppler frequency shift.

[0021] For each of the above embodiments, dedicated sensors are required. For example, solutions using PIR, cameras, and millimeter-wave radar all require additional hardware modules, increasing product costs and hindering the rapid deployment of existing consumer electronics devices such as smart speakers. Furthermore, for already deployed smart speakers and other terminal devices, since the hardware structure is fixed, it is usually impossible to add corresponding sensors through software upgrades, making it difficult to add human motion detection functionality to existing devices.

[0022] Furthermore, camera-based human motion detection solutions require continuous acquisition and processing of indoor images or video information. While they can achieve high detection accuracy, the collection of visual data related to the user's living environment can easily raise concerns about privacy breaches. In home settings where privacy is paramount, such as bedrooms, elderly people's rooms, and children's rooms, user acceptance of camera-based solutions is relatively low, thus limiting their application scope.

[0023] Furthermore, in some embodiments, human motion detection can also be achieved based on active acoustics.

[0024] In this embodiment, the active acoustic detection method can use traditional digital signal processing methods such as spectrum analysis, correlation detection, Doppler analysis, phase tracking and threshold decision to extract human motion features and realize human motion detection.

[0025] However, traditional artificial feature representation capabilities are limited. Active acoustic detection mainly relies on artificially designed features such as spectrum, phase, and Doppler, and uses fixed thresholds for judgment, which makes it difficult to fully characterize human motion features in complex environments.

[0026] To address at least one of the problems in the above embodiments, this disclosure also provides a method for high-precision, low-cost human motion detection in complex home environments that can fully utilize the existing speaker and microphone resources of smart speakers and combine deep learning to automatically extract human motion features.

[0027] Figure 1 An exemplary system architecture 100 is shown, in which embodiments of the ultrasonic anomaly detection methods, apparatuses, electronic devices, and computer-readable storage media of the present disclosure can be applied.

[0028] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0029] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various applications for enabling information communication between the terminal devices 101, 102, and 103 and server 105 can be installed. These applications include ultrasonic anomaly detection applications and instant messaging applications.

[0030] Terminal devices 101, 102, and 103 and server 105 can be either hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices with displays, including but not limited to smartphones, tablets, laptops, and desktop computers. When terminal devices 101, 102, and 103 are software, they can be installed in the aforementioned electronic devices, and can be implemented as multiple software programs or software modules, or as a single software program or software module; no specific limitation is made here. When server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When server 105 is software, it can be implemented as multiple software programs or software modules, or as a single software program or software module; no specific limitation is made here.

[0031] Server 105 can provide various services through its built-in applications. Taking an application for ultrasonic motion detection as an example, when running this application, server 105 can achieve the following: First, ultrasonic waves are emitted through the speaker on the target terminal, and ultrasonic echo signals formed by human movement are collected through the microphone on the target terminal; then, the ultrasonic echo signals are preprocessed to obtain the spectral characteristics of the ultrasonic echo signals in multiple frequency bands, and the Doppler spectrum corresponding to human movement is extracted from the spectral characteristics; finally, the Doppler spectrum is input into the detection model, and the probability of human movement is output; wherein, the detection model is used to predict the probability of human movement based on the frequency domain amplitude change characteristics of the Doppler spectrum.

[0032] It should be noted that, in addition to being obtained from terminal devices 101, 102, and 103 via network 104, ultrasonic anomaly detection tasks can also be pre-stored locally on server 105 through various means. Therefore, when server 105 detects that this data is already stored locally (e.g., when starting to process previously stored ultrasonic anomaly detection tasks), it can choose to directly obtain this data from locally. In this case, the exemplary system architecture 100 may also exclude terminal devices 101, 102, and 103 and network 104.

[0033] Because ultrasonic anomaly detection requires significant computing resources and power, the ultrasonic anomaly detection methods provided in the subsequent embodiments of this disclosure are generally executed by a server 105 with strong computing power and abundant computing resources. Correspondingly, the ultrasonic anomaly detection device is also generally located within the server 105. However, it should also be noted that when terminal devices 101, 102, and 103 also possess sufficient computing power and resources, they can also perform the aforementioned calculations performed by the server 105 through their installed ultrasonic anomaly detection applications, thereby outputting the same results as the server 105. Especially when multiple terminal devices with different computing capabilities exist simultaneously, but the ultrasonic anomaly detection application determines that the terminal device has strong computing power and abundant remaining computing resources, the terminal device can perform the aforementioned calculations, thereby appropriately reducing the computing pressure on the server 105. Accordingly, the ultrasonic anomaly detection device can also be located within the terminal devices 101, 102, and 103. In this case, the exemplary system architecture 100 may also exclude the server 105 and the network 104.

[0034] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0035] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating an ultrasonic anomaly detection method provided in an embodiment of this disclosure. This method can be applied to, for example... Figure 1 The system architecture 100 shown (e.g.) Figure 1 The server 105 shown can also be applied to processors or electronic devices, etc., but this disclosure does not limit it.

[0036] See Figure 2 The process 200 includes the following steps: Step 201: Emit ultrasonic waves through the speaker on the target terminal and collect ultrasonic echo signals generated by human movement through the microphone on the target terminal.

[0037] Specifically, in a static environment, the frequency of the reflected ultrasonic echo signal is consistently equal to the transmission frequency of the emitted ultrasonic wave. However, once a person or object moves in the space, the reflected ultrasonic echo signal will exhibit frequency shifts, signal energy fluctuations, and spectral changes. All such changes in the ultrasonic echo signal are collectively referred to as ultrasonic anomalies. In this disclosure, ultrasonic anomalies refer to changes in the ultrasonic echo signal caused by human movement.

[0038] The target terminal can be an audio-based smart terminal device that integrates a speaker and a microphone. For example, the target terminal can include a smart speaker, a desktop voice assistant, a portable Bluetooth speaker, or an in-vehicle voice interaction terminal. This type of target terminal can be equipped with a millimeter-wave radar module to collect the Doppler spectrum and complete the prediction of human motion probability based on the built-in detection model, so as to realize voice linkage control functions such as human proximity wake-up, automatic sleep when no one is around, and close-range human body sensing.

[0039] Step 202: Preprocess the ultrasonic echo signal to obtain the spectral characteristics of the ultrasonic echo signal in multiple frequency bands, and extract the Doppler spectrum corresponding to human motion from the spectral characteristics of multiple frequency bands.

[0040] The acquired ultrasonic echo signals are time-domain signals. First, the acquired raw ultrasonic echo time-domain signals are received, and noise and interference from irrelevant substrate signals generated during signal acquisition are eliminated. Then, frequency domain decomposition is performed on the processed ultrasonic echo signals to separate multiple independent frequency ranges. The amplitude distribution of the signal in each frequency range is determined, and the spectral characteristics corresponding to each frequency range are obtained. Then, combined with the inherent laws of the Doppler effect, the frequency shift change laws of the spectral characteristics of each frequency range are compared to distinguish irrelevant spectral components caused by stationary objects and environmental disturbances. From all multi-frequency spectral characteristics, spectral data with frequency shift changes caused only by human limb and trunk movements are screened and separated, and finally, the Doppler spectrum representing human movement is extracted.

[0041] Step 203: Input the Doppler spectrum into the detection model and output the probability of human motion.

[0042] The detection model is used to predict the probability of human motion based on the frequency domain amplitude variation characteristics of the Doppler spectrum.

[0043] Specifically, the Doppler spectrum of human motion extracted by ultrasonic echo processing is input into the pre-trained detection model. The detection model processes the input Doppler spectrum and finally outputs the probability of human motion occurring in the interval between 0 and 1 during the current observation period.

[0044] Human movement produces the Doppler effect, which manifests as changes in the amplitude spectrum within a frequency range. The detection model performs calculations and inferences solely based on the inherent frequency domain amplitude variation characteristics of the Doppler spectrum: by reading the amplitude values ​​corresponding to each frequency point within the spectrum and the fluctuation patterns of the frequency point amplitude over time, the model completes feature fitting and judgment based solely on the original frequency domain amplitude variation patterns of the spectrum, without introducing additional information such as distance, phase, or time-domain statistics, and directly outputs the normalized probability result of whether human movement has occurred.

[0045] In this embodiment, ultrasonic waves are emitted by a loudspeaker, and the ultrasonic echo signal generated by human movement is collected by a microphone. After processing the ultrasonic echo signal generated by human movement, the Doppler spectrum is used as the single input data of the detection model. By mining the frequency domain amplitude change information of the spectrum itself, human movement is predicted. This disclosure can make full use of the existing loudspeaker and microphone resources of the target terminal, and combine deep learning to automatically extract human movement features, realizing a high-precision and low-cost method for human movement detection in complex environments.

[0046] Furthermore, since this disclosure uses only the Doppler spectrum as the single input data for the detection model, there is no need to extract additional features of other dimensions. This eliminates complex pre-processing such as multi-band fusion, image erosion, and motion feature curve fitting, significantly simplifying the signal processing flow and reducing terminal computing power overhead. It can be adapted to low-power ultrasound terminals such as smart speakers and ensure real-time detection, and can be deployed on low-power hardware such as smart speakers and small ultrasound sensing terminals. Moreover, compared with human detection schemes that rely on multi-dimensional feature fusion and millimeter-wave radar distance information for auxiliary judgment, the embodiments of this disclosure break away from the conventional technical approach of multi-feature superposition. It extracts the Doppler spectrum from multiple frequency bands of ultrasound echo to form an integrated detection closed loop, avoiding the error superposition problem caused by multi-feature conversion and fusion. It can achieve quantitative output of human motion using only existing ultrasound hardware, without the need to add visual sensors such as cameras, which has better privacy protection advantages and can complete the human sensing function with lower hardware and algorithm deployment costs.

[0047] To meet the real-time operation requirements of target terminals (such as smart speaker terminals), the detection model disclosed herein adopts a lightweight neural network structure, which is a CNN-Frequency Attention-GRU motion detection network that integrates Convolutional Neural Networks (CNN), Frequency Attention, and Gated Recurrent Units (GRU). The CNN is used to extract spectral spatial features, the GRU performs temporal modeling of motion signals, and the frequency attention mechanism strengthens the weight representation of effective frequency domain features. The following detailed explanation of the human motion detection process based on this network structure, in conjunction with specific embodiments, further illustrates this process.

[0048] In some embodiments, inputting the Doppler spectrum into the detection model and outputting the probability of human motion includes: performing two-dimensional convolution processing on the Doppler spectrum through the feature extraction module of the detection model to obtain spectral change features; performing feature enhancement on the spectral change features in the frequency dimension through the frequency attention module of the detection model to obtain enhanced Doppler frequency shift features; performing time-dependent modeling on the enhanced Doppler frequency shift features through the temporal modeling module of the detection model to obtain time-optimized frequency shift features; and performing nonlinear mapping on the time-optimized frequency shift features through the output module of the detection model to output the probability of human motion.

[0049] First, the Doppler spectrum data is input into the feature extraction module within the detection model. This module performs two-dimensional convolution operations on the native frequency domain amplitude distribution of the Doppler spectrum to uncover the spatial correlation patterns corresponding to the amplitude distribution and fluctuations of frequency points within the spectrum, outputting spectral variation features derived from the spectrum itself. These spectral variation features characterize the differences in amplitude distribution and the spatial fluctuation patterns of frequency domain amplitude at each frequency point within the Doppler spectrum. The fluctuations and distribution patterns of these spectral variation features correspond one-to-one with the Doppler frequency shift amplitude changes caused by human limb and trunk movements. Changes in human movement speed and amplitude directly cause regular changes in the amplitude distribution of spectral frequency points. Therefore, this disclosure can utilize spectral variation features to distinguish between human activity and static environmental clutter.

[0050] Since the effective information generated by human motion is mainly concentrated in certain frequency regions, this disclosure designs a frequency-oriented attention module. This frequency attention module can automatically enhance the frequency region containing human motion information and reduce the influence of environmental noise frequencies.

[0051] Subsequently, the spectral change features are input into the frequency attention module, which calculates the weight coefficients of each frequency component along the inherent frequency dimension of the spectrum, amplifies the amplitude change information of frequency points related to human motion, weakens the feature response corresponding to irregular interference frequency points, completes the feature enhancement of the frequency dimension, and generates enhanced Doppler frequency shift features.

[0052] Next, the enhanced Doppler frequency shift feature is input into the time-series modeling module. Based on the dynamic correlation of frequency domain amplitude between consecutive frames of Doppler spectra captured by the GRU unit in the time-series modeling module over time, the time dependency of the frequency shift feature is constructed, and the time-optimized frequency shift feature is output. Compared with the single-frame judgment method, the time-series modeling module can process the Doppler spectrum corresponding to multiple consecutive frames of ultrasonic echo signals, thereby utilizing historical information to improve the stability of motion detection.

[0053] Finally, the output module of the detection model performs a nonlinear mapping operation on the time-optimized frequency shift features, transforming the frequency domain time-series features into normalized values ​​in the 0~1 interval, and finally outputting the probability of human motion occurring within the current observation window.

[0054] In some embodiments, the output module may include a fully connected layer and an activation function. The fully connected layer is used to fuse all information from the time-optimized frequency shift features. Specifically, the fully connected layer performs full-dimensional fusion of all feature nodes of the time-optimized frequency shift features output by the time-series modeling module, compressing and integrating the scattered frequency-domain time-series features into a fixed-dimensional one-dimensional feature vector, eliminating feature dimensional redundancy, and unifying the feature representation form. This fully connected layer can be a dense fully connected layer (Dense layer). The activation function maps the fused features to the 0-1 interval, outputting a normalized probability of human motion. This activation function can be a sigmoid function.

[0055] The embodiments provided in this disclosure, by employing the aforementioned neural network for human motion detection, can automatically mine the spatial distribution and temporal variation patterns of frequency domain amplitude in the Doppler spectrum, eliminating the need for manually designed complex features and reducing reliance on prior knowledge of signal processing. By extracting spectral spatial features through convolution, enhancing effective frequency domain components related to human motion through frequency attention, and modeling the temporal dependence of the spectrum using GRU, it can effectively distinguish between spectral changes caused by human motion and spurious spectral disturbances caused by environmental interference, thereby improving the ability to utilize the features of the ultrasonic Doppler spectrum. The network as a whole has a lightweight structure with controllable computational load, enabling inference to be completed on terminals with limited computing power, such as smart speakers, and outputting the probability of human motion occurrence in real time. Compared to simple threshold discrimination methods, neural networks have better generalization capabilities and are adaptable to different human motion amplitudes and different environmental noise conditions, improving the reliability of human motion detection.

[0056] In some embodiments, the ultrasonic echo signal includes multiple frames of ultrasonic signals, and each frame of ultrasonic signal is preprocessed to extract a corresponding frame of Doppler spectrum.

[0057] Specifically, for each frame of Doppler spectrum, the feature extraction module of the detection model performs two-dimensional convolution processing on the Doppler spectrum to obtain spectral change features. This includes: extracting initial frequency domain features based on the local changes between adjacent frequency points within the Doppler spectrum frame through the first convolutional layer of the feature extraction module; and performing calculations on the initial frequency domain features corresponding to the continuous frame signals through the second convolutional layer of the feature extraction module to extract local time-domain features and local frequency-domain features, and fusing the local time-domain features and local frequency-domain features to obtain spectral change features.

[0058] Specifically, the feature extraction module includes two two-dimensional convolutional layers. First, the first convolutional layer of the feature extraction module performs local feature mining on the data within a single frame of Doppler spectrum. Based on the local amplitude changes between adjacent frequency points within a single frame of Doppler spectrum, convolution operations are performed on the inherent frequency domain distribution law of the spectrum to automatically capture amplitude differences, local fluctuations, and correlation characteristics of different frequency points. This extracts the initial frequency domain features that characterize the inherent distribution characteristics of the single frame spectrum, thus completely preserving the basic frequency domain information corresponding to human motion in a single frame of Doppler spectrum.

[0059] Then, through the second convolutional layer of the feature extraction module, progressive convolution operations are performed on the initial frequency domain features corresponding to the temporally continuous multi-frame Doppler spectrum signals. On the one hand, the local frequency domain details within the features of a single frame are mined, and on the other hand, the continuous change patterns of features between adjacent spectrum frames are captured, thus extracting the corresponding temporal and frequency domain local features. At the same time, the temporal and frequency domain local features corresponding to the Doppler spectrum in the same frame are deeply fused, and the intra-frame frequency domain detail change information and inter-frame temporal dynamic change information of the Doppler spectrum are fused, finally obtaining a spectrum change feature that takes into account both frequency domain distribution characteristics and temporal change characteristics.

[0060] For example, the parameters of the first convolutional layer can be: Conv2D(64,(1,9)), where Conv2D represents a two-dimensional convolutional layer, 64 indicates that the number of convolutional kernels is 64, and (1,9) represents that the height of the convolutional kernel is 1 and the width is 9. A height of 1 indicates that the receptive field in the temporal dimension is 1, which cannot span frames and cannot capture temporal changes, and a width of 9 indicates that it covers 9 consecutive adjacent frequency points at a time.

[0061] The first convolutional layer only extracts local frequency domain features (Doppler spectrum variation features) within a single frame and has no ability to extract time domain features. It is used to extract Doppler spectrum variation features by utilizing the local variation relationship between adjacent frequency points within the frame.

[0062] The parameters for the second convolutional layer can be: Conv2D(96,(2,3)), where 96 indicates that the number of convolutional kernels is 96 and the kernels are (2,3). The height of 2 indicates that it can cover two consecutive frames at the same time, which means it can capture changes in the temporal dimension. The width of 3 indicates that it can cover three adjacent frequency points at the same time, and continue to capture local features in the frequency dimension.

[0063] The second convolutional layer can simultaneously fuse information in the temporal and frequency directions, improving the ability to express motion change features.

[0064] The first convolutional layer uses a convolutional kernel of size (1,9), which has a larger frequency domain receptive field. It can mine the variation patterns of adjacent frequency points over a large range based on the original spectrum signal, extract coarse-grained Doppler frequency domain contour features, and realize feature channel dimensionality upscaling. The second convolutional layer uses a convolutional kernel of size (2,3), taking the frequency domain features output from the first layer as input. It captures fine frequency domain changes within a smaller frequency domain receptive field, while utilizing the time dimension receptive field to cover two consecutive frames of signal, jointly mining the dynamic correlation features between the time and frequency domains.

[0065] The embodiments provided in this disclosure complete feature extraction through a two-layer convolutional progressive approach. The first layer of convolution is responsible for refining the basic features of a single frame's static Doppler spectrum, while the second layer of convolution models the joint time-frequency variation relationship on top of the basic frequency domain features, thereby improving the network's ability to learn human motion patterns and its ability to express motion change features.

[0066] Since the effective information generated by human movement is mainly concentrated in certain frequency regions, a frequency-oriented attention module was designed. This frequency attention module can automatically enhance the frequency region containing human movement information and reduce the influence of environmental noise frequencies.

[0067] In some embodiments, the frequency attention module of the detection model is used to enhance the spectral change features in the frequency dimension to obtain enhanced Doppler frequency shift features. This includes: performing linear mapping processing on the input spectral change features through the three fully connected layers of the frequency attention module to obtain a query matrix, a key matrix, and a value matrix; processing the query matrix and the key matrix through the activation layer of the frequency attention module to obtain frequency attention weights; and weighting the spectral change features in the frequency dimension based on the frequency attention weights and the value matrix to obtain enhanced Doppler frequency shift features.

[0068] Specifically, the frequency attention module utilizes three independent fully connected layers to perform linear mapping processing on the input spectral variation features, generating a query matrix, a key matrix, and a value matrix accordingly. All three matrices are obtained through linear transformation of the original spectral variation features, fully inheriting the frequency domain distribution and temporal variation information of the spectrum. The query matrix represents the feature query direction for each frequency component; the key matrix represents the feature index attributes of each frequency dimension, used for feature matching with the query matrix; and the value matrix carries the actual spectral feature information for each frequency dimension. Subsequently, the value matrix is ​​weighted based on frequency attention weights to achieve adaptive feature enhancement of the frequency dimensions.

[0069] Then, the frequency attention module performs correlation operations and non-linear activation processing on the query matrix and key matrix through a built-in activation layer, calculating the frequency attention weights corresponding to each frequency dimension based on the degree of feature correlation between different frequency components. In other words, the frequency attention weights are determined based on the feature matching degree between the query matrix and the key matrix; the higher the matching degree, the greater the weight assigned to the corresponding frequency position, and the lower the matching degree, the smaller the weight assigned. This weight is entirely derived from the query matrix and key matrix derived from the spectral change features themselves, without introducing externally set thresholds or additional prior information. This weight can autonomously characterize the contribution of each frequency feature to human motion detection, automatically highlighting effective motion frequency domain components and suppressing ineffective interference frequency domain components.

[0070] Finally, the calculated frequency attention weights and value matrix are weighted and matched, and adaptive weighted adjustment is performed on the overall spectral change features in the frequency dimension. The effective frequency domain features containing human motion information are enhanced, and the ineffective frequency domain features corresponding to environmental clutter are suppressed. In the end, enhanced Doppler frequency shift features with optimized weights and more accurate feature representation are obtained.

[0071] Specifically, suppose the spectral variation feature X output by the feature extraction module is: ; Where T is the number of time frames, F is the frequency dimension, and C is the number of channels.

[0072] First, the query matrix (Query, Q), key matrix (Key, K), and value matrix (Value, V) are obtained through the three fully connected mappings of the frequency attention module. , , , , These are the trainable weight matrices corresponding to the generated query matrix, key matrix, and value matrix, respectively.

[0073] Then, the correlation between each frequency feature is calculated:

[0074] Where A is the frequency attention weight; d is the feature dimension.

[0075] Finally, attention weights are used to weight the frequency features:

[0076] The enhanced spectral features Y are finally obtained.

[0077] In the embodiments provided in this disclosure, the frequency attention module operates on the frequency dimension and can automatically learn the correlation between different frequency regions, increase the weight of human motion-related frequency components, and suppress environmental noise and irrelevant frequency information, thereby enhancing Doppler frequency shift characteristics and improving the accuracy of human motion detection.

[0078] Furthermore, this disclosure can also compress the enhanced Doppler frequency shift features before using the time-series modeling module to model historical time-series dependencies, thereby reducing computational load and improving detection efficiency.

[0079] In some embodiments, the enhanced Doppler frequency shift features are modeled on a time-dependent basis using the temporal modeling module of the detection model to obtain time-optimized frequency shift features. This includes: compressing the dimensions of the enhanced Doppler frequency shift features using the dimensionality reduction module of the detection model to obtain a dimensionality-reduced Doppler spectrum; and modeling the historical temporal dependencies of the dimensionality-reduced Doppler spectrum using the gated recurrent unit layer of the temporal modeling module to obtain time-optimized frequency shift features.

[0080] Specifically, the dimensionality reduction module can be a convolutional layer or a fully connected layer (Dense layer). The convolutional layer can achieve channel dimension compression by setting a smaller number of output channels, while the fully connected layer maps high-dimensional features to low-dimensional space through linear mapping. Both of them complete dimensionality compression while preserving the original information of the enhanced Doppler frequency shift features, so as to adapt to the input dimension of the gated recurrent unit layer and reduce the computational amount of temporal modeling.

[0081] First, the enhanced Doppler frequency shift features are compressed using the built-in dimensionality reduction module of the detection model. While preserving all valid frequency domain information and details of human motion-related spectral changes in the enhanced Doppler frequency shift features, redundant dimensions are simplified, removing redundant information to obtain dimensionally adapted and information-complete dimensionally reduced Doppler spectral features. This effectively reduces the computational load of subsequent temporal modeling and improves model inference efficiency. Then, the dimensionality-reduced Doppler spectral features are input into the gated recurrent unit layer of the temporal modeling module. Based on the temporal memory and update mechanism of the gated recurrent unit, the Doppler spectral features corresponding to consecutive time frames are traversed and calculated. The module autonomously mines and learns the dynamic change patterns between adjacent spectral frames, models the historical temporal dependencies of Doppler spectral features over time, captures the continuous temporal changes in spectral amplitude during human motion, filters out instantaneous and irregular frequency domain disturbances, and finally outputs time-optimized frequency shift features that fuse temporal correlation information and characterize the continuous changes in human motion.

[0082] It should be noted that both the gated recurrent unit layer in the temporal modeling module and the second convolutional layer in the feature extraction module embodiment above can fuse temporal features. The difference is that the temporal modeling module can process signals from multiple consecutive frames to utilize historical information and improve motion detection stability, while the second convolutional layer can only process signals from two adjacent frames to improve the ability to express motion change features.

[0083] The embodiments provided in this disclosure introduce a dimensionality reduction module to perform dimensionality compression processing on the enhanced Doppler frequency shift features, simplify the redundant dimensions of the features, remove redundant information in the feature dimensions, thereby reducing the amount of computation and increasing the feature representation capability.

[0084] The detection models used in the above implementations are pre-trained detection models. The following examples illustrate the training steps of these detection models.

[0085] In some embodiments, the detection model is trained based on the following steps: collecting ultrasonic echo data in different scenarios, wherein the scenarios include at least one of human movement scenarios, unmanned stationary scenarios, movement scenarios at different distances, movement scenarios in different room environments, and echo scenarios; preprocessing and extracting the spectrum of the ultrasonic echo data to obtain training sample data; and training the initial model using the training sample data to obtain the detection model.

[0086] Specifically, firstly, ultrasonic echo data from various complex real-world scenarios are collected in batches, comprehensively covering all types of interference and motion conditions. The selected scenarios include at least scenarios with moving people, static scenarios without people, human movement scenarios at different distances, environmental motion scenarios with different room structures, and scenarios with multiple echo interference. By collecting data from multiple scenarios, it is possible to fully encompass various ultrasonic echo signals corresponding to static environmental clutter, spatial distance differences, indoor environmental structural differences, echo superposition interference, and real human movement, ensuring the diversity and coverage of training data and avoiding problems such as insufficient generalization ability and weak anti-interference ability of the model due to a single scenario.

[0087] Secondly, all the collected raw ultrasound echo data are subjected to unified preprocessing and multi-band spectrum extraction. Following the aforementioned signal processing flow, noise removal, signal correction, frequency domain decomposition, and human motion Doppler spectrum extraction are completed. The raw ultrasound echo data are converted into standardized and uniformly formatted Doppler spectrum samples to form training sample data that can be used for model iterative learning, ensuring the consistency of features between the training samples and the model inference input data.

[0088] Finally, the completed training sample data is input into the initial CNN-Frequency Attention-GRU network model. The trainable weight parameters of each layer of the initial model are iteratively updated and optimized. By continuously learning the characteristic differences between the Doppler spectrum of human motion and the spectrum without motion interference in different scenarios, the model accurately grasps the frequency domain amplitude change law and temporal evolution characteristics corresponding to human motion, gradually converges and completes training, and finally obtains a human motion detection model with strong scene adaptability and high anti-interference ability. The training objective is to determine whether human motion occurs in the frame, where 0 represents stillness and 1 represents motion. The loss function used can be the binary classification cross-entropy loss function.

[0089] The embodiments provided in this disclosure collect ultrasonic echo data under different environments during the training process of the detection model to ensure the diversity and coverage of training data, avoiding the problems of insufficient generalization ability and weak anti-interference ability of the model due to a single scenario; at the same time, the obtained Doppler spectrum is directly used as training sample data to train the initial model, so that the trained detection model can predict the probability of human motion based on the frequency domain amplitude change characteristics of the Doppler spectrum, simplifying the complexity of anomaly detection and improving the accuracy of detection model detection.

[0090] Since the deep learning model outputs the probability of human motion occurrence corresponding to each frame of ultrasound signal, the results of a single frame of ultrasound signal are easily affected by instantaneous noise or accidental misjudgment. To improve the stability of motion detection results, this disclosure can also perform time-domain post-processing on the output of the detection model. A specific implementation method is given below.

[0091] In some embodiments, the ultrasonic echo signal includes multiple frames of ultrasonic signals, each frame of ultrasonic signal corresponding to the output probability of human motion. That is, after the aforementioned processing, each frame of ultrasonic signal can output a corresponding probability of human motion. The method further includes: constructing a sliding time window based on the probability of human motion corresponding to multiple consecutive frames of ultrasonic signals, calculating the cumulative value of multiple probabilities within the sliding time window, and determining the detection result of human motion based on the cumulative value.

[0092] Specifically, the sliding time window is used to accommodate the probability of human motion occurrence corresponding to multiple consecutive frames of ultrasound signals in time. The window maintains a fixed number of frames. Each time a new probability of occurrence is generated, the new probability is added to the window, and the earliest probability in the window is removed, so that the window can be continuously updated along the time dimension. The probability of human motion occurrence output from each of the multiple consecutive frames of ultrasound signals is selected to construct the sliding time window; the probability values ​​corresponding to all frames within the sliding time window are collected, and the cumulative value of multiple probability occurrences within the window is calculated; then, based on the magnitude of the cumulative value, the final human motion detection result is determined.

[0093] Each frame of ultrasound signal undergoes independent signal preprocessing, Doppler spectrum extraction, and detection model inference to obtain a single-frame probability output. The single-frame probability only reflects the instantaneous detection result at that frame moment and is easily affected by fluctuations caused by instantaneous noise. In this embodiment, a sliding time window is set to include the occurrence probabilities of multiple consecutive frames within the same window, and the occurrence probabilities of each frame within the window are accumulated to obtain a cumulative value. This cumulative value integrates the model output information of multiple frames over a continuous time period, weakening the probability abrupt changes caused by instantaneous interference in a single frame. When the cumulative value meets a preset condition, it is determined that human movement exists; otherwise, it is determined that no human movement exists, thus obtaining a stable human movement detection result. This process only reuses the occurrence probabilities output by the model, does not introduce additional observation features, and relies on the statistical information of the probabilities within the time window to complete the result judgment, suppressing misjudgments caused by instantaneous noise.

[0094] Before constructing the sliding time window, the output of the detection model can be filtered for confidence, thereby updating the output results. Further time-domain post-processing of the updated output results can further improve the stability and accuracy of the detection results. The specific implementation method is given below.

[0095] In some embodiments, a sliding time window is constructed based on the probability of human motion occurring corresponding to multiple consecutive frames of ultrasound signals, and the cumulative value of multiple occurrence probabilities within the sliding time window is calculated. This includes: performing confidence screening on the probability of human motion occurring in each frame of ultrasound signals based on an output probability threshold to obtain the corresponding filtered probability; constructing a sliding time window based on the filtered probability of multiple consecutive frames of ultrasound signals, and calculating the cumulative value of multiple filtered probability within the sliding time window; wherein, for each newly added frame of ultrasound signal, the sliding time window updates the filtered probability within the sliding time window by moving forward one time frame.

[0096] Specifically, the output probability threshold can be preset based on requirements. Then, confidence filtering is performed on the probability of human motion occurrence in each frame of ultrasound signal output according to this output probability threshold to obtain the filtered probability. This filtering process filters the occurrence probability corresponding to a single frame of ultrasound signal output by the detection model, retaining only the probability value after threshold determination, without introducing additional feature information other than the occurrence probability output by the detection model. After completing the confidence filtering for each frame, a sliding time window is constructed using the filtered probabilities corresponding to multiple consecutive frames of ultrasound signals in time sequence, maintaining a fixed number of frames of data within the window. Whenever the detection model outputs a new frame of ultrasound signal corresponding to a filtered probability, the sliding time window moves forward one time frame along the time dimension, incorporating the newly generated filtered probability into the window, while removing the oldest frame of filtered probability in the window, thus updating the data within the window and ensuring that the window always retains the filtered probabilities under continuous time sequence. After completing the window update, all filtered probabilities in the current sliding time window are accumulated to obtain the cumulative value corresponding to multiple filtered probabilities within the window.

[0097] Specifically, let the probability of human motion occurring corresponding to the ultrasound signal output in frame t of the detection model be... When the detection model outputs the probability of this occurrence Below the output probability threshold If the current frame of ultrasound signal does not contain reliable motion information, the probability of human motion in the corresponding output of that frame of ultrasound signal is set to zero; otherwise, if the probability of motion in the output of the detection model is zero, the detection model will determine that the current frame of ultrasound signal does not contain reliable motion information. Greater than or equal to the output probability threshold At that time, it is determined that reliable motion information exists in the current frame of ultrasound signal, and the probability of human motion occurring corresponding to the output of that frame of ultrasound signal is retained, that is, the filtered probability. for:

[0098] After obtaining the post-filtering probabilities corresponding to each frame of ultrasound signal, the filtering results of N consecutive frames are combined into a sliding time window:

[0099] Calculate the cumulative value of motion probability within the window :

[0100] Finally, when the following conditions are met:

[0101] If the motion window length is N, then the presence of human motion is determined; otherwise, the absence of human motion in the current environment is determined. The threshold value is the average motion intensity of the window. After each judgment is completed, the sliding window moves forward one time frame and continues to perform real-time statistics on the subsequent model outputs, thereby obtaining continuous and stable human motion detection results.

[0102] The embodiments provided in this disclosure, by first performing single-frame confidence screening and then accumulating probabilities across multiple frames using a sliding time window, can filter out some invalid probability outputs with low confidence and synthesize detection results over a continuous time range. This suppresses abnormal probability fluctuations caused by instantaneous noise, providing a reliable statistical measure for determining human motion detection results based on the cumulative value. Compared to directly determining motion based on single-frame model output, this disclosure filters out low-confidence detection results by using an output probability threshold, reducing the impact of instantaneous noise and accidental false detections on the detection results. Furthermore, by using a sliding window to statistically analyze human motion features over a continuous time period, it fully utilizes the temporal continuity of human motion to improve the stability of the detection results. This disclosure can avoid frequent jumps in the detection model output at the beginning and end of motion, improving the robustness of human motion detection and user experience.

[0103] To provide high-quality input data for the detection model, this disclosure can also utilize framing, windowing, and Fourier transform to achieve joint time-frequency characterization through short-time local analysis, process non-stationary ultrasonic echo signals, and provide high-quality input for subsequent processing of human motion Doppler signals (feature extraction, action recognition, anomaly detection), thereby enabling the extraction of accurate Doppler features and motion speed information in radar human perception.

[0104] In some embodiments, the ultrasonic echo signal is preprocessed to obtain the spectral characteristics of the ultrasonic echo signal in multiple frequency bands, including: performing frame-segmentation on the ultrasonic echo signal to obtain multiple frames of ultrasonic signals; for each frame of ultrasonic signal, performing windowing on the ultrasonic signal to obtain a noise-reduced and smoothed processed signal, and performing a Fourier transform on the processed signal to obtain the spectral characteristics of the ultrasonic signal, wherein the length of the Fourier transform is determined based on the Doppler resolution requirements and computational overhead.

[0105] Specifically, framing refers to dividing a continuous time-domain signal into multiple short, overlapping segments (called "frames"). Each frame contains multiple sampling points, and the starting positions of adjacent frames are separated by multiple sampling points (frame shift).

[0106] Human motion signals (such as walking, breathing, falling, etc.) are time-varying and do not satisfy the stationary signal assumption of Fourier transform as a whole, while local signals can be considered stable. Frame length determines temporal precision; short frames capture rapid changes in a short time, while long frames improve frequency domain resolution. Therefore, frame length can be determined based on requirements.

[0107] First, the original ultrasonic echo signal is segmented into multiple frames according to a preset frame length, achieving segmented processing of the long time-series signal. Spectrum calculations are then performed frame by frame. For each frame of the segmented ultrasonic signal, windowing is applied. A window function is used to weight the time-domain signal of a single frame, reducing spectral leakage caused by signal truncation and obtaining a noise-reduced and smoothed signal. Subsequently, a Fourier transform is performed on this processed signal, converting the time-domain signal to the frequency domain and outputting the spectral characteristics corresponding to that frame of the ultrasonic signal.

[0108] The Fourier transform length is not a fixed value; it needs to balance Doppler resolution requirements with computational overhead. For Doppler resolution, a longer Fourier transform length results in a smaller frequency domain sampling interval, leading to higher Doppler frequency resolution and facilitating the differentiation of subtle Doppler frequency shifts caused by weak human movement. However, increasing the transform length increases computational load, leading to longer processing times and higher resource consumption on the device. Therefore, the Fourier transform length must be carefully chosen to balance Doppler resolution requirements with computational overhead, minimizing computational burden while meeting Doppler resolution needs and adapting to the computing power of the terminal device. The framing, windowing, and Fourier transform operations described above only apply to the raw time-domain data of the ultrasonic echo and do not introduce any external signal features.

[0109] Specifically, after acquiring the digital signal (i.e., the ultrasonic echo signal), the time-domain signal is divided into frames. For the m-th frame signal... :

[0110] Where H represents the frame shift and n represents the sampling point within the frame.

[0111] Then, a Hanning window was used to add a window:

[0112] get:

[0113] Then perform a Fourier transform:

[0114] Here, N represents the length of the Fast Fourier Transform (FFT), and k represents the frequency point search. When the FFT length is small, the Doppler frequency shift resolution is insufficient, making it impossible to effectively distinguish frequency changes caused by human motion. Therefore, considering both computational cost and effectiveness, this embodiment can use a 4096-point FFT, i.e., N equals 4096, to improve frequency resolution and enable the effective extraction of minute frequency changes around 22kHz.

[0115] In this embodiment, continuous time-domain ultrasonic echo signals are transformed into frame-level spectral features through framing, windowing, and fast Fourier transform processing. Framing enables short-term stable time-frequency analysis, adapting to the dynamic changes of human motion. Windowing effectively suppresses spectral leakage caused by signal truncation, reducing spectral distortion. Fast Fourier transform completes the conversion from the time domain to the frequency domain, obtaining spectral features containing Doppler frequency shift information. At the same time, the transform length balances Doppler resolution and computational overhead, providing reliable preprocessing input for subsequent neural network human motion detection.

[0116] Since the main frequency changes caused by human movement are concentrated near the center frequency of the transmitted ultrasound, this disclosure can avoid using the complete spectrum as input and reduce the computational load. A specific implementation method is given below.

[0117] In some embodiments, extracting the Doppler spectrum corresponding to the ultrasonic echo signal from the spectral features includes: determining the extraction frequency based on the center frequency of the emitted ultrasonic wave; determining the frequency resolution based on the sampling frequency and the length of the Fourier transform of the ultrasonic echo signal generated by human motion; and extracting the Doppler spectrum from the spectral features based on the extraction frequency and the frequency resolution.

[0118] Specifically, the center frequency of the ultrasonic waves emitted by the speaker on the target terminal is first obtained. The main frequency changes caused by human movement are concentrated around this center frequency. Based on this center frequency, the extraction frequency is determined according to the requirements. This extraction frequency corresponds to the target frequency range where the Doppler frequency shift caused by human movement is distributed. In this way, the Doppler change information caused by human movement is preserved, while low-frequency environmental noise is removed, reducing the model input dimension and computational load.

[0119] Then, based on the sampling frequency and FFT transform length of the acquired human motion ultrasound echo signal, the frequency resolution is calculated. This frequency resolution characterizes the frequency interval between two adjacent frequency points in the spectrum. The sampling frequency and FFT transform length together determine the actual physical frequency corresponding to each frequency point. After determining the extraction frequency and frequency resolution, the positions of frequency points related to the human body's Doppler frequency shift in the spectral features are located by combining the two. Data within the corresponding frequency range is extracted from the complete spectral features to obtain the Doppler spectrum corresponding to the ultrasound echo signal.

[0120] For example, assuming the center frequency of the transmitted ultrasound is 22000Hz, and the main frequency changes caused by human movement are concentrated around this frequency, this disclosure does not use the complete spectrum as input, but selects the spectral features in the range of 21000Hz≤f≤23000Hz. This can preserve the Doppler change information caused by human movement, remove low-frequency environmental noise to reduce the model input dimension, and reduce the amount of computation.

[0121] Based on sampling rate Calculate frequency resolution using 48kHz and 4096-point FFT. for:

[0122] in, =48000Hz, N=4096, thus obtaining .

[0123] Therefore, the frequency band width is: 23000-21000=2000Hz; the corresponding number of frequency points is approximately 2000 / 11.7≈170, that is, there are about 170 frequency points between 21000Hz and 23000Hz, and the amplitude spectrum of these 170 frequencies will be used as input to the deep learning model.

[0124] The embodiments provided in this disclosure utilize the center frequency of the emitted ultrasonic wave, the signal sampling frequency, and the FFT length to perform frequency band truncation on the generated spectral features. Without introducing additional signals or artificially constructed features, only the effective frequency band data corresponding to the Doppler effect of human motion is retained, irrelevant frequency components are eliminated, the amount of data processed by the subsequent model is reduced, and the interference of irrelevant frequency band noise on the detection is reduced.

[0125] In addition, the emission frequency of the emitted ultrasound may also affect the detection results.

[0126] In some embodiments, the ultrasound is determined based on at least one of the following: the sampling frequency of the microphone, the frequency response range of the speaker, the user's auditory experience, and the detection model's ability to detect Doppler frequency shift.

[0127] Specifically, the determination of the ultrasonic wave in this disclosure requires comprehensive consideration of multiple factors. On the one hand, it is necessary to combine the sampling frequency of the microphone and ensure, according to the Nyquist sampling theorem, that the ultrasonic wave frequency does not exceed half of the sampling frequency to avoid frequency aliasing. At the same time, it is necessary to match the frequency response range of the loudspeaker to ensure that the loudspeaker can stably output an ultrasonic signal with sufficient energy and without distortion. On the other hand, it is necessary to take into account the user's auditory experience, and try to select frequency bands that are not easily perceived by the human ear to eliminate auditory interference caused by sound generation. It is also necessary to adapt to the detection model's ability to detect Doppler frequency shift to ensure that the Doppler frequency shift generated by human motion falls within the detection range that the model can recognize. In this way, a balance is achieved between hardware conditions, user experience and algorithm detection performance, providing a reliable signal basis for subsequent Doppler spectrum extraction and human motion detection.

[0128] Specifically, this disclosure employs a fixed-frequency ultrasonic excitation method. The transmitted ultrasonic signal is represented as follows:

[0129] The transmission frequency f0 for emitting ultrasonic waves can be selected as 22000Hz. This frequency selection mainly takes into account the target terminal's audio hardware sampling capability, the characteristics of human hearing, and the Doppler frequency shift range caused by human movement.

[0130] Specifically, firstly, considering that existing smart speakers typically use an audio acquisition link with a 48kHz sampling rate, according to the Nyquist sampling theorem, the theoretically highest frequency that can be acquired is:

[0131] Where Fs is the sampling rate. Therefore, to avoid signal aliasing and the influence of audio hardware filtering characteristics, the transmitted ultrasonic frequency needs to be selected below 24kHz.

[0132] Secondly, to avoid the ultrasonic signal affecting the user's normal auditory experience, the transmission frequency should be selected as high as possible, above the range of frequencies perceptible to the human ear. Since the upper limit of human hearing is typically around 20kHz, setting the transmission frequency above 20kHz places it within the ultrasonic band, reducing its impact on the user experience. Furthermore, human motion detection relies on the Doppler frequency shift generated by the moving target, and the amount of this shift satisfies:

[0133] Where Δf represents the Doppler frequency shift, v represents the human motion velocity, c represents the speed of sound, and f0 represents the emitted ultrasonic frequency. The higher the emission frequency, the more significant the Doppler frequency shift, which is beneficial for improving the ability to extract motion features.

[0134] Therefore, taking into account sampling frequency limitations, hardware frequency response range, user auditory experience, and Doppler frequency shift detection capability, this disclosure can select 22000Hz as the ultrasonic transmission frequency, so that it can meet the frequency range requirements of the 48kHz sampling system and obtain more obvious human motion Doppler characteristics.

[0135] The embodiment of the determined ultrasonic frequency provided in this disclosure is compatible with the hardware transmission and acquisition capabilities of the device, avoids auditory interference, and ensures that the Doppler frequency shift generated by human movement is within the effective recognition range of the model, thereby guaranteeing the quality of the echo signal and improving the reliability of subsequent human movement detection.

[0136] Figure 3a A flowchart of another ultrasonic anomaly detection method provided in this embodiment of the disclosure. Figure 3b This is a schematic diagram of an ultrasonic anomaly detection method provided in an embodiment of this disclosure.

[0137] Please refer to Figure 3a Process 300 includes the following steps: Step 301: Selection of ultrasonic wave emission.

[0138] Specifically, in combination Figure 3b The speaker plays a 22kHz frequency signal, meaning the ultrasonic transmission frequency is selected as 22000Hz. This frequency selection primarily considers the audio hardware sampling capability, the characteristics of human hearing, and the Doppler frequency shift range caused by human movement.

[0139] Step 302: Frame splitting and windowing processing.

[0140] Step 303: Fourier transform.

[0141] Specifically, continue to combine Figure 3b To explain, the echoes generated by human movement are received by the speaker's microphone. After acquiring the digital signal, the time-domain signal is processed by frame segmentation. Then, a Hanning window is applied, followed by a Fourier transform.

[0142] Step 304: Multi-feature extraction.

[0143] Specifically, since the center frequency of the transmitted signal is 22000Hz, and the main frequency changes caused by human movement are concentrated around this frequency, this invention does not use the complete spectrum as input, but instead selects spectral features within the range of 21000Hz≤f≤23000Hz. This preserves the Doppler variation information caused by human movement, removes low-frequency environmental noise to reduce the model input dimension, and reduces computational load.

[0144] Furthermore, based on a sampling rate of 48kHz and a 4096-point FFT, the frequency resolution is: Δf = F_s / N. Given F_s = 48000Hz and N = 4096, we get Δf ≈ 11.7Hz.

[0145] Therefore, 21000Hz to 23000Hz corresponds to approximately 170 frequency points. The amplitude spectrum of these 170 frequencies will be selected from all spectral features and used as the input to the deep learning model.

[0146] Step 305: Use a neural network model to detect ultrasonic anomalies.

[0147] Specifically, to meet the real-time operation requirements of smart speaker terminals, this disclosure may adopt a lightweight neural network structure.

[0148] Figure 3c This is a schematic diagram illustrating an ultrasonic anomaly detection based on a neural network model, provided as an embodiment of this disclosure. Figure 3c As shown, the ultrasonic anomaly detection using a neural network model includes: frequency amplitude spectrum input, which is sequentially processed by the neural network model through Con2D feature extraction, batch normalization (BatchNorm) and rectified linear unit (ReLU), frequency attention and residual, bottleneck dimensionality reduction, GRU temporal modeling, and fully-connected layer (FC) and activation function (Sigmoid), finally outputting frame-level probability.

[0149] The training process for this neural network model is as follows: First, training data is prepared by collecting ultrasound echo data from different home environments, including scenarios with movement, no one at rest, movement at different distances, movement in different room environments, and echo scenarios, totaling approximately 100 decimal places of data. Then, the collected data undergoes the aforementioned FFT and spectral feature extraction before being input into the model for training. The training objective is to classify the frame as motion: 0 represents stillness, and 1 represents movement. The loss function used is the binary cross-entropy loss function.

[0150] In this disclosure, after the detection model is trained on the server side, the trained detection model parameters are converted into a lightweight inference model suitable for deployment on terminal devices, which can be deployed to smart speaker devices for real-time inference. In one embodiment, this disclosure can use TensorFlow Lite (TFLite, a framework) as the model inference framework to achieve efficient model operation on the smart speaker. It should be noted that this disclosure is not limited to using TFLite, and other deployment frameworks that support neural network inference can also be used to implement the technical solution of this disclosure.

[0151] Step 306: Post-processing.

[0152] Specifically, since deep learning models output the probability of human motion for each frame, the results for a single frame are easily affected by instantaneous noise or accidental misjudgments. To improve the stability of motion detection results, this embodiment performs time-domain post-processing on the model output.

[0153] First, the model output is filtered for confidence. When the model output probability is lower than the first preset threshold, it is considered that there is no reliable motion information in the current frame, and its output value is set to zero; otherwise, the original output probability is retained. Then, the filtering results of N consecutive frames are combined into a sliding time window, and the cumulative value of motion probability within the window is calculated.

[0154] Step 307: Exercise Results.

[0155] Specifically, if the preset conditions are met, it is determined that there is human movement; otherwise, it is determined that there is no human movement in the current environment.

[0156] After each decision is made, the sliding window moves forward one time frame and continues to perform real-time statistics on the subsequent model outputs, thereby obtaining continuous and stable human motion detection results.

[0157] Compared to directly making motion decisions based on single-frame model output, this disclosure uses the above post-processing method to filter out low-confidence detection results through a first threshold, reducing the impact of instantaneous noise and accidental false detections on the detection results; it also uses a sliding window to statistically analyze human motion features over a continuous time period, making full use of the temporal continuity of human motion to improve the stability of the detection results; and it can also avoid frequent jumps in model output at the beginning and end of motion, improving the robustness of human motion detection and user experience.

[0158] Other specific details of steps 301-307 above are the same as those in the corresponding embodiments described above. For the same parts, please refer to the corresponding parts of the corresponding embodiments described above, and they will not be repeated here.

[0159] This disclosure discloses a human motion detection method based on ultrasonic echo and neural network, which can be deployed in smart terminal devices that integrate speakers and microphones, such as smart speakers, smart screens, and smart home control terminals, to achieve human motion detection without the need for additional sensors.

[0160] During application, the smart speaker continuously plays a 48kHz single-frequency ultrasonic signal through its speaker, while the microphone simultaneously collects echo signals from the environment. The collected audio data is then preprocessed and features are extracted. The extracted features are input into a human motion detection model for real-time inference, outputting the probability of human motion. Finally, the final detection result is obtained by combining the model with a post-processing algorithm.

[0161] For example, in a home living room environment, the smart speaker continuously emits a 48 kHz ultrasonic signal. When a user enters within 3 meters of the device and moves, the human body reflection causes a change in the echoed ultrasonic signal. The device inputs the echoed signal into the human motion detection model of this invention and outputs the motion detection results in real time. When human motion is detected in multiple consecutive time windows, the system determines that human activity is present and triggers the smart speaker to enter the corresponding working mode or link with other smart home devices.

[0162] When human movement is detected, the device can perform corresponding functions according to application needs, such as: automatically waking up smart home devices; linking smart devices such as lights and air conditioners; monitoring elderly people who have fallen or have been inactive for a long time; home security intrusion detection; human presence detection and energy-saving control; and waking up or enhancing the interaction of smart speakers when they are near them.

[0163] This disclosure supports application scenarios such as elderly care, presence detection, and smart home integration. For example, in the elderly care scenario, the smart speaker can continuously sense human activity in the home environment. When it detects abnormal inactivity for an extended period, it can combine with other service capabilities to issue reminders or notifications, thereby enhancing home security.

[0164] This disclosure employs a lightweight neural network model that can be deployed on smart speaker terminals for real-time inference. It features low computational load, small storage footprint, and good real-time performance, and can meet the long-term online operation requirements of resource-constrained devices.

[0165] The embodiments disclosed herein can enhance the environmental awareness capabilities of smart speakers, enabling the product to be further expanded from a traditional voice interaction device into a smart home terminal with proactive sensing capabilities.

[0166] Compared to existing smart speakers that mainly rely on voice wake-up, cameras, or additional hardware sensors to achieve human state perception, the solution disclosed herein utilizes the device's existing speakers and microphones to achieve human motion detection. This does not require additional hardware costs or changes to the product's hardware structure, thus endowing the smart speaker with human activity perception capabilities, reducing product costs, and enhancing product competitiveness.

[0167] By employing a method based on ultrasonic echo analysis and a lightweight deep learning model, the technical solution provided in this disclosure can adapt to multipath reflection, noise interference, and changes in the movement state of different users in the home environment, thereby improving the stability and accuracy of human motion detection and enhancing the actual user experience of smart speakers in complex home environments.

[0168] Furthermore, since this disclosure is based on existing audio hardware, it does not require the addition of visual sensors such as cameras, thus offering better privacy protection. This can serve as a key selling point that distinguishes smart speakers from traditional products, thereby enhancing the added value of the product.

[0169] Compared to products that use millimeter-wave radar, infrared sensors, or camera solutions, this disclosure requires no additional hardware components, which can reduce material costs and avoid privacy issues caused by visual data acquisition, making it more suitable for long-term home deployment.

[0170] Further reference Figure 4 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of an ultrasonic anomaly detection device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0171] like Figure 4 As shown, the ultrasonic motion detection device 400 of this embodiment may include: a signal acquisition module 401, a preprocessing module 402, and a detection module 403. The signal acquisition module 401 is used to emit ultrasonic waves through a speaker on the target terminal and acquire ultrasonic echo signals generated by human motion through a microphone on the target terminal. The preprocessing module 402 is used to preprocess the ultrasonic echo signals to obtain the spectral characteristics of the ultrasonic echo signals in multiple frequency bands, and extract the Doppler spectrum corresponding to human motion from the spectral characteristics corresponding to multiple frequency bands. The detection module 403 is used to input the Doppler spectrum into a detection model and output the probability of human motion occurring. The detection model is used to predict the probability of human motion occurring based on the frequency domain amplitude variation characteristics of the Doppler spectrum.

[0172] In this embodiment, the specific processing of the signal acquisition module 401, the preprocessing module 402, and the detection module 403 in the ultrasonic anomaly detection device 400, and the resulting technical effects, can be found in the following references: Figure 2 The relevant descriptions of steps 201-203 in the corresponding embodiments will not be repeated here.

[0173] The embodiments provided in this disclosure utilize a loudspeaker to emit ultrasonic waves, which are then used to collect ultrasonic echo signals generated by human movement. After processing these echo signals, the Doppler spectrum is used as the sole input data for the detection model. By mining the frequency domain amplitude variation information of the spectrum itself, human movement is predicted. This disclosure can fully utilize the existing loudspeaker and microphone resources of the target terminal, and combine deep learning to automatically extract human movement features, achieving high-precision and low-cost human movement detection in complex environments.

[0174] In some embodiments, the detection module 403 is specifically used to: perform two-dimensional convolution processing on the Doppler spectrum through the feature extraction module of the detection model to obtain spectral change features; enhance the spectral change features in the frequency dimension through the frequency attention module of the detection model to obtain enhanced Doppler frequency shift features; model the enhanced Doppler frequency shift features in terms of time dependence through the temporal modeling module of the detection model to obtain time-optimized frequency shift features; and perform nonlinear mapping on the time-optimized frequency shift features through the output module of the detection model to output the probability of human motion.

[0175] In some embodiments, the ultrasonic echo signal includes multiple frames of ultrasonic signals. After preprocessing, each frame of ultrasonic signal corresponds to the extraction of a Doppler spectrum. For each frame of Doppler spectrum, the detection module 403 is specifically used for: the first convolutional layer of the feature extraction module extracts initial frequency domain features based on the local changes between adjacent frequency points within the Doppler spectrum frame; the second convolutional layer of the feature extraction module performs operations on the initial frequency domain features corresponding to the continuous Doppler spectra, extracts local features in the time domain and local features in the frequency domain, and fuses the local features in the time domain and local features in the frequency domain to obtain spectral change features.

[0176] In some embodiments, the detection module 403 is specifically used for: three fully connected layers of the frequency attention module, which respectively perform linear mapping processing on the input spectral change features to obtain a query matrix, a key matrix, and a value matrix; the activation layer of the frequency attention module processes the query matrix and the key matrix to obtain frequency attention weights; and based on the frequency attention weights and the value matrix, adaptively weighting and enhancing the spectral change features in the frequency dimension to obtain enhanced Doppler frequency shift features.

[0177] In some embodiments, the detection module 403 is specifically used for: the dimensionality reduction module of the detection model compresses the enhanced Doppler frequency shift features to obtain the dimensionality-reduced Doppler spectrum; the gated recurrent unit layer of the time series modeling module models the historical time series dependency relationship of the dimensionality-reduced Doppler spectrum to obtain the time-optimized frequency shift features.

[0178] In some embodiments, the ultrasonic echo signal includes multiple frames of ultrasonic signals, each frame of ultrasonic signal corresponding to the probability of human motion. The device also includes a post-processing module for constructing a sliding time window based on the probability of human motion corresponding to multiple consecutive frames of ultrasonic signals, calculating the cumulative value of multiple probabilities within the sliding time window, and determining the detection result of human motion based on the cumulative value.

[0179] In some embodiments, the post-processing module is specifically used to: perform confidence screening on the probability of human motion occurrence in each frame of ultrasound signal based on the output probability threshold to obtain the corresponding filtered probability; construct a sliding time window with the filtered probabilities of multiple consecutive frames of ultrasound signals, and calculate the cumulative value of multiple filtered probabilities within the sliding time window; wherein, for each new frame of ultrasound signal, the sliding time window updates the filtered probability within the sliding time window by moving forward one time frame.

[0180] In some embodiments, the preprocessing module 402 is specifically used to: perform frame-segmentation processing on the ultrasonic echo signal to obtain multiple frames of ultrasonic signals; for each frame of ultrasonic signal, perform windowing processing on the ultrasonic signal to obtain a noise-reduced and smoothed processed signal, and perform Fourier transform on the processed signal to obtain the spectral features corresponding to the ultrasonic signal, wherein the length of the Fourier transform is determined based on the Doppler resolution requirements and computational overhead.

[0181] In some embodiments, the preprocessing module 402 is specifically used to: determine the extraction frequency based on the center frequency of the emitted ultrasonic wave; determine the frequency resolution based on the sampling frequency and the length of the Fourier transform of the ultrasonic echo signal generated by human motion; and extract the Doppler spectrum from the spectral features based on the extraction frequency and the frequency resolution.

[0182] In some embodiments, the ultrasound is determined based on at least one of the following: the sampling frequency of the microphone, the frequency response range of the speaker, the user's auditory experience, and the detection model's ability to detect Doppler frequency shift.

[0183] In some embodiments, the detection model is trained based on the following steps: collecting ultrasonic echo data in different scenarios, wherein the scenarios include at least one of human movement scenarios, unmanned stationary scenarios, movement scenarios at different distances, movement scenarios in different room environments, and echo scenarios; preprocessing and extracting the spectrum of the ultrasonic echo data to obtain training sample data; and training the initial model using the training sample data to obtain the detection model.

[0184] This embodiment is a device embodiment corresponding to the method embodiment described above. The ultrasonic motion detection device provided in this embodiment uses a loudspeaker to emit ultrasonic waves, which are then used to collect ultrasonic echo signals generated by human movement. After processing the ultrasonic echo signals generated by human movement, the Doppler spectrum is used as the sole input data for the detection model. By mining the frequency domain amplitude change information of the spectrum itself, human movement is predicted. This disclosure can fully utilize the existing loudspeaker and microphone resources of the target terminal, and combine deep learning to automatically extract human movement features, achieving high-precision and low-cost human movement detection in complex environments.

[0185] According to embodiments of this disclosure, this disclosure also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to implement the ultrasonic anomaly detection method described in any of the above embodiments.

[0186] According to embodiments of this disclosure, this disclosure also provides a readable storage medium storing computer instructions that, when executed by a computer, enable the ultrasonic anomaly detection method described in any of the above embodiments.

[0187] According to embodiments of this disclosure, this disclosure also provides a computer program product that, when executed by a processor, can implement the ultrasonic anomaly detection method described in any of the above embodiments.

[0188] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0189] like Figure 5 As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. The RAM 503 may also store various programs and data required for the operation of the electronic device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0190] Multiple components in electronic device 500 are connected to I / O interface 505, including: input unit 505, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows electronic device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0191] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the ultrasonic anomaly detection method. For example, in some embodiments, the ultrasonic anomaly detection method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the ultrasonic anomaly detection method described above can be performed. Alternatively, in other embodiments, the computing unit 501 can be configured to perform the ultrasonic anomaly detection method by any other suitable means (e.g., by means of firmware).

[0192] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0193] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0194] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0195] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0196] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0197] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0198] According to the technical solution of this disclosure, ultrasonic waves are emitted by a loudspeaker, and ultrasonic echo signals generated by human movement are collected by a microphone. After processing the ultrasonic echo signals generated by human movement, the Doppler spectrum is used as the single input data of the detection model. By mining the frequency domain amplitude change information of the spectrum itself, human movement is predicted. This disclosure can make full use of the existing loudspeaker and microphone resources of the target terminal, and automatically extract human movement features by combining deep learning, achieving high-precision and low-cost human movement detection in complex environments.

[0199] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0200] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for detecting ultrasonic anomalies, comprising: Ultrasonic waves are emitted through a speaker on the target terminal, and ultrasonic echo signals generated by human movement are collected through a microphone on the target terminal. The ultrasonic echo signal is preprocessed to obtain the spectral characteristics of the ultrasonic echo signal in multiple frequency bands, and the Doppler spectrum corresponding to the human motion is extracted from the spectral characteristics of the multiple frequency bands. The Doppler spectrum is input into the detection model, which outputs the probability of human motion. The detection model is used to predict the probability of human motion based on the frequency domain amplitude variation characteristics of the Doppler spectrum.

2. The method according to claim 1, wherein, The step of inputting the Doppler spectrum into the detection model and outputting the probability of human motion includes: The Doppler spectrum is processed by a two-dimensional convolution module of the detection model to obtain spectral change features. The frequency attention module of the detection model is used to enhance the spectral change features in the frequency dimension to obtain enhanced Doppler frequency shift features. The time-dependent modeling module of the detection model is used to model the enhanced Doppler frequency shift features to obtain time-optimized frequency shift features. The output module of the detection model performs a nonlinear mapping on the time-optimized frequency shift feature to output the probability of human motion.

3. The method according to claim 2, wherein, The ultrasonic echo signal includes multiple frames of ultrasonic signals. After preprocessing, a corresponding Doppler spectrum is extracted from each frame. For each Doppler spectrum, the feature extraction module of the detection model performs two-dimensional convolution processing on the Doppler spectrum to obtain spectral change features, including: The initial frequency domain features are extracted through the first convolutional layer of the feature extraction module based on the local changes between adjacent frequency points within the Doppler spectrum frame. The second convolutional layer of the feature extraction module performs calculations on the initial frequency domain features corresponding to the continuous Doppler spectrum, extracts local features in the time domain and local features in the frequency domain, and fuses the local features in the time domain and the local features in the frequency domain to obtain the spectral change features.

4. The method according to claim 2, wherein, The frequency attention module of the detection model enhances the spectral change features in the frequency dimension to obtain enhanced Doppler frequency shift features, including: The frequency attention module uses three fully connected layers to perform linear mapping on the input spectral variation features, resulting in a query matrix, a key matrix, and a value matrix. The query matrix and key matrix are processed through the activation layer of the frequency attention module to obtain frequency attention weights; and based on the frequency attention weights and the value matrix, the spectral change features are weighted and enhanced in the frequency dimension to obtain enhanced Doppler frequency shift features.

5. The method according to claim 2, wherein, The time-series modeling module of the detection model performs time-dependent modeling on the enhanced Doppler frequency shift features to obtain time-optimized frequency shift features, including: The enhanced Doppler frequency shift feature is compressed in dimension by the dimensionality reduction module of the detection model to obtain the dimensionality-reduced Doppler spectrum; By using the gated loop unit layer of the time-series modeling module, the historical time-series dependency relationship of the dimensionality-reduced Doppler spectrum is modeled to obtain time-optimized frequency shift features.

6. The method according to claim 1, wherein, The ultrasonic echo signal includes multiple frames of ultrasonic signals, each frame of ultrasonic signal corresponding to the probability of human motion occurring; the method further includes: A sliding time window is constructed based on the probability of human motion occurring corresponding to multiple consecutive frames of ultrasound signals. The cumulative value of multiple occurrence probabilities within the sliding time window is calculated, and the detection result of human motion is determined based on the cumulative value.

7. The method according to claim 6, wherein, The step of constructing a sliding time window based on the probability of human motion occurring corresponding to multiple consecutive frames of ultrasound signals, and calculating the cumulative value of multiple occurrence probabilities within the sliding time window, includes: The probability of human motion in each frame of ultrasound signal is filtered by confidence based on the output probability threshold to obtain the corresponding filtered probability. A sliding time window is constructed using the post-filtering probabilities of multiple consecutive ultrasound signals, and the cumulative value of multiple post-filtering probabilities within the sliding time window is calculated; wherein, for each new ultrasound signal, the sliding time window updates the post-filtering probabilities within the sliding time window by moving forward one time frame.

8. The method according to claim 1, wherein, The preprocessing of the ultrasonic echo signal to obtain the spectral characteristics of the ultrasonic echo signal in multiple frequency bands includes: The ultrasonic echo signal is processed by framing to obtain multiple frames of ultrasonic signal; For each frame of ultrasound signal, the ultrasound signal is windowed to obtain a noise-reduced and smoothed processed signal, and a Fourier transform is performed on the processed signal to obtain the spectral features corresponding to the ultrasound signal. The length of the Fourier transform is determined based on the Doppler resolution requirements and computational overhead.

9. The method according to claim 8, wherein, Extracting the Doppler spectrum corresponding to the ultrasonic echo signal from the spectral features includes: The extraction frequency is determined based on the center frequency of the emitted ultrasonic wave; The frequency resolution is determined based on the sampling frequency of the ultrasonic echo signal generated by human movement and the length of the Fourier transform. Based on the extracted frequency and the frequency resolution, the Doppler spectrum is extracted from the spectral features.

10. The method according to claim 1, wherein, The ultrasound wave is determined based on at least one of the following: the sampling frequency of the microphone, the frequency response range of the speaker, the user's auditory experience, and the detection model's ability to detect Doppler frequency shift.

11. The method according to claim 1, wherein, The detection model was trained based on the following steps: Collect ultrasonic echo data under different scenarios, wherein the scenarios include at least one of the following: a scenario with people moving, a scenario with no one stationary, a scenario with people moving at different distances, a scenario with people moving in different room environments, and an echo scenario; The ultrasonic echo data is preprocessed and its spectrum is extracted to obtain training sample data; The initial model is trained using the training sample data to obtain the detection model.

12. An ultrasonic anomaly detection device, comprising: The signal acquisition module is used to emit ultrasonic waves through a speaker on the target terminal and to acquire ultrasonic echo signals generated by human movement through a microphone on the target terminal. The preprocessing module is used to preprocess the ultrasonic echo signal to obtain the spectral characteristics of the ultrasonic echo signal in multiple frequency bands, and to extract the Doppler spectrum corresponding to human motion from the spectral characteristics of the multiple frequency bands. The detection module is used to input the Doppler spectrum into the detection model and output the probability of human motion; wherein, the detection model is used to predict the probability of human motion based on the frequency domain amplitude variation characteristics of the Doppler spectrum.

13. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the ultrasonic anomaly detection method according to any one of claims 1-11.

14. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the ultrasonic anomaly detection method according to any one of claims 1-11.

15. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the ultrasonic anomaly detection method according to any one of claims 1-11.

Images