Distance sensing method and system based on single audio signal feature

Abstract

The invention belongs to but not limited to the technical field of distance sensing, and discloses a distance sensing method and system based on a single audio signal characteristic, and the method comprises the steps: carrying out the data preprocessing of an original data stream of a single base station audio Chirp signal, converting a time domain audio signal into an energy density map EDM, and carrying out the analysis of the characteristics; analyzing the influence of the equipment difference on the EDM characteristics; constructing an EDM data set based on the mixed data source; based on EDM feature analysis and in combination with a space attention mechanism, a single base station distance sensing network DPNet based on a deep learning method is constructed. According to the method, on the basis of numerical features of the EDM, a linear distance sensing model of the normalized EDM in a physical distance and LoS state is constructed, so that the DRR is easier to understand and express; a deep learning network based on a residual attention mechanism is designed, and the distance sensing performance of a single base station in an LoS state is optimized.

Description

Technical Field

[0001] The present invention belongs to but is not limited to the field of distance perception technology, and in particular relates to a distance perception method and system based on a single audio signal feature. Background Art

[0002] In indoor positioning technology with ranging as the observation condition, 4 base stations are the most common basic positioning unit. However, the actual indoor space is not always suitable for the deployment of 4 base stations, especially in small-area rooms / hallways and corridors. The deployment of 4 base stations often fails to meet the requirements of cost control or convenient construction. The observation data obtained by forcibly applying this solution is prone to negative redundancy, and the indication of the terminal location is not clear enough. For such scenarios, the industry has generally preferred Bluetooth Beacon (ranging or fingerprint) technology with low-cost equipment deployment characteristics. However, as the implementation of this technology has expanded, people have found that its performance has gradually failed to meet the increasingly high standards of accuracy in the process of advancing scene intelligence. When the audio signal group of the 4 base stations is taken as a whole, the temporal relationship between its independent signals provides a reference for the geometric method and fingerprint method to solve the position; and when a single signal is stripped out of it, its supporting spatial positioning features have not been fully explored.

[0003] Single base station positioning is often seen in RF array technology, which usually uses a multi-beam signal source to measure angles and distances, and locate the terminal in a polar coordinate framework. In the absence of spatial correlation, a single audio signal is affected by the frequency response differences of the device microphone, and it is easy to show different distance perception capabilities under different terminals.

[0004] The ratio of the direct path component to the reverberation component of the audio signal is an important factor in determining distance judgment. When the environmental topology does not change, the reverberation energy will remain stable and evenly distributed in the closed space. At this time, the perceived distance of the terminal is closely related to the energy of the direct path component.

[0005] In view of the above analysis, the technical problems that urgently need to be solved in the existing technology are: the limited number of signals and poor observation topology caused by the system laying cost in scenarios such as small-area rooms / hallways and corridors commonly seen in indoor spaces. Summary of the invention

[0006] In view of the problems existing in the prior art, the present invention provides a distance perception method and system based on a single audio signal feature.

[0007] The present invention is implemented as follows: a distance perception method based on a single audio signal feature comprises the following steps:

[0008] Step 1: Preprocess the original data stream of the single-base-station audio Chirp signal, convert the time-domain audio signal into an energy density map (EDM), and analyze its characteristics;

[0009] Step 2: Analyze the impact of equipment differences on EDM characteristics;

[0010] Step 3: Construct an EDM dataset based on hybrid data sources;

[0011] Step 4: Based on the feature analysis of EDM and combined with the spatial attention mechanism, a single-base station distance perception network (DPNet) based on deep learning method is constructed.

[0012] Furthermore, the Chirp signal used in step 1 can be expressed as:

[0013]

[0014] Where A(t) is the adjustable signal amplitude, T is the signal duration, f0 and f e are the start frequency and cut-off frequency of the signal respectively.

[0015] Furthermore, in step 1, the audio signal is preprocessed by a short-time Fourier transform (STFT) with a window length of l and an overlap rate of k, and a Chirp signal time-frequency diagram with a pixel size of R×C is calculated. The numerical value of each frequency point represents the signal energy level, so the time-frequency diagram is called an energy density diagram (EDM).

[0016] Furthermore, the relationship between EDM and perception distance in step 2 is affected by both the distance from the terminal to the base station and the differences in the response of different terminals to the signal. The former is reflected in the different changes in the overall energy of the signal represented by EDM at different positions in space, and the latter is reflected in the different signal frequency responses and sensitivities of different sampling devices at the same position. The absolute logarithm of the pixel value of each frequency point in EDM is defined as the relative energy size of the audio signal at each frequency point, and the relative energy range of EDM is defined as the response domain. The domain value of this domain can intuitively reflect the above two differences.

[0017] Furthermore, in step 2, the differences between different devices are controlled to the maximum value based on the maximum value normalization method. ±1 m level; EDM is normalized as follows:

[0018]

[0019] Among them, i and j are the row and column index numbers of EDM respectively. It represents the normalized energy of the frequency point, and its range is [0, 1]. The stronger the signal energy, the larger the value of the corresponding pixel.

[0020] Furthermore, in step 3, a high-quality dataset containing enough difference elements is constructed, which is labeled with different physical reference distances (unit: meter). The single base station audio Chirp signal is collected in two different typical indoor scenes of corridor and office, and finally processed into normalized EDM data.

[0021] Furthermore, the architecture of the single-base-station distance perception network (DPNet) in step 4 adaptively perceives the EDM image features that can represent the law of spatial distance variation in the form of residual attention by stacking two spatial attention modules;

[0022] DPNet adopts a "sandwich" structure of preamble, main body, and suffix, and the core component is the pre-activation bottleneck residual block (Pre-activation Bottleneck ResBlock);

[0023] In the leading part, a 7×7 convolution layer and a 3×3 maximum pooling layer are first used to quickly reduce the dimension of the feature map and increase its receptive field;

[0024] The main part consists of two spatial attention modules and a connected residual block. The number of output channels of each connected residual block is doubled to infer deeper features. In the spatial attention module, the mask branch uses the same number of residual blocks as the trunk branch, and the number of input and output channels of all residual blocks is consistent. Since the mask branch contains more pooling layers, its receptive field is larger than that of the trunk branch, which generates denser features in the process of back-propagating the network to update the gradient to infer the weight of each pixel for the perceived distance.

[0025] For the suffix part, an adaptive average pooling layer is used to fix the feature map size to 1×1, and it is sent to the fully connected layer for network output.

[0026] Furthermore, in DPNet, each attention module consists of a trunk branch and a mask branch, where the trunk branch is used to process network features and increase the receptive field; the mask branch adds soft weights to the features, and controls the flow of weight information to various objects through bottom-up feedforward and top-down feedback. When the mask branch output after the sigmoid layer normalization covers the trunk branch, the dot multiplication operation will cause the attenuation of the global features of the image and may destroy the original data identity mapping characteristics of the trunk branch. Therefore, the input and output of the attention module are connected with a secondary residual connection. This residual attention learning structure helps the network capture information of different scales and continuously improves the ability of the entire network as the network depth increases.

[0027] Furthermore, during the entire model training process, the input EDM is gradually decomposed into pixel sets of smaller and smaller sizes through the feed-forward structure, generating a low-resolution feature tensor with strong semantic information.

[0028] Another object of the present invention is to provide a distance perception method based on a single audio signal feature and a distance perception system based on a single audio Chirp signal feature, comprising:

[0029] The data preprocessing module performs data preprocessing on the original data stream of the single base station audio Chirp signal, converts the time domain audio signal into an energy density map (EDM), and analyzes its characteristics;

[0030] Normalization module, analyzes the impact of equipment differences on EDM characteristics;

[0031] EDM dataset construction module, which constructs EDM dataset based on hybrid data sources;

[0032] The single-base station distance perception network building module is based on the feature analysis of EDM and combined with the spatial attention mechanism to construct a single-base station distance perception network (DPNet) based on deep learning methods.

[0033] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0034] First, the present invention proposes a distance perception network (DPNet) based on the distance perception technology of a single audio signal.

[0035] The audio Chirp signal used in this invention has time-frequency characteristics that are highly correlated with spatial distance. These characteristics can be mainly reflected in the ratio of the direct path of the signal to the reverberation component (DRR). In order to make DRR easier to understand and express, the audio data is first preprocessed into an energy density map (EDM) with a granularity of 1.3ms / pixel, and the specific characteristics of EDM are analyzed from three dimensions: physical distance, occlusion, and equipment. The maximum value normalization method is proposed to suppress equipment differences. Based on the numerical characteristics of EDM, a linear distance perception model of physical distance and normalized EDM is constructed.

[0036] In order to optimize the distance perception performance of a single base station, a deep learning network based on the residual attention mechanism is designed. The network uses the spatial attention module of trunk branch feature extraction and mask branch weight allocation, and utilizes the relative gradient change of the direct path energy in EDM. In a mixed data set of 21,840 frames formed by 3 typical scenes and 7 devices, the distance perception accuracy of RMSE in the training process is about 0.9m.

[0037] Experiments on test datasets in different scenarios and on different devices used in the training datasets show that DPNet achieves distance perception accuracy and stability with an RMSE of about 1m and a standard deviation of about 0.6m with a reasonable architecture design, and it demonstrates good generalization capabilities in three test devices and two types of test scenarios. In addition, since DPNet is a dataset-customized classification model, it comprehensively outperforms the reference RANet in terms of parameter quantity and computational complexity.

[0038] Second, the expected benefits and commercial value of the technical solution of the present invention after transformation are as follows: Distance perception based on a single base station reduces the hardware cost, construction cost and operating cost of base station equipment in a small space compared to the traditional 4-base station technology. Distance perception based on audio signals is different from radio frequency signals. The receivers and transmitters of audio signals are widely present in various consumer and professional terminals, such as smart phones, smart watches, walkie-talkies, law enforcement devices, etc., which can be directly applied to the industry without changing or adding any equipment hardware, with low promotion resistance and high market acceptance.

[0039] The technical solution of the present invention fills the technical gap in the industry at home and abroad: Indoor distance perception mostly uses radio frequency technology, such as Bluetooth, ultra-wideband (UWB), Wi-Fi RTT, etc., but the core patents or authorizations of these technologies are all in the hands of foreign technology companies or alliances. In addition, audio signals are terminal-friendly. The present invention adopts a technical route different from sonar round-trip ranging. The base station only broadcasts signals and the terminal only receives signals, which greatly improves the user capacity of the system.

[0040] The technical solution of the present invention solves the technical problem that people have been eager to solve but have never succeeded: the technology based on single base station distance perception is mostly limited to radio frequency signals, such as Bluetooth, ultra-wideband (UWB), Wi-Fi RTT, etc. However, radio frequency signals are electromagnetic waves with strong penetration, which are easy to take effect within the range where undesired signals take effect, causing problems such as wall penetration or increased errors that are difficult to solve from a technical level; audio signals are mechanical waves with weak penetration, which are easy to control the effective range of the signal within a specified area without interfering with other areas.

[0041] The technical solution of the present invention overcomes technical prejudice: in the public's perception, audio signals are easily disturbed by environmental noise. In fact, the present invention adopts a special signal modulation method with obvious signal characteristics; and for the sub-ultrasonic to ultrasonic frequency band, there is no environmental and human pollution. In addition, the deep learning network proposed in the present invention is based on spatial attention, and with the assistance of prior knowledge of a large number of data sets, it can effectively extract signal features in environmental noise.

[0042] Third, technical problems existing in existing technologies

[0043] 1) Insufficient accuracy of distance perception

[0044] Existing distance perception methods based on audio signals usually rely on complex multi-base station positioning solutions. However, this method requires the synchronous collaboration of multiple devices and is significantly affected by environmental noise and multipath interference, resulting in insufficient ranging accuracy, especially in complex environments. It is more difficult to meet actual needs.

[0045] 2) Interference of equipment differences on ranging results

[0046] Differences in hardware characteristics of different audio acquisition devices (such as microphones) can introduce systematic biases, leading to inconsistent outputs of audio signal processing and ranging models, and a lack of standardized correction mechanisms between devices.

[0047] 3) Insufficient real-time and robustness

[0048] The application of existing deep learning models in audio signal distance perception has the problem of high computational complexity and is difficult to meet real-time requirements. At the same time, the model has poor robustness in the face of environmental noise or signal interference, resulting in insufficient ranging stability.

[0049] 4) Insufficient multi-source datasets and low model generalization ability

[0050] In the prior art, the training data set of audio signals usually comes from a single source and cannot cover a variety of usage scenarios, resulting in insufficient generalization ability of the model in practical applications and difficulty in coping with complex situations in different environments and devices.

[0051] Technical advances of the present invention:

[0052] 1) High-precision single base station distance perception

[0053] The present invention accurately captures the correspondence between signal energy distribution and distance through the generation of energy density map (EDM) and feature analysis of deep learning network. It can achieve high-precision distance perception without the collaboration of multiple base stations, effectively solving the deficiency of existing technology that relies on multi-base station positioning.

[0054] 2) Compensation and normalization of device differences

[0055] The present invention introduces a normalization module to standardize the hardware characteristics of different acquisition devices, effectively reducing the interference of device differences on the ranging results. The normalized data ensures the adaptability and consistency of the model among multiple devices.

[0056] 3) Improvement of robustness and real-time performance

[0057] By introducing the spatial attention mechanism and the optimized convolutional neural network structure, the present invention significantly improves the robustness of the model to environmental noise and multipath interference. At the same time, through the lightweight model design, the real-time performance of the system in practical applications is ensured to meet the low latency requirements of industrial-grade scenarios.

[0058] 4) Constructing EDM datasets from mixed data sources

[0059] The present invention constructs a diversified EDM data set through hybrid data source acquisition and data enhancement technology, covering a variety of environments, equipment and signal conditions, significantly improving the generalization ability and adaptability of the model, and ensuring that the system can maintain a high ranging accuracy in different scenarios.

[0060] 5) Simple and efficient deployment and application

[0061] The present invention is based on the structural design of a single base station, does not require complex deployment of additional hardware, reduces system costs, and facilitates rapid promotion and application in scenarios such as indoor positioning and distance perception between IoT devices.

[0062] Significant effects of industrial applications:

[0063] 1) Improved ranging accuracy

[0064] Through the technical solution of the present invention, the distance perception error of the audio signal is reduced to 10 cm level, meeting the high-precision ranging requirements such as indoor precise positioning and environmental monitoring.

[0065] 2) Strong adaptability and wide application scenarios

[0066] The present invention adapts to different devices, environments and signal conditions through normalization modules and data enhancement technology, and can be widely used in smart home, industrial automation, indoor positioning and unmanned equipment navigation and other fields.

[0067] 3) System cost reduction

[0068] The technical architecture based on a single base station eliminates the need for coordination among multiple base stations, greatly simplifying hardware deployment and maintenance costs while lowering the threshold for industrialization.

[0069] 4) Real-time response and high efficiency

[0070] The system supports millisecond-level response time and can perform reliable distance perception in real-time dynamic environments, improving user experience and system performance and meeting the efficiency requirements of industrial applications.

[0071] 5) Promote technological development

[0072] The present invention realizes the innovative integration of audio signal processing and deep learning technology, provides a new solution for accurate distance perception technology based on audio signals, and promotes the application and development of intelligent perception technology in multiple fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 is a flow chart of a distance perception method based on a single audio Chirp signal feature provided by an embodiment of the present invention;

[0074] Figure 2 is a reference spectrum diagram of a single base station audio Chirp signal provided by an embodiment of the present invention;

[0075] Figure 3 3D visualization and energy distribution contour diagram of a single signal EDM provided by an embodiment of the present invention; (a) 1m; (b) 35m;

[0076] Figure 4 Schematic diagram of the gradient field distribution of the single signal EDM relative energy provided by an embodiment of the present invention; (a) 1m; (b) 35m;

[0077] Figure 5 Schematic diagram of the EDM response domain variation range of three different brands of test machines at different distances provided by an embodiment of the present invention;

[0078] Figure 6 It is a schematic diagram of the range change of the EDM normalized response domain at different base station distances provided by an embodiment of the present invention;

[0079] Figure 7 Schematic diagram of the DPNet static data set acquisition process provided by an embodiment of the present invention; (a) corridor; (b) office;

[0080] Figure 8 It is a schematic diagram of a portion of a DPNet static data set provided by an embodiment of the present invention;

[0081] Fig. 9 is a distance perception network structure diagram based on EDM provided by an embodiment of the present invention;

[0082] Fig.10 is a structural diagram of a distance perception system based on a single audio Chirp signal feature provided by an embodiment of the present invention;

[0083] Fig.11 This is a test data set collection environment provided by an embodiment of the present invention, (a) corridor; (b) conference room;

[0084] Fig.12 The static distance perception performance of three test devices provided in the embodiments of the present invention are: (a) Nova8Pro; (b) Mate30; (c) Reno5;

[0085] Fig.13 It is a schematic diagram of distance perception error of three test equipment sample sets and different environment sample sets under static conditions provided by an embodiment of the present invention;

[0086] Fig.14 It is a schematic diagram of the classification probability of DPNet corresponding to the correct distance label at the marked point position in the three-round dynamic distance perception provided by the embodiment of the present invention;

[0087] Fig.15 It is a schematic diagram of the overall error distribution of three-wheel dynamic distance perception at the marking point provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0088] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0089] like Figure 1 As shown, an embodiment of the present invention provides a distance perception method based on a single audio signal feature, comprising the following steps:

[0090] Step 1: Preprocess the original data stream of the single-base-station audio Chirp signal, convert the time-domain audio signal into an energy density map (EDM), and analyze its characteristics;

[0091] Step 2: Analyze the impact of equipment differences on EDM characteristics;

[0092] Step 3: Construct an EDM dataset based on hybrid data sources;

[0093] Step 4: Based on the feature analysis of EDM and combined with the spatial attention mechanism, a single-base station distance perception network (DPNet) based on deep learning method is constructed.

[0094] (1) Single-base station audio chirp signal preprocessing and feature analysis

[0095] The Chirp signal used in the present invention can be expressed as:

[0096]

[0097] Where A(t) is the adjustable signal amplitude, T is the signal duration, f0 and f e are the signal start frequency and cut-off frequency respectively. In the embodiment of the present invention, T=45ms, f0=19.5k / 23.5k Hz, f e =16.5k / 20.5k.

[0098] The audio signal is preprocessed by short-time Fourier transform (STFT) with a window length of l and an overlap rate of k, and the Chirp signal time-frequency diagram with a pixel size of R×C is calculated. The numerical value of each frequency point represents the signal energy level, so this time-frequency diagram is called energy density diagram (EDM).

[0099] In the embodiment of the present invention, when the STFT window length parameter l is 512 and the overlap rate parameter k is 87.5%, the calculated EDM size R×C is 33×68, and the single frequency point pixel distance resolution is 45.77 cm. Figure 2 At this resolution, when the chirp signal in EDM is misjudged by 2 to 3 frequency pixels (about 5%), the error in the signal arrival time (Time of arrival, ToA) is about 2 to 4 ms, which is converted to a distance of about 1 m. The impact on subsequent modeling and distance perception is relatively controllable.

[0100] When facing the base station, the direct component energy of the terminal receiving the signal significantly exceeds the reverberation energy and the ambient noise energy. The relative energy amplitude of each frequency point is the z-axis. Figure 3 The three-dimensional visualization and energy distribution contour results of EDM at actual distances of 1m and 35m from the base station are shown.

[0101] Figure 3In the figure, the relative energy amplitude of EDM rises rapidly at the moment the Chirp signal is received, and maintains a high level during the duration of the direct path; then as the direct path passes the peak, it gradually transitions to the reverberation part until the signal disappears. To quantify the trend of the value change, the gradient of the EDM relative energy is calculated along the positive direction of the x-axis and the y-axis according to the following formula, and the result is as follows: Figure 4 shown.

[0102]

[0103] It means to find the partial derivative in the x direction of each pixel of EDM. It means to find the partial derivative in the y direction of each pixel of EDM, and ▽E is the EDM gradient.

[0104] Figure 4 The arrows in the figure represent the gradient vectors of each pixel, where the direction of the arrow represents the gradient direction, and the length of the arrow represents the gradient magnitude. Whether it is the short distance of 1m or the farthest distance of 35m, the energy amplitude produces a gradient mutation in the direct path part, which undoubtedly provides a powerful feature identification for EDM image processing.

[0105] (2) Analyze the impact of device differences on perceived distance

[0106] The relationship between EDM and perception distance is affected by both the distance from the terminal to the base station and the differences in the response of different terminals to the signal. The former is reflected in the different changes in the overall energy of the signal represented by EDM at different locations in space, and the latter is reflected in the different signal frequency responses and sensitivities of different sampling devices at the same location. The absolute logarithm of the pixel value of each frequency point in EDM is defined as the relative energy size of the audio signal at each frequency point, and the relative energy range of EDM is defined as the response domain. The domain value of this domain can intuitively reflect the above two differences.

[0107] Generally speaking, the clearer the boundary intervals of the response domains of the same device within an adjacent 1m range, the higher the accuracy of its perceived distance; conversely, if the response domain at a certain distance is very wide, it is easy to overlap with the response domains 1m before and after.

[0108] Three smartphones of different brands, Vivo S10, Huawei Nova8 Pro, and GooglePixel3, were used as test equipment. Audio data facing the base station was collected at intervals of 1m for 60 seconds on the central axis of a 2m×40m corridor. The EDM response domain range was used as the statistical object. Figure 5 Results from three of the test machines are shown.

[0109] observe Figure 5From the data, it is easy to find that the response domain of a single-frame EDM is linearly inversely proportional to the distance from the terminal to the base station in the range of 1 to 35 m, and the slope coefficients are similar, which shows that although there are obvious differences in the EDM energy levels of different devices at the same distance, the response domain changes of different devices at the same distance interval are similar relative to the overall energy range of the device.

[0110] Therefore, the maximum value normalization method is used for EDM, so that it has the possibility of weakening the differences between devices under a unified standard. Specifically, the minimum energy value of each frequency point of all collected data is set as the normalization lower limit E lower , any energy value below this threshold is considered as noise and is assigned a value of 0; the maximum energy value of the frequency point at the test distance closest to the base station is set as the normalized upper limit E upper , any energy value above this threshold is assigned a value of 1; the EDM is normalized as follows: Figure 6 As shown:

[0111]

[0112] Among them, i and j are the row and column index numbers of EDM respectively. It represents the normalized energy of the frequency point, and its range is [0, 1]. The stronger the signal energy, the larger the value of the corresponding pixel.

[0113] It can be found that although the normalized EDM in the LoS state has a deviation of about ±1m in the overall value, after excluding the weak regular signal beyond 35m caused by the reduced energy variation amplitude of the direct path component, the EDM normalized response domain range of the three test machines and the actual physical distance all approximately show the following linear relationship:

[0114]

[0115] Among them, the independent variable is the EDM normalized energy value, and D is the estimated distance. Table 1 further compares the physical distances corresponding to the three test machines at different response domain values. The results show that the maximum value normalization method controls the differences between different devices to the maximum. ±1 m level.

[0116] Table 1 Corresponding ground reference distances of three tested mobile phones at different normalized response domain values

[0117]

[0118] (3) Constructing an EDM dataset based on hybrid data sources

[0119] A good network model is not only related to the rationality of its own architecture and task fit, but the balance and representativeness of the training data also account for a considerable proportion in the generalization ability and repeatability of the final model. Different from some typical network targets with strong generalization ability, the distance-aware network (DPNet) proposed in this patent focuses more on the classification performance of audio Chirp signals rather than the classification ability on public data sets. Therefore, in order to improve the robustness and generalization of the model, a high-quality data set containing enough difference elements is constructed. It is labeled with different physical reference distances (unit: meter), and the audio Chirp signal of a single base station is collected in two different typical indoor scenes, corridors and offices, and finally processed into normalized EDM data. Among them, 7 devices of different brands or models were used for data acquisition, including: Huawei Nova7, Huawei P40, Xiaomi 10Ultra, Redmi 12C, Oppo Find X6, Vivo X50 pro, Google Pixel3. Such as Figure 7 As shown, appropriate locations are selected in the three scenarios to set up 1.2m high brackets to place base stations, so that the base stations broadcast Chirp signals at a frequency of 1Hz.

[0120] In the process of data collection, the first thing to consider is how to choose the data collection point spacing and sampling method. For the former, sparse point spacing is conducive to improving classification accuracy, but it is difficult to respond smoothly to distance changes; while dense point spacing is conducive to improving the resolution of distance perception, but both the difficulty of model training and the data collection cost will increase.

[0121] The mapping relationship between the normalized EDM and the physical distance has an error of 1 to 1.5 m only from the energy value level. Therefore, considering the accuracy, sensitivity and cost of distance perception, the present invention selects 1 m as the data acquisition point spacing. For the latter, the mainstream methods include dynamic sampling and static sampling. The dynamic method usually uses a high-precision IMU or portable SLAM to quickly measure the true value of the distance, and associates the audio data with the high-resolution distance tag through interpolation. When a pedestrian walks at a speed of 1.5 m / s, the frequency domain of the target signal frequency band will produce a frequency offset of about 48.5 to 57.4 Hz and a time offset of 0.65 to 0.77 ms. In the dynamic positioning process, the random and changeable movement direction and speed of pedestrians will introduce different degrees of Doppler error. It is quite difficult and impractical to make the training data set completely cover the Doppler frequency shift error. At the same time, the dynamic sampling method will also bring additional errors to static positioning, so this method is not suitable for audio distance perception. In view of the above-mentioned Doppler error of less than 1 ms, this paper chooses the time-consuming but better overall effect static sampling method to collect data:

[0122] In the range of 1 to 35 meters from the base station, 1.2m high brackets were arranged every 1m to place data acquisition equipment, and the ground reference distance was measured by the SNDWAY-100G laser rangefinder (nominal accuracy 3mm±50ppm); at the same time, 60s audio signals were collected at each distance. In addition, data acquisition was performed separately for conventional environments without signals to expand the adaptability of the data set to noise.

[0123] In the office scene, the data acquisition distance range is 1 to 15m due to the size of the site. After batch processing, the normalized EDM data with a total of 21,852 frames will be archived in the format of "scene-device-distance-cycle index"; at the same time, a copy of the data set is made to mix the results of different scenes and devices, and only the "distance" label (0 to 35, 0 represents no signal) is used for model training. The average number of EDM frames corresponding to each of the 36 labels is 607. The final data set of the mixed data source is as follows: Figure 8 shown.

[0124] (4) Building a single base station distance sensing network

[0125] In deep learning, attention mechanism refers to the weight allocation of different parts of objects or features in the input target, including spatial attention, channel attention, temporal attention, and mixed attention. Some classic attention mechanism-based networks or modules use the form of self-attention to add self-attention modules in parallel during the hidden layer stacking process, calculate the attention score through the key vector (Key), value vector (Value), and query vector (Query), and perform weighted summation on the input to dynamically adjust the attention to different domains of interest.

[0126] 1) DPNet model architecture

[0127] In step 2, a concise linear function model of physical distance and received audio signal is established based on the maximum normalized EDM obtained by data preprocessing. In order to further optimize the performance of the model in terms of absolute ranging accuracy and device differences, this step constructs a single base station distance perception network based on deep learning methods.

[0128] Fig. 9 The architecture of the single-base-station distance perception network (DPNet) proposed in this paper is demonstrated. By stacking two spatial attention modules, the EDM image features that can represent the law of spatial distance change are adaptively perceived in the form of residual attention. The architecture details are shown in Table 2.

[0129] As can be seen from the architecture diagram, DPNet adopts a "sandwich" structure of leader, main body, and suffix. The entire network depth is 34 (trunk branch depth), and the core component is the pre-activated bottleneck residual block (Pre-activation BottleneckResBlock, hereinafter referred to as residual block). Although the standard residual block has largely solved the optimization problem of deep networks through the method of jump connection, it has increased network parameters and reduced computational efficiency; the bottleneck residual block introduces two 1×1 convolutions to increase and decrease the dimension of the data, and obtains a smaller dimensional convolution layer in the middle layer to reduce the number of parameters and computational cost, and accelerate the convergence of network training; the pre-activated bottleneck residual block further places BatchNorm and Relu operations before the convolution layer, and no longer uses the activation function after performing the element addition operation, achieving a purer "direct connection" effect, enhancing the identity mapping characteristics when extracting features, better propagating gradients, and improving training effects.

[0130] In the leading part, a 7×7 convolution layer and a 3×3 maximum pooling layer are first used to quickly reduce the dimension of the feature map and increase its receptive field.

[0131] The main part consists of two spatial attention modules and connected residual blocks. The number of output channels of each connected residual block is doubled to infer deeper features. In the spatial attention module, the mask branch uses the same number of residual blocks as the trunk branch, and the number of input and output channels of all residual blocks is consistent. Since the mask branch contains more pooling layers, its receptive field is larger than that of the trunk branch, which generates denser features in the process of back-propagating the network to update the gradient to infer the weight of each pixel for the perceived distance.

[0132] For the suffix part, an adaptive average pooling layer is used to fix the feature map size to 1×1, and it is sent to the fully connected layer for network output.

[0133] In DPNet, each attention module consists of a trunk branch and a mask branch. The trunk branch is used to process network features and increase the receptive field; the mask branch adds soft weights to the features, and controls the flow of weight information to various objects through bottom-up feedforward and top-down feedback. When the mask branch output after the sigmoid layer normalization covers the trunk branch, the dot multiplication operation will cause the attenuation of the global features of the image and may destroy the original data identity mapping characteristics of the trunk branch. Therefore, the input and output of the attention module are connected with a secondary residual connection. This residual attention learning structure helps the network capture information of different scales and continuously improves the ability of the entire network as the network depth increases.

[0134] During the entire model training process, the input EDM is gradually decomposed into smaller and smaller pixel sets through the feedforward structure, generating a low-resolution feature tensor with strong semantic information. Among them, the feature map output by the spatial attention module with the largest depth is 3×5, which is the smallest unit that can fully reflect the local characteristics of the signal at the perceived distance.

[0135] Table 2 DPNet model architecture details

[0136]

[0137] 2) DPNet model training

[0138] Since the periodic chirp signals broadcast by the single-audio base station are independent of each other and these signals are uniformly preprocessed, the entire data set can be randomly divided into a training set (17472 frames of EDM, 36×485) and a validation set (4368 frames of EDM, 36×121) in a ratio of 80% and 20%.

[0139] The original output obtained from each round of DPNet training is converted into a probability distribution through the Softmax activation function, so that the output can correspond to the probability of each label distance. For the i-th frame EDM input, its predicted distance for:

[0140]

[0141] Among them, x i is the original output vector of the model, L label is the corresponding label distance vector. The training is to minimize the absolute error between the predicted distance and the label distance as the loss function:

[0142]

[0143] in, For the iThe true label distance corresponding to the frame EDM data input. Adam (AdaptiveGradient Algorithm) and Stochastic Gradient Descent with momentum (SGDM) were selected as optimizers, where the first-order momentum of Adam was set to 0.9, the second-order momentum was set to 0.999, and the first-order momentum of SGDM was set to 0.9; batch size and learning rate were adjusted as two important hyperparameters, the former was fixed to 4, and the latter was initialized to 0.1. In the early stage of training, Adam with adaptive learning rate was used to quickly converge the model parameters. When the learning rate was less than 0.0001 for the first time, it was switched to SGDM to maintain convergence stability, and the learning rate did not change until the model converged. The training environment was built under the PyTorch framework, where the host was configured with a single NVIDIA TITAN Xp GPU (44GB memory) and CUDA version 11.5.5. The average loss value of each batch obtained in the first round of epoch training of the model is 6.70m. When the loss value optimization of 5 consecutive epochs does not exceed 0.05m, the model training is stopped. The loss values ​​of the model on the training set and the validation set are 0.82m and 0.92m respectively.

[0144] like Fig.10 As shown, the embodiment of the present invention provides a distance perception method based on a single audio signal feature and a distance perception system based on a single audio Chirp signal feature, including:

[0145] The data preprocessing module performs data preprocessing on the original data stream of the single base station audio Chirp signal, converts the time domain audio signal into an energy density map (EDM), and analyzes its characteristics;

[0146] Normalization module, analyzes the impact of equipment differences on EDM characteristics;

[0147] EDM dataset construction module, which constructs EDM dataset based on hybrid data sources;

[0148] The single-base station distance perception network building module is based on the feature analysis of EDM and combined with the spatial attention mechanism to construct a single-base station distance perception network (DPNet) based on deep learning methods.

[0149] The data preprocessing module analyzes the audio Chirp signal collected by a single base station and converts the original time domain signal into an energy density map (EDM). The specific process is as follows:

[0150] First, a bandpass filter is used to filter the audio signal to remove non-target frequency band noise in the environment and extract the effective frequency band characteristics of the Chirp signal.

[0151] Perform short-time Fourier transform (STFT) on the filtered signal to convert the time domain signal into a time-frequency map (Time-Frequency Map) to obtain the energy distribution of the signal in time and frequency.

[0152] Finally, the time-frequency diagram undergoes specific energy statistical processing to generate an energy density map (EDM), which expresses the frequency characteristics and energy distribution of the signal in an intuitive way for subsequent feature extraction and modeling.

[0153] The normalization module eliminates the impact of different hardware devices on EDM features by analyzing the differences in device hardware characteristics (such as microphone sensitivity and sampling frequency).

[0154] The frequency distribution range of EDM is standardized, and its characteristic quantities are uniformly mapped to the [0,1] interval to reduce the interference of equipment differences on signal energy characteristics.

[0155] By comparing the device response characteristics with the standard signal, a device calibration factor is generated and the signal is compensated to ensure that the audio signals collected by different devices have consistent characteristics.

[0156] The normalized EDM data is more suitable for processing by deep learning networks, improving the generalization ability and prediction accuracy of the model.

[0157] The EDM dataset construction module is responsible for collecting and organizing diverse chirp signal data sources to form a mixed dataset to improve the adaptability of the model.

[0158] The data sources include chirp signals from different devices, different environments (indoor, outdoor) and various distances (near, medium and far).

[0159] The collected original signals are grouped and annotated, and the EDM features are associated with the corresponding actual distances to generate complete training and test data sets.

[0160] Data augmentation technology was introduced to expand the size of the data set and improve the model's robustness to environmental noise and signal interference by performing operations such as time-frequency domain translation, rotation, and scaling on the signal.

[0161] The single-base station distance perception network (DPNet) based on deep learning predicts distance by analyzing EDM features:

[0162] First, the input layer of DPNet receives the normalized EDM features and extracts the spatial characteristics and local patterns of the signal through a convolutional neural network (CNN).

[0163] The introduction of the spatial attention mechanism enables the model to focus on the frequency band features that have a significant impact on distance prediction and reduce the impact of background interference on the results.

[0164] The fully connected layer integrates the convolutional features and predicts the distance of the signal source through the regression model. The model is trained and optimized through a large-scale annotated data set, ultimately achieving high-precision distance perception capabilities.

[0165] The output layer of DPNet directly provides the predicted target distance and supports real-time inference to meet the low latency requirements of practical applications.

[0166] The present invention is applicable to applications and products such as geographic fencing of smart mobile terminals or wearable devices.

[0167] For example, in elderly care, with the audio transmission base station as the center, a specific distance is defined as a virtual fence, and single-base station ranging is continuously performed on the terminal or device. When the ranging result is greater than the distance threshold, it is considered that the object has left the fence range, and an alarm is given in the background system.

[0168] For keyless entry, a specific distance is defined as a virtual fence with the audio transmission base station as the center, and single-base station ranging is continuously performed on the terminal or device. When the ranging result is less than the distance threshold, it is considered that the object has entered the fence range, and entry permission is automatically granted, triggering logical operations such as opening the door.

[0169] The experiment designed by this invention will verify the performance of the proposed distance perception method based on a single base station in terms of ranging and positioning in different small-area rooms / hallways and corridors, including the dynamic and static distance perception accuracy of DPNet, the generalization ability of DPNet in the face of different devices and environments, etc. This section first introduces the experimental data.

[0170] In addition to the 21852 frames of EDM data used for model training described in step 3, three different models of devices, Huawei Nova8Pro, Huawei Mate30, and Oppo Reno5, are added to collect new experimental data. In order to test whether DPNet has environmental adaptability, data collection is performed in corridors and office scenes that are different from those when training data was collected, including:

[0171] Test data 1. Fig.11(Left), a base station is set up at one end of a corridor with a length of 40m and an average width of 2.6m. According to the data acquisition method in step 3, 60s of LoS data facing the base station and soft-NLoS data facing the back of the base station are collected statically every 1m away from the base station until 35m. A total of 12960 frames of EDM (including no signal) are collected to test the generalization ability of the model to perceive distance. At the same time, starting from 35m away from the base station, walk towards the base station at a speed of about 1m / s and collect dynamic audio LoS ​​data. Repeat three rounds, in which marks are set at intervals of 1m on the ground, and the timestamp record is manually triggered when moving to the mark. Among them, the position of the ground mark point is measured by the SNDWAY-100G laser rangefinder (nominal accuracy 3mm±50ppm).

[0172] Test data 2. Fig.11 (Right) A base station is set up in a corner of a conference room that is 9m long and 8m wide. A total of 3960 frames of static EDM (including no signal) are collected along the diagonal line with a maximum distance of 10m according to the data acquisition method in step 3. The ground marker setting method and reference true value calculation method are the same as test data 1.

[0173] DPNet model rationality evaluation

[0174] Five network models with different configurations were designed, including: ① RANet with hybrid / channel / spatial attention modules; ② RANet with only hybrid / spatial attention modules; ③ DPNet with one / two / three spatial attention modules. With the help of torchstat tool, Table 3 details the parameters and computational complexity of each network model. At the same time, the table uses the Top-1 error rate and Top-5 error rate to express the classification accuracy of each model for a total of 6480 frames of LoS data from three devices.

[0175] Table 3 Main attributes and static distance classification accuracy of different configuration models

[0176]

[0177] The following control group was set up to explore the design rationality and performance of DPNet:

[0178] In control group 1, the Top-1 and Top-5 error rates of model 1 and model 2 without the channel attention module are compared. There is no significant difference between the two, and the model capability has not been significantly improved.

[0179] Control group 2, comparing model 4 with model 2, both contain two spatial attention modules, but DPNet-34 uses fewer residual blocks in the suffix part of the network, so both the number of parameters and the amount of calculation are greatly reduced, and the distance classification accuracy is slightly improved.

[0180] Control group 3 compares model 4 with models 3 and 5. The former has a poor effect due to its shallow network depth and insufficient feature extraction, while the latter has a slight overfitting phenomenon due to the degradation of useful information caused by the extra attention dot product, which breaks up the granularity of the basic feature units.

[0181] In control group 4, the Top-5 error rates of the models with and without attention mechanism (WA) are compared. The former is significantly better than the latter. Having only trunk branches but lacking mask branches greatly affects the distance perception ability of the model.

[0182] The above control experiments show that for EDM data sources, it is effective to extract spatial attention features by soft masking, while channel attention has little effect on distance perception of single-channel EDM data. Compared with the reference RANet, the DPNet proposed in this paper borrows the advantages of its residual attention connection and optimizes the network architecture. Under the same model scale, the minimum feature map of weight allocation is controlled to 3×5 through two spatial attention modules, which has more advantages in terms of classification accuracy of distance labels and computational efficiency.

[0183] DPNet distance perception performance evaluation

[0184] The EDM in the test data set is sent to DPNet to estimate the perceived distance value, and the Euclidean distance between it and the reference ground distance is used as the error value. The samples are divided according to different test devices and different test scenarios. The statistical results of all static samples are as follows: Fig.12 and Fig.13 shown.

[0185] Table 4 Distance perception performance of DPNet on static test dataset

[0186]

[0187] The detailed performance statistics of several sample comparisons are shown in Table 4.

[0188] According to the results of different test distances, DPNet has a slightly weaker ability to distinguish medium-distance samples (16-25m) and is more likely to output an overly large distance estimate; short-distance samples (1-15m) and long-distance samples (26-35m) have more stable error distributions than medium-distance samples.

[0189] According to the results of different test devices, the distance perception error distribution of the three mobile phones is similar, and Nova8Pro is slightly better than the other two mobile phones with an RMSE reduction of 7.8% and 11.3%.

[0190] According to the results of different test environments, within the same close distance range, the corridor and conference room scenes have almost the same performance, both achieving an RMSE of around 0.7m.

[0191] It is also worth noting that in the test data set, the signal-free part was randomly collected at different physical distances, and it achieved a Top-1 error rate of 1% as a noise control group, indicating that DPNet is capable of processing EDM data that does not contain Chirp signals.

[0192] In general, the output of DPNet on the static test data set tends to be larger than the reference true value, with an RMSE of 1.01m. About 95% of the distance perception errors are distributed in the range of -1 to 2m, and the remaining 5% of errors exceeding 2m can be identified and eliminated through gross error detection and other methods in the actual positioning process.

[0193] For dynamic ranging, the key to DPNet's distance perception performance is not only reflected in the Top-5 error rate, but also in the probability score of correctly classifying the distance when triggering the mark. Fig.14 and Fig.15 The output probability of DPNet corresponding to the correct distance label at the marked point and the distribution of all distance perception errors during the three rounds of dynamic data acquisition are shown respectively.

[0194] The probability output of dynamic distance perception for correctly classifying the label distance in the close and long distance ranges is above 0.6, while the results in the medium distance range hover between 0.5 and 0.6 or even lower, which directly leads to the probability weighted distance estimate being more likely to be larger or smaller. However, comparing the three rounds of probability outputs at the same label point, the average standard deviation is 0.05, which proves the feature repeatability of EDM at the same spatial position and the ability of DPNet to extract similar features. Similar to the conclusions in the static test, the mean error of dynamic distance perception at all marked points is about 0.83m, the RMSE is 0.92m, and the standard deviation is 0.52m.

[0195] The above dynamic and static evaluation results show that when DPNet processes the EDM data of the Chirp signal of a single base station, it not only achieves a comprehensive distance perception accuracy and stability of RMSE of approximately 1m and std of approximately 0.6m, but also demonstrates the ability to generalize to different devices and scenarios.

[0196] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0197] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A distance perception method based on a single audio signal feature, characterized in that: The following steps are involved: Step 1: Preprocess the original data stream of the single-base station audio Chirp signal, convert the time-domain audio signal into an energy density map (EDM), and analyze its characteristics; Step 2: Analyze the impact of equipment differences on EDM characteristics; Step 3: Construct an EDM dataset based on hybrid data sources; Step 4: Based on the feature analysis of EDM and combined with the spatial attention mechanism, a single-base station distance perception network DPNet based on deep learning method is constructed.

2. The distance perception method based on a single audio Chirp signal feature as claimed in claim 1, characterized in that: The Chirp signal used in step 1 is expressed as: Where A(t) is the adjustable signal amplitude, T is the signal duration, f0 and f e are the start frequency and cut-off frequency of the signal respectively.

3. The distance perception method based on a single audio Chirp signal feature as claimed in claim 1, characterized in that: In step 1, the audio signal is preprocessed by short-time Fourier transform (STFT) with a window length of l and an overlap rate of k, and a Chirp signal time-frequency diagram with a pixel size of R×C is calculated. The numerical value of each frequency point represents the signal energy level, so the time-frequency diagram is called energy density diagram (EDM).

4. The distance perception method based on a single audio Chirp signal feature as claimed in claim 1, characterized in that: The relationship between EDM and perception distance in step 2 is affected by both the distance from the terminal to the base station and the differences in the responses of different terminals to signals. The former is reflected in the different changes in the overall energy of the signal represented by EDM at different positions in space, and the latter is reflected in the different signal frequency responses and sensitivities of different sampling devices at the same position. The absolute logarithm of the pixel value of each frequency point in EDM is defined as the relative energy size of the audio signal at each frequency point, and the relative energy range of EDM is defined as the response domain. The domain value of this domain can intuitively reflect the above two differences.

5. The distance perception method based on a single audio Chirp signal feature as claimed in claim 1, characterized in that: In step 2, the difference between different devices is controlled to a maximum level of ±1m based on the maximum value normalization method; the EDM is normalized according to the following formula: Among them, i and j are the row and column index numbers of EDM respectively. It represents the normalized energy of the frequency point, and its range is [0, 1]. The stronger the signal energy, the larger the value of the corresponding pixel.

6. The distance perception method based on a single audio Chirp signal feature as claimed in claim 1, characterized in that: In step 3, a high-quality dataset containing enough difference elements is constructed, which is labeled with different physical reference distances in meters. The single-base station audio Chirp signal is collected in two different typical indoor scenes: corridor and office, and finally processed into normalized EDM data.

7. The distance perception method based on a single audio Chirp signal feature as claimed in claim 1, characterized in that: The architecture of the single-base-station distance perception network DPNet in step 4 adaptively perceives the EDM image features that can represent the law of spatial distance change in the form of residual attention by stacking two spatial attention modules; DPNet adopts a "sandwich" structure of preamble, main body, and tail suffix, and the core component is the pre-activated bottleneck residual block; In the leading part, a 7×7 convolution layer and a 3×3 maximum pooling layer are first used to quickly reduce the dimension of the feature map and increase its receptive field; The main part consists of two spatial attention modules and a connected residual block. The number of output channels of each connected residual block is doubled to infer deeper features. In the spatial attention module, the mask branch uses the same number of residual blocks as the trunk branch, and the number of input and output channels of all residual blocks is consistent. Since the mask branch contains more pooling layers, its receptive field is larger than that of the trunk branch, which generates denser features in the process of back-propagating the network to update the gradient to infer the weight of each pixel for the perceived distance. For the suffix part, an adaptive average pooling layer is used to fix the feature map size to 1×1, and it is sent to the fully connected layer for network output.

8. The distance perception method based on a single audio Chirp signal feature as claimed in claim 7, characterized in that: In DPNet, each attention module consists of a trunk branch and a mask branch. The trunk branch is used to process network features and increase the receptive field; the mask branch adds soft weights to the features, and controls the flow of weight information to various objects through bottom-up feedforward and top-down feedback; when the output of the mask branch normalized by the Sigmoid layer covers the trunk branch, the input and output of the attention module are connected with a secondary residual connection.

9. The distance perception method based on a single audio Chirp signal feature as claimed in claim 8, characterized in that: During the entire model training process, the input EDM is gradually decomposed into a set of smaller and smaller pixels through the feedforward structure, generating a low-resolution feature tensor with strong semantic information; among them, the spatial attention module with the largest depth outputs a feature map of size 3×5, which is the smallest unit that can fully reflect the local characteristics of the signal at the perceptual distance.

10. A distance perception system based on a single audio Chirp signal feature of a distance perception method based on a single audio Chirp signal feature as claimed in any one of claims 1 to 8, characterized in that: include: The data preprocessing module performs data preprocessing on the original data stream of the single base station audio Chirp signal, converts the time domain audio signal into an energy density map (EDM), and analyzes its characteristics; Normalization module, analyzes the impact of equipment differences on EDM characteristics; EDM dataset construction module, which constructs EDM dataset based on hybrid data sources; The single-base station distance perception network building module is based on the feature analysis of EDM and combined with the spatial attention mechanism to construct a single-base station distance perception network DPNet based on deep learning methods.

Images