Property service response method and system based on voice robot

Abstract

The present invention discloses a property service response method and system based on a voice robot, relating to the field of speech recognition technology. The method comprises collecting voice signals and generating spectrogram features, generating keyword category predictions based on the spectrogram features, and performing signal screening based on the prediction results; locating the signal source direction based on the screening results, extracting the target voice signal, performing voice enhancement on the target voice signal, converting the voice into text, performing semantic analysis on the text, and generating user instructions. The present invention improves signal acquisition accuracy through voice signal acquisition and adaptive noise suppression using a multi-channel microphone array. Based on an improved spectrogram feature generation mechanism, the method enhances the feature expression capability of voice signals, significantly improving service response speed and accuracy.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to a property service response method and system based on a speech robot. Background Art

[0002] Voice interaction technology, a key research area in human-computer interaction, has experienced rapid development in recent years. With the gradual maturity of deep learning, natural language processing, and speech recognition technologies, voice interaction has been widely adopted in scenarios such as smart homes, customer service systems, and industrial control. In this context, speech recognition algorithms have evolved from traditional hidden Markov models and dynamic time warping to highly efficient models such as deep neural networks, recurrent neural networks, transformers, and transformers. These models have significantly improved the accuracy of speech recognition and semantic analysis. Simultaneously, advances in hardware technology have made multi-microphone arrays a key component supporting real-time sound source localization and signal enhancement, providing the hardware foundation for multi-scenario voice-based applications. In the property management sector, traditional operations rely primarily on manual operation, resulting in low service efficiency, particularly in emergency response situations, where rapid response is difficult. In recent years, intelligent robots and voice assistants have been gradually introduced into this field to achieve intelligent and automated services. However, the application of existing voice robots in property management still faces numerous challenges due to complex environmental noise, unstable voice signal quality, and limitations in semantic understanding. Summary of the Invention

[0003] In view of the problems existing in the above-mentioned existing property service response methods and systems based on voice robots, the present invention is proposed.

[0004] Therefore, the problem to be solved by the present invention is that due to the complex environmental noise, unstable voice signal quality and limitations of semantic understanding, the application of existing voice robots in property services still faces many challenges.

[0005] To solve the above technical problems, the present invention provides the following technical solutions: a property service response method based on a voice robot, comprising collecting voice signals and generating spectrogram features, generating keyword category predictions based on the spectrogram features, and performing signal screening based on the prediction results;

[0006] Based on the screening results, the direction of the signal sound source is located, the target voice signal is extracted, the target voice signal is enhanced, the voice is converted into text, the text is semantically analyzed, and user instructions are generated;

[0007] Transmit user instructions to property management personnel for execution.

[0008] As a preferred embodiment of the voice robot-based property service response method of the present invention, the method of collecting voice signals and generating spectrogram features comprises collecting voice signals using a 6-channel microphone array, attaching a timestamp to the recording of each channel signal, and encoding the signals into mono PCM, performing clock synchronization correction on the collected multi-channel signals, outputting a multi-channel synchronization signal matrix X(t), and performing preliminary processing on the ambient noise using an adaptive noise suppression algorithm to obtain a noise-reduced multi-channel signal matrix X1(x);

[0009] Perform fast Fourier transform on multi-channel signals to generate the spectrum of each channel;

[0010] The spectrum information of all channels is combined to generate a comprehensive spectrum S(f), and the energy peak in the spectrum is detected based on the amplitude of the comprehensive spectrum;

[0011] Based on the energy peak and frequency interval, the frequency bands are automatically divided and the frequency band boundaries are smoothed, and a bandpass filter is designed for each segment:

[0012]

[0013] Where f is the frequency component, u m is the lower bound of the current segment frequency interval, u m+1 is the upper bound of the current segment frequency interval, ξ is the bandwidth adjustment coefficient, β is the smoothing factor, ψ m (f) is the filter value;

[0014] The integrated spectrum is passed through each bandpass filter to extract the segmented signal, and each segmented signal is subjected to inverse Fourier transform to restore the time domain signal. Based on the segmented signal, the empirical wavelet feature is generated;

[0015] Generate an empirical wavelet matrix based on the empirical wavelet features, normalize the empirical wavelet feature matrix, and discretize the normalized feature matrix;

[0016] Assign the discretized feature values ​​to the pattern set, calculate the pattern probability, and calculate the reverse discrete entropy RDE of each segment feature based on the pattern probability;

[0017] Set a threshold Q, select feature segments with inverse discrete entropy lower than the threshold Q as candidate feature segments, and perform time-frequency reconstruction on the candidate feature segments using time domain S transform;

[0018] The time-frequency reconstruction results of each segment are arranged into a two-dimensional matrix. After the two-dimensional matrix is ​​normalized, the feature matrix is ​​reduced in dimension using principal component analysis. The reduced feature vector is remapped into a two-dimensional spectrum form to obtain the spectrum feature matrix.

[0019] As a preferred embodiment of the voice robot-based property service response method of the present invention, generating keyword category predictions based on spectrogram features and performing signal screening based on the prediction results refers to adjusting the dimension of the spectrogram feature matrix by linear interpolation, and inputting the adjusted spectrogram feature matrix into the first convolutional module of the MobileNetV2 network to extract local features;

[0020] Use the depthwise separable convolution of the MobileNetV2 network to perform independent convolution operations on the features of each channel, divide the convolved feature matrix into l blocks, calculate the attention weight of each block separately, and calculate the query vector, key vector, and value vector for the elements in each block;

[0021] The similarity between the query and the key is calculated by dot product, and then the attention distribution is obtained by Softmax normalization;

[0022] Use the attention distribution weighted value vector to generate updated features, flatten the updated features and input them into the fully connected layer. Apply the Softmax function to the output of the fully connected layer to generate the probability distribution of keyword categories.

[0023] According to the probability distribution of keyword categories, extract the confidence value corresponding to each keyword category and find the keyword category index corresponding to the confidence value;

[0024] Set a threshold K. If the keyword confidence is greater than or equal to the threshold K, the signal is marked as "target signal" and the keyword category and corresponding confidence are recorded. Otherwise, the signal is marked as "background noise".

[0025] As a preferred solution of the property service response method based on the voice robot of the present invention, wherein: the positioning signal sound source direction based on the screening result refers to calculating the time delay R between the microphone pairs using the generalized cross-correlation method according to the target keyword signal. vl (ε);

[0026] Calculate the time delay value of each pair of microphones one by one, find the time delay corresponding to the peak of the cross-correlation function, divide the directional space according to the geometric arrangement of the microphone array, and for each direction b, use the SRP-PHAT method to calculate the power response P(b) of the target signal in the spatial direction based on the time delay results;

[0027] The direction corresponding to the maximum value is selected from the directional power response and output as the target sound source direction.

[0028] As a preferred embodiment of the voice robot-based property service response method of the present invention, the method of extracting the target voice signal and performing voice enhancement on the target voice signal refers to using the time delay information of the target sound source direction, aligning the multi-channel signals in time, and using delay-sum beamforming to enhance the voice signal y[n] in the target direction;

[0029] The noise spectrum of the current frame is dynamically estimated using spectral subtraction, and the noise spectrum is used to enhance the target signal to obtain the enhanced speech signal.

[0030] As a preferred solution of the voice robot-based property service response method of the present invention, wherein: converting speech into text, performing semantic analysis on the text, and generating user instructions refers to slicing the spectral features corresponding to the enhanced speech signal by time frame, constructing feature blocks with a length of W frames, and constructing a slicing feature matrix based on the sliced ​​feature blocks;

[0031] The slice feature matrix is ​​input into the Transformer model. A multi-head self-attention mechanism is used to extract the global contextual relationship of the time series to obtain a new slice feature matrix. The slice feature matrix is ​​then subjected to nonlinear feature transformation through a feedforward neural network. Multiple self-attention layers and feedforward network layers are stacked in the Transformer model. Each layer models the input features. In the last layer of the Transformer model, a linear classification head is used to map the features to a vocabulary. The Softmax function is used to normalize the vocabulary scores to a probability distribution. Greedy decoding is used to select the token with the highest probability at each time step. The generated tokens are then concatenated into a complete text sequence based on the time step. Duplicate tokens in the text sequence are cleaned and padded.

[0032] Use natural language processing tools to segment text into key semantic units and filter out stop words. Then, use the Conformer model embedding layer to convert the segmented text into a fixed-dimensional vector representation. Use a convolutional network to extract local semantic features of the text. Use a multi-head self-attention mechanism to capture the contextual relationships between long-range words in the text. Stack multiple Conformer layers to deeply model semantic features, capturing textual relationships and implicit semantics to generate a semantic feature vector.

[0033] Based on the semantic feature vector, the user's operation type and operation object are determined by the classifier to obtain the user's intention and goal. The parsing results are generated into a standardized structured data format to obtain the user's instructions. If the semantic parsing fails to extract the instructions, an error message is returned and the user is prompted to re-enter.

[0034] As a preferred solution of the voice robot-based property service response method described in the present invention, the method of transmitting user instructions to property management personnel for execution refers to encapsulating the obtained user instruction information into a structured data packet, and transmitting the task data packet to the property management system using the HTTP / HTTPS protocol or a message queue. According to the task type and location, the task is assigned to the corresponding property management personnel. After the task is completed, the management personnel submits execution feedback through the mobile terminal and stores the feedback information in the database.

[0035] Another object of the present invention is to provide a property service response system based on a voice robot, which includes:

[0036] Voice acquisition module, used to collect user voice signals and perform clock synchronization correction;

[0037] Feature generation module, used to analyze the speech signal and generate a spectrogram feature matrix;

[0038] The signal screening module is used to analyze the feature matrix of the spectrum graph, identify keywords and screen the target signal in the speech signal;

[0039] A positioning enhancement module is used to locate the sound source based on the target signal and enhance the speech signal in the target direction;

[0040] Semantic analysis module, which is used to analyze the enhanced speech signal to generate a text sequence and generate user instructions by determining the user's intention and goal through a classifier;

[0041] The instruction execution module is used to allocate tasks according to the generated user instructions.

[0042] A computer device comprises: a memory and a processor; the memory stores a computer program, and the processor implements the steps of a property service response method based on a voice robot when executing the computer program.

[0043] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a property service response method based on a voice robot.

[0044] The beneficial effects of the present invention are as follows: the present invention improves the signal acquisition accuracy through voice signal acquisition and adaptive noise suppression of a multi-channel microphone array, enhances the feature expression ability of the voice signal based on an improved spectrum graph feature generation mechanism, combines the attention mechanism and sound source localization algorithm in deep learning, strengthens the robustness of keyword extraction and sound source localization, enhances the target voice signal through beamforming and spectral subtraction, and finally realizes semantic parsing and user intent extraction based on the Transformer and Conformer models, and quickly distributes them to the property management system, thereby significantly improving the service response speed and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 The figure is a flowchart of a property service response method based on a voice robot.

[0047] Figure 2 This is a structural diagram of the property service response system based on voice robots. DETAILED DESCRIPTION

[0048] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0049] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0050] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it constitute a separate or selective embodiment that is mutually exclusive with other embodiments.

[0051] Example 1, with reference to Figure 1 , which is the first embodiment of the present invention, provides a property service response method based on a voice robot. The property service response method based on a voice robot includes:

[0052] S1. Collect voice signals and generate spectrogram features, generate keyword category predictions based on the spectrogram features, and perform signal screening based on the prediction results;

[0053] Specifically, collecting speech signals and generating spectrogram features involves using a 6-channel microphone array to collect speech signals, attaching a timestamp to the recording of each channel signal, and encoding it into mono PCM. Clock synchronization correction is performed on the collected multi-channel signals to eliminate the time delay caused by the difference in microphone positions, and the multi-channel synchronous signal matrix X(t) is output. The ambient noise is preliminarily processed using an adaptive noise suppression algorithm to obtain the denoised multi-channel signal matrix X1(t):

[0054] x1(t)=x(t)-αn(t)

[0055] Where x1(t) is the denoised multi-channel signal, x(t) is the original signal, n(t) is the background noise estimate, and α is the noise suppression weight, which is dynamically adjusted by the minimum mean square error method.

[0056] Perform a fast Fourier transform on a multichannel signal to generate a spectrum for each channel:

[0057]

[0058] Where S i (f) is the spectral feature matrix of each channel, X i (t) is the time domain signal, f is the frequency, t is the time, and j is the imaginary unit;

[0059] Combine the spectrum information of all channels to generate the comprehensive spectrum S(f). Based on the amplitude of the comprehensive spectrum, detect the energy peak in the spectrum:

[0060]

[0061] Where P(f) is the frequency point corresponding to the energy peak in the spectrum, Indicates a maximum value search for the variable f, where f is the frequency component;

[0062] Based on the energy peak and frequency interval, the frequency bands are automatically divided and the frequency band boundaries are smoothed, and a bandpass filter is designed for each segment:

[0063]

[0064] Where f is the frequency component, u m is the lower bound of the current segment frequency interval, u m+1 is the upper bound of the current segment frequency interval, ξ is the bandwidth adjustment coefficient, β is the smoothing factor, ψ m (f) is the filter value, (1-ξ)u m≤|f|≤(1+ξ)u m+1 It means f is within the main frequency band of the current filter, (1+ξ)u m <|f|<(1-ξ)u m+1 It means f is located in the transition region of the filter, |f|<(1-ξ)u m or|f|>(1+ξ)u m+1 It means that f is outside the main frequency band and transition region of the filter;

[0065] In the existing technology, EWT decomposes signals by constructing bandpass filters that adapt to the signal's spectral distribution. The boundaries of these filters are usually determined by the signal's spectral energy distribution. In property service scenarios, the frequency distribution of voice signals may be relatively wide (including background noise, interference frequencies, etc.), and the spectrum segmentation needs to be adaptively adjusted according to different scenarios. The filter boundary adjustment and the smoothness of the transition region directly affect the accuracy of the segmentation results. In this solution, the filter boundary is increased by a bandwidth adjustment coefficient ξ, allowing dynamic adjustment at the fundamental frequency boundary. The original transition region only relies on a single cosine function. After improvement, the width and strength of the transition region are controlled by adding a smoothing factor β to adapt to the needs of multiple scenarios.

[0066] The integrated spectrum is passed through each bandpass filter to extract the segmented signal, and each segmented signal is inverse Fourier transformed to restore the time domain signal. Based on the segmented signal, the empirical wavelet feature is generated:

[0067]

[0068] Where W f (f,t) is the piecewise wavelet feature matrix, x'(t) is the time domain signal, is the complex conjugate of the empirical wavelet filter, t is the current time point, and τ is the central time point of the empirical wavelet filter;

[0069] Generate an empirical wavelet matrix based on the empirical wavelet features, normalize the empirical wavelet feature matrix, and discretize the normalized feature matrix:

[0070]

[0071] Where z(i) is the discretized eigenvalue after normalization, W n (f,t) is the spectrum feature matrix, min(W n ) is W n The minimum value of all elements in (f, t), max(W n ) is W n The maximum value of all elements in (f,t);

[0072] Assign the discretized eigenvalues ​​to the pattern set and calculate the pattern probability P(pi ):

[0073]

[0074] In the formula, c(p i ) is the pattern p i The number of occurrences, N is the total number of all patterns;

[0075] Calculate the reverse discrete entropy RDE of each segment feature based on the pattern probability:

[0076]

[0077] Where, P(p i ) is the discrete mode p i The probability of M is the total number of discrete modes;

[0078] A statistical analysis is performed on the historical RDE values ​​to set a threshold Q, and feature segments with reverse discrete entropy lower than the threshold Q are selected as candidate feature segments. The time domain S transform is used to reconstruct the candidate feature segments in time and frequency to enhance their time and frequency resolution:

[0079]

[0080] Where ST(τ,f) is the signal component strength at time τ and frequency component f, and W n (t) is the normalized value of the time domain signal;

[0081] The time-frequency reconstruction results of each segment are arranged into a two-dimensional matrix. After the two-dimensional matrix is ​​normalized, the feature matrix is ​​reduced in dimension using principal component analysis. The reduced feature vector is remapped into a two-dimensional spectrum form to obtain the spectrum feature matrix.

[0082] The 6-channel microphone array design enables multi-channel signal acquisition, providing spatial information and redundant data support, while clock synchronization correction ensures time alignment of signals across all channels. The generation of a multi-channel signal matrix facilitates subsequent spatial signal processing and precise localization of the sound source. This process significantly improves the robustness of speech signal processing, particularly in noisy and echoy environments. The adaptive noise suppression algorithm dynamically adjusts noise reduction parameters to significantly reduce ambient noise interference, laying the foundation for high-quality signals for spectral analysis. The Fourier transform converts time-domain signals into frequency-domain signals, providing the foundation for subsequent spectral information processing and energy distribution analysis. By generating spectrograms and extracting integrated spectra, the system identifies the primary energy distribution within the signal, enhancing its ability to express characteristic features. The frequency band division method based on energy peaks and frequency intervals avoids discontinuities at the band boundaries through smoothing, improving the rationality of the band division. The bandpass filter design extracts features within a specific frequency range for each segmented signal. This refined processing of the segmented signal enhances the representation of the target signal and suppresses out-of-band noise. The empirical wavelet features capture local signal variations and frequency information at different scales, accurately describing the detailed characteristics of different speech components in the speech signal. Normalization of the empirical wavelet matrix eliminates the impact of signal amplitude differences on subsequent analysis, making the features more dimensional and uniform, facilitating discretization and pattern probability calculation. Feature segments are screened using the RDE value, ensuring that subsequent processing focuses only on candidate feature segments with high information content and low redundancy, thereby reducing computational resource waste. The time-domain S-transform performs time-frequency reconstruction on the selected feature segments, more comprehensively integrating the signal's time-varying characteristics with frequency information, providing high-quality input for subsequent principal component analysis and dimensionality reduction. Principal component analysis (PCA) preserves the key information of the spectral feature matrix during dimensionality reduction, removes redundant features, and reduces the computational burden of the model. The generation of a two-dimensional spectrogram feature matrix provides rich and representative feature representation for subsequent keyword recognition and speech signal screening, improving the recognition and processing efficiency of the entire system.

[0083] Furthermore, generating keyword category predictions based on spectrogram features and performing signal screening based on the prediction results involves adjusting the dimension of the spectrogram feature matrix through linear interpolation, and inputting the adjusted spectrogram feature matrix into the first convolutional module of the MobileNetV2 network to extract local features;

[0084] Adjusting the dimensions of the spectrogram feature matrix through linear interpolation avoids feature loss caused by direct cropping or stretching of spectral features, improves the compatibility of the feature matrix with the MobileNetV2 network input, and reduces error accumulation in the preprocessing stage.

[0085] Use the depthwise separable convolution of the MobileNetV2 network to perform independent convolution operations on the features of each channel, divide the convolved feature matrix into l blocks, calculate the attention weight of each block separately, and calculate the query vector, key vector, and value vector for the elements in each block;

[0086] The similarity between the query and the key is calculated by dot product, and then the attention distribution is obtained by Softmax normalization;

[0087] Use the attention distribution weighted value vector to generate updated features, flatten the updated features and input them into the fully connected layer. Apply the Softmax function to the output of the fully connected layer to generate the probability distribution of keyword categories.

[0088] The features of MobileNetV2 significantly reduce the computational complexity of the model, making it suitable for resource-constrained embedded devices. The segmented two-dimensional self-attention mechanism enhances the ability to capture features in the time and frequency dimensions, improving the accuracy and robustness of keyword recognition.

[0089] Dynamically adjust feature importance through the attention mechanism, enhance the feature expression related to keywords, reduce the interference of redundant information and background noise, and optimize the signal screening effect;

[0090] According to the probability distribution of keyword categories, extract the confidence value corresponding to each keyword category and find the keyword category index corresponding to the confidence value;

[0091] The threshold K is set through ROC curve analysis. If the keyword confidence is greater than or equal to the threshold K, the signal is marked as "target signal" and the keyword category and corresponding confidence are recorded. Otherwise, the signal is marked as "background noise";

[0092] Screening target signals according to confidence improves the accuracy of target signal screening, significantly reduces the false recognition rate, reduces the interference of background noise, and provides high-quality input signals for subsequent sound source localization and speech enhancement.

[0093] By innovatively combining a deep learning network (MobileNetV2) with an attention mechanism, the system effectively addresses issues such as insufficient feature extraction and insufficient robustness of signal screening in existing speech recognition systems. Precise feature processing and confidence screening significantly improve the accuracy of keyword recognition and the system's noise resistance. Furthermore, the optimized signal screening results provide high-quality input for subsequent sound source localization and speech enhancement, demonstrating superior performance in multiple scenarios. This system not only offers advantages in computational efficiency and resource consumption, but also demonstrates strong applicability, meeting the high-performance requirements of multi-task voice interaction and noisy environments, providing solid technical support for the application of voice interaction technology in areas such as property management, smart homes, and public services.

[0094] S2. Locate the direction of the signal source based on the screening results, extract the target voice signal, perform voice enhancement on the target voice signal, convert the voice into text, perform semantic analysis on the text, and generate user instructions;

[0095] Specifically, locating the direction of the signal sound source based on the screening results means calculating the time delay R between the microphone pairs using the generalized cross-correlation method based on the target keyword signal. vl (ε):

[0096]

[0097] Where T is the total duration of the signal, x v (t) is the target signal received by microphone v at time t, ε is the time delay, x l (t+ε) is the target signal received by microphone l at time t+ε;

[0098] Calculate the time delay value of each pair of microphones one by one, find the time delay corresponding to the peak of the cross-correlation function, divide the directional space according to the geometric arrangement of the microphone array, and for each direction b, combine the time delay results and use the SRP-PHAT method to calculate the power response P(b) of the target signal in the spatial direction:

[0099]

[0100] Where H is the total number of microphones, R vl (τ vl (b)) is the cross-correlation value of microphone pair v,l in direction b, σ vl is the standard deviation of the cross-correlation of the microphone pair v,l;

[0101] The direction corresponding to the maximum value is selected from the directional power response and output as the target sound source direction.

[0102] Through keyword recognition, high-confidence target signals are screened out, background noise and irrelevant signals are eliminated, and the accuracy of subsequent calculations is ensured. Only the direction of the target signal is estimated, which significantly reduces the computational complexity and improves the real-time performance of the system. The time delay data of multiple microphone pairs are used to comprehensively calculate the sound source direction, reducing the possible errors of a single microphone pair. The weighted average of multiple pairs of time delays is used to reduce the interference of environmental noise and echo on the calculation. Through direction division, the power calculation range is limited, and the calculation time of invalid directions is reduced. The angular interval of the direction division is matched with the geometric structure of the microphone array to maximize the accuracy of direction estimation. The influence of environmental noise and multipath interference is reduced through phase normalization (PHAT). The data of multiple pairs of microphones are accumulated to improve the confidence of the sound source direction estimation. The direction of maximum power response usually corresponds to the true position of the target signal, ensuring the accuracy of sound source positioning. Directly selecting the maximum value simplifies the subsequent analysis process and meets real-time requirements.

[0103] Furthermore, extracting the target speech signal and performing speech enhancement on the target speech signal means using the time delay information of the target sound source direction to align the multi-channel signals in time and using delay-sum beamforming to enhance the speech signal y[n] in the target direction:

[0104]

[0105] Where x m is the time domain signal received by the mth microphone, s m is the time delay in the target direction, H is the number of microphones, and n is the sampling point index in discrete time;

[0106] The noise spectrum of the current frame is dynamically estimated using spectral subtraction, and the noise spectrum is used to enhance the target signal to obtain the enhanced speech signal.

[0107] Aligning multi-channel signals using time delay information ensures that the target speech signal is phase-aligned across the multi-channel data. This not only enhances the target signal's energy but also significantly suppresses interference noise and multipath effects from other directions. Beamforming leverages the principles of spatial filtering to effectively boost the target speech signal's energy while simultaneously reducing noise and interference from other directions. By enhancing the target signal and suppressing signals from other directions, the signal-to-noise ratio (SNR) is significantly improved, enabling more accurate subsequent speech processing (such as speech recognition and semantic parsing). Spectral subtraction dynamically estimates the noise spectrum of each frame and adjusts the noise suppression strength in real time based on environmental changes, providing consistent noise reduction in changing environments. Smoothing and spectrum preservation minimize speech signal distortion, preserving the clarity and naturalness of the target speech signal. The combined use of beamforming and spectral subtraction significantly enhances the output speech signal's clarity, making it suitable for subsequent speech-to-text and semantic analysis modules. The enhanced signal reduces the computational complexity of the subsequent speech recognition model and improves the overall system's responsiveness.

[0108] S3, assigning task priorities and planning task execution paths based on user instructions;

[0109] Specifically, the speech is converted into text, the text is semantically analyzed, and the user instruction is generated by slicing the spectral features corresponding to the enhanced speech signal by time frame, constructing feature blocks with a length of W frames, and constructing a slicing feature matrix based on the slicing feature blocks;

[0110] The slice feature matrix is ​​input into the Transformer model. A multi-head self-attention mechanism is used to extract the global contextual relationship of the time series to obtain a new slice feature matrix. The slice feature matrix is ​​then subjected to nonlinear feature transformation through a feedforward neural network. Multiple self-attention layers and feedforward network layers are stacked in the Transformer model. Each layer models the input features. In the last layer of the Transformer model, a linear classification head is used to map the features to a vocabulary. The Softmax function is used to normalize the vocabulary scores to a probability distribution. Greedy decoding is used to select the token with the highest probability at each time step. The generated tokens are then concatenated into a complete text sequence based on the time step. Duplicate tokens in the text sequence are cleaned and padded.

[0111] Use natural language processing tools to segment text into key semantic units and filter out stop words. Then, use the Conformer model embedding layer to convert the segmented text into a fixed-dimensional vector representation. Use a convolutional network to extract local semantic features of the text. Use a multi-head self-attention mechanism to capture the contextual relationships between long-range words in the text. Stack multiple Conformer layers to deeply model semantic features, capturing textual relationships and implicit semantics to generate a semantic feature vector.

[0112] Based on the semantic feature vector, the user's operation type and operation object are determined by the classifier to obtain the user's intention and goal. The parsing results are generated into a standardized structured data format to obtain the user's instructions. If the semantic parsing fails to extract the instructions, an error message is returned and the user is prompted to re-enter.

[0113] Time frame slicing preserves the time series characteristics of speech signals, enabling the model to better capture dynamic changes in speech signals. The slicing feature matrix represents frame features through multidimensional vectors, helping subsequent models capture inter-frame relationships. The Transformer captures global contextual relationships through a multi-head self-attention mechanism, making the modeling of long-distance dependencies in speech signals more accurate. The feedforward neural network provides nonlinear transformations to the feature matrix, enhancing feature expression. Greedy decoding significantly improves decoding efficiency by gradually selecting the highest-probability tokens. Cleaning up duplicate and padding tokens generates more coherent and readable text sequences. Text segmentation and stop word filtering reduce invalid information and improve the accuracy of semantic parsing. The Conformer model combines local convolutional feature extraction with a global self-attention mechanism for more comprehensive semantic feature modeling. Standardized data formats ensure the compatibility and portability of user commands across different systems.

[0114] Furthermore, transmitting the user instructions to the property management personnel for execution means encapsulating the obtained user instruction information into a structured data packet, transmitting the task data packet to the property management system using the HTTP / HTTPS protocol or message queue, and assigning the task to the corresponding property management personnel based on the task type and location. After the task is completed, the management personnel submit execution feedback through the mobile terminal and store the feedback information in the database.

[0115] Encapsulating user instructions as unstructured data packets can improve the flexibility of data transmission and avoid information loss caused by format restrictions. Unstructured data packets support a variety of content formats, such as text, timestamps, geographic locations, etc., which can meet the complex needs of different tasks. The use of HTTP / HTTPS protocols ensures the stability and security of task data packet transmission. In particular, the encryption mechanism of HTTPS effectively prevents sensitive data from being intercepted or tampered with. The message queue can realize asynchronous transmission, reduce the system pressure of synchronous task scheduling, and avoid performance bottlenecks in high-concurrency environments. By parsing user instructions, the system can automatically identify the type of task (such as equipment maintenance, security inspection) and location, thereby assigning the task to the most suitable manager.

[0116] Example 2, reference Figure 2 , which is the second embodiment of the present invention, is different from the previous embodiment and provides a property service response system based on a voice robot, which includes:

[0117] Voice acquisition module, used to collect user voice signals and perform clock synchronization correction;

[0118] Feature generation module, used to analyze the speech signal and generate a spectrogram feature matrix;

[0119] The signal screening module is used to analyze the feature matrix of the spectrum graph, identify keywords and screen the target signal in the speech signal;

[0120] A positioning enhancement module is used to locate the sound source based on the target signal and enhance the speech signal in the target direction;

[0121] Semantic analysis module, which is used to analyze the enhanced speech signal to generate a text sequence and generate user instructions by determining the user's intention and goal through a classifier;

[0122] The instruction execution module is used to allocate tasks according to the generated user instructions.

[0123] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0124] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0125] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0126] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

Claims

1. A property service response method based on a voice robot, characterized by: include, Speech signals are collected and spectrogram features are generated. Based on the spectrogram features, property keyword category predictions are generated using the MobileNetV2 network and the attention mechanism. Signals are then filtered based on the prediction results, with the "target signals" whose confidence level is higher than a threshold K being selected. Locating the direction of the signal sound source based on the screening results, including performing SRP-PHAT directional power response calculation only on the speech segment containing the target keyword, selecting the direction corresponding to the maximum value from the directional power response as the target sound source direction output, extracting the target speech signal, performing speech enhancement on the target speech signal and converting the speech into text, performing semantic analysis on the text, and generating user instructions, including segmenting the spectral features corresponding to the enhanced speech signal into slices according to time frames and inputting them into the Transformer model to generate new text, segmenting the text and inputting it into the Conformer model to capture text relationships and implicit semantics to obtain a semantic feature vector, determining the user's operation type and operation object to obtain the user's intention and goal and generate user instructions; Transmit user instructions to property management personnel for execution; The collecting of speech signals and generating spectrogram features refers to collecting speech signals using a 6-channel microphone array, attaching a timestamp to the recording of each channel signal, and encoding it into mono PCM, performing clock synchronization correction on the collected multi-channel signals, outputting a multi-channel synchronization signal matrix X(t), and performing preliminary processing on the ambient noise through an adaptive noise suppression algorithm to obtain a noise-reduced multi-channel signal matrix X1(t); Perform fast Fourier transform on multi-channel signals to generate the spectrum of each channel; The spectrum information of all channels is combined to generate a comprehensive spectrum S(f), and the energy peak in the spectrum is detected based on the amplitude of the comprehensive spectrum; Automatically divide the frequency bands according to the energy peak and frequency interval, smooth the frequency band boundaries, and design a bandpass filter for each segment; The integrated spectrum is passed through each bandpass filter to extract the segmented signal, and each segmented signal is subjected to inverse Fourier transform to restore the time domain signal. Based on the segmented signal, the empirical wavelet feature is generated; Generate an empirical wavelet matrix based on the empirical wavelet features, normalize the empirical wavelet feature matrix, and discretize the normalized feature matrix; Assign the discretized feature values ​​to the pattern set, calculate the pattern probability, and calculate the reverse discrete entropy RDE of each segment feature based on the pattern probability; Set a threshold Q, select feature segments with inverse discrete entropy lower than the threshold Q as candidate feature segments, and perform time-frequency reconstruction on the candidate feature segments using time domain S transform; The time-frequency reconstruction results of each segment are arranged into a two-dimensional matrix. After the two-dimensional matrix is ​​normalized, the feature matrix is ​​reduced in dimension using principal component analysis. The reduced feature vector is remapped into a two-dimensional spectrum form to obtain the spectrum feature matrix.

2. The property service response method based on a voice robot according to claim 1, characterized in that: The bandpass filter designed for each segment is expressed as: Where f is the frequency component, u m is the lower bound of the current segment frequency interval, u m+1 is the upper bound of the current segment frequency interval, ξ is the bandwidth adjustment coefficient, β is the smoothing factor, ψ m (f) is the filter value.

3. The property service response method based on a voice robot according to claim 2, characterized in that: Generating keyword category predictions based on spectrogram features and performing signal screening based on the prediction results refers to adjusting the dimension of the spectrogram feature matrix by linear interpolation, and inputting the adjusted spectrogram feature matrix into the first convolution module of the MobileNetV2 network to extract local features; Use the depthwise separable convolution of the MobileNetV2 network to perform independent convolution operations on the features of each channel, divide the convolved feature matrix into l blocks, calculate the attention weight of each block separately, and calculate the query vector, key vector, and value vector for the elements in each block; The similarity between the query and the key is calculated by dot product, and then the attention distribution is obtained by Softmax normalization; Use the attention distribution weighted value vector to generate updated features, flatten the updated features and input them into the fully connected layer. Apply the Softmax function to the output of the fully connected layer to generate the probability distribution of keyword categories. According to the probability distribution of keyword categories, extract the confidence value corresponding to each keyword category and find the keyword category index corresponding to the confidence value; Set a threshold K. If the keyword confidence is greater than or equal to the threshold K, the signal is marked as "target signal" and the keyword category and corresponding confidence are recorded. Otherwise, the signal is marked as "background noise".

4. The property service response method based on a voice robot according to claim 3, characterized in that: The positioning of the signal sound source direction based on the screening result refers to calculating the time delay R between the microphone pairs using the generalized cross-correlation method according to the target keyword signal. vl (ε); Calculate the time delay value of each pair of microphones one by one, find the time delay corresponding to the peak of the cross-correlation function, divide the directional space according to the geometric arrangement of the microphone array, and for each direction b, use the SRP-PHAT method to calculate the power response P(b) of the target signal in the spatial direction based on the time delay results; The direction corresponding to the maximum value is selected from the directional power response and output as the target sound source direction.

5. The property service response method based on a voice robot according to claim 4, characterized in that: Extracting the target speech signal and performing speech enhancement on the target speech signal refers to using the time delay information of the target sound source direction to align the multi-channel signals in time and using delay-sum beamforming to enhance the speech signal y[n] in the target direction; The noise spectrum of the current frame is dynamically estimated using spectral subtraction, and the noise spectrum is used to enhance the target signal to obtain the enhanced speech signal.

6. The property service response method based on a voice robot according to claim 5, characterized in that: Converting speech to text, performing semantic analysis on the text, and generating user instructions refers to slicing the spectral features corresponding to the enhanced speech signal by time frame, constructing feature blocks with a length of W frames, and constructing a slicing feature matrix based on the slicing feature blocks; The slice feature matrix is ​​input into the Transformer model. A multi-head self-attention mechanism is used to extract the global contextual relationship of the time series to obtain a new slice feature matrix. The slice feature matrix is ​​then subjected to nonlinear feature transformation through a feedforward neural network. Multiple self-attention layers and feedforward network layers are stacked in the Transformer model. Each layer models the input features. In the last layer of the Transformer model, a linear classification head is used to map the features to a vocabulary. The Softmax function is used to normalize the vocabulary scores to a probability distribution. Greedy decoding is used to select the token with the highest probability at each time step. The generated tokens are then concatenated into a complete text sequence based on the time step. Duplicate tokens in the text sequence are cleaned and padded. Use natural language processing tools to segment text into key semantic units and filter out stop words. Then, use the Conformer model embedding layer to convert the segmented text into a fixed-dimensional vector representation. Use a convolutional network to extract local semantic features of the text. Use a multi-head self-attention mechanism to capture the contextual relationships between long-range words in the text. Stack multiple Conformer layers to deeply model semantic features, capturing textual relationships and implicit semantics to generate a semantic feature vector. Based on the semantic feature vector, the user's operation type and operation object are determined by the classifier to obtain the user's intention and goal. The parsing results are generated into a standardized structured data format to obtain the user's instructions. If the semantic parsing fails to extract the instructions, an error message is returned and the user is prompted to re-enter.

7. The property service response method based on a voice robot according to claim 6, characterized in that: The transmitting of user instructions to property management personnel for execution refers to encapsulating the obtained user instruction information into a structured data packet, transmitting the task data packet to the property management system using the HTTP / HTTPS protocol or message queue, and assigning the task to the corresponding property management personnel based on the task type and location. After the task is completed, the management personnel submits execution feedback through the mobile terminal and stores the feedback information in the database.

8. A voice robot-based property service response system based on the voice robot-based property service response method according to any one of claims 1 to 7, characterized in that: include, Voice acquisition module, used to collect user voice signals and perform clock synchronization correction; Feature generation module, used to analyze the speech signal and generate a spectrogram feature matrix; The signal screening module is used to analyze the feature matrix of the spectrum graph, identify keywords and screen the target signal in the speech signal; A positioning enhancement module is used to locate the sound source based on the target signal and enhance the speech signal in the target direction; Semantic analysis module, which is used to analyze the enhanced speech signal to generate a text sequence and generate user instructions by determining the user's intention and goal through a classifier; The instruction execution module is used to allocate tasks according to the generated user instructions.

9. A computer device comprising: memory and processor; The memory stores a computer program, characterized in that when the processor executes the computer program, the steps of the property service response method based on a voice robot described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the property service response method based on a voice robot according to any one of claims 1 to 7 are implemented.

Images