Property service response method and system based on voice robot

Abstract

The invention discloses a property service response method and system based on a voice robot, and relates to the technical field of voice recognition, and the method comprises the steps: collecting a voice signal, generating a spectrogram feature, generating keyword category prediction based on the spectrogram feature, and carrying out the signal screening according to a prediction result; and positioning a signal sound source direction based on a screening result, extracting a target voice signal, performing voice enhancement on the target voice signal, converting the voice into a text, performing semantic analysis on the text, and generating a user instruction. Through voice signal acquisition and adaptive noise suppression of the multi-channel microphone array, the signal acquisition precision is improved, based on an improved spectrogram feature generation mechanism, the feature expression ability of voice signals is improved, and the service response speed and accuracy are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to a property service response method and system based on a speech robot. Background Art

[0002] As an important research direction in the field of human-computer interaction, voice interaction technology has developed rapidly in recent years. With the gradual maturity of deep learning, natural language processing and speech recognition technology, voice interaction has been widely used in scenarios such as smart homes, customer service systems and industrial control. In this context, speech recognition algorithms have gradually evolved from traditional hidden Markov models and dynamic time warping to efficient models represented by deep neural networks, recurrent neural networks, Transformers and Conformers, which have greatly improved the accuracy of speech recognition and semantic analysis. At the same time, the advancement of hardware technology has made multi-microphone arrays a key device to support real-time sound source localization and signal enhancement, providing a hardware foundation for multi-scenario applications based on voice. In the field of property services, the traditional working mode mainly relies on manual operation, and the service efficiency is low, especially in emergency tasks, it is difficult to achieve rapid response. In recent years, intelligent robots and voice assistants have been gradually introduced into this field to realize the intelligence and automation of services. However, due to the complex environmental noise, unstable voice signal quality and limitations of semantic understanding, the application of existing voice robots in property services still faces many challenges. Summary of the invention

[0003] In view of the problems existing in the above-mentioned existing property service response method and system based on voice robot, the present invention is proposed.

[0004] Therefore, the problem to be solved by the present invention is that due to the complex environmental noise, unstable voice signal quality and limitations of semantic understanding, the application of existing voice robots in property services still faces many challenges.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: a property service response method based on a voice robot, which includes collecting voice signals and generating spectrogram features, generating keyword category predictions based on the spectrogram features and performing signal screening according to the prediction results;

[0006] Based on the screening results, the direction of the signal sound source is located, the target voice signal is extracted, the target voice signal is enhanced, the voice is converted into text, the text is semantically analyzed, and user instructions are generated;

[0007] Transmit user instructions to property management personnel for execution.

[0008] As a preferred solution of the property service response method based on a voice robot described in the present invention, wherein: the collecting of voice signals and generating spectrogram features refers to using a 6-channel microphone array to collect voice signals, attaching a timestamp to the record of each channel signal, and encoding it into a mono PCM, performing clock synchronization correction on the collected multi-channel signals, outputting a multi-channel synchronization signal matrix X(t), and performing preliminary processing on the environmental noise through an adaptive noise suppression algorithm to obtain a multi-channel signal matrix X1(x) after noise reduction;

[0009] Perform fast Fourier transform on multi-channel signals to generate the spectrum of each channel;

[0010] The spectrum information of all channels is combined to generate a comprehensive spectrum S(f), and the energy peak in the spectrum is detected based on the amplitude of the comprehensive spectrum;

[0011] According to the energy peak and frequency interval, the frequency bands are automatically divided and the frequency band boundaries are smoothed, and a bandpass filter is designed for each segment:

[0012]

[0013] Where f is the frequency component, u m is the lower bound of the current segment frequency interval, u m+1 is the upper bound of the current segment frequency interval, ξ is the bandwidth adjustment coefficient, β is the smoothing factor, ψ m (f) is the filter value;

[0014] The integrated spectrum is passed through each bandpass filter to extract segmented signals, each segmented signal is inversely Fourier transformed to restore the time domain signal, and based on the segmented signal, the empirical wavelet feature is generated;

[0015] Generate an empirical wavelet matrix based on the empirical wavelet feature, normalize the empirical wavelet feature matrix, and discretize the normalized feature matrix;

[0016] Assign the discretized feature values ​​to the pattern set, calculate the pattern probability, and calculate the reverse discrete entropy RDE of each feature segment based on the pattern probability;

[0017] Set a threshold Q, select feature segments with inverse discrete entropy lower than the threshold Q as candidate feature segments, and perform time-frequency reconstruction on the candidate feature segments using time domain S transform;

[0018] The time-frequency reconstruction results of each segment are arranged into a two-dimensional matrix. After the two-dimensional matrix is ​​normalized, the principal component analysis is used to reduce the dimension of the feature matrix. The reduced feature vector is remapped into a two-dimensional spectrum form to obtain the spectrum feature matrix.

[0019] As a preferred solution of the property service response method based on a voice robot of the present invention, wherein: the generating keyword category prediction based on the spectrogram features and performing signal screening according to the prediction results refers to adjusting the dimension of the spectrogram feature matrix by linear interpolation, and inputting the adjusted spectrogram feature matrix into the first layer convolution module of the MobileNetV2 network to extract local features;

[0020] Use the deep separable convolution of the MobileNetV2 network to perform independent convolution operations on the features of each channel, divide the convolved feature matrix into l blocks, calculate the attention weight of each block separately, and calculate the query vector, key vector, and value vector for the elements in each block;

[0021] The similarity between query and key is calculated by dot product, and then the attention distribution is obtained by Softmax normalization;

[0022] Use the attention distribution weighted value vector to generate updated features, flatten the updated features and input them into the fully connected layer, apply the Softmax function to the output of the fully connected layer to generate the probability distribution of keyword categories;

[0023] According to the probability distribution of keyword categories, the confidence value corresponding to each keyword category is extracted, and the keyword category index corresponding to the confidence value is found;

[0024] Set a threshold K. If the keyword confidence is greater than or equal to the threshold K, the signal is marked as a "target signal" and the keyword category and the corresponding confidence are recorded. Otherwise, the signal is marked as "background noise".

[0025] As a preferred solution of the property service response method based on voice robot of the present invention, wherein: the positioning signal sound source direction based on the screening result refers to calculating the time delay R between the microphone pairs using the generalized cross-correlation method according to the target keyword signal. vl (ε);

[0026] Calculate the time delay value of each pair of microphones one by one, find the time delay corresponding to the peak of the cross-correlation function, divide the directional space according to the geometric arrangement of the microphone array, and for each direction b, combine the time delay results and use the SRP-PHAT method to calculate the power response P(b) of the target signal in the spatial direction;

[0027] The direction corresponding to the maximum value is selected from the directional power response and output as the target sound source direction.

[0028] As a preferred solution of the property service response method based on a voice robot of the present invention, wherein: the said extracting the target voice signal and performing voice enhancement on the target voice signal refers to using the time delay information of the target sound source direction, aligning the multi-channel signals in time, and using delay-sum beamforming to enhance the voice signal y[n] in the target direction;

[0029] The noise spectrum of the current frame is dynamically estimated using spectral subtraction, and the noise spectrum is used to enhance the target signal to obtain an enhanced speech signal.

[0030] As a preferred solution of the property service response method based on a voice robot of the present invention, wherein: the converting of voice into text, performing semantic analysis on the text, and generating user instructions refers to slicing the spectral features corresponding to the enhanced voice signal according to time frames, constructing feature blocks with a length of W frames, and constructing a slicing feature matrix based on the slicing feature blocks;

[0031] The slice feature matrix is ​​input into the Transformer model. The multi-head self-attention mechanism is used to extract the global contextual relationship of the time series to obtain a new slice feature matrix. The slice feature matrix is ​​transformed nonlinearly through a feedforward neural network. Multiple self-attention layers and feedforward network layers are stacked in the Transformer model. Each layer models the input features. In the last layer of the Transformer model, the features are mapped to the vocabulary through a linear classification head. The vocabulary scores are normalized to probability distributions using the Softmax function. Greedy decoding is used to select the token with the highest probability at each time step. The generated tokens are spliced ​​into a complete text sequence according to the time step. Duplicate tokens in the text sequence are cleaned and filled with tokens.

[0032] Use natural language processing tools to segment text into key semantic units and filter stop words. Convert the segmented text into a fixed-dimensional vector representation through the Conformer model embedding layer. Use a convolutional network to extract local semantic features of the text. Use a multi-head self-attention mechanism to capture the contextual relationship of long-distance words in the text. Stack multiple Conformer layers to perform deep modeling of semantic features, capture text relationships and implicit semantics, and obtain a semantic feature vector.

[0033] Based on the semantic feature vector, the user's operation type and operation object are determined by the classifier to obtain the user's intention and goal. The parsing result is generated into a standardized structured data format to obtain the user's instructions. If the semantic parsing fails to extract the instructions, an error message is returned and the user is prompted to re-enter.

[0034] As a preferred solution of the voice robot-based property service response method described in the present invention, the transmitting of user instructions to property management personnel for execution refers to encapsulating the obtained user instruction information into a structured data packet, transmitting the task data packet to the property management system using the HTTP / HTTPS protocol or a message queue, and assigning the task to the corresponding property management personnel according to the task type and location. After the task is completed, the management personnel submits execution feedback through the mobile terminal and stores the feedback information in the database.

[0035] Another object of the present invention is to provide a property service response system based on a voice robot, which comprises:

[0036] Voice collection module, used to collect user voice signals and perform clock synchronization correction;

[0037] A feature generation module is used to analyze the speech signal and generate a spectrum feature matrix;

[0038] The signal screening module is used to analyze the feature matrix of the spectrum graph and screen the target signal in the speech signal after identifying the keywords;

[0039] A positioning enhancement module is used to locate the sound source based on the target signal and enhance the speech signal in the target direction;

[0040] Semantic analysis module, used to analyze the enhanced speech signal to generate a text sequence and generate user instructions by determining the user's intention and goal through a classifier;

[0041] The instruction execution module is used to distribute tasks according to the generated user instructions.

[0042] A computer device includes: a memory and a processor; the memory stores a computer program, and the processor implements the steps of a property service response method based on a voice robot when executing the computer program.

[0043] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a property service response method based on a voice robot.

[0044] The beneficial effects of the present invention are as follows: the present invention improves the signal acquisition accuracy through voice signal acquisition and adaptive noise suppression of a multi-channel microphone array, improves the feature expression ability of voice signals based on an improved spectrum graph feature generation mechanism, combines the attention mechanism and sound source localization algorithm in deep learning, strengthens the robustness of keyword extraction and sound source localization, enhances the target voice signal through beamforming and spectral subtraction, and finally realizes semantic parsing and user intent extraction based on Transformer and Conformer models, and quickly distributes them to the property management system, thereby significantly improving the service response speed and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0046] Figure 1 The figure is a flowchart of a property service response method based on a voice robot.

[0047] Figure 2 This is a structural diagram of a property service response system based on a voice robot. DETAILED DESCRIPTION

[0048] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0049] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0050] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.

[0051] Example 1, reference Figure 1 , which is the first embodiment of the present invention, and which provides a property service response method based on a voice robot. The property service response method based on a voice robot includes:

[0052] S1, collect speech signals and generate spectrogram features, generate keyword category predictions based on the spectrogram features, and perform signal screening based on the prediction results;

[0053] Specifically, collecting speech signals and generating spectrogram features refers to using a 6-channel microphone array to collect speech signals, attaching a timestamp to the record of each channel signal, and encoding it into mono PCM, performing clock synchronization correction on the collected multi-channel signals, eliminating the time delay caused by the difference in microphone positions, and outputting a multi-channel synchronization signal matrix X(t). The environmental noise is preliminarily processed by an adaptive noise suppression algorithm to obtain a multi-channel signal matrix X1(t) after noise reduction:

[0054] x1(t)=x(t)-αn(t)

[0055] Where x1(t) is the multi-channel signal after denoising, x(t) is the original signal, n(t) is the background noise estimate, and α is the noise suppression weight, which is dynamically adjusted by the minimum mean square error method;

[0056] Perform a fast Fourier transform on a multichannel signal to generate a spectrum for each channel:

[0057]

[0058] In the formula, S i (f) is the spectral feature matrix of each channel, X i (t) is the time domain signal, f is the frequency, t is the time, and j is the imaginary unit;

[0059] The spectrum information of all channels is combined to generate a comprehensive spectrum S(f). Based on the amplitude of the comprehensive spectrum, the energy peak in the spectrum is detected:

[0060]

[0061] Where P(f) is the frequency point corresponding to the energy peak in the spectrum, It means to search for the maximum value of variable f, where f is the frequency component;

[0062] According to the energy peak and frequency interval, the frequency bands are automatically divided and the frequency band boundaries are smoothed, and a bandpass filter is designed for each segment:

[0063]

[0064] Where f is the frequency component, u m is the lower bound of the current segment frequency interval, u m+1 is the upper bound of the current segment frequency interval, ξ is the bandwidth adjustment coefficient, β is the smoothing factor, ψ m (f) is the filter value, (1-ξ)u m≤|f|≤(1+ξ)u m+1 It means that f is within the main frequency band of the current filter, (1+ξ)u m <|f|<(1-ξ)u m+1 It means f is in the transition region of the filter, |f|<(1-ξ)u m or|f|>(1+ξ)u m+1 It means that f is outside the main frequency band and transition region of the filter;

[0065] In the prior art, EWT decomposes signals by constructing bandpass filters that adapt to the signal spectrum distribution. The boundaries of these filters are usually determined by the spectral energy distribution of the signal. In the property service scenario, the frequency distribution of the voice signal may be relatively wide (including background noise, interference frequency, etc.), and the spectrum segmentation needs to be adaptively adjusted according to different scenarios. The boundary adjustment of the filter and the smoothness of the transition area directly affect the accuracy of the segmentation results. The boundary of the filter in this solution adds a bandwidth adjustment coefficient ξ, allowing dynamic adjustment on the basic frequency boundary. The original transition area only relies on a single cosine function. After improvement, the width and strength of the transition area are controlled by increasing the smoothing factor β to adapt to the needs of multiple scenarios.

[0066] The integrated spectrum is passed through each bandpass filter to extract the segmented signal, and each segmented signal is inversely Fourier transformed to restore the time domain signal. Based on the segmented signal, the empirical wavelet feature is generated:

[0067]

[0068] Where W f (f, t) is the piecewise wavelet feature matrix, x'(t) is the time domain signal, is the complex conjugate of the empirical wavelet filter, t is the current time point, and τ is the central time point of the empirical wavelet filter;

[0069] Generate an empirical wavelet matrix based on the empirical wavelet features, normalize the empirical wavelet feature matrix, and discretize the normalized feature matrix:

[0070]

[0071] Where z(i) is the discretized eigenvalue after normalization, W n (f, t) is the spectral feature matrix, min(W n ) is W n The minimum value of all elements in (f, t), max(W n ) is W n The maximum value of all elements in (f,t);

[0072] Assign the discretized eigenvalues ​​to the pattern set and calculate the pattern probability P(pi ):

[0073]

[0074] In the formula, c(p i ) is the pattern p i The number of occurrences, N is the total number of all patterns;

[0075] The reverse discrete entropy RDE is calculated for each segment feature based on the pattern probability:

[0076]

[0077] In the formula, P(p i ) is the discrete mode p i The probability of, M is the total number of discrete modes;

[0078] The historical RDE values ​​are statistically analyzed to set the threshold Q, and the feature segments whose reverse discrete entropy is lower than the threshold Q are selected as candidate feature segments. The time domain S transform is used to reconstruct the candidate feature segments in time and frequency to enhance their time and frequency resolution:

[0079]

[0080] Where ST(τ,f) is the signal component strength at time τ and frequency component f, and W n (t) is the normalized value of the time domain signal;

[0081] The time-frequency reconstruction results of each segment are arranged into a two-dimensional matrix. After the two-dimensional matrix is ​​normalized, the principal component analysis is used to reduce the dimension of the feature matrix. The reduced feature vector is remapped into a two-dimensional spectrum form to obtain the spectrum feature matrix.

[0082] Through the design of a 6-channel microphone array, multi-channel signal acquisition provides spatial information and redundant data support, while clock synchronization correction ensures the time alignment of each channel signal. The generation of a multi-channel signal matrix is ​​helpful for subsequent spatial signal processing and precise positioning of the sound source direction. This process significantly improves the robustness of speech signal processing, especially in noise interference and echo environments. The adaptive noise suppression algorithm significantly reduces the interference of environmental noise by dynamically adjusting the noise reduction parameters, laying a high-quality signal foundation for spectrum analysis. The Fourier transform can convert time domain signals into frequency domain signals, providing a basis for subsequent spectrum information processing and energy distribution analysis. Through the generation of spectrum graphs and the extraction of comprehensive spectra, the system can identify the main energy distribution in the signal and improve the signal's feature expression ability. The frequency band division method based on energy peaks and frequency intervals avoids the discontinuity of frequency band boundaries through smoothing processing and improves the rationality of frequency band division. The design of the bandpass filter extracts the features of a specific frequency range for each segmented signal. This refined processing of the segmented signal enhances the representation ability of the target signal and suppresses the noise interference outside the frequency band. The empirical wavelet feature captures the local change trend of the signal and the frequency information of different scales, and can accurately describe the detailed features of different speech components in the speech signal. The normalization of the empirical wavelet matrix eliminates the influence of the signal amplitude difference on the subsequent analysis, making the features more unified in dimension, which facilitates the discretization allocation and pattern probability calculation. The feature segments are screened by the RDE value to ensure that the subsequent processing only focuses on the candidate feature segments with high information content and low redundancy, thereby reducing the waste of computing resources. The time domain S transform reconstructs the time-frequency of the screened feature segments, more comprehensively integrates the time-varying characteristics of the signal with the frequency information, and provides high-quality input for the subsequent principal component analysis and dimensionality reduction. The principal component analysis (PCA) retains the main information of the spectrum feature matrix in the dimensionality reduction process, removes redundant features, and reduces the computational burden of the model. The generation of a two-dimensional spectrogram feature matrix provides rich and representative feature representation for subsequent keyword recognition and speech signal screening, improving the recognition and processing efficiency of the entire system.

[0083] Further, generating keyword category prediction based on the spectrogram features and performing signal screening according to the prediction results refers to adjusting the dimension of the spectrogram feature matrix by linear interpolation, and inputting the adjusted spectrogram feature matrix into the first convolution module of the MobileNetV2 network to extract local features;

[0084] The dimension of the feature matrix of the spectrum graph is adjusted through linear interpolation to avoid feature loss caused by direct cropping or stretching of spectrum features, improve the adaptability of the feature matrix to the MobileNetV2 network input, and reduce error accumulation in the preprocessing stage;

[0085] Use the deep separable convolution of the MobileNetV2 network to perform independent convolution operations on the features of each channel, divide the convolved feature matrix into l blocks, calculate the attention weight of each block separately, and calculate the query vector, key vector, and value vector for the elements in each block;

[0086] The similarity between query and key is calculated by dot product, and then the attention distribution is obtained by Softmax normalization;

[0087] Use the attention distribution weighted value vector to generate updated features, flatten the updated features and input them into the fully connected layer, apply the Softmax function to the output of the fully connected layer to generate the probability distribution of keyword categories;

[0088] The characteristics of MobileNetV2 significantly reduce the computational complexity of the model, making it suitable for resource-constrained embedded devices. The segmented two-dimensional self-attention mechanism enhances the feature capture capability in the time and frequency dimensions, improving the accuracy and robustness of keyword recognition.

[0089] Dynamically adjust feature importance through the attention mechanism, enhance the feature expression related to keywords, reduce the interference of redundant information and background noise, and optimize the signal screening effect;

[0090] According to the probability distribution of keyword categories, the confidence value corresponding to each keyword category is extracted, and the keyword category index corresponding to the confidence value is found;

[0091] The threshold K is set through ROC curve analysis. If the keyword confidence is greater than or equal to the threshold K, the signal is marked as "target signal" and the keyword category and corresponding confidence are recorded. Otherwise, the signal is marked as "background noise";

[0092] Screening target signals according to confidence improves the accuracy of target signal screening, significantly reduces the misrecognition rate, reduces the interference of background noise, and provides high-quality input signals for subsequent sound source localization and speech enhancement.

[0093] By innovatively combining the deep learning network (MobileNetV2) with the attention mechanism, the existing speech recognition system effectively solves the problems of insufficient feature extraction and insufficient robustness of signal screening. Through precise feature processing and confidence screening, the accuracy of keyword recognition and the system's anti-noise ability are significantly improved. In addition, the optimized signal screening results provide high-quality input for subsequent sound source localization and speech enhancement, thereby demonstrating excellent performance in multiple scenarios. It not only has advantages in computing efficiency and resource consumption, but also demonstrates strong applicability, which can meet the high-performance requirements of multi-task voice interaction and noisy environments, and provides solid technical support for the application of voice interaction technology in property management, smart homes, and public services.

[0094] S2, based on the screening results, locate the direction of the signal sound source, extract the target voice signal, perform voice enhancement on the target voice signal, convert the voice into text, perform semantic analysis on the text, and generate user instructions;

[0095] Specifically, locating the signal sound source direction based on the screening result means calculating the time delay R between the microphone pairs using the generalized cross-correlation method according to the target keyword signal. vl (ε):

[0096]

[0097] Where T is the total duration of the signal, x v (t) is the target signal received by microphone v at time t, ε is the time delay, x l (t+ε) is the target signal received by microphone l at time t+ε;

[0098] Calculate the time delay value of each pair of microphones one by one, find the time delay corresponding to the peak of the cross-correlation function, divide the directional space according to the geometric arrangement of the microphone array, and for each direction b, combine the time delay results and use the SRP-PHAT method to calculate the power response P(b) of the target signal in the spatial direction:

[0099]

[0100] Where H is the total number of microphones, R vl (τ vl (b)) is the cross-correlation value of microphone pair v,l in direction b, σ vl is the standard deviation of the cross-correlation of the microphone pair v,l;

[0101] The direction corresponding to the maximum value is selected from the directional power response and output as the target sound source direction.

[0102] Through keyword recognition, high-confidence target signals are screened out, background noise and irrelevant signals are eliminated to ensure the accuracy of subsequent calculations. Only the direction of the target signal is estimated, which significantly reduces the calculation complexity and improves the real-time performance of the system. The time delay data of multiple microphone pairs are used to comprehensively calculate the sound source direction to reduce the possible errors of a single microphone pair. The weighted average of multiple pairs of time delays is used to reduce the interference of environmental noise and echo on the calculation. Through direction division, the power calculation range is limited to reduce the calculation time of invalid directions. The angular interval of the direction division is matched with the geometric structure of the microphone array to maximize the accuracy of direction estimation. The impact of environmental noise and multipath interference is reduced through phase normalization (PHAT). The data of multiple pairs of microphones are accumulated to improve the confidence of the sound source direction estimation. The direction of maximum power response usually corresponds to the true position of the target signal, ensuring the accuracy of sound source positioning. Directly selecting the maximum value simplifies the subsequent analysis process and meets real-time requirements.

[0103] Furthermore, the target speech signal is extracted and speech enhancement is performed on the target speech signal, which means using the time delay information of the target sound source direction, aligning the multi-channel signals in time, and using delay-sum beamforming to enhance the speech signal y[n] in the target direction:

[0104]

[0105] In the formula, x m is the time domain signal received by the mth microphone, s m is the time delay in the target direction, H is the number of microphones, and n is the sampling point index in discrete time;

[0106] The noise spectrum of the current frame is dynamically estimated using spectral subtraction, and the noise spectrum is used to enhance the target signal to obtain an enhanced speech signal.

[0107] By aligning multi-channel signals with time delay information, the phase of the speech signal in the target direction can be ensured in the multi-channel data. This not only enhances the energy of the target signal, but also significantly suppresses the interference noise and multipath effect in other directions. Beamforming uses the principle of spatial filtering to effectively enhance the energy of the speech signal in the target direction while reducing the noise and interference in other directions. By enhancing the target direction signal and suppressing the signal in other directions, the signal-to-noise ratio is significantly improved, making subsequent speech processing (such as speech recognition and semantic parsing) more accurate. Spectral subtraction can dynamically estimate the noise spectrum of each frame and adjust the noise suppression intensity in real time according to environmental changes, thereby providing consistent noise reduction effects in a changing environment. Through smoothing and spectrum retention mechanisms, the distortion of the speech signal is minimized, and the clarity and naturalness of the target speech signal are retained. Through the combined use of beamforming and spectral subtraction, the clarity of the output speech signal is significantly improved, which is suitable for subsequent speech-to-text and semantic analysis module processing. The enhanced signal reduces the computational complexity of the subsequent speech recognition model and improves the response speed of the overall system.

[0108] S3, assigning task priorities and planning task execution paths based on user instructions;

[0109] Specifically, the speech is converted into text, the text is semantically analyzed, and the user instruction is generated by slicing the spectral features corresponding to the enhanced speech signal according to the time frame, constructing a feature block with a length of W frames, and constructing a slicing feature matrix based on the slicing feature blocks;

[0110] The slice feature matrix is ​​input into the Transformer model. The multi-head self-attention mechanism is used to extract the global contextual relationship of the time series to obtain a new slice feature matrix. The slice feature matrix is ​​transformed nonlinearly through a feedforward neural network. Multiple self-attention layers and feedforward network layers are stacked in the Transformer model. Each layer models the input features. In the last layer of the Transformer model, the features are mapped to the vocabulary through a linear classification head. The vocabulary scores are normalized to probability distributions using the Softmax function. Greedy decoding is used to select the token with the highest probability at each time step. The generated tokens are spliced ​​into a complete text sequence according to the time step. Duplicate tokens in the text sequence are cleaned and filled with tokens.

[0111] Use natural language processing tools to segment text into key semantic units and filter stop words. Convert the segmented text into a fixed-dimensional vector representation through the Conformer model embedding layer. Use a convolutional network to extract local semantic features of the text. Use a multi-head self-attention mechanism to capture the contextual relationship of long-distance words in the text. Stack multiple Conformer layers to perform deep modeling of semantic features, capture text relationships and implicit semantics, and obtain a semantic feature vector.

[0112] Based on the semantic feature vector, the user's operation type and operation object are determined by the classifier to obtain the user's intention and goal. The parsing result is generated into a standardized structured data format to obtain the user's instructions. If the semantic parsing fails to extract the instructions, an error message is returned and the user is prompted to re-enter.

[0113] Time frame slicing preserves the time series characteristics of speech signals, enabling the model to better capture dynamic changes in speech signals. The slicing feature matrix represents frame features through multi-dimensional vectors, which helps subsequent models capture inter-frame relationships. Transformer captures global contextual relationships through a multi-head self-attention mechanism, making the modeling of long-distance dependencies in speech signals more accurate. Feedforward neural networks provide nonlinear transformations for feature matrices and enhance feature expression capabilities. Greedy decoding significantly improves decoding efficiency by gradually selecting the highest probability tags. Cleaning up duplicate tags and padding tags can generate more coherent and readable text sequences. Text segmentation and stop word filtering reduce invalid information and improve the accuracy of semantic parsing. The Conformer model combines local convolutional feature extraction and global self-attention mechanisms to make semantic feature modeling more comprehensive. Standardized data formats ensure the compatibility and portability of user instructions between different systems.

[0114] Furthermore, transmitting the user instructions to the property management personnel for execution means encapsulating the obtained user instruction information into a structured data packet, transmitting the task data packet to the property management system using the HTTP / HTTPS protocol or a message queue, and assigning the task to the corresponding property management personnel based on the task type and location. After the task is completed, the management personnel submits execution feedback through the mobile terminal and stores the feedback information in the database.

[0115] Encapsulating user instructions as unstructured data packets can improve the flexibility of data transmission and avoid information loss caused by format restrictions. Unstructured data packets support a variety of content forms, such as text, timestamps, geographic locations, etc., which can meet the complex requirements of different tasks. The use of HTTP / HTTPS protocol ensures the stability and security of task data packet transmission. In particular, the encryption mechanism of HTTPS effectively prevents sensitive data from being intercepted or tampered with. The message queue can realize asynchronous transmission, reduce the system pressure of synchronous task scheduling, and avoid performance bottlenecks in high-concurrency environments. By parsing user instructions, the system can automatically identify the type of task (such as equipment maintenance, security inspection) and location, thereby assigning tasks to the most suitable managers.

[0116] Example 2, reference Figure 2 , which is the second embodiment of the present invention, which is different from the previous embodiment and provides a property service response system based on a voice robot, which includes:

[0117] Voice collection module, used to collect user voice signals and perform clock synchronization correction;

[0118] A feature generation module is used to analyze the speech signal and generate a spectrum feature matrix;

[0119] The signal screening module is used to analyze the feature matrix of the spectrum graph and screen the target signal in the speech signal after identifying the keywords;

[0120] A positioning enhancement module is used to locate the sound source based on the target signal and enhance the speech signal in the target direction;

[0121] Semantic analysis module, used to analyze the enhanced speech signal to generate a text sequence and generate user instructions by determining the user's intention and goal through a classifier;

[0122] The instruction execution module is used to distribute tasks according to the generated user instructions.

[0123] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0124] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0125] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0126] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

Claims

1. A property service response method based on a voice robot, characterized in that: include, Collect speech signals and generate spectrogram features, generate keyword category predictions based on the spectrogram features, and perform signal screening based on the prediction results; Based on the screening results, the direction of the signal sound source is located, the target voice signal is extracted, the target voice signal is enhanced, the voice is converted into text, the text is semantically analyzed, and user instructions are generated; Transmit user instructions to property management personnel for execution.

2. The property service response method based on a voice robot according to claim 1, characterized in that: The collecting of speech signals and generating spectrogram features refers to collecting speech signals using a 6-channel microphone array, attaching a timestamp to the record of each channel signal, and encoding it into mono PCM, performing clock synchronization correction on the collected multi-channel signals, outputting a multi-channel synchronization signal matrix X(t), and performing preliminary processing on the environmental noise through an adaptive noise suppression algorithm to obtain a multi-channel signal matrix X1(t) after noise reduction; Perform fast Fourier transform on multi-channel signals to generate the spectrum of each channel; The spectrum information of all channels is combined to generate a comprehensive spectrum S(f), and the energy peak in the spectrum is detected based on the amplitude of the comprehensive spectrum; According to the energy peak and frequency interval, the frequency bands are automatically divided and the frequency band boundaries are smoothed, and a bandpass filter is designed for each segment: Where f is the frequency component, u m is the lower bound of the current segment frequency interval, u m+1 is the upper bound of the current segment frequency interval, ξ is the bandwidth adjustment coefficient, β is the smoothing factor, ψ m (f) is the filter value; The integrated spectrum is passed through each bandpass filter to extract segmented signals, each segmented signal is inversely Fourier transformed to restore the time domain signal, and based on the segmented signal, the empirical wavelet feature is generated; Generate an empirical wavelet matrix based on the empirical wavelet feature, normalize the empirical wavelet feature matrix, and discretize the normalized feature matrix; Assign the discretized feature values ​​to the pattern set, calculate the pattern probability, and calculate the reverse discrete entropy RDE of each feature segment based on the pattern probability; Set a threshold Q, select feature segments with inverse discrete entropy lower than the threshold Q as candidate feature segments, and perform time-frequency reconstruction on the candidate feature segments using time domain S transform; The time-frequency reconstruction results of each segment are arranged into a two-dimensional matrix. After the two-dimensional matrix is ​​normalized, the principal component analysis is used to reduce the dimension of the feature matrix. The reduced feature vector is remapped into a two-dimensional spectrum form to obtain the spectrum feature matrix.

3. The property service response method based on a voice robot as claimed in claim 2, characterized in that: Generating keyword category prediction based on spectrogram features and performing signal screening according to the prediction results refers to adjusting the dimension of the spectrogram feature matrix by linear interpolation, and inputting the adjusted spectrogram feature matrix into the first convolution module of the MobileNetV2 network to extract local features; Use the deep separable convolution of the MobileNetV2 network to perform independent convolution operations on the features of each channel, divide the convolved feature matrix into l blocks, calculate the attention weight of each block separately, and calculate the query vector, key vector, and value vector for the elements in each block; The similarity between query and key is calculated by dot product, and then the attention distribution is obtained by Softmax normalization; Use the attention distribution weighted value vector to generate updated features, flatten the updated features and input them into the fully connected layer, apply the Softmax function to the output of the fully connected layer to generate the probability distribution of keyword categories; According to the probability distribution of keyword categories, the confidence value corresponding to each keyword category is extracted, and the keyword category index corresponding to the confidence value is found; Set a threshold K. If the keyword confidence is greater than or equal to the threshold K, the signal is marked as "target signal" and the keyword category and corresponding confidence are recorded. Otherwise, the signal is marked as "background noise".

4. The property service response method based on a voice robot as claimed in claim 3, characterized in that: The signal source direction based on the screening result is to calculate the time delay R between the microphone pairs using the generalized cross-correlation method according to the target keyword signal. vl (ε); Calculate the time delay value of each pair of microphones one by one, find the time delay corresponding to the peak of the cross-correlation function, divide the directional space according to the geometric arrangement of the microphone array, and for each direction b, combine the time delay results and use the SRP-PHAT method to calculate the power response P(b) of the target signal in the spatial direction; The direction corresponding to the maximum value is selected from the directional power response and output as the target sound source direction.

5. The property service response method based on a voice robot according to claim 4, characterized in that: Extracting the target speech signal and performing speech enhancement on the target speech signal refers to using the time delay information of the target sound source direction to align the multi-channel signals in time and using delay-sum beamforming to enhance the speech signal y[n] in the target direction; The noise spectrum of the current frame is dynamically estimated using spectral subtraction, and the noise spectrum is used to enhance the target signal to obtain an enhanced speech signal.

6. The property service response method based on a voice robot according to claim 5, characterized in that: The converting speech into text, performing semantic analysis on the text, and generating user instructions refers to slicing the spectral features corresponding to the enhanced speech signal according to time frames, constructing feature blocks with a length of W frames, and constructing a slicing feature matrix based on the slicing feature blocks; The slice feature matrix is ​​input into the Transformer model. The multi-head self-attention mechanism is used to extract the global contextual relationship of the time series to obtain a new slice feature matrix. The slice feature matrix is ​​transformed nonlinearly through a feedforward neural network. Multiple self-attention layers and feedforward network layers are stacked in the Transformer model. Each layer models the input features. In the last layer of the Transformer model, the features are mapped to the vocabulary through a linear classification head. The vocabulary scores are normalized to probability distributions using the Softmax function. Greedy decoding is used to select the token with the highest probability at each time step. The generated tokens are spliced ​​into a complete text sequence according to the time step. Duplicate tokens in the text sequence are cleaned and filled with tokens. Use natural language processing tools to segment text into key semantic units and filter stop words. Convert the segmented text into a fixed-dimensional vector representation through the Conformer model embedding layer. Use a convolutional network to extract local semantic features of the text. Use a multi-head self-attention mechanism to capture the contextual relationship of long-distance words in the text. Stack multiple Conformer layers to perform deep modeling of semantic features, capture text relationships and implicit semantics, and obtain a semantic feature vector. Based on the semantic feature vector, the user's operation type and operation object are determined by the classifier to obtain the user's intention and goal. The parsing result is generated into a standardized structured data format to obtain the user's instructions. If the semantic parsing fails to extract the instructions, an error message is returned and the user is prompted to re-enter.

7. The property service response method based on a voice robot according to claim 6, characterized in that: The transmitting of user instructions to property management personnel for execution refers to encapsulating the obtained user instruction information into a structured data packet, transmitting the task data packet to the property management system using the HTTP / HTTPS protocol or a message queue, and assigning the task to the corresponding property management personnel based on the task type and location. After the task is completed, the management personnel submits execution feedback through the mobile terminal and stores the feedback information in the database.

8. A voice robot-based property service response system based on the voice robot-based property service response method according to any one of claims 1 to 7, characterized in that: include, Voice collection module, used to collect user voice signals and perform clock synchronization correction; A feature generation module is used to analyze the speech signal and generate a spectrum feature matrix; The signal screening module is used to analyze the feature matrix of the spectrum graph and screen the target signal in the speech signal after identifying the keywords; A positioning enhancement module is used to locate the sound source based on the target signal and enhance the speech signal in the target direction; Semantic analysis module, used to analyze the enhanced speech signal to generate a text sequence and generate user instructions by determining the user's intention and goal through a classifier; The instruction execution module is used to distribute tasks according to the generated user instructions.

9. A computer device comprising: Memory and processor; The memory stores a computer program, characterized in that when the processor executes the computer program, the steps of the property service response method based on a voice robot described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the property service response method based on a voice robot described in any one of claims 1 to 7 are implemented.

Images