Building equipment lightweight intelligent auscultation system fused with model distillation
By using model distillation technology in building equipment fault diagnosis, high-precision teacher model knowledge is migrated to lightweight models, the low accuracy problem caused by limited computing capabilities of edge equipment is solved, and efficient and real-time fault diagnosis is achieved.
Patent Information
- Application Number
- CN202510185837.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-20
- Publication Date
- 2025-06-10
AI Technical Summary
In complex built environments, the computing power of edge devices is limited, resulting in lightweight models with low accuracy in building equipment fault diagnosis and cannot meet high-precision requirements.
The fusion model distillation technology is used to migrate high-precision teacher model knowledge to lightweight student models, and real-time fault diagnosis is achieved by deploying optimized lightweight models on edge devices.
While ensuring real-time performance, it improves the accuracy and computing efficiency of building equipment fault diagnosis, and reduces resource consumption and cost.
Smart Images
Figure CN120123974A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a lightweight intelligent stethoscope system for building equipment integrating model distillation. Background Art
[0002] With the increase in the number and complexity of building equipment (such as precision air conditioners, elevators, boilers, transformers, etc.), equipment failures or performance changes will directly affect the safety, environmental quality, and energy efficiency of buildings. To ensure the normal operation of equipment, it is crucial to detect potential failures in a timely manner. In recent years, equipment fault diagnosis technology based on voiceprint recognition has gradually attracted attention. By monitoring the voiceprint signals during equipment operation, abnormalities in the equipment status can be detected in a timely manner.
[0003] However, the voiceprint recognition task of equipment usually requires the system to have low latency and real-time performance. Especially in a complex environment such as a building, a large amount of data collection and processing is required, and edge computing has become an ideal choice. However, the computing power of edge devices is limited. Although using lightweight models can improve the inference speed and save computing resources, their accuracy is low and cannot meet the requirements of high-precision fault diagnosis. Therefore, how to improve the computing accuracy of lightweight models has become a bottleneck in the development of technology. Summary of the Invention
[0004] The purpose of the present invention is to provide a lightweight intelligent stethoscope system for building equipment integrating model distillation.
[0005] To solve the above problems, the present invention provides a lightweight intelligent stethoscope system for building equipment integrating model distillation, including:
[0006] A voiceprint data acquisition module for real-time acquisition of voiceprint data and equipment status data generated during the operation of building equipment;
[0007] A voiceprint data processing and feature extraction module for preprocessing and feature extraction of the original voiceprint data;
[0008] A lightweight model construction module for constructing a lightweight model suitable for edge devices based on the extracted features and equipment status data as a lightweight student model;
[0009] A model distillation module for transferring the knowledge of the teacher model to the lightweight student model through distillation;
[0010] A lightweight model optimization module for further optimizing the lightweight student model to obtain an optimized model;
[0011] An edge-side deployment module for deploying the optimized model to edge devices and performing real-time inference on whether the building equipment is faulty through the edge devices.
[0012] Further, in the above system, the voiceprint data acquisition module is used to collect voiceprint audio signals and device status data generated by construction equipment in real time through highly sensitive microphones or sensor arrays installed around the construction equipment, and convert the voiceprint audio signals from analog to digital to obtain voiceprint digital signals.
[0013] Further, in the above system, the voiceprint data acquisition module is also used to transmit the voiceprint digital signals and device status data to the voiceprint data processing and feature extraction module of the cloud or edge device through a wireless transmission protocol.
[0014] Further, in the above system, the voiceprint data processing and feature extraction module is used to perform noise suppression and filtering on the voiceprint digital signals to remove the interference of background noise on voiceprint recognition, so as to obtain the processed voiceprint digital signals; perform standardization processing on the device status data to ensure the unity between different data sources, so as to obtain the processed device status data.
[0015] Further, in the above system, the voiceprint data processing and feature extraction module is also used to convert the processed voiceprint digital signals into Mel spectrograms as the feature input for voiceprint recognition.
[0016] Further, in the above system, the voiceprint data processing and feature extraction module uses the short-time Fourier transform (STFT) to convert the audio signal into a spectrogram, and maps the spectrogram according to the Mel scale to obtain the Mel spectrogram.
[0017] Further, in the above system, the voiceprint data processing and feature extraction module is used to apply the discrete cosine transform to the Mel spectrogram to extract the Mel frequency cepstral coefficients as the voiceprint feature vector.
[0018] Further, in the above system, the lightweight model construction module is used to transmit the preprocessed voiceprint data to the edge device, and a lightweight voiceprint recognition model MobileNetV3 is deployed on the edge device. The voiceprint recognition model MobileNetV3 matches the voiceprint feature vector with the device status to preliminarily judge the working status of the device or whether there is a fault.
[0019] Further, in the above system, the model distillation module is used to train a teacher model with higher accuracy based on the voiceprint feature vector and device status on the cloud or local server, and use the teacher model to generate soft labels; on the edge device, use the distillation algorithm to transfer the knowledge of the teacher model to the lightweight student model. The lightweight student model updates the parameters of the lightweight student model by learning the soft labels generated by the teacher model, so as to obtain the distilled lightweight student model.
[0020] Further, in the above system, the lightweight model optimization module is used to prune the distilled lightweight student model, removing neurons or convolutional layers whose contribution to performance is less than a preset threshold to obtain an optimized lightweight model;
[0021] The edge-side deployment module is used to convert the optimized lightweight model into a format suitable for edge devices; the edge device monitors the status of construction equipment in real time by running the lightweight model after format conversion, infers whether there is a fault in the construction equipment, and issues an alarm or notification according to the inference result.
[0022] In summary, the present invention proposes a lightweight intelligent stethoscope system for construction equipment combining edge computing and model distillation. By using distillation technology, the knowledge of the high-precision teacher model is transferred to the lightweight student model, thereby improving the real-time performance, accuracy, and computing efficiency of the construction equipment stethoscope system while ensuring real-time performance. The present invention is applicable to the fault diagnosis scenario of high-risk construction equipment capable of collecting sound data.
[0023] A lightweight intelligent stethoscope system for construction equipment integrating model distillation proposed by the present invention has the following advantages:
[0024] 1. Efficient real-time fault diagnosis
[0025] Through the combination of edge computing and lightweight models, combined with voiceprint recognition technology, real-time monitoring and fault diagnosis of construction equipment are realized. Voiceprint recognition can quickly capture the operating characteristics of equipment, significantly reduce the diagnostic delay, and efficiently process data on edge devices to ensure timely response to equipment status.
[0026] 2. Balance between accuracy and computing efficiency
[0027] The lightweight convolutional neural network is optimized using model distillation technology to improve the fault diagnosis accuracy while maintaining low computational complexity, enabling the system to have high recognition accuracy and low computational resource requirements in an edge computing environment. Voiceprint recognition can accurately distinguish the operating states of different devices in a multi-device monitoring scenario, ensuring early warning of equipment failures.
[0028] 3. Reducing resource consumption and costs
[0029] By moving data processing and inference tasks to edge devices, the computing burden and data transmission requirements on the cloud are reduced, bandwidth and storage costs are lowered, and energy consumption is saved. Voiceprint recognition not only reduces the dependence on hardware but also can efficiently monitor multiple devices through a small number of sensors and data collection points, improving the scalability, intelligence level, and maintenance efficiency of the system. Description of the Drawings
[0030] Figure 1 It is a schematic diagram of a lightweight intelligent stethoscope system for building equipment with fused model distillation according to an embodiment of the present invention. Detailed implementation manners
[0031] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation manners.
[0032] As Figure 1 shown, the present invention provides a lightweight intelligent stethoscope system for building equipment with fused model distillation, including:
[0033] A voiceprint data acquisition module for real-time acquisition of voiceprint data and equipment status data generated by building equipment during operation;
[0034] A voiceprint data processing and feature extraction module for preprocessing and feature extraction of the original voiceprint data;
[0035] A lightweight model construction module for constructing a lightweight model suitable for edge devices based on the extracted features and equipment status data as a lightweight student model;
[0036] Preferably, the lightweight model is MobileNetV3;
[0037] A model distillation module for transferring the knowledge of the teacher model to the lightweight student model through distillation;
[0038] A lightweight model optimization module for further optimizing the lightweight student model to obtain an optimized model and reduce the calculation and storage overhead;
[0039] An edge-side deployment module for deploying the optimized model to edge devices and performing real-time inference on whether the building equipment is faulty through the edge devices.
[0040] In an embodiment of the lightweight intelligent stethoscope system for building equipment with fused model distillation of the present invention, step 1, voiceprint data acquisition;
[0041] Step 1.1, equipment voiceprint data acquisition:
[0042] The voiceprint data acquisition module is used to collect voiceprint audio signals and equipment status data generated by building equipment in real time through high-sensitivity microphones or sensor arrays installed around the building equipment, and convert the voiceprint audio signals into voiceprint digital signals through analog-to-digital conversion (ADC);
[0043] Optionally, the building equipment includes: air conditioners, elevators, boilers, transformers, etc.
[0044] Step 1.2, data transmission and storage:
[0045] The voiceprint data acquisition module is further configured to transmit the voiceprint digital signal and the device status data to the voiceprint data processing and feature extraction module of the cloud or edge device through a wireless transmission protocol.
[0046] Here, the voiceprint digital signal and the device status data can be stored in a cloud database or local storage for subsequent data processing and analysis.
[0047] Optionally, the wireless transmission protocol includes Wi-Fi, Bluetooth protocol, etc.
[0048] Step 2, Data processing and feature extraction:
[0049] Step 2.1, Data preprocessing:
[0050] The voiceprint data processing and feature extraction module is configured to perform noise suppression and filtering on the voiceprint digital signal to remove the interference of background noise on voiceprint recognition, so as to obtain the processed voiceprint digital signal; perform standardization processing on the device status data to ensure the unity between different data sources, so as to obtain the processed device status data.
[0051] Step 2.2, Mel spectrogram generation:
[0052] The voiceprint data processing and feature extraction module is further configured to convert the processed voiceprint digital signal into a Mel spectrogram as the feature input for voiceprint recognition;
[0053] Preferably, the voiceprint data processing and feature extraction module uses the short-time Fourier transform (STFT) to convert the audio signal into a spectrogram, and maps the spectrogram according to the Mel scale to obtain the Mel spectrogram.
[0054] Preferably, the calculation formula of the Mel spectrogram is as follows:
[0055]
[0056] Where f is the frequency and M(f) is the Mel scale frequency.
[0057] Step 2.3, Mel-frequency cepstral coefficient (MFCC) extraction:
[0058] The voiceprint data processing and feature extraction module is configured to apply the discrete cosine transform (DCT) to the Mel spectrogram to extract the Mel-frequency cepstral coefficient (MFCC) as the voiceprint feature vector.
[0059] Preferably, the calculation formula of the Mel-frequency cepstral coefficient (MFCC) is:
[0060]
[0061] Among them, S(k) is the spectrum, and MFCC(n) is the Mel Frequency Cepstral Coefficient.
[0062] Step 3, lightweight model inference:
[0063] The lightweight model construction module is used to transmit the preprocessed voiceprint data to the edge device. A lightweight voiceprint recognition model MobileNetV3 is deployed on the edge device. The voiceprint recognition model MobileNetV3 matches the voiceprint feature vector with the device status to preliminarily judge the working status of the device or whether there is a fault.
[0064] Step 4, model distillation:
[0065] The model distillation module is used to train a teacher model with higher accuracy based on the voiceprint feature vector and the device status on the cloud or local server, and use the teacher model to generate soft labels; then, on the edge device, use the distillation algorithm to transfer the knowledge of the teacher model to the lightweight student model. The lightweight student model updates the parameters of the lightweight student model by learning the soft labels generated by the teacher model, so as to obtain a distilled lightweight student model with higher accuracy.
[0066] Preferably, the distillation loss function of the distillation algorithm is as follows:
[0067]
[0068] Among them, L hard is the loss of the lightweight student model for the true label, L soft is the loss of the lightweight student model for the soft label output by the teacher model, and α is the weight coefficient of the true label (hard label) and the soft label.
[0069] Step 5, lightweight model optimization:
[0070] The lightweight model optimization module is used to prune the distilled lightweight student model, removing those neurons or convolutional layers whose contribution to performance is less than the preset threshold, so as to obtain an optimized lightweight model.
[0071] Step 6, edge-side deployment:
[0072] Step 6.1, model conversion and deployment:
[0073] The edge-side deployment module is used to convert the optimized lightweight model into a format suitable for edge devices, such as TensorFlow Lite or ONNX format, so as to obtain a lightweight model after format conversion.
[0074] Here, the edge devices deployed (such as embedded systems, Edge TPU, Raspberry Pi, etc.) are used to achieve real-time inference.
[0075] Step 6.2, Real-time Inference and Fault Diagnosis:
[0076] The edge device runs the lightweight model after format conversion to perform real-time monitoring of the status of construction equipment, infer whether there is a fault in the construction equipment, and give an alarm or notification according to the inference result.
[0077] In summary, the present invention proposes a lightweight intelligent stethoscope system for construction equipment that combines edge computing and model distillation. By using the distillation technology, the knowledge of the high-precision teacher model is transferred to the lightweight student model, thereby improving the real-time performance, accuracy, and computing efficiency of the construction equipment stethoscope system while ensuring real-time performance. The present invention is applicable to the fault diagnosis scenario of high-risk construction equipment that can collect sound data.
[0078] A lightweight intelligent stethoscope system for construction equipment that integrates model distillation proposed by the present invention has the following advantages:
[0079] 1. Efficient Real-time Fault Diagnosis
[0080] Through the combination of edge computing and lightweight models, combined with voiceprint recognition technology, real-time monitoring and fault diagnosis of construction equipment are achieved. Voiceprint recognition can quickly capture the operating characteristics of the equipment, significantly reduce the diagnostic delay, and efficiently process data on edge devices to ensure timely response to the equipment status.
[0081] 2. Balance between Precision and Computing Efficiency
[0082] The lightweight convolutional neural network is optimized by using model distillation technology to improve the fault diagnosis accuracy while maintaining low computational complexity, enabling the system to have high recognition accuracy and low computational resource requirements in the edge computing environment. Voiceprint recognition can accurately distinguish the operating states of different devices in the multi-device monitoring scenario, ensuring early warning of equipment faults.
[0083] 3. Reducing Resource Consumption and Costs
[0084] By moving data processing and inference tasks to edge devices, the computing burden and data transmission requirements on the cloud are reduced, bandwidth and storage costs are lowered, and energy consumption is saved at the same time. Voiceprint recognition not only reduces the dependence on hardware but also can efficiently monitor multiple devices through a small number of sensors and data collection points, improving the scalability, intelligence level, and maintenance efficiency of the system.
[0085] In this specification, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts among the embodiments can be referred to each other.
[0086] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.
[0087] Obviously, those skilled in the art can make various modifications and variations to the invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these modifications and variations.
Claims
1. A lightweight intelligent stethoscope system for building equipment integrating model distillation, characterized in that: include: Voiceprint data collection module, used to collect voiceprint data and equipment status data generated by construction equipment during operation in real time; Voiceprint data processing and feature extraction module, used for preprocessing and feature extraction of raw voiceprint data; A lightweight model building module is used to build a lightweight model suitable for edge devices based on the extracted features and device status data, as a lightweight student model; Model distillation module, used to transfer the knowledge of the teacher model to the lightweight student model through distillation; A lightweight model optimization module, used to further optimize the lightweight student model to obtain an optimized model; The edge-side deployment module is used to deploy the optimized model to the edge device and perform real-time reasoning on whether the building equipment is faulty through the edge device.
2. The lightweight intelligent stethoscope system for building equipment based on the fusion model distillation as claimed in claim 1, characterized in that: The voiceprint data acquisition module is used to collect voiceprint audio signals and equipment status data generated by construction equipment in real time through high-sensitivity microphones or sensor arrays installed around the construction equipment, and convert the voiceprint audio signals into voiceprint digital signals through analog-to-digital conversion.
3. The lightweight intelligent auscultation system for building equipment based on the fusion model distillation as claimed in claim 1 is characterized in that: The voiceprint data acquisition module is also used to transmit the voiceprint digital signal and device status data to the voiceprint data processing and feature extraction module of the cloud or edge device through a wireless transmission protocol.
4. The lightweight intelligent stethoscope system for building equipment based on the fusion model distillation as claimed in claim 1, characterized in that: The voiceprint data processing and feature extraction module is used to suppress noise and filter the voiceprint digital signal to remove the interference of background noise on voiceprint recognition to obtain the processed voiceprint digital signal; and to standardize the device status data to ensure the uniformity between different data sources to obtain the processed device status data.
5. The lightweight intelligent stethoscope system for building equipment based on the fusion model distillation as claimed in claim 4, characterized in that: The voiceprint data processing and feature extraction module is also used to convert the processed voiceprint digital signal into a Mel-spectrogram as a feature input for voiceprint recognition.
6. The lightweight intelligent auscultation system for building equipment based on the fusion model distillation as claimed in claim 5 is characterized in that: The voiceprint data processing and feature extraction module uses short-time Fourier transform (STFT) to convert the audio signal into a spectrogram, and maps the spectrogram according to the Mel scale to obtain a Mel spectrogram.
7. The lightweight intelligent auscultation system for building equipment based on the fusion model distillation as claimed in claim 5 is characterized in that: The voiceprint data processing and feature extraction module is used to apply discrete cosine transform to the Mel frequency spectrum to extract Mel frequency cepstrum coefficients as voiceprint feature vectors.
8. The lightweight intelligent auscultation system for building equipment based on the fusion model distillation as claimed in claim 7 is characterized in that: The lightweight model building module is used to transmit the preprocessed voiceprint data to the edge device, and the lightweight voiceprint recognition model MobileNetV3 is deployed on the edge device. The voiceprint recognition model MobileNetV3 matches the voiceprint feature vector with the device status to preliminarily determine the working status of the device or whether there is a fault.
9. The lightweight intelligent auscultation system for building equipment based on the fusion model distillation as claimed in claim 7, characterized in that: The model distillation module is used to train a teacher model with higher accuracy based on the voiceprint feature vector and the device status on the cloud or local server, and use the teacher model to generate soft labels; the distillation algorithm is used on the edge device to transfer the knowledge of the teacher model to the lightweight student model, and the lightweight student model updates the parameters of the lightweight student model by learning the soft labels generated by the teacher model, thereby obtaining the distilled lightweight student model.
10. The lightweight intelligent stethoscope system for building equipment based on fusion model distillation as claimed in claim 1, characterized in that: The lightweight model optimization module is used to prune the lightweight student model after distillation, and remove neurons or convolutional layers whose contribution to performance is less than a preset threshold, so as to obtain an optimized lightweight model; The edge side deployment module is used to convert the optimized lightweight model into a format suitable for edge devices; The edge device conducts real-time monitoring of the status of building equipment by running the lightweight model after format conversion, infers whether there is any fault in the building equipment, and issues alarms or notifications based on the inference results.
Citation Information
Cited By
Method and device for identifying low-current single-phase earth fault of power distribution network based on cloud-side cooperation
CN120744634A
Carrier roller fault monitoring method and system based on optical fiber auscultation
CN121493540A