Methods, devices, equipment, and media for identifying and locating leaks in public building restrooms
By processing the acoustic signals of public building restrooms and applying the CNN-Transformer model, accurate leakage identification of restroom water-using equipment was achieved, solving the problems of inaccurate positioning and detection lag in existing technologies, and improving detection efficiency and intelligence level.
Patent Information
- Application Number
- CN202610307272.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-13
- Publication Date
- 2026-05-26
AI Technical Summary
Existing technologies struggle to accurately pinpoint leaks in specific water-using equipment in public building restrooms, particularly lacking sensitivity to detect minute leaks and offering limited remote monitoring and intelligent alarm capabilities.
The acoustic signal, after being processed by wavelet denoising, pre-emphasis, and frame-segmentation windowing, is used to construct a log-Mel spectrogram through short-time Fourier transform and Mel filter bank. Feature extraction and classification are then performed using a CNN-Transformer hybrid model to achieve accurate identification of leaky devices.
It enables accurate leakage identification of equipment such as squat toilets, faucets, and urinals, improving detection sensitivity, reducing water waste and building damage, and enhancing detection efficiency and intelligence.
Smart Images

Figure CN122084202A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of water leakage detection, and in particular to a method for identifying and locating water leakage in public building restrooms, a corresponding device, electronic equipment, and a computer-readable storage medium. Background Technology
[0002] Public building restrooms, as core functional areas for water use, are frequently used and involve a concentration of water-using equipment, including water supply pipes, squat toilets, washbasins, and urinals. Due to their consistently damp environment and frequent use, issues such as aging pipe joints, failed seals, and equipment damage are unavoidable, making them high-risk areas for leaks in public buildings. Leaks not only waste water resources but can also cause secondary problems such as dampness damage to surrounding building structures, mold growth on walls, and corrosion of building materials, affecting the lifespan of public buildings and indoor environmental quality. Furthermore, against the backdrop of the nation's vigorous promotion of "dual-carbon" goals, conserving water resources, reducing energy consumption, and achieving green and low-carbon development have become important tasks for the construction industry. Accurate monitoring and timely remediation of leaks in public building restrooms are of great significance for reducing water waste, lowering building operation and maintenance costs, and promoting energy conservation and emission reduction.
[0003] Currently, the main technical solutions for building leak detection include Building Information Modeling (BIM) visualization and location methods, flow and pressure monitoring methods, and infrared thermal imaging detection methods. For example, patent CN104614126A uses BIM technology and sensor networks to achieve visualized location of building leaks; patent CN115760070A uses pressure sensors and flow sensors to determine the location of leaks and generates emergency plans based on the BIM model; and patent CN121185517A uses infrared thermal imaging and immersion re-testing methods to detect external leakage in building walls.
[0004] However, existing methods still have the following technical shortcomings, including:
[0005] Firstly, leak detection is mostly aimed at external seepage of building structures or macroscopic areas, making it difficult to apply to the detection and precise location of leaks in specific water-using equipment in indoor water-using places such as bathrooms.
[0006] Secondly, the accuracy of leak location is limited to the pipe or area level, lacking the ability to locate leaks with fine precision.
[0007] Third, it lacks sufficient sensitivity in detecting minute leaks, and exhibits significant detection lag.
[0008] Fourth, the lack of proactive monitoring strategies during periods of low water usage makes it difficult to effectively integrate with cloud platforms, thus hindering remote monitoring and intelligent alarm management.
[0009] In summary, existing technologies for leak detection primarily target external leaks or macroscopic areas of building structures, making them difficult to apply to the detection and precise location of leaks in specific water-using equipment in indoor spaces such as bathrooms. Furthermore, they suffer from insufficient sensitivity to detect minute leaks and significant detection lag. Therefore, the applicant has undertaken this exploration to address these issues. Summary of the Invention
[0010] The purpose of this application is to solve the above-mentioned problems by providing a method for identifying and locating leaks in public building toilets, a corresponding device, electronic equipment, and a computer-readable storage medium.
[0011] To achieve the various objectives of this application, the following technical solution is adopted:
[0012] A method for identifying and locating leaks in public building restrooms, proposed to meet one of the purposes of this application, includes:
[0013] Acquire the acoustic signals to be identified for each toilet to be inspected in a public building suspected of leaking water;
[0014] After performing wavelet denoising, pre-emphasis, and frame-windowing processing on the acoustic signal to be identified, the power spectrum is calculated using short-time Fourier transform. After filtering by a Mel filter bank, logarithmic compression is performed to determine the log-Mel spectral features, thereby constructing the log-Mel spectrum corresponding to the acoustic signal to be identified. Figure 2 The dimensional feature matrix, and the log-Mel spectrum Figure 2 The feature matrix is subjected to global mean-variance normalization.
[0015] Log-Mel spectrum after global mean-variance normalization Figure 2 The 3D feature matrix is input into a pre-trained leaky device identification model, and the CNN feature extraction module processes the log-Mel spectrum. Figure 2 The 3D feature matrix is used to extract local time-frequency features and transform them into a 1D time-series feature sequence. After being processed by a linear projection layer and a position coding layer, it is input into a Transformer encoder. The global temporal dependencies of the 1D time-series feature sequence are captured by a multi-head self-attention sublayer and then processed by a feedforward neural network sublayer to determine the first output feature sequence.
[0016] The first output feature sequence is input into the Transformer decoder. The second output feature sequence is determined by modeling the temporal causal relationship through the masked multi-head self-attention sub-layer and fusing global time-frequency features through the cross-attention sub-layer. After processing by the global average pooling layer and the fully connected layer, the normalized classification probability distribution is output by the Softmax layer.
[0017] If the maximum classification probability is higher than the preset confidence threshold, the category of the leaking device corresponding to the maximum classification probability is taken as the category of the leaking device corresponding to the toilet to be detected, so as to complete the identification and location of the leaking toilet in the public building.
[0018] Optionally, the basic network architecture of the leaky device identification model is a CNN-Transformer hybrid model, wherein the CNN-Transformer hybrid model includes an input layer, a CNN feature extraction module, a linear projection layer, a position encoding layer, a Transformer encoder module, a Transformer decoder module, a global average pooling layer, a Dropout layer, a fully connected layer, a Softmax layer, and a classification output layer connected in sequence.
[0019] The CNN feature extraction module is composed of multiple convolutional blocks cascaded together, wherein each convolutional block includes a CNN convolutional layer, a batch normalization layer, a GELU activation function layer, and a max pooling layer connected in sequence.
[0020] The position coding layer includes a sinusoidal position coding layer;
[0021] The Transformer encoder module comprises multiple identical Transformer encoder layers stacked together. Each Transformer encoder layer uses a Pre-Norm residual structure to sequentially complete feature processing. It includes a multi-head self-attention sub-layer and a feedforward neural network sub-layer. Both the multi-head self-attention sub-layer and the feedforward neural network sub-layer are provided with layer normalization and residual connection layers. The feedforward neural network sub-layer includes two linear transformation layers and a GELU activation function layer.
[0022] The Transformer decoder module is composed of multiple identical Transformer decoder layers stacked together. Each Transformer decoder layer uses a Pre-Norm residual structure to complete feature processing sequentially. It includes a masked multi-head self-attention sub-layer, a cross-attention sub-layer, and a feedforward neural network sub-layer. The masked multi-head self-attention sub-layer, the cross-attention sub-layer, and the feedforward neural network sub-layer are all equipped with layer normalization and residual connection layers.
[0023] The categories of leaking equipment include no leaks, leaking squat toilets, leaking accessories, leaking urinals, or leaking faucets.
[0024] Optionally, after performing wavelet denoising, pre-emphasis, and frame-windowing processing on the acoustic signal to be identified, the power spectrum is calculated using short-time Fourier transform, filtered by a Mel filter bank, and then logarithmically compressed to determine the log-Mel spectral features, thereby constructing the log-Mel spectrum corresponding to the acoustic signal to be identified. Figure 2The steps for constructing a 3D feature matrix include:
[0025] Wavelet denoising, pre-emphasis, and frame-segmentation windowing are sequentially performed on the acoustic signal to be identified in the toilet to be detected. Short-time Fourier transform is performed on the acoustic signal to be identified after frame-segmentation windowing to calculate the power spectrum of each frame of the acoustic signal to be identified.
[0026] The power spectrum is filtered by a Mel filter bank containing multiple triangular bandpass filters with center frequencies uniformly distributed according to Mel scale. The cumulative energy value of each frame of the acoustic signal to be identified in each frequency band of each triangular bandpass filter is calculated to obtain the output energy value of each frame of the acoustic signal to be identified in each triangular bandpass filter.
[0027] The output energy value is logarithmically compressed to determine the log-Mel spectrum features. The log-Mel spectrum features of all frames of the acoustic signal to be identified are arranged in chronological order to construct the log-Mel spectrum corresponding to the acoustic signal to be identified. Figure 2 3D feature matrix.
[0028] Optionally, the log-Mel spectrum is processed by the CNN feature extraction module. Figure 2 The steps for extracting local time-frequency features from a 3D feature matrix and converting it into a one-dimensional time-series feature sequence include:
[0029] Log-Mel spectrum after global mean-variance normalization Figure 2 The 3D feature matrix is input into the CNN feature extraction module;
[0030] Each convolutional block in the CNN feature extraction module sequentially processes the log-Mel spectrum. Figure 2 The log-Mel spectrum is extracted by performing two-dimensional convolution, batch normalization, GELU activation function processing, and max pooling on the feature matrix. Figure 2 Local time-frequency features of the 3D feature matrix;
[0031] After all convolutional blocks are cascaded, the output two-dimensional time-frequency feature map is compressed in the frequency dimension and transformed into a one-dimensional time-series feature sequence.
[0032] Optionally, the step of processing the sequence through a linear projection layer and a position encoding layer, inputting it to a Transformer encoder to capture the global temporal dependencies of the one-dimensional temporal feature sequence through a multi-head self-attention sublayer, and then processing it through a feedforward neural network sublayer to determine the first output feature sequence includes:
[0033] The one-dimensional temporal feature sequence is input into the linear projection layer, and the dimension of the one-dimensional temporal feature sequence is mapped to the model feature dimension of the Transformer encoder through linear transformation to obtain the dimension-adapted feature sequence.
[0034] The dimension-adapted feature sequence is input into the sinusoidal position coding layer, and a position coding value generated based on sinusoidal position coding is added to the dimension-adapted feature sequence to obtain the dimension-position fusion feature sequence.
[0035] The dimension-location fusion feature sequence is input into a Transformer encoder consisting of multiple identical Transformer encoder layers stacked together. For each Transformer encoder layer, the attention weights between each time position in the dimension-location fusion feature sequence are first calculated through its corresponding multi-head self-attention sub-layer to capture the global temporal dependency of the one-dimensional temporal feature sequence and obtain the intermediate feature sequence with fused global dependency.
[0036] The intermediate feature sequence is subjected to nonlinear feature transformation through its corresponding feedforward neural network sublayer to enhance the nonlinear expressive power of the features, thereby obtaining a feature sequence optimized by the feedforward neural network sublayer.
[0037] After being processed by stacking all Transformer encoder layers, the output is a feature sequence that integrates global temporal dependency information and nonlinear features, which serves as the first output feature sequence of the Transformer encoder.
[0038] Optionally, the step of inputting the first output feature sequence into the Transformer decoder, refining the features by modeling temporal causality through a masked multi-head self-attention sublayer and fusing global time-frequency features through a cross-attention sublayer to determine the second output feature sequence includes:
[0039] The first output feature sequence of the Transformer encoder is input into the Transformer decoder module, which is composed of multiple identical Transformer decoder layers stacked together, as the initial input feature of the masked multi-head self-attention sub-layer in each Transformer decoder layer;
[0040] The first output feature sequence is temporally modeled by the masked multi-head self-attention sub-layer of each Transformer decoder layer in the Transformer decoder, which masks the future time location feature information to construct a temporal causal relationship and obtain the masked self-attention output features.
[0041] Through the cross-attention sub-layer of each Transformer decoder layer, the mask self-attention output feature is used as the query feature, and the first output feature sequence output by the Transformer encoder module is used as the key feature and value feature. Attention calculation is performed, and global time-frequency feature information is fused to obtain the cross-attention output feature.
[0042] After all Transformer decoder layers in the Transformer decoder complete the temporal causal relationship modeling and global time-frequency feature fusion, the second output feature sequence with refined features is obtained.
[0043] Optionally, before the step of obtaining the corresponding acoustic signals to be identified for each toilet to be tested in a public building suspected of leaking, the following steps are included:
[0044] The instantaneous flow rate value in the smart water meter of the main water supply pipe of the target public building is continuously monitored. When the instantaneous flow rate value is continuously higher than the leakage judgment threshold within the leakage judgment time range, the target public building is judged as a suspected leakage public building.
[0045] A collection command is sent to the acoustic signal collectors deployed in each toilet to be tested within the suspected leaking public building, in order to collect the corresponding acoustic signals to be identified in each toilet to be tested within the suspected leaking public building.
[0046] A public building toilet leak detection and location device provided for another purpose of this application includes:
[0047] The module for acquiring acoustic signals to be identified is set to acquire the acoustic signals to be identified for each toilet to be tested in a public building suspected of leaking water.
[0048] The Mel spectrum construction module is configured to sequentially perform wavelet denoising, pre-emphasis, and frame-windowing processing on the acoustic signal to be identified, then calculate the power spectrum using short-time Fourier transform, filter it using a Mel filter bank, and perform logarithmic compression to determine the log-Mel spectral features, thereby constructing the log-Mel spectrum corresponding to the acoustic signal to be identified. Figure 2 The dimensional feature matrix, and the log-Mel spectrum Figure 2 The feature matrix is subjected to global mean-variance normalization.
[0049] The first feature sequence determination module is configured to process the log-Mel spectrum after global mean-variance normalization. Figure 2 The 3D feature matrix is input into a pre-trained leaky device identification model, and the CNN feature extraction module processes the log-Mel spectrum. Figure 2 The 3D feature matrix is used to extract local time-frequency features and transform them into a 1D time-series feature sequence. After being processed by a linear projection layer and a position coding layer, it is input into a Transformer encoder. The global temporal dependencies of the 1D time-series feature sequence are captured by a multi-head self-attention sublayer and then processed by a feedforward neural network sublayer to determine the first output feature sequence.
[0050] The second feature sequence determination module is configured to input the first output feature sequence into the Transformer decoder, and refine the features by modeling the temporal causal relationship through the masked multi-head self-attention sub-layer and fusing global time-frequency features through the cross-attention sub-layer to determine the second output feature sequence. After processing by the global average pooling layer and the fully connected layer, the normalized classification probability distribution is output by the Softmax layer.
[0051] The leak detection and location module is configured to use the category of the leaking device corresponding to the maximum classification probability as the category of the leaking device corresponding to the toilet to be detected if the maximum classification probability is higher than a preset confidence threshold, so as to complete the identification and location of leaks in public building toilets.
[0052] An electronic device provided for another purpose of this application includes a central processing unit and a memory, the central processing unit being configured to invoke and run a computer program stored in the memory to perform the steps of the public building toilet leak identification and location method of this application.
[0053] A computer-readable storage medium is provided for another purpose of this application, which stores, in the form of computer-readable instructions, a computer program implemented according to the method for identifying and locating leaks in public building toilets, which, when invoked by a computer, executes the steps included in the corresponding method.
[0054] Compared to existing technologies, this application addresses the problems of existing leak detection methods, which primarily target external leaks or macroscopic areas of building structures, making them difficult to apply to the detection and precise location of leaks in specific water-using equipment in indoor water-using areas such as bathrooms. It also suffers from insufficient sensitivity in detecting minute leaks and significant detection lag. This application offers the following advantages, including but not limited to:
[0055] Firstly, the public building toilet leakage identification and localization method of this application effectively filters out noise and retains leakage characteristic sound signals through a series of acoustic signal preprocessing processes such as wavelet denoising, pre-emphasis, and frame-segmentation windowing. Combined with the construction and normalization of log-Mel spectrum, it lays a high-quality feature foundation for model recognition. Relying on the advantages of the CNN-Transformer hybrid model, CNN extracts local time-frequency features, Transformer captures global temporal dependencies and completes feature refinement. Finally, the Softmax layer achieves accurate classification of leakage equipment categories, greatly improving the accuracy of leakage equipment identification.
[0056] Secondly, the public building toilet leak identification and location method of this application can accurately determine the specific type of leaking equipment such as squat toilets, faucets, urinals, and pipe fittings, breaking through the limitation of traditional leak detection that can only locate areas or pipes, providing clear direction for maintenance work and significantly improving the efficiency of leak treatment.
[0057] Thirdly, the public building toilet leakage identification and location method of this application is based on acoustic feature identification, which can capture the characteristic sound signals generated by tiny leaks in the water-using equipment in the toilet. Compared with traditional detection methods, it has higher sensitivity to detect tiny leaks, enabling early detection of leakage problems and avoiding secondary problems such as water waste and building structure damage caused by the expansion of leakage.
[0058] Fourth, the public building toilet leakage identification and location method of this application uses a pre-trained leakage equipment identification model to realize the automated analysis and classification of sound signals. The entire process from sound signal processing to leakage equipment location can be completed without manual intervention, replacing the inefficient mode of traditional manual inspection and improving the intelligence level and detection efficiency of public building toilet leakage detection.
[0059] Furthermore, by accurately and quickly identifying and locating leaks in public building restrooms, it is possible to promptly repair and handle leaking equipment, effectively reducing the ineffective waste of water resources, lowering the operation and maintenance costs of public buildings, and simultaneously aligning with the national requirements for water conservation, emission reduction, green and low-carbon development, thus promoting the green and intelligent operation and maintenance of public buildings. Attached Figure Description
[0060] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:
[0061] Figure 1 This is a flowchart illustrating the method for identifying and locating leaks in public building restrooms in this application.
[0062] Figure 2 This is a diagram illustrating the effect of the real-time monitoring module in the embodiments of this application;
[0063] Figure 3 This is a diagram illustrating the effect of the device management module in an embodiment of this application.
[0064] Figure 4 This is a diagram illustrating the effect of the intelligent triggering module in the embodiments of this application;
[0065] Figure 5 This is a diagram illustrating the effect of the water leakage detection module in the embodiments of this application;
[0066] Figure 6 This is a diagram illustrating the effect of the acoustic signal storage module in the embodiments of this application;
[0067] Figure 7 This is a diagram illustrating the effect of the alarm management module in this embodiment of the application.
[0068] Figure 8This is a diagram illustrating the effect of the statistical report module in this application embodiment;
[0069] Figure 9 This is a diagram illustrating the effect of the system configuration module in this embodiment of the application.
[0070] Figure 10 This is a schematic diagram of the acoustic signal collector in an embodiment of this application;
[0071] Figure 11 This is a flowchart of the acoustic signal preprocessing and log-Mel spectrum feature extraction in the embodiments of this application;
[0072] Figure 12 This is a schematic diagram of the structure of the CNN-Transformer model in the embodiments of this application;
[0073] Figure 13 This is a schematic diagram of the public building toilet leak detection and location device in the embodiments of this application;
[0074] Figure 14 This is a schematic diagram of the structure of the computer device in the embodiments of this application. Detailed Implementation
[0075] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.
[0076] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.
[0077] Those skilled in the art will understand that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.
[0078] Unless otherwise specified, all data involved in this application may be stored remotely on a server or on a local terminal device, as long as it is suitable for use by the technical solution of this application.
[0079] Those skilled in the art will understand that although the various methods in this application are described based on the same concept and thus present commonality among them, they can be performed independently unless otherwise specified. Similarly, the various embodiments disclosed in this application are all based on the same inventive concept; therefore, concepts expressed in the same way, as well as concepts that are appropriately changed for convenience but are expressed differently, should be understood equivalently.
[0080] Unless otherwise expressly stated, the various embodiments disclosed in this application can be combined in a cross-cutting manner to flexibly construct new embodiments, as long as such combination does not depart from the inventive spirit of this application and can meet the needs of the prior art or solve a certain deficiency in the prior art. Those skilled in the art should be aware of such modifications.
[0081] Please see Figure 1 In one embodiment of the public building toilet leak identification and location method of this application, the method includes:
[0082] Step S10: Obtain the corresponding acoustic signals to be identified for each toilet to be tested in the public building suspected of leaking water;
[0083] The public building toilet leak identification and positioning system in the terminal device can acquire the corresponding acoustic signals to be identified for each toilet in a public building suspected of having a leak. The public building toilet leak identification and positioning system adopts a microservice architecture design and is deployed on a cloud server cluster. It includes a real-time monitoring module, a device management module, an intelligent triggering module, a leak identification module, an acoustic signal storage module, an alarm management module, a statistical report module, and a system configuration module.
[0084] In some embodiments, before the step of acquiring the acoustic signals to be identified for each toilet to be tested in a public building suspected of leaking, the following steps are included:
[0085] Step S101: Continuously monitor the instantaneous flow rate value in the smart water meter of the main water supply pipe of the target public building. When the instantaneous flow rate value is continuously higher than the leakage judgment threshold within the leakage judgment time range, the target public building is judged as a suspected leakage public building.
[0086] Step S102: Send a collection command to the acoustic signal collectors deployed in each toilet to be tested in the suspected leaking public building, so as to collect the acoustic signals to be identified corresponding to each toilet to be tested in the suspected leaking public building.
[0087] The public building toilet leak identification and location method of this application can be implemented based on a public building toilet leak identification and location system. This system receives monitoring data reported by front-end devices and performs intelligent analysis and alarm management. The front-end devices include smart water meters deployed on the main water supply pipes of each public building and acoustic signal collectors deployed in each toilet. The public building toilet leak identification and location system includes a real-time monitoring module, an equipment management module, an intelligent triggering module, a leak identification module, an acoustic signal storage module, an alarm management module, a statistical report module, and a system configuration module, etc.
[0088] Please see Figures 2 to 9 ,in, Figure 2 This is a diagram illustrating the effect of the real-time monitoring module in the embodiments of this application; Figure 3 This is a diagram illustrating the effect of the device management module in an embodiment of this application. Figure 4 This is a diagram illustrating the effect of the intelligent triggering module in the embodiments of this application; Figure 5 This is a diagram illustrating the effect of the water leakage detection module in the embodiments of this application; Figure 6 This is a diagram illustrating the effect of the acoustic signal storage module in the embodiments of this application; Figure 7 This is a diagram illustrating the effect of the alarm management module in this embodiment of the application. Figure 8 This is a diagram illustrating the effect of the statistical report module in this application embodiment; Figure 9 This is a diagram illustrating the effect of the system configuration module in this embodiment.
[0089] In some embodiments, the real-time monitoring module is configured to receive, store, and display monitoring data from each smart water meter in real time, and includes a data receiving and storage unit and a data display unit.
[0090] The data receiving and storage unit establishes a communication connection with each smart water meter and receives flow data reported by each smart water meter according to a preset period. The flow data includes instantaneous flow value, cumulative flow value, water meter operating status, and data reporting time. The data receiving and storage unit assigns a unique device identifier to each smart water meter and establishes a correspondence between the device identifier and public buildings. It stores the received flow data in time sequence to form historical flow data records for each smart water meter.
[0091] The data display unit uses a visual interface to display the current flow status of each smart water meter in real time, including instantaneous flow value, daily cumulative flow value, and smart water meter online status. The data display unit supports grouping and displaying smart water meters by building, using different colors or icons to indicate the working status of the smart water meters, including normal status, abnormal status, and offline status, and provides a flow data query function, supporting the retrieval of historical flow data by device number, public building to which it belongs, time range, and other conditions.
[0092] In some embodiments, the device management module is configured to manage front-end devices in a unified manner, enabling device information viewing, status monitoring, and maintenance reminders, including a smart water meter management unit, a sound signal collector management unit, and a device maintenance reminder unit.
[0093] The smart water meter management unit stores and displays basic information for each smart water meter, including at least the device number, device model, measuring range, accuracy class, installation location, installation date, and the number of the public building to which it belongs. The smart water meter management unit displays the online status and communication quality of each smart water meter in real time, records the cumulative runtime and cumulative flow rate of each smart water meter, automatically generates a device anomaly record when a smart water meter experiences communication anomalies, metering anomalies, or goes offline, and provides functions for adding, modifying, deleting, and querying smart water meter information. The acoustic signal collector management unit stores and displays basic information for each acoustic signal collector, including at least the device number, device model, installation location, installation date, and associated smart water meter number.
[0094] The acoustic signal collector management unit receives and displays the remaining power information of each acoustic signal collector in real time, sets a power warning threshold, and automatically generates a low power reminder when the remaining power of a certain acoustic signal collector falls below the power warning threshold. The power warning threshold can be configured by the administrator, and the default power warning threshold is preferably 20%. The acoustic signal collector management unit records the online status, the most recent collection time, and the cumulative number of collections for each acoustic signal collector. When an acoustic signal collector experiences a communication abnormality or goes offline, it automatically generates a device abnormality record and provides functions for adding, modifying, deleting, and querying acoustic signal collector information.
[0095] The device maintenance reminder unit aggregates the abnormal status and low battery information of all front-end devices, generating a device maintenance reminder list. This list includes device number, device type, abnormality type, occurrence time, and device location, and supports filtering by abnormality type, device type, and occurrence time. The device maintenance reminder unit prominently displays the number of devices awaiting maintenance on the cloud-based backend interface and can optionally send device maintenance notifications to maintenance personnel through preset channels.
[0096] In some embodiments, the intelligent triggering module is configured to monitor the status of the smart water meter during a preset monitoring period and automatically trigger the acquisition of sound signals when abnormal flow is detected, including a monitoring period control unit, an anomaly determination unit, and an acquisition command issuing unit.
[0097] The monitoring period control unit has a built-in system clock. It determines whether the current period is a leakage monitoring period based on preset monitoring period parameters, including the monitoring start and end times. These parameters can be set by administrators through the system configuration module. The default monitoring period is preferably from 02:00 to 05:00 AM. The monitoring period control unit supports setting multiple monitoring periods to adapt to the water usage patterns of different building types. The leakage monitoring mode is automatically activated when the system clock enters a monitoring period and automatically deactivated when the system clock leaves the monitoring period.
[0098] The anomaly detection unit, in leak monitoring mode, polls the real-time flow data of each smart water meter at a preset detection cycle. When it detects that a smart water meter is continuously rotating within a leak detection time range, the leak detection time range is the observation period for determining whether the smart water meter is continuously rotating, which can be 15 to 30 minutes. The anomaly detection unit sets a leak detection threshold. When the instantaneous flow value of the smart water meter exceeds the leak detection threshold within the leak detection time range, it determines that there is a suspected leak in the public building corresponding to that smart water meter. The leak detection threshold can be configured according to the accuracy of the smart water meter and actual needs, and can be 0.5 liters / hour. The anomaly detection unit generates a suspected leak event, recording the trigger time, the smart water meter identifier, and the corresponding public building information.
[0099] After a suspected water leakage event is generated, the data acquisition instruction issuing unit determines all sound signal collectors in the public building based on the correspondence between the smart water meter and the corresponding public building, as well as the deployment information of the sound signal collectors in the public building, and issues sound signal acquisition instructions to them. The acquisition instructions include acquisition duration parameters, acquisition accuracy parameters, and data feedback address, and record the instruction issuance time and target device information for subsequent data association.
[0100] In some embodiments, the leak detection module is configured to receive acoustic signal data uploaded by an acoustic signal collector and determine the type of leaking device through a preset leak detection model, including an acoustic signal receiving unit, a device identification unit, and an identification result output unit.
[0101] The acoustic signal receiving unit receives acoustic signal data collected and uploaded by the acoustic signal collector according to the acquisition command. Based on the suspected water leakage event number carried in the acquisition command, the acoustic signal data is bound to the suspected water leakage event that triggered this acquisition. The acoustic signal data is validated to determine whether the data integrity and signal quality meet the recognition requirements. The validated valid acoustic signal data is then transmitted to the acoustic signal storage module for storage.
[0102] The device identification unit has a pre-trained leak detection model built in. This model is constructed using a deep learning algorithm and trained on sample data containing sound signals from various bathroom water-using devices, including squat toilets, angle valves and other accessories, urinals, and faucets. The device identification unit inputs the received sound signal data into the leak detection model for analysis and processing, outputting the leak detection device type identification result and its corresponding confidence level. When the confidence level is higher than or equal to a preset threshold, the identification result is output as a valid result and transmitted to the alarm management module for further processing; when the confidence level is lower than the preset threshold, the identification result is marked as requiring manual review.
[0103] The identification result output unit integrates the identification results with the suspected water leakage event to generate a water leakage identification record. The water leakage identification record includes the water leakage time, water leakage location, water leakage device type, identification confidence level, and original flow data summary. The water leakage identification record is stored in the database and transmitted to the alarm management module for further processing.
[0104] In some embodiments, the acoustic signal storage module is configured to automatically store and manage the collected acoustic signal data to facilitate subsequent query analysis and model optimization, and includes an automatic storage unit, a data indexing unit, and a storage management unit.
[0105] The automatic storage unit receives the acoustic signal data transmitted by the leak detection module and automatically stores it in the cloud storage space. It generates a unique storage identifier for each acoustic signal data piece and stores the data in a standardized audio format, preferably WAV or MP3. The automatic storage unit synchronously stores the acquisition metadata of the acoustic signal data, including acquisition time, acquisition device number, acquisition duration, sampling rate, associated smart water meter number, and associated toilet location.
[0106] The data indexing unit establishes a multi-dimensional index structure for acoustic signal data. The multi-dimensional index includes at least an index by acquisition time, an index by acquisition device, an index by spatial location, and an index by recognition result. It supports the rapid retrieval of acoustic signal data based on the above index conditions and associates the acoustic signal storage identifier with the corresponding water leakage recognition record and alarm record to form a complete data link.
[0107] The storage management unit monitors cloud storage space usage, sets a storage capacity warning threshold, and automatically generates a storage capacity warning when the storage space usage exceeds the warning threshold. The storage management unit provides a batch export function for audio signal data, supporting the export of audio signal data under specified conditions for model training or offline analysis. It also sets a data retention strategy and automatically cleans up expired audio signal data according to a preset retention period. The retention period can be configured by administrators, with a default retention period of one year.
[0108] In some embodiments, the alarm management module is configured to generate alarm information based on the water leakage identification result and to send and display alarms, including an alarm generation unit, an alarm sending unit, and an alarm display unit.
[0109] The alarm generation unit receives the leak identification records output by the leak identification module and generates alarm records according to preset alarm rules. Each alarm record includes at least the following fields: alarm number, alarm time, alarm location, leaking device type, alarm level, and associated sound signal storage identifier. The alarm generation unit determines the alarm level according to the leaking device type and identification confidence level, based on preset rules. The alarm levels are divided into three levels: Level 1 alarm is urgent, Level 2 alarm is important, and Level 3 alarm is general.
[0110] The alarm sending unit sends a notification according to the alarm level and preset notification rules. The alarm notification content includes at least the alarm location, the type of leaking equipment, and the alarm time.
[0111] The alarm display unit displays all alarm records in a list format on the cloud backend interface. It supports filtering and sorting by alarm level, alarm time, alarm location, and other conditions. It provides an alarm details viewing function to display complete alarm information and associated traffic data and sound signal recognition results. It also provides online playback and download functions for associated sound signal data and a historical alarm query function to support searching historical alarms by time range, location, device type, and other conditions.
[0112] In some embodiments, the statistical report module is configured to automatically generate periodic leakage statistical reports to provide data support for management decisions, including a data statistics unit, a report generation unit, and a report query unit.
[0113] The data statistics unit summarizes and analyzes leakage data according to a preset statistical period. Preferably, the statistical period is one week, i.e., a leakage statistics report is automatically generated weekly. The statistical content of the data statistics unit includes at least the total number of leakage alarms in the current period, the number and percentage of alarms classified by leakage device type, the distribution of alarms classified by spatial location, the number and percentage of alarms classified by alarm level, a comparison and trend of leakage numbers with the previous period, and a statistically estimated leakage volume.
[0114] The report generation unit automatically generates a leakage statistics report at the end of each statistical period. The statistics report includes data tables and visualization charts, including but not limited to bar charts, pie charts, and trend line charts. The statistics report is output in a standardized format and supports online viewing and file download.
[0115] In some embodiments, the system configuration module is configured to uniformly manage the operating parameters of the public building toilet leak identification and positioning system, including a monitoring parameter configuration unit, an alarm parameter configuration unit, an equipment parameter configuration unit, a storage parameter configuration unit, and a report parameter configuration unit.
[0116] The monitoring parameter configuration unit configures the start and end times of the leakage monitoring period, supports setting multiple monitoring periods and their corresponding activation status, configures the leakage judgment threshold and leakage judgment duration range, and configures the flow data acquisition cycle and anomaly detection cycle.
[0117] The device parameter configuration unit configures the power warning threshold of the sound signal collector, configures the correspondence between the smart water meter and the public building and the sound signal collector, and configures the mapping relationship between the device and the spatial location.
[0118] The storage parameter configuration unit configures the storage format and sampling rate requirements of the acoustic signal data, configures the storage capacity warning threshold, and configures the retention period of the acoustic signal data.
[0119] In some embodiments, based on modules such as a real-time monitoring module, an equipment management module, an intelligent triggering module, a leak detection module, an acoustic signal storage module, an alarm management module, a statistical report module, and a system configuration module, the leak detection workflow of the public building restroom leak detection and location system is as follows:
[0120] The real-time monitoring module continuously receives and stores the flow data reported by each smart water meter, and displays it in real time. The device management module continuously monitors the status of each front-end device and the remaining power of the sound signal collector, and generates a maintenance reminder when an abnormality occurs or the power is insufficient. When the system clock enters the preset monitoring period, the smart trigger module starts the leakage monitoring mode. When the smart trigger module detects that a smart water meter is continuously rotating during the monitoring period and the instantaneous flow rate continuously exceeds the leakage judgment threshold, it determines that there is a suspected leakage in the public building. The smart trigger module sends a collection command to the sound signal collectors in each toilet in the public building. The leakage identification module receives the sound signal data uploaded by the sound signal collector. The sound signal storage module automatically stores the received sound signal data and its collection metadata. The leakage identification module calls the leakage device identification model to identify the leakage device type and outputs the confidence level. The alarm management module generates alarm information based on the identification results. The statistical report module automatically generates a leakage statistical report according to a preset period.
[0121] In a specific embodiment, the real-time monitoring module establishes a communication connection with each smart water meter and receives the flow data reported by the meters in real time. The data receiving and storage unit stores the received data in a database, and the data fields include: device ID, timestamp, instantaneous flow value, cumulative flow value, and device status code. The data display unit implements a visual interface, displaying the real-time status of the smart water meters in each public building in the form of a list of public buildings. Normal status is displayed as a first icon (e.g., a green icon), abnormal status as a second icon (e.g., a red icon), and offline status as a third icon (e.g., a gray icon). Clicking on any water meter icon allows viewing the meter's detailed information and historical flow curves.
[0122] Furthermore, the workflow of the intelligent triggering module includes: the monitoring period control unit periodically checks the system clock to determine whether it is currently in a monitoring period. When a monitoring period begins, a start command is sent to the anomaly determination unit. The anomaly determination unit polls the flow data of each smart water meter at a preset period. For each water meter, the instantaneous flow value within the leakage determination time range is obtained. If the instantaneous flow value is continuously greater than the leakage determination threshold, it is determined that there is a suspected leakage in the public building corresponding to that water meter. After the suspected leakage determination, the acquisition command issuing unit searches for all sound signal collectors in the public building according to the preset water meter-public building-sound signal collector correspondence table, and issues sound signal acquisition commands to them. The command content includes: acquisition duration parameters, acquisition accuracy parameters, and the return server address.
[0123] Furthermore, the leak detection module is deployed on a cloud server. The sound signal receiving unit receives audio data uploaded by the sound signal collector through a network interface, performs data integrity verification, and transmits the data to the device identification unit after successful verification. The device identification unit loads a pre-trained CNN-Transformer model, performs preprocessing, log-Melogram feature extraction, and model inference on the audio data in sequence, and outputs probability values for each category and the final identification result. If the maximum probability is higher than a preset threshold, the identification result is considered valid and transmitted to the alarm management module for further processing; if the maximum probability is lower than the preset threshold, the identification result is marked as "insufficient confidence, manual review recommended". The identification result output unit integrates the identification result, confidence level, original audio file path, and other information into a leak detection record, stores it in the database, and transmits it to the alarm management module.
[0124] Furthermore, the alarm level classification rules of the alarm management module are as follows: Level 1 alarm (urgent) is a squat toilet leak with a confidence level greater than or equal to 0.9; Level 2 alarm (important) is other equipment leak with a confidence level greater than or equal to 0.8, or a squat toilet leak with a confidence level between 0.7 and 0.9; Level 3 alarm (general) is any equipment leak with a confidence level between 0.5 and 0.8, or if the confidence level is insufficient, manual review is recommended.
[0125] Furthermore, the alarm display unit presents the alarm list in a table format on the web interface, supporting filtering by alarm level, time range, building, and floor. Clicking on an alarm record will display a details page, which shows: complete alarm information, the traffic data curve that triggered the alarm, details of the sound signal recognition results, and an online player for the original audio.
[0126] Furthermore, the statistical reporting module automatically executes statistical tasks according to a preset period to generate leakage statistical reports. The report content includes: the total number of leakage alarms and a comparison with the previous period; the number of alarms categorized by leakage device type; the distribution of alarm numbers categorized by building and floor; the distribution of alarm levels; the trend of leakage alarm numbers; and an estimated leakage volume. Simultaneously, a list of historical reports is displayed in the system interface's reporting module, supporting online viewing and downloading.
[0127] In some embodiments, the following example of a complete monitoring process illustrates the operation of a public building restroom leak detection and location system:
[0128] (1) Equipment deployment phase: A smart water meter (equipment number D1-WM) was deployed on the main water supply pipe of a commercial office building, and sound signal collectors were deployed in each restroom. Among them, a sound signal collector (equipment number D1-5F-M01-SC) was deployed in the men's restroom on the 5th floor. The corresponding relationship between each device was established in the cloud background. There are 3 squat toilets, 4 washbasins and 5 urinals in the men's restroom on the 5th floor.
[0129] (2) Daily monitoring stage: The smart water meter D1-WM reports flow data to the cloud backend according to the preset cycle. The cloud backend real-time monitoring module receives and stores the data, and displays the real-time status of the water meter on the visualization interface.
[0130] (3) Anomaly detection phase: At 02:30 a.m. on a certain day, the intelligent trigger module entered the monitoring period and began to detect anomalies in the flow data of each water meter. At 03:00, the anomaly judgment unit detected that the instantaneous flow of D1-WM had continuously exceeded the leakage judgment threshold in the past 30 minutes, and judged that there was a suspected leakage in the public building.
[0131] (4) Sound signal acquisition stage: The acquisition command issuing unit sends sound signal acquisition commands to all sound signal collectors in the public building. Taking D1-5F-M01-SC in the men's restroom on the 5th floor as an example, after receiving the command, it starts the microphone module to acquire sound signals. After the acquisition is completed, the WAV format audio file is uploaded to the cloud server through the 4G network.
[0132] (5) Leakage identification stage: The leakage identification module receives audio data and performs preprocessing, log-Melogram feature extraction, and CNN-Transformer model inference in sequence. The model output results are: Category 1 (squat toilet leakage) probability 0.87, Category 2 (washbasin leakage) probability 0.05, and other categories have probabilities less than 0.05. The final identification result is [squat toilet leakage], with a confidence level of 0.87.
[0133] (6) Alarm Generation Stage: The alarm management module generates alarm records based on the identification results. Since the leaking device is a squat toilet and the confidence level is 0.87 (between 0.7 and 0.9), the alarm level is determined to be Level 2 (Important). The alarm content includes: alarm time 03:05, location 5th floor men's restroom, leaking device squat toilet, confidence level 87%.
[0134] (7) Alarm handling stage: During working hours, the property maintenance personnel went to the men's restroom on the 5th floor to check based on the alarm information and found that the water inlet valve of squat toilet No. 2 was leaking. After replacing the water inlet valve, the leak was eliminated. The maintenance personnel marked the alarm as "handled" in the cloud backend and filled in the handling result remarks.
[0135] (8) Statistical summary stage: On Sunday morning, the statistical report module automatically generates a weekly water leakage statistical report. The alarm is included in the statistics. The report shows that a total of 3 water leakage alarms occurred this week, including 2 leaks in the squat toilet and 1 leak in the washbasin.
[0136] In some embodiments, the smart water meter is deployed on the main water supply pipe of a public building, with one meter per building, to monitor changes in the total water supply flow of the entire building and determine whether there is a leak within the building. The acoustic signal collector is deployed inside the restroom to collect acoustic signals generated by water-using devices within the restroom. The leak analysis model is deployed on a cloud server to extract features and classify the acoustic signals. The cloud backend is used to receive monitoring data, execute leak detection logic, generate alarm information, and perform data statistics and display.
[0137] In a specific embodiment, a 20-story commercial office building is used as an example. Each floor of the building has one public restroom for men and one for women, totaling 40 restrooms. The building is equipped with a smart water meter installed on the main water supply pipe to monitor the total water flow of the entire building; each restroom is equipped with a sound signal collector installed on the ceiling. All devices communicate with the cloud backend via a 4G wireless network.
[0138] The smart water meter mentioned is a commercially available product that meets the functional requirements of this system. The smart water meter selected in this application should have high metering accuracy and high sensitivity, and be able to detect minute changes in flow in the pipeline network; it should have remote wireless data transmission capabilities, supporting at least one of the communication methods such as NB-IoT, 4G LTE, and LoRa; and it should support both timed reporting and real-time reporting modes.
[0139] Currently, there are various commercial smart water meter products on the market that meet the above requirements, such as smart water meters using ultrasonic flow metering technology. These meters calculate flow velocity by measuring the time difference of ultrasonic waves propagating in the upstream and downstream directions, offering advantages such as high metering accuracy, low starting flow rate, and no mechanical wear. The selected smart water meter should have data processing and storage functions, capable of flow calculation, data timestamp marking, and responding to configuration and query commands issued by the cloud backend.
[0140] Furthermore, each smart water meter has a unique device number, which is formatted as [Building Number - Device Type Code]. For example, [D6-WM] indicates a smart water meter for building number 6. The correspondence between the device number and the building is entered into the device management module in the cloud backend during device installation.
[0141] In some embodiments, please refer to Figure 10The acoustic signal collector includes a housing and an acoustic sound collection cavity, a main control board, a microphone module, a 4G communication module, a battery module, and an antenna disposed inside the housing.
[0142] Specifically, the housing is made of engineering plastic and has a corresponding protection level, capable of adapting to the humid environment of a bathroom. The housing surface has heat dissipation holes and sound inlet holes, with the sound inlet holes equipped with dustproof and waterproof mesh covers. A mounting bracket is provided on the back of the housing, supporting wall-mounted or ceiling-mounted installation. In this embodiment, the sound signal collector is installed in the bathroom ceiling area, with the installation location chosen in a suitable area within the bathroom to ensure effective reception of sound signals generated by various water-using devices.
[0143] Furthermore, the acoustic sound-collecting cavity is located inside the housing and adopts a parabolic acoustic optimization structure design, with its inner wall forming a parabolic shape. The microphone module is installed in the sound wave converging area where the focal point of the parabola is located.
[0144] The working principle of the acoustic sound collecting cavity includes: ambient sound waves enter through the sound inlet of the shell, are reflected and focused by the parabolic inner wall of the cavity, and utilize the geometric properties of the parabola to converge parallel incident sound waves to the focal point, efficiently converging sound waves from all directions to the microphone. This design can significantly enhance the received sound signal and improve the signal-to-noise ratio of the sound signal acquisition. The inner wall of the acoustic sound collecting cavity can be coated with a sound-absorbing material layer to absorb reflected noise inside the cavity, reduce sound reverberation and interference, and improve the clarity of the acquired signal.
[0145] Furthermore, the microphone module employs a high-sensitivity, high-signal-to-noise ratio digital microphone. This digital microphone directly outputs digital audio signals, simplifying circuit design and improving anti-interference capabilities. The microphone module connects to the main control board via a digital interface, allowing the main control board to directly read the digitized sound sampling data.
[0146] Furthermore, the main control board uses a low-power microprocessor as its core and integrates a storage module for program storage and data caching. The main control board performs the following functions:
[0147] (1) Timed control function, automatically starts and stops the microphone module according to the preset time strategy;
[0148] (2) Data acquisition function: reads digital audio data output from the microphone;
[0149] (3) Signal preprocessing function: to filter and normalize the original audio signal; data temporary storage function: to temporarily store the preprocessed audio data;
[0150] (4) Data transmission function: audio data is uploaded to the cloud server via the 4G communication module;
[0151] (5) Power management function: monitors battery level and enters low-power sleep mode to extend battery life.
[0152] Furthermore, the 4G communication module supports multiple mobile network standards, meeting the bandwidth requirements for audio data upload. The battery module uses a rechargeable lithium-ion battery pack with sufficient capacity to support long-term stable operation of the device. The battery module supports charging via an external charger.
[0153] Step S20: After performing wavelet denoising, pre-emphasis, and frame-windowing processing on the acoustic signal to be identified, the power spectrum is calculated using short-time Fourier transform. After filtering by a Mel filter bank, logarithmic compression is performed to determine the log-Mel spectral features, thereby constructing the log-Mel spectrum corresponding to the acoustic signal to be identified. Figure 2 The dimensional feature matrix, and the log-Mel spectrum Figure 2 The feature matrix is subjected to global mean-variance normalization.
[0154] After acquiring the acoustic signals to be identified from each toilet in a suspected leaking public building, the acoustic signals are sequentially processed by wavelet denoising, pre-emphasis, and frame-segmentation windowing. The power spectrum is then calculated using short-time Fourier transform, filtered by a Mel filter bank, and logarithmically compressed to determine the log-Mel spectral characteristics, thereby constructing the log-Mel spectrum corresponding to the acoustic signals to be identified. Figure 2 The dimensional feature matrix, and the log-Mel spectrum Figure 2 The feature matrix is subjected to global mean-variance normalization.
[0155] The feature extraction of the acoustic signal to be identified can use a log-Mel spectrogram as the input feature representation of the leaky device identification model in this application. Compared with traditional MFCC features, the log-Mel spectrogram retains relatively complete spectral details and avoids the spectral information compression loss caused by discrete cosine transform (DCT), making it more suitable as the input of a convolutional neural network. The log-Mel spectrogram is presented in the form of a two-dimensional time-frequency graph, with the horizontal axis representing the time frame and the vertical axis representing the Mel frequency channels. CNNs can simultaneously extract local feature patterns in both the time and frequency dimensions from it.
[0156] In some embodiments, after performing wavelet denoising, pre-emphasis, and frame-windowing processing sequentially on the acoustic signal to be identified, the power spectrum is calculated using short-time Fourier transform, filtered by a Mel filter bank, and then logarithmically compressed to determine the log-Mel spectral features, thereby constructing the log-Mel spectrum corresponding to the acoustic signal to be identified. Figure 2 The steps for constructing a 3D feature matrix include:
[0157] Step S201: Perform wavelet denoising, pre-emphasis and frame windowing processing on the sound signal to be identified in the toilet to be detected in sequence. Perform short-time Fourier transform on the sound signal to be identified after frame windowing processing to calculate the power spectrum of the sound signal to be identified in each frame.
[0158] Step S202: Filter the power spectrum using a Mel filter bank containing multiple triangular bandpass filters with center frequencies uniformly distributed according to Mel scale, calculate the cumulative energy value of each frame of the acoustic signal to be identified in each frequency band of each triangular bandpass filter, and obtain the output energy value of each frame of the acoustic signal to be identified in each triangular bandpass filter.
[0159] Step S203: Perform logarithmic compression on the output energy value to determine the log-Mel spectrum features. Arrange the log-Mel spectrum features of all frames of the sound signal to be identified in chronological order to construct the log-Mel spectrum corresponding to the sound signal to be identified. Figure 2 3D feature matrix.
[0160] Specifically, please refer to Figure 11 The acoustic signal to be identified is sequentially subjected to wavelet denoising, pre-emphasis, and frame-segmentation windowing. Wavelet denoising can be achieved by performing multi-level wavelet decomposition on the original acoustic signal using wavelet basis functions to obtain low-frequency approximate components and multiple high-frequency detail components. A soft thresholding function is applied to each level of high-frequency components. After soft thresholding, signal reconstruction is performed based on the wavelet coefficients after soft thresholding to obtain the denoised acoustic signal to be identified. The soft thresholding function for wavelet denoising is described below. The calculation formula is expressed as:
[0161]
[0162] in, This represents the soft threshold for wavelet denoising, used for thresholding high-frequency detail components; This is an estimate of the noise standard deviation, characterizing the degree of dispersion of noise in the acoustic signal to be identified; The signal length is the total number of sampling points of the acoustic signal to be identified.
[0163] Furthermore, pre-emphasis is used to compensate for the attenuation of high-frequency components during sound signal propagation, making the signal spectrum flatter. A first-order high-pass filter can be used for pre-emphasis processing of the signal. The transfer function of a first-order high-pass filter is expressed as:
[0164]
[0165] in, Indicates the pre-emphasis filter Domain transfer function; This is the pre-emphasis coefficient, used to compensate for the attenuation of high-frequency components during the propagation of the acoustic signal to be identified; This represents the unit delay operator, which characterizes the first-order delay of a signal.
[0166] Furthermore, the frame segmentation and windowing is used to reduce spectral leakage during the framing process and improve the frequency domain characteristics of the signal. Frame segmentation and windowing involves dividing the continuous acoustic signal into several short frames for analysis. The frame length and frame shift are set according to the actual sampling rate, and a certain proportion of overlap is maintained between adjacent frames. Each frame signal is multiplied by a Hamming window function, where the Hamming window function is expressed as:
[0167]
[0168] in, Indicates the first The Hamming window weight values corresponding to each sampling point; This represents the intra-frame sampling point number, with values ranging from 0, 1, 2, ... ; The frame length is the total number of sampling points for each short-time acoustic signal.
[0169] Furthermore, a short-time Fourier transform is performed on the frame-windowed acoustic signal to be identified to calculate the power spectrum of each frame; a fast Fourier transform (FFT) is performed on each windowed acoustic signal to obtain the complex spectrum of each frame, and the power spectrum is calculated from the FFT result. The calculation formula is as follows:
[0170]
[0171] in, Indicates the first The power spectrum value corresponding to each frequency point represents the signal energy at that frequency point; For the first The Fast Fourier Transform (FFT) of each frequency point is a complex value containing amplitude and phase information.
[0172] Furthermore, the power spectrum is filtered by a Mel filter bank containing multiple triangular bandpass filters with center frequencies uniformly distributed according to the Mel scale. The cumulative energy value of each frame of the acoustic signal to be identified in each frequency band of each triangular bandpass filter is calculated to obtain the output energy value of each frame of the acoustic signal to be identified in each triangular bandpass filter. A design is then developed that includes... A Mel filter bank with three triangular bandpass filters, the center frequencies of which are uniformly distributed according to the Mel scale. The conversion relationship between the Mel scale and the Hertz frequency is expressed as:
[0173]
[0174] in, Represents Hertz frequency The corresponding Mel frequency value conforms to the non-linear perception characteristics of sound by the human ear; is the actual Hertz frequency, and is the physical frequency of the acoustic signal to be identified.
[0175] Each triangular filter The response is triangular, with the edges of adjacent filters overlapping. The cumulative energy value of the acoustic signal to be identified in each frame is calculated across each frequency band in each triangular bandpass filter, yielding the output energy value of the acoustic signal to be identified in each frame in each triangular bandpass filter. The output energy value of each triangular bandpass filter in the Mel filter bank is expressed as:
[0176]
[0177] in, Indicates the first Output energy value of a triangular bandpass filter; This represents the total number of triangular bandpass filters in the Mel filter bank; Indicates the first The triangular bandpass filter at the ... Response values at each frequency point; Represents the summation operator, for From 0 to Sum all the values.
[0178] Furthermore, the output energy value is logarithmically compressed to determine the log-Mel spectrum characteristics. Logarithmic compression simulates the nonlinear perception of sound intensity by the human ear and compresses the dynamic range of the characteristics, which is beneficial to the stability of model training. The logarithm of the output energy value of each triangular bandpass filter in the Mel filter bank is taken to obtain the log-Mel spectrum characteristics, where the calculation formula for the log-Mel spectrum characteristics is expressed as:
[0179]
[0180] in, Indicates the first Log-Mel spectrum eigenvalues corresponding to each triangular bandpass filter; It is a very small positive number, used to avoid Numerical overflow in logarithmic operations.
[0181] Furthermore, by arranging the log-Mel spectrum features of the acoustic signals to be identified in all frames in chronological order, the log-Mel spectrum corresponding to the acoustic signals to be identified is constructed. Figure 2 3D characteristic matrix, construct log-Mel spectrum Figure 2 3D feature matrix ,in, This represents the total number of triangular bandpass filters in the Mel filter bank, which is also the number of Mel frequency channels. This is the number of time frames. The logarithmic Mel spectrum. Figure 2 The 3D feature matrix can be viewed as a single-channel spectral image, with the horizontal axis representing the time dimension and the vertical axis representing the frequency dimension. Pixel values are the logarithmic energy values corresponding to the time-frequency positions. For the log-Mel spectrum... Figure 2 The feature matrix is subjected to global mean-variance normalization to eliminate the differences in feature distribution under different recording conditions and improve the generalization ability of the model.
[0182] Step S30: Normalize the log-Mel spectrum after global mean-variance processing. Figure 2 The 3D feature matrix is input into a pre-trained leaky device identification model, and the CNN feature extraction module processes the log-Mel spectrum. Figure 2 The 3D feature matrix is used to extract local time-frequency features and transform them into a 1D time-series feature sequence. After being processed by a linear projection layer and a position coding layer, it is input into a Transformer encoder. The global temporal dependencies of the 1D time-series feature sequence are captured by a multi-head self-attention sublayer and then processed by a feedforward neural network sublayer to determine the first output feature sequence.
[0183] After performing wavelet denoising, pre-emphasis, and frame-windowing processing on the acoustic signal to be identified, the power spectrum is calculated using short-time Fourier transform. After filtering by a Mel filter bank, logarithmic compression is performed to determine the log-Mel spectral features, thereby constructing the log-Mel spectrum corresponding to the acoustic signal to be identified. Figure 2 The dimensional feature matrix, and the log-Mel spectrum Figure 2 After performing global mean-variance normalization on the eigenvalue matrix, the log-Mel spectrum of the normalized eigenvalue matrix is then analyzed. Figure 2 The 3D feature matrix is input into a pre-trained leaky device identification model, and the CNN feature extraction module processes the log-Mel spectrum. Figure 2 The 3D feature matrix is used to extract local time-frequency features and transform them into a 1D time-series feature sequence. After being processed by a linear projection layer and a position coding layer, it is input into a Transformer encoder. The global temporal dependencies of the 1D time-series feature sequence are captured by a multi-head self-attention sublayer and then processed by a feedforward neural network sublayer to determine the first output feature sequence.
[0184] Specifically, the CNN-Transformer hybrid model is used for deep learning classification of the extracted log-Mel spectrogram features to determine whether the sound signal is a water leakage sound signal and to identify the type of leaking device. This method integrates the local feature extraction capability of convolutional neural networks (CNNs) with the global modeling capability of Transformers to achieve joint modeling of the time-frequency features of water leakage sound signals. Specifically, the CNN module extracts local time-frequency patterns (such as energy texture in specific frequency bands, short-term spectral changes, etc.) from the log-Mel spectrogram through multi-layer convolution and pooling operations. The Transformer encoder module, on the other hand, performs global temporal dependency modeling on the high-level feature sequence output by the CNN through a self-attention mechanism, capturing the dynamic evolution of the water leakage sound signal over a longer time span (such as the persistence and periodic fluctuations of the water leakage sound). The Transformer decoder module models the causal temporal relationship of the sequence through a masked self-attention mechanism and integrates the global feature information extracted by the encoder into the decoding process through an encoder-decoder cross-attention mechanism, achieving further refinement and enhancement of the sound signal features.
[0185] In some embodiments, please refer to Figure 12The basic network architecture of the leaky device identification model is a CNN-Transformer hybrid model, which includes, in sequence, an input layer, a CNN feature extraction module, a linear projection layer, a position encoding layer, a Transformer encoder module, a Transformer decoder module, a global average pooling layer, a Dropout layer, a fully connected layer, a Softmax layer, and a classification output layer. The CNN feature extraction module consists of multiple cascaded convolutional blocks, each including a CNN convolutional layer, a batch normalization layer, a GELU activation function layer, and a max pooling layer. The position encoding layer includes a sinusoidal position encoding layer. The Transformer encoder module includes multiple identical Transformer encoder layers. The Transformer decoder module is composed of multiple identical Transformer decoder layers stacked together. Each Transformer decoder layer uses a Pre-Norm residual structure to perform feature processing sequentially. It includes a multi-head self-attention sub-layer and a feedforward neural network sub-layer. The multi-head self-attention sub-layer and the feedforward neural network sub-layer are equipped with layer normalization and residual connection layers. The feedforward neural network sub-layer includes two linear transformation layers and a GELU activation function layer. The Transformer decoder module is composed of multiple identical Transformer decoder layers stacked together. Each Transformer decoder layer uses a Pre-Norm residual structure to perform feature processing sequentially. It includes a masked multi-head self-attention sub-layer, a cross-attention sub-layer, and a feedforward neural network sub-layer. The masked multi-head self-attention sub-layer, the cross-attention sub-layer, and the feedforward neural network sub-layer are equipped with layer normalization and residual connection layers.
[0186] In some embodiments, the input layer of the CNN-Transformer hybrid model receives a log-Mel spectrum after global mean-variance normalization. Figure 2 The dimensional feature matrix, whose size is expressed as ,in, Number of Mel frequency channels, 1 represents the number of time frames and 1 represents the number of input channels. The input layer normalizes the feature data to eliminate the dimensional differences between different feature dimensions, thereby improving the convergence speed and stability of model training.
[0187] In some embodiments, the log-Mel spectrum is processed by the CNN feature extraction module. Figure 2 The steps for extracting local time-frequency features from a 3D feature matrix and converting it into a one-dimensional time-series feature sequence include:
[0188] Step S301: Normalize the log-Mel spectrum after global mean-variance processing. Figure 2 The 3D feature matrix is input into the CNN feature extraction module;
[0189] Step S302: Each convolutional block in the CNN feature extraction module sequentially processes the log-Mel spectrum. Figure 2 The log-Mel spectrum is extracted by performing two-dimensional convolution, batch normalization, GELU activation function processing, and max pooling on the feature matrix. Figure 2 Local time-frequency features of the 3D feature matrix;
[0190] Step S303: After all convolutional blocks are cascaded, the output two-dimensional time-frequency feature map is compressed in the frequency dimension and transformed into a one-dimensional time-series feature sequence.
[0191] Specifically, the CNN feature extraction module consists of multiple cascaded convolutional blocks, each including a CNN convolutional layer, a batch normalization (BN) layer, a GELU activation function layer, and a max pooling layer connected in sequence. The function of the CNN feature extraction module is to extract local time-frequency features from the log-Melogram and reduce the time-frequency resolution of the feature map through layer-by-layer pooling, compressing the high-dimensional spectrogram into a compact feature sequence.
[0192] Convolutional layers employ "same" padding to preserve spatial dimensions. The convolution operation slides a learnable 2D convolutional kernel across a log-Melogram, calculating a weighted sum of the kernel and its covered local receptive field at each location. For multi-channel input, the convolution operation is expressed as:
[0193]
[0194] in, To output the feature map in spatial location First The activation value of each output channel; For the log-Mel spectrum in spatial location First The value of each input channel; These are the convolution kernel weight parameters, corresponding to the first... The output channel, the first One input channel, convolution kernel position ; The number of input channels for the log-Mel spectrum; Let be the side length of the square convolution kernel; Indicates the index of the output channel; Indicates the index of the input channel; Indicates the pixel index within the convolution kernel; This represents the spatial location index of the feature map.
[0195] Furthermore, a batch normalization (BN) layer is introduced after each CNN convolutional layer to accelerate training convergence and improve model stability. Batch normalization normalizes the features of each channel on the current mini-batch, making the input distribution of each network layer more stable by eliminating internal covariate shift, thus allowing for a higher learning rate and accelerating the training process.
[0196] Each convolutional block employs a GELU (Gaussian Error Linear Unit) activation function layer. Compared to the ReLU activation function, the GELU activation function has a smooth transition region near zero, better preserving near-zero feature information. It is widely used in the Transformer architecture and has been proven to improve model performance.
[0197] Spatial downsampling is performed using a max pooling layer, and its calculation formula is expressed as follows:
[0198]
[0199] in, This indicates the position of the output feature map after max pooling. Eigenvalues at; This represents the maximum value operator, used to select the maximum value within a specified range; Represents the spatial coordinates on the input feature map; Indicates output position The pooling window region centered on the input feature map is composed of a set of... Coordinate structure; Indicates traversing the pooled window region All input locations ; This indicates that the input feature map is at the location The eigenvalue at that location.
[0200] Max pooling operates by progressively downsampling along the frequency dimension, eventually compressing the frequency dimension to 1, so that the output of the CNN feature extraction module becomes a one-dimensional temporal feature sequence arranged along the time axis.
[0201] After processing through all convolutional blocks, the output two-dimensional time-frequency feature map is compressed along the frequency dimension, transforming it into a one-dimensional time-series feature sequence. After further processing by multiple convolutional blocks, the CNN feature extraction module extracts the feature map of size [size missing]. The log-Mel spectrum is converted into a one-dimensional temporal feature sequence, which is then mapped to the model feature dimension required by the Transformer encoder through a linear projection layer. .
[0202] In a further embodiment, the step of determining the first output feature sequence after processing by a linear projection layer and a position encoding layer, and then inputting the sequence to a Transformer encoder to capture the global temporal dependencies of the one-dimensional temporal feature sequence through a multi-head self-attention sublayer and processing it through a feedforward neural network sublayer, includes:
[0203] Step S3001: Input the one-dimensional temporal feature sequence into the linear projection layer, and map the dimension of the one-dimensional temporal feature sequence to the model feature dimension of the Transformer encoder through linear transformation to obtain the dimension-adapted feature sequence.
[0204] Step S3002: Input the dimension-adapted feature sequence into the sinusoidal position coding layer, add position coding values generated based on sinusoidal position coding to the dimension-adapted feature sequence, and obtain the dimension-position fusion feature sequence;
[0205] Step S3003: Input the dimension-location fusion feature sequence into a Transformer encoder composed of multiple identical Transformer encoder layers stacked together. For each Transformer encoder layer, first calculate the attention weight between each time position in the dimension-location fusion feature sequence through its corresponding multi-head self-attention sub-layer, capture the global temporal dependency of the one-dimensional temporal feature sequence, and obtain the intermediate feature sequence with fused global dependency.
[0206] Step S3004: For the intermediate feature sequence, perform nonlinear feature transformation through its corresponding feedforward neural network sub-layer to enhance the nonlinear expressive power of the features and obtain a feature sequence optimized by the feedforward neural network sub-layer.
[0207] Step S3005: After all Transformer encoder layers are stacked one by one, a feature sequence that integrates global temporal dependency information and nonlinear features is output as the first output feature sequence of the Transformer encoder.
[0208] Specifically, the one-dimensional temporal feature sequence is input into a linear projection layer, and a linear transformation maps the dimension of the one-dimensional temporal feature sequence to the model feature dimension of the Transformer encoder. We obtain a dimension-adapted feature sequence. The Transformer's self-attention mechanism itself does not contain positional information. Therefore, positional encoding is needed to introduce positional awareness into the input sequence.
[0209] A sinusoidal positional encoding layer can be used to add positional encoding values generated based on sinusoidal positional encoding to the dimension-adapted feature sequence, resulting in a dimension-positional fusion feature sequence; wherein, the calculation formula for sinusoidal positional encoding includes:
[0210]
[0211]
[0212] in, Represents sequence position pos and dimension index The positional encoding value corresponding to (even-dimensional) dimensions; Represents sequence position pos and dimension index (Odd-dimensional) The positional encoding value; pos is the position index in the dimension-adapted feature sequence, representing the order of elements in the dimension-adapted feature sequence; Index for the model's feature dimensions; This represents the model feature dimension of the Transformer encoder. This represents a sine function used to generate periodic position codes; This represents the cosine function, used to generate periodic position codes.
[0213] The advantages of sinusoidal positional encoding are: its encoded values are bounded and do not increase with the sequence length; the relative distance between different positions can be represented by a linear transformation, enabling the model to learn relative positional relationships; it does not contain learnable parameters and does not increase model complexity. The positional encoding is added element-wise to the feature sequence output by the CNN.
[0214] Furthermore, the Transformer encoder module is the core global dependency modeling component of the CNN-Transformer hybrid model, used to capture long-range temporal dependencies in the feature sequences of acoustic signals.
[0215] Transformer encoder is composed of The Transformer encoder layer is composed of stacked identical Transformer layers. Each Transformer encoder layer contains a Multi-Head Self-Attention (MHSA) sublayer and a Feed-Forward Network (FFN) sublayer. Both the MHSA and FFN sublayers employ residual connections and layer normalization. Layer normalization (LN) normalizes all feature dimensions of a single sample. Unlike batch normalization, layer normalization does not depend on the statistics of mini-batch samples, thus it is more stable in scenarios with variable sequence lengths. Each Transformer encoder layer uses a pre-norm residual structure to perform feature processing sequentially; that is, layer normalization is performed first, followed by computation in the MHSA and FFN sublayers.
[0216] The Multi-Head Self-Attention (MHSA) sublayer is a core component of the Transformer encoder layer, modeling global dependencies by calculating attention weights between positions in the sequence. For the input sequence, it is first mapped to query, key, and value matrices using three linear projection matrices. The formula for Scaled Dot-Product Attention is as follows:
[0217]
[0218] in, , , These represent the query matrix, key matrix, and value matrix, respectively, obtained by projecting from linear projection matrices. A matrix representing the transpose of the key matrix; For each attention head, the feature dimension is composed of... The calculation is determined; The number of attention heads. This is a scaling factor used to prevent the dot product value from becoming too large when the dimension is large. The function enters a saturation region where the gradient is minimal; This represents the normalization function, which converts the dot product result into attention weights between 0 and 1, with the weights summing to 1.
[0219] The multi-head self-attention (MHSA) sublayer will query the matrix. Key matrix Value matrix The process involves splitting the attention into h attention heads, computing them in parallel, and then concatenating the outputs of each attention head and projecting them linearly. The Multi-Head Self-Attention (MHSA) mechanism enables the model to simultaneously focus on information at different positions in the sequence across different representation subspaces, enhancing the model's expressive power. In the task of identifying leaking sound signals, MHSA can capture global dependencies between different temporal locations, such as the persistence pattern and periodic fluctuations of the leaking sound.
[0220] The Transformer encoder module comprises multiple identical Transformer encoder layers stacked together. Each Transformer encoder layer uses a Pre-Norm residual structure to sequentially complete feature processing, including a multi-head self-attention sub-layer and a feedforward neural network sub-layer. Both the multi-head self-attention sub-layer and the feedforward neural network sub-layer are configured with layer normalization and residual connection layers. Furthermore, the feedforward neural network (FFN) sub-layer independently performs nonlinear transformations on the features at each time position. The feedforward neural network sub-layer includes two linear transformation layers and a GELU activation function layer. The calculation formula for the feedforward neural network sub-layer is expressed as follows:
[0221]
[0222] in, This represents the input feature vector of a feedforward neural network (FFN) sublayer; , Let the weight matrix represent the linear transformation, where, Represents an upgraded matrix. Represents a dimension-reduced matrix; , The bias vector represents the linear transformation; express The Gaussian Error Linear Unit (Gaussian Error Linear Unit) activation function preserves near-zero feature information; the hidden layer dimension of a feedforward network is typically set to... Feed-Forward Network (FFN) sublayers enhance the nonlinear fitting ability of the model through a structure that first expands and then compresses.
[0223] Each Transformer encoder layer employs a Pre-Norm residual connection structure. The residual connection allows the gradient to propagate directly back to the shallow layers, effectively alleviating the gradient vanishing problem in deep networks. At the same time, it allows the network to learn identity mappings, reducing the optimization difficulty.
[0224] Step S40: Input the first output feature sequence into the Transformer decoder. The second output feature sequence is determined by modeling the temporal causal relationship through the masked multi-head self-attention sub-layer and fusing global time-frequency features through the cross-attention sub-layer. After processing through the global average pooling layer and the fully connected layer, the normalized classification probability distribution is output by the Softmax layer.
[0225] Step S50: If the maximum classification probability is higher than the preset confidence threshold, then the leakage equipment category corresponding to the maximum classification probability is taken as the leakage equipment category corresponding to the toilet to be detected, so as to complete the identification and location of water leakage in public building toilets.
[0226] Log-Mel spectrum after global mean-variance normalization Figure 2 The 3D feature matrix is input into a pre-trained leaky device identification model, and the CNN feature extraction module processes the log-Mel spectrum. Figure 2 The 3D feature matrix is used to extract local time-frequency features and transform them into a 1D time-series feature sequence. After processing by a linear projection layer and a position encoding layer, it is input to a Transformer encoder. A multi-head self-attention sublayer captures the global temporal dependencies of the 1D time-series feature sequence, and after processing by a feedforward neural network sublayer, the first output feature sequence is determined. The first output feature sequence is then input to a Transformer decoder. A masked multi-head self-attention sublayer models temporal causality, and a cross-attention sublayer fuses global time-frequency features for feature refinement to determine the second output feature sequence. After processing by a global average pooling layer and a fully connected layer, a normalized classification probability distribution is output by a Softmax layer. If the maximum classification probability is higher than a preset confidence threshold, the category of the leaking device corresponding to the maximum classification probability is taken as the category of the leaking device corresponding to the toilet to be detected, so as to complete the identification and location of water leakage in public building toilets. The categories of leaking devices include no leakage, squat toilet leakage, accessory leakage, urinal leakage, or faucet leakage.
[0227] In some embodiments, the step of inputting the first output feature sequence into a Transformer decoder, refining the features by modeling temporal causality through a masked multi-head self-attention sublayer and fusing global time-frequency features through a cross-attention sublayer to determine the second output feature sequence includes:
[0228] Step S401: Input the first output feature sequence of the Transformer encoder into the Transformer decoder module, which is composed of multiple identical Transformer decoder layers stacked together, as the initial input feature of the masked multi-head self-attention sub-layer in each Transformer decoder layer;
[0229] Step S402: Through the masked multi-head self-attention sub-layer of each Transformer decoder layer in the Transformer decoder, perform temporal modeling on the first output feature sequence, mask future time location feature information, so as to construct temporal causal relationship and obtain masked self-attention output features;
[0230] Step S403: Through the cross-attention sub-layer of each Transformer decoder layer, using the mask self-attention output feature as the query feature and the first output feature sequence output by the Transformer encoder module as the key feature and value feature, attention calculation is performed to fuse global time-frequency feature information and obtain cross-attention output features;
[0231] Step S404: After all Transformer decoder layers in the Transformer decoder complete the temporal causal relationship modeling and global time-frequency feature fusion, the second output feature sequence with refined features is obtained.
[0232] The Transformer decoder module works in conjunction with the encoder module to perform further sequence modeling and feature refinement on the global feature representation extracted by the encoder. The Transformer decoder module consists of... The same Transformer decoder layers are stacked together. Each decoder layer contains a masked multi-head self-attention sublayer, an encoder-decoder attention sublayer, and a feed-forward network (FFN) sublayer. Each sublayer employs residual connections and layer normalization.
[0233] The Masked Multi-Head Self-Attention (MFA) sublayer is the first sublayer in the Transformer decoder. Its basic computation mechanism is the same as the MFA in the encoder, but a mask matrix is introduced during attention calculation. The mask matrix sets the attention weights for all positions after the current position to negative infinity. After Softmax normalization, these attention weights approach zero, ensuring that the decoder can only focus on the current position and its preceding positions when generating the output, and cannot access information about future positions. The calculation formula for the Masked Multi-Head Self-Attention sublayer is expressed as follows:
[0234]
[0235] in, It is a mask matrix, with the upper triangular elements being negative infinity (-∞) and the lower triangular and diagonal elements being zero; , , These represent the query matrix, key matrix, and value matrix, respectively, obtained by projecting from linear projection matrices. A matrix representing the transpose of the key matrix; For each attention head, the feature dimension is composed of... The calculation is determined; The number of attention heads. This is a scaling factor used to prevent the dot product value from becoming too large when the dimension is large. The function enters a saturation region where the gradient is minimal; The normalization function is represented, which converts the dot product result into attention weights between 0 and 1, with the weights summing to 1. This masking mechanism enables the decoder to progressively model sequence features in an autoregressive manner, which helps capture the causal evolution of sound signal features along the time axis in water leakage sound signal recognition tasks. The output of the masked self-attention sublayer is passed to the encoder-decoder attention sublayer after residual connection and layer normalization.
[0236] The encoder-decoder attention sublayer is the second sublayer of the Transformer decoder layer, also known as the cross-attention sublayer. The query matrix of this cross-attention sublayer comes from the output of the previous sublayer of the decoder (i.e., the masked multi-head self-attention sublayer), while the key and value matrices come from the output of the last layer of the Transformer encoder module. The calculation formula for the encoder-decoder attention sublayer is expressed as follows:
[0237]
[0238] in, The query matrix is obtained by linear projection of the mask self-attention sublayer output in the Transformer decoder layer. This represents the key matrix obtained by linear projection of the final output of the Transformer encoder layer; This represents the value matrix obtained by linear projection of the final output of the Transformer encoder layer.
[0239] Through the Cross-Attention sublayer, each position of the Transformer decoder layer can perform weighted aggregation on all positions output by the Transformer encoder module, thereby integrating the global time-frequency feature information extracted by the Transformer encoder module into the sequence modeling process of the Transformer decoder layer.
[0240] In the task of identifying water leakage sound signals, the encoder-decoder attention sublayer enables the Transformer decoder layer to fully utilize the global spectral patterns and long-range dependency information captured by the Transformer encoder module during temporal inference, thereby enhancing the model's ability to distinguish the differences in sound signal characteristics of different water leakage devices. The output of this encoder-decoder attention sublayer is also passed to the feedforward neural network sublayer after residual connection and layer normalization.
[0241] The feedforward neural network (FFN) sublayer in the Transformer decoder layer has the same structure as the feedforward neural network (FFN) sublayer in the Transformer encoder layer. It consists of two linear transformations and a GELU activation function, performing nonlinear transformations independently on the features at each time point. The output of the feedforward neural network (FFN) sublayer is then passed to the next Transformer decoder layer as the final output after residual connections and layer normalization.
[0242] Each Transformer decoder layer also employs a pre-norm residual connection structure, meaning that layer normalization is performed before computation in each sub-layer. Residual connections ensure efficient gradient propagation in deep networks, while layer normalization maintains the stability of the input distribution across layers. The decoder layer processes the data layer by layer. The decoder module outputs a refined feature sequence, which integrates the global time-frequency features extracted by the encoder with the temporal causal information modeled by the decoder. This feature sequence is then fed into the global average pooling layer for further processing.
[0243] Furthermore, the Global Average Pooling (GAP) layer averages the feature sequence output by the Transformer decoder over time, compressing the variable-length feature sequence into a fixed-length feature vector. Global Average Pooling does not introduce additional learnable parameters and is robust to changes in the length of the input sequence.
[0244] The fully connected layer maps the feature vectors after global average pooling to the target class space, and its calculation formula is expressed as:
[0245]
[0246] in, This represents the output vector of the fully connected layer, with the dimension being the number of target categories. ; This is the weight matrix of the fully connected layer; This represents a fixed-length feature vector output by the global average pooling layer. This is the bias vector of the fully connected layer; The number of target categories.
[0247] A Dropout layer can be added before the fully connected layer to prevent overfitting by randomly discarding some of the neuron outputs. For the task of identifying water leakage sound signals, the number of output units of the fully connected layer can be set to 5, corresponding to the five target categories of "no leakage", "squat toilet leakage", "accessory leakage", "urinal leakage" and "faucet leakage".
[0248] Furthermore, the Softmax layer transforms the original class scores output by the fully connected layer into a normalized probability distribution, giving the model output explicit probabilistic semantics. For target class classification problems, the Softmax function converts the original scores into probability values, with all output values being positive and located in the (0,1) interval. The sum of the output probabilities for all classes is strictly equal to 1. The Softmax layer outputs five probability values, representing the probability that the sample belongs to one of the five categories: "no leaks," "leaky toilet," "leaky accessories," "leaky urinal," and "leaky faucet."
[0249] Furthermore, the classification output layer is the final layer of the CNN-Transformer hybrid model, responsible for calculating the loss function value based on the probability distribution output by the Softmax layer and generating the final classification prediction result. This embodiment uses the multi-class cross-entropy loss function as the optimization objective during model training, measuring the difference between the predicted probability distribution and the true label distribution. For the target category classification problem, the formula for calculating the cross-entropy loss function is:
[0250]
[0251] in, This represents the cross-entropy loss value, which characterizes the difference between the model's predicted probability and the true label. The smaller the value, the more accurate the prediction. This represents the number of target categories, which can take a value of 5. The first one-hot encoding of the sample's true label One element; The output of the Softmax layer The predicted probabilities of each category; The loss function is the natural logarithm. When the predicted probability distribution perfectly matches the true label distribution, the loss value approaches 0; as the difference between the two increases, the loss value increases.
[0252] During the model inference stage, the classification output layer selects the category of the leaking equipment corresponding to the maximum classification probability output by the Softmax layer as the final prediction result, which is the category of the leaking equipment corresponding to the toilet to be detected.
[0253] As can be seen from the above embodiments, compared with the prior art, this application addresses the problems of existing technologies, which mostly target external leakage or macroscopic areas of building structures, making it difficult to apply to the detection and precise location of leakage in specific water-using equipment in indoor water-using places such as bathrooms, as well as insufficient sensitivity for detecting minute leaks and significant detection lag. This application has, but is not limited to, the following beneficial effects:
[0254] Firstly, the public building toilet leakage identification and localization method of this application effectively filters out noise and retains leakage characteristic sound signals through a series of acoustic signal preprocessing processes such as wavelet denoising, pre-emphasis, and frame-segmentation windowing. Combined with the construction and normalization of log-Mel spectrum, it lays a high-quality feature foundation for model recognition. Relying on the advantages of the CNN-Transformer hybrid model, CNN extracts local time-frequency features, Transformer captures global temporal dependencies and completes feature refinement. Finally, the Softmax layer achieves accurate classification of leakage equipment categories, greatly improving the accuracy of leakage equipment identification.
[0255] Secondly, the public building toilet leak identification and location method of this application can accurately determine the specific type of leaking equipment such as squat toilets, faucets, urinals, and pipe fittings, breaking through the limitation of traditional leak detection that can only locate areas or pipes, providing clear direction for maintenance work and significantly improving the efficiency of leak treatment.
[0256] Thirdly, the public building toilet leakage identification and location method of this application is based on acoustic feature identification, which can capture the characteristic sound signals generated by tiny leaks in the water-using equipment in the toilet. Compared with traditional detection methods, it has higher sensitivity to detect tiny leaks, enabling early detection of leakage problems and avoiding secondary problems such as water waste and building structure damage caused by the expansion of leakage.
[0257] Fourth, the public building toilet leakage identification and location method of this application uses a pre-trained leakage equipment identification model to realize the automated analysis and classification of sound signals. The entire process from sound signal processing to leakage equipment location can be completed without manual intervention, replacing the inefficient mode of traditional manual inspection and improving the intelligence level and detection efficiency of public building toilet leakage detection.
[0258] Furthermore, by accurately and quickly identifying and locating leaks in public building restrooms, it is possible to promptly repair and handle leaking equipment, effectively reducing the ineffective waste of water resources, lowering the operation and maintenance costs of public buildings, and simultaneously aligning with the national requirements for water conservation, emission reduction, green and low-carbon development, thus promoting the green and intelligent operation and maintenance of public buildings.
[0259] Please see Figure 13A public building toilet leak detection and location device, provided to meet one of the purposes of this application, includes a target acoustic signal acquisition module 1100, a Mel spectrogram construction module 1200, a first feature sequence determination module 1300, a second feature sequence determination module 1400, and a leak detection and location module 1500. The target acoustic signal acquisition module 1100 is configured to acquire the target acoustic signals corresponding to each toilet to be detected in a public building suspected of leaking. The Mel spectrogram construction module 1200 is configured to sequentially perform wavelet denoising, pre-emphasis, and frame-windowing processing on the target acoustic signals, then calculate the power spectrum using short-time Fourier transform, filter the signals using a Mel filter bank, and perform logarithmic compression to determine the logarithmic Mel spectrum features, thereby constructing the logarithmic Mel spectrum corresponding to the target acoustic signals. Figure 2 The dimensional feature matrix, and the log-Mel spectrum Figure 2 The feature matrix is subjected to global mean-variance normalization; the first feature sequence determination module 1300 is configured to perform global mean-variance normalization on the log-Mel spectrum. Figure 2 The 3D feature matrix is input into a pre-trained leaky device identification model, and the CNN feature extraction module processes the log-Mel spectrum. Figure 2 The first output feature sequence is determined by extracting local time-frequency features from the 1D feature matrix and converting it into a 1D time-series feature sequence. After processing by a linear projection layer and a position encoding layer, it is input into a Transformer encoder. The encoder captures the global temporal dependencies of the 1D time-series feature sequence through a multi-head self-attention sublayer and then processes it through a feedforward neural network sublayer to determine the first output feature sequence. The second feature sequence determination module 1400 is configured to input the first output feature sequence into a Transformer decoder. The decoder then models the temporal causal relationship through a masked multi-head self-attention sublayer and refines the features by fusing global time-frequency features through a cross-attention sublayer to determine the second output feature sequence. After processing by a global average pooling layer and a fully connected layer, the normalized classification probability distribution is output by a Softmax layer. The leak identification and location module 1500 is configured to use the category of the leaking device corresponding to the maximum classification probability as the category of the leaking device corresponding to the toilet to be detected if the maximum classification probability is higher than a preset confidence threshold, so as to complete the identification and location of leaks in public building toilets.
[0260] Based on any embodiment of this application, please refer to Figure 14 Another embodiment of this application also provides an electronic device, which can be implemented by a computer device, such as... Figure 14The diagram shows the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable storage medium stores an operating system, a database, and computer-readable instructions. The database may store control information sequences. When the processor executes the computer-readable instructions, it enables the processor to implement a method for identifying and locating leaks in public building restrooms. The processor provides computing and control capabilities, supporting the operation of the entire computer device. The memory stores computer-readable instructions, which, when executed by the processor, enable the processor to execute the public building restroom leak identification and location method of this application. The network interface of the computer device is used for communication with a terminal. Those skilled in the art will understand that… Figure 14 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0261] In this embodiment, the processor is used to execute... Figure 13 The memory stores the specific functions of each module, and stores the program code and various data required to execute the above modules or sub-modules. The network interface is used for data transmission between the user terminal and the server. In this embodiment, the memory stores the program code and data required to execute all modules in the public building toilet leak detection and positioning device of this application, and the server can call the server's program code and data to execute the functions of all modules.
[0262] This application also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the public building toilet leakage identification and location method described in any embodiment of this application.
[0263] This application also provides a computer program product, including a computer program / instructions, which, when executed by one or more processors, implement the steps of the public building toilet leakage identification and location method described in any embodiment of this application.
[0264] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.
[0265] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A method for identifying and locating leaks in public building restrooms, characterized in that, include: Acquire the acoustic signals to be identified for each toilet to be inspected in a public building suspected of leaking water; After wavelet denoising, pre-emphasis and frame windowing are performed on the acoustic signal to be identified, the power spectrum is calculated by short-time Fourier transform, filtered by Mel filter bank and then logarithmically compressed to determine the log-Mel spectrum features, so as to construct the two-dimensional feature matrix of the log-Mel spectrum corresponding to the acoustic signal to be identified, and the two-dimensional feature matrix of the log-Mel spectrum is subjected to global mean-variance normalization. The log-Mel spectrogram two-dimensional feature matrix after global mean-variance normalization is input into the pre-trained leaky device identification model. The CNN feature extraction module performs local time-frequency feature extraction on the log-Mel spectrogram two-dimensional feature matrix and converts it into a one-dimensional time-series feature sequence. After processing by the linear projection layer and the position coding layer, it is input into the Transformer encoder. The multi-head self-attention sublayer captures the global temporal dependency of the one-dimensional time-series feature sequence, and after processing by the feedforward neural network sublayer, the first output feature sequence is determined. The first output feature sequence is input into the Transformer decoder. The second output feature sequence is determined by modeling the temporal causal relationship through the masked multi-head self-attention sub-layer and fusing global time-frequency features through the cross-attention sub-layer. After processing by the global average pooling layer and the fully connected layer, the normalized classification probability distribution is output by the Softmax layer. If the maximum classification probability is higher than the preset confidence threshold, the category of the leaking device corresponding to the maximum classification probability is taken as the category of the leaking device corresponding to the toilet to be detected, so as to complete the identification and location of the leaking toilet in the public building.
2. The method for identifying and locating leaks in public building restrooms according to claim 1, characterized in that, The basic network architecture of the leaky device identification model is a CNN-Transformer hybrid model, which includes an input layer, a CNN feature extraction module, a linear projection layer, a position encoding layer, a Transformer encoder module, a Transformer decoder module, a global average pooling layer, a Dropout layer, a fully connected layer, a Softmax layer, and a classification output layer connected in sequence. The CNN feature extraction module is composed of multiple convolutional blocks cascaded together, wherein each convolutional block includes a CNN convolutional layer, a batch normalization layer, a GELU activation function layer, and a max pooling layer connected in sequence. The position coding layer includes a sinusoidal position coding layer; The Transformer encoder module comprises multiple identical Transformer encoder layers stacked together. Each Transformer encoder layer uses a Pre-Norm residual structure to sequentially complete feature processing. It includes a multi-head self-attention sub-layer and a feedforward neural network sub-layer. Both the multi-head self-attention sub-layer and the feedforward neural network sub-layer are provided with layer normalization and residual connection layers. The feedforward neural network sub-layer includes two linear transformation layers and a GELU activation function layer. The Transformer decoder module is composed of multiple identical Transformer decoder layers stacked together. Each Transformer decoder layer uses a Pre-Norm residual structure to complete feature processing sequentially. It includes a masked multi-head self-attention sub-layer, a cross-attention sub-layer, and a feedforward neural network sub-layer. The masked multi-head self-attention sub-layer, the cross-attention sub-layer, and the feedforward neural network sub-layer are all equipped with layer normalization and residual connection layers. The categories of leaking equipment include no leaks, leaking squat toilets, leaking accessories, leaking urinals, or leaking faucets.
3. The method for identifying and locating leaks in public building restrooms according to claim 1, characterized in that, The steps of sequentially performing wavelet denoising, pre-emphasis, and frame-windowing processing on the acoustic signal to be identified, calculating the power spectrum using short-time Fourier transform, filtering it with a Mel filter bank, and then performing logarithmic compression to determine the log-Mel spectral features, in order to construct the two-dimensional feature matrix of the log-Mel spectrum corresponding to the acoustic signal to be identified, include: Wavelet denoising, pre-emphasis, and frame-segmentation windowing are sequentially performed on the acoustic signal to be identified in the toilet to be detected. Short-time Fourier transform is performed on the acoustic signal to be identified after frame-segmentation windowing to calculate the power spectrum of each frame of the acoustic signal to be identified. The power spectrum is filtered by a Mel filter bank containing multiple triangular bandpass filters with center frequencies uniformly distributed according to Mel scale. The cumulative energy value of each frame of the acoustic signal to be identified in each frequency band of each triangular bandpass filter is calculated to obtain the output energy value of each frame of the acoustic signal to be identified in each triangular bandpass filter. The output energy value is logarithmically compressed to determine the log-Mel spectrum features. The log-Mel spectrum features of all frames of the acoustic signal to be identified are arranged in chronological order to construct a two-dimensional feature matrix of the log-Mel spectrum corresponding to the acoustic signal to be identified.
4. The method for identifying and locating leaks in public building restrooms according to claim 2, characterized in that, The steps of extracting local time-frequency features from the two-dimensional feature matrix of the log-Mel spectrum using the CNN feature extraction module and converting it into a one-dimensional time-series feature sequence include: The two-dimensional feature matrix of the log-Mel spectrogram after global mean-variance normalization is input into the CNN feature extraction module; In the CNN feature extraction module, each convolutional block sequentially performs two-dimensional convolution, batch normalization, GELU activation function processing, and max pooling processing on the two-dimensional feature matrix of the log-Mel spectrogram to extract the local time-frequency features of the two-dimensional feature matrix of the log-Mel spectrogram. After all convolutional blocks are cascaded, the output two-dimensional time-frequency feature map is compressed in the frequency dimension and transformed into a one-dimensional time-series feature sequence.
5. The method for identifying and locating leaks in public building restrooms according to claim 2, characterized in that, After processing by a linear projection layer and a position encoding layer, the input to a Transformer encoder captures the global temporal dependencies of the one-dimensional temporal feature sequence through a multi-head self-attention sublayer, and after processing by a feedforward neural network sublayer, determines the first output feature sequence. This process includes: The one-dimensional temporal feature sequence is input into the linear projection layer, and the dimension of the one-dimensional temporal feature sequence is mapped to the model feature dimension of the Transformer encoder through linear transformation to obtain the dimension-adapted feature sequence. The dimension-adapted feature sequence is input into the sinusoidal position coding layer, and a position coding value generated based on sinusoidal position coding is added to the dimension-adapted feature sequence to obtain the dimension-position fusion feature sequence. The dimension-location fusion feature sequence is input into a Transformer encoder consisting of multiple identical Transformer encoder layers stacked together. For each Transformer encoder layer, the attention weights between each time position in the dimension-location fusion feature sequence are first calculated through its corresponding multi-head self-attention sub-layer to capture the global temporal dependency of the one-dimensional temporal feature sequence and obtain the intermediate feature sequence with fused global dependency. The intermediate feature sequence is subjected to nonlinear feature transformation through its corresponding feedforward neural network sublayer to enhance the nonlinear expressive power of the features, thereby obtaining a feature sequence optimized by the feedforward neural network sublayer. After being processed by stacking all Transformer encoder layers, the output is a feature sequence that integrates global temporal dependency information and nonlinear features, which serves as the first output feature sequence of the Transformer encoder.
6. The method for identifying and locating leaks in public building restrooms according to claim 2, characterized in that, The steps of inputting the first output feature sequence into the Transformer decoder, refining the features by modeling temporal causality through a masked multi-head self-attention sublayer and fusing global time-frequency features through a cross-attention sublayer to determine the second output feature sequence include: The first output feature sequence of the Transformer encoder is input into the Transformer decoder module, which is composed of multiple identical Transformer decoder layers stacked together, as the initial input feature of the masked multi-head self-attention sub-layer in each Transformer decoder layer; The first output feature sequence is temporally modeled by the masked multi-head self-attention sub-layer of each Transformer decoder layer in the Transformer decoder, which masks the future time location feature information to construct a temporal causal relationship and obtain the masked self-attention output features. Through the cross-attention sub-layer of each Transformer decoder layer, the mask self-attention output feature is used as the query feature, and the first output feature sequence output by the Transformer encoder module is used as the key feature and value feature. Attention calculation is performed, and global time-frequency feature information is fused to obtain the cross-attention output feature. After all Transformer decoder layers in the Transformer decoder complete the temporal causal relationship modeling and global time-frequency feature fusion, the second output feature sequence with refined features is obtained.
7. The method for identifying and locating leaks in public building restrooms according to any one of claims 1 to 6, characterized in that, Before obtaining the acoustic signals to be identified for each toilet in a public building suspected of leaking, the following steps are included: The instantaneous flow rate value in the smart water meter of the main water supply pipe of the target public building is continuously monitored. When the instantaneous flow rate value is continuously higher than the leakage judgment threshold within the leakage judgment time range, the target public building is judged as a suspected leakage public building. A collection command is sent to the acoustic signal collectors deployed in each toilet to be tested within the suspected leaking public building, in order to collect the corresponding acoustic signals to be identified in each toilet to be tested within the suspected leaking public building.
8. A device for identifying and locating leaks in public building restrooms, characterized in that, include: The module for acquiring acoustic signals to be identified is set to acquire the acoustic signals to be identified for each toilet to be tested in a public building suspected of leaking water. The Mel spectrogram construction module is configured to sequentially perform wavelet denoising, pre-emphasis, and frame-windowing processing on the acoustic signal to be identified, then calculate the power spectrum using short-time Fourier transform, filter it through a Mel filter bank, and perform logarithmic compression to determine the log-Mel spectral features, thereby constructing a two-dimensional feature matrix of the log-Mel spectrogram corresponding to the acoustic signal to be identified, and then perform global mean-variance normalization processing on the two-dimensional feature matrix of the log-Mel spectrogram. The first feature sequence determination module is configured to input the two-dimensional feature matrix of the log-Mel spectrogram after global mean-variance normalization into the pre-trained leaky device identification model. The CNN feature extraction module performs local time-frequency feature extraction on the two-dimensional feature matrix of the log-Mel spectrogram and converts it into a one-dimensional time-series feature sequence. After processing by the linear projection layer and the position coding layer, it is input into the Transformer encoder. The multi-head self-attention sublayer captures the global temporal dependency of the one-dimensional time-series feature sequence, and after processing by the feedforward neural network sublayer, the first output feature sequence is determined. The second feature sequence determination module is configured to input the first output feature sequence into the Transformer decoder, and refine the features by modeling the temporal causal relationship through the masked multi-head self-attention sub-layer and fusing global time-frequency features through the cross-attention sub-layer to determine the second output feature sequence. After processing by the global average pooling layer and the fully connected layer, the normalized classification probability distribution is output by the Softmax layer. The leak detection and location module is configured to use the category of the leaking device corresponding to the maximum classification probability as the category of the leaking device corresponding to the toilet to be detected if the maximum classification probability is higher than a preset confidence threshold, so as to complete the identification and location of leaks in public building toilets.
9. An electronic device comprising a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, It stores, in the form of computer-readable instructions, a computer program implemented according to any one of claims 1 to 7, which, when invoked by a computer, executes the steps included in the corresponding method.
Citation Information
Patent Citations
Leakage monitoring method for building in steel-frame structure based on BIM (Building Information Modeling) technology
CN104614126A
Building leakage detection device and detection method
CN121185517A
