Electrical key equipment fault early warning system and method based on voiceprint recognition and infrared temperature measurement

Abstract

The invention relates to an electrical key equipment fault early warning system and method based on voiceprint recognition and infrared temperature measurement. The method comprises the following steps: collecting voiceprint signals and infrared images of electrical equipment, extracting voiceprint features and infrared image features of different scales by using related technologies, carrying out feature fusion based on a cascade channel attention and space attention mechanism, establishing a feature fusion fault diagnosis model, establishing a detection and segmentation multi-task network by using two multi-layer perception mechanisms, and carrying out fault diagnosis on the electrical equipment. Therefore, the fault type and position are predicted. According to the system and method, infrared image information and voiceprint information are organically combined, the diagnosis accuracy of complex faults can be improved, accurate and rapid diagnosis of the faults of the electrical key equipment can be achieved, the safety and reliability of the equipment can be effectively improved, and the system and method are suitable for state monitoring and fault early warning of the electrical equipment such as transformers and switch cabinets. The method has a wide application prospect and an important practical value.

Description

Technical Field

[0001] The present invention relates to the technical field of fault warning systems for electrical key equipment based on voiceprint recognition and infrared temperature measurement, and specifically to a fault warning system and method for electrical key equipment based on voiceprint recognition and infrared temperature measurement. Background Art

[0002] In the field of industrial production, the stable operation of electrical key equipment is crucial for the continuity and safety of the entire production process. Traditional methods for detecting electrical equipment faults mainly rely on manual inspections and regular maintenance, and this approach has many drawbacks. On the one hand, it is difficult to achieve real-time monitoring through manual inspections, and equipment may malfunction during the interval between two inspections without being detected in time, resulting in production interruptions and huge economic losses. On the other hand, for some electrical equipment with complex installation locations or in high-voltage, high-risk environments, manual detection is not only difficult to operate but also poses safety risks to the detection personnel.

[0003] With the development of technology, although some single monitoring technologies such as simple infrared temperature measurement or simple vibration monitoring have been applied, these methods have limitations. For example, relying solely on infrared temperature measurement technology may not be able to accurately distinguish between normal temperature fluctuations of equipment and temperature anomalies caused by internal potential faults in some cases, prone to false alarms or missed alarms. Moreover, for some early faults, their temperature changes are not obvious, and simple infrared temperature measurement is difficult to detect problems in the budding stage of faults. Similarly, traditional vibration monitoring has poor detection effects for some non-mechanical structure faults or cases where vibration characteristics are not obvious in the initial stage of faults.

[0004] In recent years, the demand for intelligent monitoring of electrical equipment has been increasing day by day, but the existing monitoring systems and methods still need to be improved in terms of accuracy, reliability, and comprehensiveness, and cannot meet the requirements of efficient and accurate fault warning for electrical key equipment in complex industrial environments. Summary of the Invention

[0005] The purpose of the present invention is to provide a fault warning system and method for electrical key equipment based on voiceprint recognition and infrared temperature measurement to solve the problems raised in the above background art.

[0006] To achieve the above purpose, the present invention provides the following technical solution: A fault warning system for electrical key equipment based on voiceprint recognition and infrared temperature measurement, comprising:

[0007] A voiceprint acquisition module, configured to acquire the sound signals generated during the operation of electrical key equipment and convert them into digital voiceprint data;

[0008] Infrared temperature measurement module, whose structure includes an image acquisition part and a data processing part. The image acquisition part consists of a visible light channel and an infrared channel. The visible light channel acquires the image of the device under natural light, and the infrared channel acquires the image of the device under infrared light and transmits it to the processing unit through a high-speed data transmission interface. A germanium glass is added to the front end of the infrared channel to increase the permeability of the infrared spectral signal. The acquisition part transmits the optical signal to the CMOS sensor to generate an electrical signal and processes it in the ISP chip. Finally, the visible light image and the infrared image are stitched and generated in the VPU chip. This module is used to collect the infrared thermal image data of electrical key equipment;

[0009] Storage module, used to store the digital voiceprint data and infrared thermal image data, and is coupled and configured with the processing module. It stores the standard voiceprint feature library and temperature threshold range data under the normal operating state of electrical key equipment, as well as image processing and temperature measurement related programs;

[0010] Processing module, including a voiceprint recognition unit and a temperature analysis unit. The voiceprint recognition unit compares and analyzes the collected digital voiceprint data with the standard voiceprint feature library. The temperature analysis unit calculates the accurate temperature of the device based on the image processing result and the data of the ambient temperature sensor using a temperature measurement algorithm, and compares it with the stored temperature threshold range;

[0011] Warning module, connected to the processing module. When the voiceprint recognition unit determines that the voiceprint is abnormal or the temperature analysis unit detects that the temperature exceeds the threshold range, it issues a fault warning signal. The warning signal is transmitted to the relevant monitoring system or the terminal of the equipment management personnel through the CAN bus or other communication methods.

[0012] Furthermore, the voiceprint acquisition module includes at least two high-sensitivity microphones. The microphones are evenly distributed at different positions around the electrical key equipment, and each microphone is equipped with an independent signal amplifier and anti-interference filter; by collecting sound signals from different angles through multiple microphones, the sound characteristics generated during the operation of the equipment can be captured more comprehensively, avoiding information loss caused by the position limitation of a single microphone; the independent signal amplifier can enhance the intensity of weak sound signals and ensure the accuracy of the collected voiceprint data; the anti-interference filter can effectively remove electromagnetic interference and mechanical vibration noise in the surrounding environment, further improving the purity of the voiceprint data, making the comparison and analysis results of the subsequent voiceprint recognition unit more reliable, and reducing the possibility of misjudgment caused by noise interference.

[0013] Furthermore, both the visible light channel and the infrared channel of the infrared temperature measurement module adopt adjustable focal length lenses and have an automatic focusing function. In different installation environments and equipment sizes, the adjustable focal length lenses can flexibly adjust the shooting range and clarity to ensure that the acquired images are complete and the details are clear. The automatic focusing function enables the system to quickly autofocus during the operation of the equipment, even if the equipment position or imaging conditions change due to vibration or temperature change factors, and always maintain a clear image acquisition effect. This helps to improve the accuracy of temperature measurement because clear and accurate images are the basis for accurately extracting temperature information, reducing temperature measurement errors caused by blurred images, and thus enhancing the reliability of the entire system's temperature monitoring of electrical key equipment.

[0014] Furthermore, the storage module adopts a redundant storage design, including at least two independent storage units. One storage unit serves as the main storage area for real-time storage of the acquired digital voiceprint data, infrared thermal image data, as well as intermediate data and results during the operation of the system. The other storage unit serves as the backup storage area, regularly mirroring and backing up the data in the main storage area. When the main storage unit fails, the backup storage unit can quickly take over the work to ensure the integrity and continuity of the data. This redundant storage design can effectively prevent data loss caused by storage failures, ensure the stability and reliability of the system, enable the system to always store and call data normally during long-term operation, and provide solid data support for fault warning.

[0015] Furthermore, the voiceprint recognition unit of the processing module constructs a voiceprint recognition model using deep learning algorithms. This model is trained with a large amount of voiceprint data of electrical key equipment in normal and faulty states. Deep learning algorithms can automatically learn the complex features and patterns in voiceprint data and have stronger adaptability and accuracy compared to traditional rule-based or simple feature matching methods. Through learning a large amount of sample data, the model can accurately distinguish the voiceprint features during normal operation of the equipment from the voiceprint changes corresponding to different types of faults, effectively improving the accuracy of voiceprint anomaly judgment. For motor equipment, it can accurately identify the unique voiceprint changes caused by bearing wear and rotor imbalance faults, and thus issue warning signals in a timely manner, providing a more accurate basis for equipment maintenance.

[0016] Furthermore, when calculating the device temperature, the temperature analysis unit of the processing module comprehensively analyzes in combination with the device's historical temperature data, the current ambient temperature, and the load condition; in addition to relying on the infrared thermogram data and the ambient temperature sensor data, introducing the device's historical temperature data can analyze the temperature change trend to determine whether the current temperature increase is a normal fluctuation or an abnormal rise; at the same time, the load condition of the device is considered because the load change will directly affect the heat generation of the device; when the device load suddenly increases, the temperature may rise accordingly, but if it is within a reasonable range, it does not belong to a fault state; through this comprehensive analysis method, it is possible to more accurately determine whether the device temperature truly exceeds the normal threshold range, reduce the occurrence of false alarms, and improve the reliability and accuracy of fault warning.

[0017] A fault warning method for electrical key equipment based on voiceprint recognition and infrared temperature measurement, comprising the following steps:

[0018] Use the voiceprint acquisition module to acquire the sound signal during the operation of the electrical key equipment and convert it into digital voiceprint data;

[0019] At the same time, use the infrared temperature measurement module to acquire the infrared thermogram data of the electrical key equipment. The acquisition process of this module is as follows: the visible light channel and the infrared channel respectively obtain the images of the equipment under natural light and infrared light and transmit them to the processing unit, and after being processed by the sensor and the chip, a fused image is generated in the VPU chip;

[0020] Compare and analyze the acquired digital voiceprint data with the standard voiceprint feature library in the storage module to determine whether the voiceprint is abnormal;

[0021] Use the temperature measurement algorithm to calculate the accurate temperature of the device based on the image processing result of the infrared thermogram data and the ambient temperature sensor data, and compare it with the temperature threshold range in the storage module to determine whether the temperature exceeds the threshold;

[0022] If the voiceprint is abnormal or the temperature exceeds the threshold, the warning module issues a fault warning signal and transmits the warning signal to the relevant monitoring system or the terminal of the device management personnel through the CAN bus or other communication methods. Among them, during the whole process, the image processing program in the storage module processes the acquired infrared images, including performing brightness equalization processing on the visible light image, converting the infrared light image into a grayscale image and enhancing it before fusing it with the visible light image to assist in temperature measurement and fault judgment.

[0023] Furthermore, before collecting the sound signals and infrared thermal image data of electrical key equipment, the equipment is pre-diagnostically scanned first. Through low-resolution infrared thermal imaging and simple voiceprint monitoring, it is preliminarily judged whether there are obvious abnormalities in the equipment. If abnormal signs are found during the pre-diagnostic scan, the collection frequency and accuracy are increased, and more detailed data is collected for analysis. This pre-diagnostic scan method can quickly screen out equipment that may have problems without consuming too many system resources, and collect and analyze data in a targeted manner, improving the detection efficiency of the system. In a large electrical equipment group, the equipment with potential fault risks can be quickly located through pre-diagnostic scanning, avoiding high-frequency and high-precision data collection for all equipment, saving system resources and time, and ensuring that potential faulty equipment is not missed.

[0024] Furthermore, during the voiceprint recognition process, the collected digital voiceprint data is segmented, and the dynamic time warping algorithm (DTW) is used for feature matching. Since the sound signals may have inconsistent durations during the operation of electrical equipment, the DTW algorithm can perform elastic matching on voiceprint data of different lengths on the time axis to find the best matching path, thereby more accurately comparing voiceprint features. Through segmentation processing, the analysis can be further refined, and the ability to capture subtle changes in voiceprints can be improved. For equipment with periodic operation characteristics, by segmenting and matching the voiceprint data according to the period, abnormal changes in the voiceprint within each period can be detected more sensitively, enhancing the accuracy and reliability of voiceprint recognition and effectively improving the accuracy of fault warning.

[0025] Furthermore, after the warning module issues a fault warning signal, the system automatically starts the data recording and analysis program, records the voiceprint data, infrared thermal image data, and equipment operation parameters for a period of time before the warning, and uses data analysis tools to deeply analyze these data to generate a fault report. The fault report includes the time of fault occurrence, possible fault types, and trend information of relevant data changes. This helps equipment maintenance personnel quickly understand the fault situation, accurately judge the cause of the fault, and formulate effective maintenance plans. The system stores the fault report in the local database and can transmit it to the remote maintenance center through the network, facilitating subsequent fault tracing and statistical analysis, and providing strong support for improving the overall maintenance level of the equipment.

[0026] Compared with the prior art, the beneficial effects of the present invention are:

[0027] The present invention combines voiceprint recognition and infrared temperature measurement technology to give full play to the advantages of both technologies. Voiceprint recognition can capture the subtle changes in the sound during the operation of the device, and these changes are often closely related to the states of the internal mechanical structure, electrical connections, etc. of the device. Infrared temperature measurement can directly monitor the surface temperature of the device and reflect the heating condition of the device. By working together and complementing each other, they can more accurately determine whether the device has a fault. For example, for a motor device, in the initial stage of bearing wear, abnormal sounds with specific frequencies may be generated, and at the same time, due to increased friction, local temperature rise will also occur. This system can detect these two abnormal signals simultaneously, avoiding misjudgments that may occur in single-technology detection and greatly improving the accuracy of fault detection.

[0028] During the development process of electrical equipment faults, the symptoms are often not obvious in the early stage, but there will be weak abnormal manifestations in voiceprint and temperature. With its highly sensitive voiceprint acquisition module and high-precision infrared temperature measurement module, this system can keenly capture these early abnormal signals. Through real-time monitoring and intelligent analysis algorithms, it can issue early warnings when the fault is in its infancy, providing sufficient time for equipment maintenance personnel to carry out maintenance and repair, effectively preventing the further deterioration of the fault, reducing equipment downtime and maintenance costs, and improving production efficiency.

[0029] In the hardware design of the system, a variety of reliability enhancement measures such as redundant storage design and adjustable-focus lenses are adopted, and in the software algorithm, advanced technologies such as deep learning algorithms and dynamic time warping algorithms are used to ensure the accuracy and stability of data acquisition, processing, and analysis. Even in a complex industrial environment, such as a strong electromagnetic interference and large equipment vibration, the system can still operate stably, continuously and effectively monitor key electrical equipment, reduce the risk of monitoring errors caused by system failures or environmental factors, and ensure the safe and reliable operation of the production process.

[0030] After the system issues a fault warning, the automatically generated detailed fault report contains rich information such as the time of fault occurrence, possible fault types, and the change trends of relevant data. Equipment maintenance personnel can quickly locate the cause of the fault based on this information, formulate targeted maintenance plans, avoid blindly troubleshooting faults, and greatly shorten the maintenance time. At the same time, the system's function of recording and analyzing historical data also helps maintenance personnel summarize the laws of equipment faults, optimize the equipment maintenance plan, and further improve the overall efficiency and quality of equipment maintenance. Brief Description of the Drawings

[0031] Figure 1 is a schematic diagram of the system flow of the present invention;

[0032] Figure 2 is a schematic diagram of the method flow of the present invention. Detailed Embodiments

[0033] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0034] Please refer to Figure 1 —2. The present invention provides a technical solution: an implementation manner of an electrical key equipment fault warning system and method based on voiceprint recognition and infrared temperature measurement

[0035] The system hardware includes:

[0036] Voiceprint acquisition module

[0037] The voiceprint acquisition module mainly consists of multiple high-sensitivity microphones, signal amplifiers, anti-interference filters, and data acquisition cards. Capacitive microphones are selected for the microphones, which have the characteristics of high sensitivity and wide frequency response, and can capture the weak sound signals generated during the operation of electrical key equipment. At least three microphones are reasonably arranged around the electrical equipment, such as at the front, rear, and side of the motor, etc., to ensure that sounds are collected from different directions and form a multi-dimensional voiceprint information acquisition system.

[0038] Each microphone is connected to an independent signal amplifier. The signal amplifier uses a low-noise amplifier chip, such as OPA2134 of Texas Instruments (TI), etc., which can amplify the weak electrical signals collected by the microphone, while maintaining a low noise level and improving the signal-to-noise ratio of the signal. The anti-interference filter uses a band-pass filter, which is designed according to the frequency range of the sound signals during the operation of the electrical equipment. For example, for common motor equipment, the passband frequency range of the band-pass filter can be set between 20 Hz and 20 kHz, effectively filtering out low-frequency electromagnetic interference and high-frequency noise in the environment and ensuring the purity of the collected voiceprint signals.

[0039] The signals after amplification and filtering are transmitted to the storage module and processing module of the system through the data acquisition card. A data acquisition card with a high sampling rate and high precision is selected, such as the PCI-6221 data acquisition card of NI, whose sampling rate can reach 250 kS / s, which can meet the high-speed acquisition requirements of voiceprint signals and ensure the integrity and distortion-free of the collected voiceprint data.

[0040] Infrared temperature measurement module

[0041] Both the visible light channel and the infrared channel of the infrared temperature measurement module are equipped with adjustable focus lenses, such as the EF series lenses of Canon, which have a wide focal length adjustment range and can meet the monitoring requirements of electrical equipment with different distances and sizes. The lens adopts an electric zoom method and can be remotely and automatically adjusted through system control to ensure clear images of the equipment are obtained.

[0042] The image sensor in the visible light channel selects a high-resolution CMOS sensor, such as the IMX series sensors of Sony, which can capture detailed information of the equipment under natural light. The sensor in the infrared channel adopts an infrared pyroelectric sensor, such as the LHI series sensors of Heimann in Germany, which has high sensitivity to infrared radiation and can accurately sense the temperature distribution on the surface of the equipment. The germanium glass at the front end of the sensor is made of high-purity germanium material, which has good permeability to the infrared spectrum, can effectively enhance the permeability of the infrared spectrum signal, and improve the accuracy of temperature measurement.

[0043] The optical signals collected by the sensors are first transmitted to the ISP chips in their respective channels, such as the A5S chips of Ambarella. The ISP chips perform a series of processing on the image signals, including brightness equalization processing, which adjusts the brightness of different regions of the image to make the illumination of the whole image uniform; grayscale processing, which converts the color image into a grayscale image for subsequent temperature analysis; and image enhancement processing, which uses algorithms such as histogram equalization and filtering to enhance the contrast and clarity of the image and highlight the temperature characteristics of the equipment.

[0044] The visible light image and the infrared image processed by the ISP chip are stitched and fused in the VPU chip. The VPU chip uses chips with powerful image processing capabilities such as the Jetson TX2 of NVIDIA. It weights and fuses the visible light image and the infrared image according to preset algorithms and parameters to generate a fused image containing the appearance and temperature information of the equipment, and transmits it to the storage module and the processing module for subsequent analysis.

[0045] Storage module

[0046] The storage module adopts a redundant storage design and consists of two high-performance solid-state drives (SSDs). The main storage SSD selects the 870 EVO series of Samsung, which has the characteristics of large capacity, high-speed read and write, and is used to store the collected digital voiceprint data, infrared thermal image data, as well as the intermediate data and results during the operation of the system in real time. The backup storage SSD uses the same model or a product with similar performance as the main storage, and regularly mirrors and backs up the data in the main storage area through a dedicated data backup software and hardware controller.

[0047] The storage module also has a built-in database management system, such as MySQL or PostgreSQL, which is used to classify, index, and manage the stored data. The database stores a standard voiceprint feature library and temperature threshold range data for electrical critical equipment under normal operating conditions. These data are obtained by collecting, analyzing, and statistically processing the voiceprint and temperature data of a large number of equipment of the same type during normal operation. The standard voiceprint feature library stores the feature information of the voiceprint, such as frequency, amplitude, and harmonics, in the form of feature vectors. The temperature threshold range sets reasonable upper and lower limits according to factors such as the type, specifications, and operating environment of the equipment, providing a basis for subsequent fault judgment.

[0048] Processing module

[0049] The processing module is built based on a high-performance multi-core processor, such as the Core i7 or i9 series processors of Intel. It has powerful computing capabilities and multi-threaded processing capabilities, enabling it to quickly process a large amount of voiceprint and temperature data. The processing module is equipped with a large-capacity high-speed memory, such as DDR4 memory, with a capacity of up to 16GB or 32GB, which is used to temporarily store data and run programs to ensure the smooth operation of the system. At the same time, it is also equipped with a professional graphics processing unit (GPU), such as the GeForce RTX series GPUs of NVIDIA. When performing image processing and deep learning algorithm operations, the parallel computing capabilities of the GPU are utilized to accelerate the calculation process and improve the response speed of the system.

[0050] The voiceprint recognition unit constructs a voiceprint recognition model using deep learning algorithms, specifically selecting the convolutional neural network (CNN) architecture. First, a large amount of voiceprint data of electrical critical equipment in normal and faulty states collected is preprocessed, including operations such as audio signal framing, windowing, and fast Fourier transform (FFT), to convert the audio signal into a spectrogram. Then, the spectrogram is used as the input of the CNN model and trained through multiple convolutional layers, pooling layers, and fully connected layers. The convolutional layer extracts the local features of the voiceprint through convolutional kernels of different sizes. The pooling layer performs downsampling to reduce the data volume and prevent overfitting. The fully connected layer integrates and classifies the extracted features.

[0051] During the training process, optimization algorithms such as the cross-entropy loss function and stochastic gradient descent (SGD) are used to optimize the model, continuously adjusting the model's parameters to enable it to accurately distinguish the voiceprint features during normal equipment operation from the voiceprint changes corresponding to different types of faults. After iterative training with a large amount of training data, the model achieves high accuracy and generalization ability, and can quickly and accurately identify and classify the collected voiceprint data.

[0052] When calculating the temperature of the device, the temperature analysis unit conducts a comprehensive analysis by combining the device's historical temperature data, the current ambient temperature, and the load conditions. First, it obtains the temperature data of the current environment where the device is located by connecting to the ambient temperature sensor. At the same time, it retrieves the device's historical temperature data from the storage module and uses time series analysis algorithms such as ARIMA or LSTM to analyze the changing trends and patterns of the device's temperature.

[0053] Regarding the load conditions of the device, it obtains real-time load information by communicating with the device's control system or power monitoring module. According to the load-temperature characteristic curve of the device, a mathematical model of temperature and load is established, and the load factor is taken into account when calculating whether the actual temperature of the device exceeds the threshold. For example, for a transformer device, when the load increases, its heat generation will increase accordingly. Through the pre-established model, it can more accurately determine whether the current temperature rise is within the normal range, avoiding misjudgment caused by load changes.

[0054] Early warning module

[0055] The early warning module consists of an audible and visual alarm and a communication module. The audible and visual alarm selects a high-brightness LED light and a high-power buzzer, which can emit a strong audible and visual alarm when receiving the fault early warning signal sent by the processing module, attracting the attention of on-site staff. The communication module uses industrial Ethernet or wireless communication technologies such as Wi-Fi6 or 5G modules to transmit the early warning signal and relevant device information (such as device name, fault type, occurrence time, etc.) to the relevant monitoring system or the terminal devices of equipment management personnel (such as mobile phones, tablets, etc.) through the CAN bus or network. During the communication process, data encryption technologies such as the AES algorithm are used to ensure the security and reliability of data transmission, preventing data leakage and tampering.

[0056] System software

[0057] Data acquisition and preprocessing software

[0058] The voiceprint data acquisition program runs on the data acquisition card of the voiceprint acquisition module, responsible for controlling the microphone to collect sound signals and converting them into digital signals. The program first initializes and configures the data acquisition card, setting parameters such as the sampling rate, sampling accuracy, and number of channels. For example, the sampling rate is set to 44.1 kHz, the sampling accuracy is set to 16 bits, and the number of channels is configured according to the actual number of microphones.

[0059] During the acquisition process, an interrupt-driven method is adopted. When a sound signal is input, the interrupt service program is triggered, and the acquired audio data is stored in a predefined buffer. Meanwhile, the acquisition program performs preliminary filtering on the audio data to remove the DC component and part of the high-frequency noise, improving the data quality. After a certain amount of data is acquired, the data is quickly transferred to the storage module through the DMA method, reducing the burden on the CPU.

[0060] The infrared image acquisition and processing program runs on the VPU chip of the infrared temperature measurement module, responsible for controlling the lenses and sensors of the visible light channel and the infrared channel to perform image acquisition, and processing and fusing the acquired images. The program first automatically calculates and adjusts the focal length of the lens according to the distance and size of the device to ensure clear images. Then, the sensor is started for image acquisition, and the acquired image data is first transferred to the ISP chip for processing.

[0061] During the processing in the ISP chip, the program controls the ISP chip to perform operations such as brightness equalization, grayscale processing, and image enhancement according to the preset algorithms. For example, in the brightness equalization process, an algorithm based on the histogram is adopted to statistically analyze the brightness distribution of the image, and then the grayscale values of the image are adjusted to make the brightness uniform. The processed visible light image and infrared image are fused in the VPU chip according to a certain weight, and the fusion weight can be adjusted according to the actual application scenario and requirements. For example, in the night or low-light environment, the weight of the infrared image can be appropriately increased to highlight the temperature information. The fused image data is transferred to the storage module through the network or bus.

[0062] Voiceprint Recognition and Temperature Analysis Software

[0063] The voiceprint recognition algorithm is implemented in the voiceprint recognition unit of the processing module, based on the convolutional neural network model of deep learning. In the model training stage, programming is carried out using the Python language and deep learning frameworks such as TensorFlow or PyTorch. First, the preprocessed voiceprint data set is loaded, and the data set is divided into a training set, a validation set, and a test set according to a certain ratio. For example, 70% of the data is used as the training set, 20% of the data is used as the validation set, and 10% of the data is used as the test set.

[0064] Then, construct the CNN model architecture, including defining parameters such as the number of convolutional layers, the size of the convolutional kernels, the type and stride of the pooling layers, and the number of nodes in the fully connected layers. For example, construct a model with 3 convolutional layers, 2 pooling layers, and 2 fully connected layers. The size of the convolutional kernels in the first convolutional layer is 3x3, the stride is 1, the pooling layer uses max pooling, and the pooling window size is 2x2. During the training process, set the number of training epochs to 100, the size of each batch of data to 32, and use the Adam optimizer and the cross-entropy loss function to train the model. During the training process, monitor the accuracy and loss value of the validation set in real time. When the accuracy of the validation set no longer improves and the loss value tends to stabilize, stop the training and save the trained model parameters.

[0065] In the voiceprint recognition stage, first preprocess the collected real-time voiceprint data, convert it into a spectrogram, and then input it into the trained CNN model. The model outputs the classification result of the voiceprint to determine whether the voiceprint is abnormal. If the voiceprint has a large difference from the normal voiceprint features in the standard voiceprint feature library and exceeds the preset threshold, it is determined that the voiceprint is abnormal.

[0066] The temperature analysis algorithm is implemented in the temperature analysis unit of the processing module, and comprehensive analysis is carried out in combination with the device's historical temperature data, the current ambient temperature, and the load situation. First, write a program in C++ to read the device's historical temperature data, current ambient temperature data, and load data from the storage module. For the historical temperature data, use a data fitting algorithm, such as the least squares method, to fit the temperature change curve over time and analyze the temperature change trend.

[0067] According to the device's load-temperature characteristic curve, establish a linear or non-linear regression model, use the load data as the input, and predict the normal temperature range of the device under the current load. For example, for a certain motor device, through experiments and data analysis, the relationship between its load and temperature is a quadratic function relationship, and the theoretical normal temperature range is calculated based on the current load value. Finally, compare the actual temperature measured by the infrared temperature measurement module with the predicted normal temperature range. If the actual temperature exceeds the normal temperature range and lasts for a certain period of time (such as 5 minutes), it is determined that the temperature exceeds the limit.

[0068] Early Warning and Data Management Software

[0069] The early warning trigger and notification program runs in the communication module of the early warning module, and monitors the fault early warning signals sent by the processing module in real time. When receiving the early warning signals of abnormal voiceprint or temperature exceeding the limit, the program first controls the audible and visual alarm to sound an alarm, and at the same time obtains the relevant information of the device, such as the device name, fault type, occurrence time, etc.

[0070] Then, according to the preset communication protocol and the target address, the warning information is sent to the monitoring system or the terminal device of the equipment management personnel through the industrial Ethernet or the wireless communication network. During the sending process, data formats such as JSON or XML are used to encapsulate the warning information to ensure the structuring and readability of the information. The following is an example of a warning information in JSON format:

[0071] json

[0072] {

[0073] "device_name":"Transformer01",

[0074] "fault_type":"Overheating",

[0075] "occur_time":"2023-08-15 10:25:30",

[0076] "temperature":85,

[0077] "soundwave_anomaly":false

[0078] }

[0079] ```

[0080] The data management and analysis program runs on the database management system of the storage module and is responsible for managing and analyzing the collected data such as voiceprint data, infrared thermal image data, and equipment operation parameters. The program provides functions such as data storage, query, deletion, and update. For example, maintenance personnel can use SQL query statements to obtain the temperature data and voiceprint data of a certain device within a certain period from the database for fault analysis and equipment performance evaluation.

[0081] The program also has the function of data analysis. For example, data mining algorithms are used to analyze a large amount of historical data to mine the potential rules and characteristics of equipment failures. For example, the association rule mining algorithm is used to discover the association relationships between certain voiceprint characteristics and temperature change patterns and specific fault types, providing a reference basis for fault diagnosis and prediction. In addition, the program is also responsible for generating fault reports. When a fault warning occurs, it automatically extracts relevant data from the database, generates a report containing detailed fault information, and stores it locally and transmits it to a remote server for subsequent fault tracing and statistical analysis.

[0082] Select a suitable installation location around the electrical critical equipment to install the microphone of the voiceprint acquisition module, ensuring that the microphone can clearly collect the sound during the operation of the equipment, while avoiding interference from other strong noise sources. Use a fixed bracket to fix the microphone on the wall, rack or specially designed mounting base near the equipment, and adjust the angle and height of the microphone so that it is aligned with the main sound-emitting part of the equipment. Connect the cables between the microphone, signal amplifier, anti-interference filter and data acquisition card, ensure the connection is firm, and the cable routing should avoid crossing other power lines or signal lines to prevent electromagnetic interference.

[0083] Install the infrared temperature measurement module at a position where the surface temperature of the equipment can be comprehensively monitored, generally installed above or on the side of the equipment, ensuring that the infrared lens can clearly capture the key parts of the equipment. Use the mounting bracket to fix the infrared temperature measurement module and adjust the angle and focal length of the lens so that the equipment is within the field of view of the lens. Connect the communication cables between the infrared temperature measurement module, storage module and processing module, such as Ethernet cables or CAN bus cables, to ensure stable data transmission.

[0084] Install the host equipment of the storage module, processing module and warning module in the control room or a suitable location, ensuring that the equipment is installed in an environment with good ventilation, dryness and appropriate temperature. Connect the power lines and data lines between each module, and make the correct connection according to the interface identification and instruction manual of the equipment to ensure the correct electrical connection of the system.

[0085] Install and configure the system software, install the data acquisition and preprocessing software, voiceprint recognition and temperature analysis software, warning and data management software, etc. on the host equipment of the processing module. Install according to the software installation wizard and set relevant parameters, such as database connection parameters, communication port numbers, etc. After the installation is completed, perform the initialization configuration of the software and import the relevant information of the electrical critical equipment, standard voiceprint feature library, temperature threshold range and other data.

[0086] First, debug the voiceprint acquisition module. After starting the system, check whether the microphone is working properly. Ensure the signal is normal by observing the indicator light of the data acquisition card or using an oscilloscope to detect the electrical signal output by the microphone. Adjust the gain of the signal amplifier to make the intensity of the collected voiceprint signal moderate, avoiding signal saturation or being too weak. Check the filtering effect of the anti-interference filter. You can use a spectrum analyzer to observe the signal spectrum before and after filtering to ensure that the environmental noise is effectively suppressed.

[0087] Debug the infrared temperature measurement module, check whether the lenses of the visible light channel and the infrared channel have clear images, and check the image quality by observing the image output by the VPU chip or connecting a monitor. Adjust the focal length and aperture of the lens to make the image clear and of appropriate brightness. Use a standard blackbody radiation source to calibrate the infrared temperature measurement module, compare the temperature measured by the infrared temperature measurement module with the known temperature of the standard blackbody, adjust the parameters of the infrared temperature measurement module, ensure the accuracy of the temperature measurement, and the calibration error should be controlled within the allowable range. For example, for applications with high precision requirements, the error should be less than ±1°C.

[0088] Check the hardware connection and working status of the storage module and processing module, and ensure that each hardware device is recognized and running normally by checking the device manager or system log. Perform performance tests on the multi-core processor, GPU, and memory of the processing module. You can use professional testing software such as Prime95 to test CPU performance, 3DMark to test GPU performance, and MemTest to test memory stability to ensure that the hardware performance meets the system operation requirements.

[0089] Test the sound and light alarm and communication module of the early warning module, trigger the early warning signal, check whether the sound and light alarm sound the alarm normally, whether the sound is loud and clear, and whether the light flashes obviously. By sending test data, check whether the communication module can accurately send the early warning information to the target device, and check whether the receiving end can correctly receive and parse the data.

[0090] Debug the data acquisition and preprocessing software, check whether the voiceprint data acquisition program and infrared image acquisition and processing program can run normally, and whether the collected data is accurate and complete. Observe the waveform and spectrum of the collected voiceprint data to check whether there is abnormal noise or distortion; check whether the clarity, brightness and temperature distribution of the infrared image are normal. Adjust the parameters of the acquisition program, such as sampling rate, image resolution, etc., to optimize the data acquisition effect.

[0091] Debug the voiceprint recognition and temperature analysis software. First, use known normal and faulty voiceprint data and temperature data to test the models and algorithms in the software. Check the accuracy of the voiceprint recognition model. Compare the recognition results output by the model with the known true results, calculate the accuracy, recall rate and other indicators, and adjust the parameters and structure of the model according to the test results, such as increasing the depth of the convolution layer and adjusting the connection method of neurons, to improve the performance of the model. For the temperature analysis algorithm, verify its accuracy in temperature judgment under different loads and environmental conditions, check whether there are misjudgments or missed judgments, and optimize the temperature threshold setting and load-temperature model in the algorithm according to the actual test results.

[0092] Debugging warning and data management software, check whether the warning trigger and notification program can send warning signals in a timely and accurate manner and send information to the target device. Simulate a fault situation to trigger a warning and observe whether the monitoring system or terminal device can receive the warning information in a timely manner and whether the information content is complete and accurate. Conduct functional tests on the data management and analysis program, check whether operations such as data storage, query, deletion, and update are normal, use test data for data analysis, and verify whether the mined association rules and fault patterns are reasonable and effective to ensure that the software can meet the system requirements.

Claims

1. A fault warning system for key electrical equipment based on voiceprint recognition and infrared temperature measurement, characterized in that: include: The voiceprint collection module is used to collect the sound signals generated by the operation of key electrical equipment and convert them into digital voiceprint data; The infrared temperature measurement module includes an image acquisition unit and a data processing unit. The image acquisition unit consists of a visible light channel and an infrared channel. The visible light channel acquires the image of the device under natural light, and the infrared channel acquires the image of the device under infrared light and transmits it to the processing unit through a high-speed data transmission interface. Germanium glass is added to the front end of the infrared channel to increase the transparency of the infrared spectrum signal. The acquisition unit transmits the optical signal to the CMOS sensor to generate an electrical signal and processes it in the ISP chip. Finally, the visible light image and the infrared image are spliced ​​and generated in the VPU chip. This module is used to collect infrared thermal imaging data of key electrical equipment; A storage module, which is used to store the digital voiceprint data and infrared thermal imaging data, and is coupled with the processing module to store a standard voiceprint feature library and temperature threshold range data under normal operating conditions of key electrical equipment, as well as image processing and temperature measurement related programs; A processing module, including a voiceprint recognition unit and a temperature analysis unit. The voiceprint recognition unit compares and analyzes the collected digital voiceprint data with a standard voiceprint feature library. The temperature analysis unit uses a temperature measurement algorithm to calculate the accurate temperature of the device based on the image processing result and the ambient temperature sensor data, and compares it with the stored temperature threshold range. The early warning module is connected to the processing module. When the voiceprint recognition unit determines that the voiceprint is abnormal or the temperature analysis unit detects that the temperature exceeds the threshold range, a fault early warning signal is issued. The early warning signal is transmitted to the relevant monitoring system or equipment management personnel terminal via the CAN bus or other communication methods.

2. A method for early warning of electrical key equipment failure based on voiceprint recognition and infrared temperature measurement adopted by the system according to claim 1, characterized in that: The following steps are involved: The voiceprint collection module is used to collect the sound signals of key electrical equipment during operation and convert them into digital voiceprint data; At the same time, the infrared temperature measurement module collects infrared thermal imaging data of key electrical equipment. The acquisition process of this module is as follows: the visible light channel and the infrared channel respectively obtain images of the equipment under natural light and infrared light and transmit them to the processing unit. After being processed by the sensor and chip, a fused image is generated in the VPU chip; Compare and analyze the collected digital voiceprint data with the standard voiceprint feature library in the storage module to determine whether there is any abnormality in the voiceprint; The temperature measurement algorithm is used to calculate the accurate temperature of the device based on the image processing results of the infrared thermal imaging data and the ambient temperature sensor data, and is compared with the temperature threshold range in the storage module to determine whether the temperature exceeds the threshold; If the voiceprint is abnormal or the temperature exceeds the threshold, the early warning module will send out a fault warning signal and transmit the warning signal to the relevant monitoring system or equipment management personnel terminal through the CAN bus or other communication methods. During the whole process, the image processing program in the storage module processes the collected infrared image, including brightness balancing of the visible light image, converting the infrared image into a grayscale image and fusing it with the visible light image after enhancement processing to assist in temperature measurement and fault judgment.

3. The electrical key equipment fault early warning system based on voiceprint recognition and infrared temperature measurement according to claim 1 is characterized in that: The voiceprint collection module includes at least two high-sensitivity microphones, which are evenly distributed at different positions around the key electrical equipment, and each microphone is equipped with an independent signal amplifier and anti-interference filter; by collecting sound signals from different angles with multiple microphones, it is possible to more comprehensively capture the sound characteristics generated when the equipment is running, avoiding information loss due to the position limitation of a single microphone; the independent signal amplifier can enhance the strength of weak sound signals to ensure the accuracy of the collected voiceprint data; the anti-interference filter can effectively remove electromagnetic interference and mechanical vibration noise in the surrounding environment, further improve the purity of the voiceprint data, make the comparison and analysis results of the subsequent voiceprint recognition unit more reliable, and reduce the possibility of misjudgment due to noise interference.

4. The electrical key equipment fault early warning system based on voiceprint recognition and infrared temperature measurement according to claim 1 is characterized in that: The visible light channel and infrared channel of the infrared temperature measurement module both use adjustable focal length lenses and have an automatic focusing function; under different installation environments and equipment sizes, the adjustable focal length lens can flexibly adjust the shooting range and clarity to ensure that the acquired image is complete and the details are clear; the automatic focusing function enables the system to quickly automatically focus during the operation of the equipment, even if the equipment position or imaging conditions change due to vibration or temperature changes, and always maintain a clear image acquisition effect; this helps to improve the accuracy of temperature measurement, because clear and accurate images are the basis for accurately extracting temperature information, reducing temperature measurement errors caused by image blur, thereby improving the reliability of the entire system for temperature monitoring of key electrical equipment.

5. The electrical key equipment fault early warning system based on voiceprint recognition and infrared temperature measurement according to claim 1 is characterized in that: The storage module adopts a redundant storage design and includes at least two independent storage units; One of the storage units is used as the main storage area to store the collected digital voiceprint data, infrared thermal imaging data, and intermediate data and results during the system operation in real time; the other storage unit is used as the backup storage area to regularly perform mirror backup of the data in the main storage area; When the main storage unit fails, the backup storage unit can quickly take over to ensure the integrity and continuity of the data; this redundant storage design can effectively prevent data loss due to storage failure, ensure the stability and reliability of the system, and enable the system to always store and call data normally during long-term operation, providing solid data support for fault warning.

6. The electrical key equipment fault early warning system based on voiceprint recognition and infrared temperature measurement according to claim 1 is characterized in that: The voiceprint recognition unit of the processing module adopts a deep learning algorithm to construct a voiceprint recognition model, and the model is trained with a large amount of voiceprint data of key electrical equipment in normal and faulty states; the deep learning algorithm can automatically learn the complex features and patterns in the voiceprint data, and has stronger adaptability and accuracy than the traditional rule-based or simple feature matching methods; by learning a large amount of sample data, the model can accurately distinguish the voiceprint features during normal operation of the equipment and the voiceprint changes corresponding to different types of faults, effectively improving the accuracy of voiceprint abnormality judgment; for motor equipment, it can accurately identify the unique voiceprint changes caused by bearing wear and rotor imbalance faults, thereby issuing early warning signals in time, providing a more accurate basis for equipment maintenance.

7. The electrical key equipment fault early warning system based on voiceprint recognition and infrared temperature measurement according to claim 1 is characterized in that: When calculating the device temperature, the temperature analysis unit of the processing module combines the historical temperature data of the device with the current ambient temperature and load conditions for comprehensive analysis. In addition to infrared thermal imaging data and ambient temperature sensor data, the introduction of the historical temperature data of the device can analyze the temperature change trend and determine whether the current temperature increase is a normal fluctuation or an abnormal increase. At the same time, the load condition of the device is considered, because the load change will directly affect the heat generation of the device. When the device load suddenly increases, the temperature may rise accordingly, but it is not a fault state within a reasonable range. Through this comprehensive analysis method, it is possible to more accurately determine whether the device temperature actually exceeds the normal threshold range, reduce the occurrence of false alarms, and improve the reliability and accuracy of fault warnings.

8. The electrical key equipment fault early warning method based on voiceprint recognition and infrared temperature measurement according to claim 2 is characterized in that: Before collecting sound signals and infrared thermal imaging data of key electrical equipment, a pre-diagnosis scan is first performed on the equipment to preliminarily determine whether there are obvious abnormalities in the equipment through low-resolution infrared thermal imaging and simple voiceprint monitoring; if the pre-diagnosis scan finds signs of abnormality, the collection frequency and accuracy are increased, and further detailed data is collected for analysis; this pre-diagnosis scanning method can quickly screen out equipment that may have problems without increasing excessive system resource consumption, conduct targeted data collection and analysis, and improve the detection efficiency of the system; in large-scale electrical equipment groups, pre-diagnosis scanning is first used to quickly locate equipment that may be at risk of failure, avoiding high-frequency, high-precision data collection for all equipment, saving system resources and time, and ensuring that potential faulty equipment is not missed.

9. The electrical key equipment fault early warning method based on voiceprint recognition and infrared temperature measurement according to claim 2 is characterized in that: In the voiceprint recognition process, the collected digital voiceprint data is segmented and processed, and the dynamic time warping algorithm (DTW) is used for feature matching. Since the sound signal may have inconsistent duration when the electrical equipment is running, the DTW algorithm can flexibly match voiceprint data of different lengths on the time axis and find the best matching path, so as to compare the voiceprint features more accurately. Through segmented processing, the analysis can be further refined and the ability to capture subtle changes in voiceprints can be improved. For equipment with periodic operation characteristics, the voiceprint data can be segmented by period and then matched. This can more keenly detect abnormal changes in the voiceprint within each period, enhance the accuracy and reliability of voiceprint recognition, and effectively improve the accuracy of fault warning.

10. The electrical key equipment fault early warning method based on voiceprint recognition and infrared temperature measurement according to claim 2 is characterized in that: When the early warning module sends out a fault warning signal, the system automatically starts the data recording and analysis program to record the voiceprint data, infrared thermal imaging data and equipment operating parameters in the period before the warning, and uses data analysis tools to conduct in-depth analysis of these data to generate a fault report; the fault report includes the time when the fault occurred, possible fault types, and change trend information of related data; this helps equipment maintenance personnel to quickly understand the fault situation, accurately determine the cause of the fault, and formulate effective maintenance plans; the system stores the fault report in the local database and can transmit it to the remote maintenance center via the network, which is convenient for subsequent fault tracing and statistical analysis, and provides strong support for improving the overall maintenance level of equipment.

Images