Elevator anomaly detection method and device, computer device and storage medium

Abstract

The application discloses an elevator anomaly detection method and device, computer equipment and a storage medium. The method is based on audio data and non-audio data of elevator operation, identifies the audio data and non-audio data respectively, obtains audio recognition results and non-audio recognition results, and then performs voting through a model-based voting method based on the audio recognition results and non-audio recognition results to obtain a final detection result. Compared with a traditional elevator fault detection method, the method can more flexibly adapt to different elevator data features based on the calculation of an adversarial network model and Gaussian distribution, and improve the accuracy of elevator anomaly detection. In addition, the method considers the mutual relationship and time sequence information between the audio data and non-audio data during elevator operation, and can also effectively detect anomalies of new audio data and non-audio data.

Description

Technical Field

[0001] This invention relates to the field of elevator operation monitoring technology, and in particular to elevator anomaly detection methods, devices, computer equipment, and storage media. Background Technology

[0002] With societal development, elevators have become increasingly common. However, as more elevators are used, the number of elevator malfunctions is also rising. This is mainly due to the reliance on manual elevator inspections and reports from property management or passengers, which can lead to missed, delayed, and false reports, resulting in a buildup of malfunctions. Traditional methods involve monitoring sensor data during elevator operation to determine if any abnormalities exist. However, relying solely on this sensor data for anomaly detection has certain limitations. Summary of the Invention

[0003] The present invention provides an elevator anomaly detection method and related apparatus, which aims to solve the technical problem that the prior art cannot accurately detect elevator operation faults.

[0004] In a first aspect, embodiments of the present invention provide an elevator anomaly detection method, comprising:

[0005] Audio data and non-audio data of the elevator operation are collected separately. The non-audio data includes the elevator's speed information, acceleration information and position information during operation.

[0006] The audio data and non-audio data are preprocessed to obtain audio processed data and non-audio processed data, and features are extracted from the audio processed data and non-audio processed data respectively to obtain audio features and corresponding non-audio features;

[0007] The audio features are embedded to obtain the target voiceprint vector, and the target voiceprint vector is input into a pre-trained adversarial network model for voiceprint recognition to obtain the audio recognition result, which is either "elevator normal" or "elevator abnormal".

[0008] Calculate the probability density of the mean, covariance matrix, and mixing coefficient of each non-audio feature in a preset benchmark Gaussian distribution, and sum all probability densities by weight to obtain a comprehensive density; determine the non-audio recognition result based on the comprehensive density, wherein the non-audio recognition result is elevator normal or elevator abnormal; the benchmark Gaussian distribution is obtained by constructing a Gaussian mixture model from the non-audio features of the sample when the elevator is normal, and estimate the mean, covariance matrix, and mixing coefficient of each benchmark Gaussian distribution using the EM algorithm;

[0009] Based on the audio recognition results and non-audio recognition results, a voting method is used to obtain the final detection result of elevator operation.

[0010] Secondly, embodiments of the present invention provide an elevator anomaly detection device, comprising:

[0011] The acquisition module is used to acquire audio data and non-audio data of the elevator operation, including speed information, acceleration information and position information of the elevator during operation.

[0012] The feature extraction module is used to preprocess the audio data and non-audio data to obtain audio processed data and non-audio processed data, and to extract features from the audio processed data and non-audio processed data respectively to obtain audio features and corresponding non-audio features;

[0013] An audio recognition module is used to embed the audio features to obtain a target voiceprint vector, and input the target voiceprint vector into a pre-trained adversarial network model for voiceprint recognition to obtain an audio recognition result, wherein the audio recognition result is that the elevator is normal or the elevator is abnormal.

[0014] The non-audio recognition module is used to calculate the probability density of the mean, covariance matrix, and mixing coefficients of each non-audio feature in a preset benchmark Gaussian distribution, and to perform a weighted summation of all probability densities to obtain a comprehensive density; based on the comprehensive density, the non-audio recognition result is determined, which is either the elevator is normal or the elevator is abnormal. The benchmark Gaussian distribution is obtained by constructing a Gaussian mixture model from the non-audio features of the sample when the elevator is normal, and the mean, covariance matrix, and mixing coefficients of each benchmark Gaussian distribution are estimated by the EM algorithm.

[0015] The detection module is used to obtain the final detection result of elevator operation by voting through model integration voting based on the audio recognition results and non-audio recognition results.

[0016] Thirdly, embodiments of the present invention also provide a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the elevator anomaly detection method described in the first aspect.

[0017] Fourthly, embodiments of the present invention also provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to perform the elevator anomaly detection method described in the first aspect.

[0018] This invention provides an elevator anomaly detection method, apparatus, computer equipment, and storage medium. The method identifies both audio and non-audio data based on elevator operation, obtaining audio and non-audio recognition results. Then, based on these results, a model-based voting method is used to obtain the final detection result. Compared to traditional elevator fault detection methods, this method, based on an adversarial network model and Gaussian distribution calculations, can more flexibly adapt to different elevator data characteristics, improving the accuracy of elevator anomaly detection. Furthermore, considering the interrelationships and temporal information between audio and non-audio data during elevator operation, it can also effectively detect anomalies in new audio and non-audio data. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating an embodiment of the elevator anomaly detection method provided by the present invention.

[0021] Figure 2 This is a schematic diagram of an elevator anomaly detection device provided in an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0024] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0025] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0026] Please see Figure 1 As shown, Figure 1 This is a flowchart illustrating an elevator anomaly detection method provided in an embodiment of the present invention, which includes steps S110 to S150.

[0027] Step S110: Collect audio data and non-audio data of the elevator operation respectively. The non-audio data includes speed information, acceleration information and position information of the elevator during operation.

[0028] In this embodiment, audio and non-audio data of the elevator operation are collected separately. The audio data includes sound data during normal and abnormal elevator operation; the non-audio data includes speed, acceleration, and position information during normal and abnormal elevator operation. Data collection can be achieved by installing microphones, monitors, or other audio acquisition devices on the elevator to collect and record audio data during operation. Additionally, various non-audio data during elevator operation can be acquired through sensors, including speed sensors, acceleration sensors, and position sensors.

[0029] In one embodiment, step S110 includes:

[0030] Step S201: Record the audio generated by the elevator operation using an audio acquisition device, and acquire the audio using a data acquisition card to obtain the audio data;

[0031] Step S202: Monitor the elevator operation data through sensors, save the sensor data using the sensor interface, and obtain the non-audio data.

[0032] In this embodiment, based on the audio generated by the elevator operation recorded on the audio acquisition device, the audio data is obtained by acquiring the audio data from the audio acquisition device through the data acquisition card; at the same time, the sensor data monitored by the sensor is saved to the terminal or cloud server through the sensor interface to obtain non-audio data.

[0033] Step S120: Preprocess the audio data and non-audio data to obtain audio processed data and non-audio processed data, and extract features from the audio processed data and non-audio processed data respectively to obtain audio features and corresponding non-audio features;

[0034] In this embodiment, after acquiring audio and non-audio data, preprocessing is required. Specifically, this includes: firstly, vectorizing the audio data to obtain audio vector data; since both audio vector data and non-audio data contain noise or bias, which affects the accuracy and stability of the data, Gaussian filtering is used to denoise the audio vector data and non-audio data. The one-dimensional Gaussian kernel and convolution operation during Gaussian filtering are expressed by the following formula:

[0035]

[0036]

[0037] Where x represents the time offset, σ represents the standard deviation, I(x) represents the input voiceprint data, I′(x) represents the output voiceprint feature value, G(i) is the value of the Gaussian kernel at the i-th position, k represents half of the Gaussian kernel, e represents a constant, and G(x) represents the Gaussian kernel operation.

[0038] Next, since non-audio data have different units of measurement, they need to be normalized. Max-min normalization or Z-score normalization methods can be used to convert each non-audio data point to the same range. Furthermore, since non-audio data has issues such as jitter, it needs to be smoothed using moving average or exponential average methods. Finally, both vectorized audio data (audio vector data) and non-audio data may have deviations or errors during the acquisition process, requiring sampling verification and data correction. Correction methods include interpolation and extrapolation.

[0039] Furthermore, feature extraction is first performed on the preprocessed audio data, using the following method:

[0040] (1) The spectrum corresponding to the preprocessed audio data is divided into short-time windows, and a short-time Fourier transform is performed on each short-time window to obtain spectral features (frequency, amplitude, etc.). This method can be understood as converting the audio signal from the time domain to the frequency domain, thereby obtaining the energy distribution of the sound at different frequencies. The specific process is as follows: Given a discrete signal x(n) of length N, where n = 0, 1, ..., N-1; set the length of each short-time window to M, and the total number of non-overlapping segments to K. For the k-th segment, the window function can be set to w(n), then the segmented signal is:

[0041] X k(n) = x(kR+n)·w(n), where k = 0, 1, ..., K-1, and R is the step size of each window. Next, a discrete Fourier transform is performed on each segment of the signal to obtain the spectral sequence:

[0042]

[0043] Where m = 0, 1, ..., M-1, and i is the complex unit, i.e. Ultimately, X was obtained. k(m) It refers to the energy distribution of an audio signal at different frequencies, which can be converted into the frequency, amplitude, and other characteristics of audio data.

[0044] (2) Next, calculate the zero-crossing rate of each short-time window, that is, calculate the number of times the audio signal crosses zero in the time domain. This is useful for describing the rapid changes of the audio signal. The higher the value of this feature, the more frequently the signal changes.

[0045] Spectral characteristics and signal variation characteristics are used as audio features of audio data.

[0046] Next, feature extraction is performed on the preprocessed non-audio data, as follows:

[0047] (1) Calculate the statistical characteristics of non-audio data, including the mean, variance, skewness, and kurtosis. For example, the mean, variance, skewness, and kurtosis of velocity, and the mean, variance, skewness, and kurtosis of acceleration. It should be noted that the mean, variance, skewness, and kurtosis are not calculated for location.

[0048] (2) Calculate the transform domain features of non-audio data. The transform domain features include discrete cosine transform and principal component analysis (PCA) of non-audio data. For example, calculate the discrete cosine transform and principal component analysis (PCA) of speed.

[0049] Step S130: Embed the audio features to obtain the target voiceprint vector, and input the target voiceprint vector into a pre-trained adversarial network model for voiceprint recognition to obtain the audio recognition result, which is either "elevator normal" or "elevator abnormal".

[0050] In this embodiment, the training process of the adversarial network model includes: obtaining sample voiceprint vectors corresponding to sample audio data of the elevator; using the DBSCAN algorithm to cluster the sample voiceprint vectors to obtain voiceprint vectors of different categories (normal elevator voiceprint vectors and abnormal elevator voiceprint vectors); inputting the voiceprint vectors of different categories into the generator to generate simulated voiceprint vectors of the corresponding categories; finally, based on the sample voiceprint vectors and the corresponding simulated voiceprint vectors, calculating the model loss of the adversarial network model according to a preset loss function, and iteratively optimizing the model parameters of the initial adversarial network model based on the model loss, so that the adversarial network model can better capture the latent representation of voiceprint features, and obtaining the final adversarial network model.

[0051] Furthermore, for clustering sample voiceprint vectors, the type of each sample voiceprint vector can be determined by calculating its neighborhood and core points, and clustering can be performed accordingly. The formula for the DBSCAN algorithm is as follows:

[0052] For each sample speaker vector f in the sample audio data, its ε neighborhood is represented by N(f), which is all feature vectors contained within a spherical region with radius ε centered on the feature vector f, as shown in the following formula:

[0053] N(f)={g|distance(f,g)≤ε},

[0054] If the ε-neighborhood of the audio voiceprint vector f contains at least MinPts feature vectors (this parameter is adjustable), then vector f can be considered as the core point, and we have:

[0055] |N(f)|≥MinPts.

[0056] In this embodiment, the adversarial network model consists of two parts: a generator and a discriminator. The generator is responsible for generating realistic voiceprint features, while the discriminator distinguishes between the generated voiceprint features and real voiceprint features. Specifically, the generator G generates a synthetic voiceprint feature vector G(z) based on an input random vector z; a voiceprint feature vector x is input to the discriminator D, and the discriminator D outputs a probability value D(x) representing its authenticity. The model loss includes generator loss, discriminator loss, and adversarial loss, and the generator loss Loss is calculated using the following formula. G (D,G):

[0057]

[0058] Where E represents the expected value of the probability, and E[log(1-D(G(Z)))] represents the expected value of the logarithm of the probability that the discriminator D considers G(z) to be a false sample for the simulated sample G(z) generated by the generator G.

[0059] The discriminator loss (Loss) is calculated using the following formula. D (D,G):

[0060] Loss D (D,G)=-E[log(D(x))]-E[log(1-D(G(z)))],

[0061] Where -E[log(D(x))] represents the negative log-expected probability that the discriminator D considers x to be a real sample, and -E[log(1-D(G(Z)))] represents the negative log-expected probability that the discriminator considers G(z) to be a fake sample, given the simulated sample G(z) generated by the generator G.

[0062] The adversarial loss Loss is calculated using the following formula. A (D,G):

[0063] Loss A (D,G)=E[log(D(x))]+E[log(1-D(G(z)))].

[0064] It's important to understand that by optimizing the loss functions of the generator and discriminator, the generator can produce more realistic voiceprint features, while the discriminator can more accurately distinguish between real and generated samples. Through an adversarial training process, the generator and discriminator compete with each other, ultimately enabling the generator to learn to generate samples that closely resemble realistic voiceprint features.

[0065] Step S140: Calculate the probability density of the mean, covariance matrix, and mixing coefficient of each non-audio feature in the preset benchmark Gaussian distribution, and perform a weighted summation of all probability densities to obtain a comprehensive density; determine the non-audio recognition result based on the comprehensive density, wherein the non-audio recognition result is elevator normal or elevator abnormal, and the benchmark Gaussian distribution is obtained by constructing a Gaussian mixture model from the non-audio features of the sample when the elevator is normal, and estimate the mean, covariance matrix, and mixing coefficient of each benchmark Gaussian distribution through the EM algorithm;

[0066] In this embodiment, a segment of elevator operation status data is pre-selected from the collected non-audio data as the baseline data for the Gaussian distribution, and the mean, covariance matrix, and mixing coefficient of each Gaussian distribution are estimated using the EM algorithm. After constructing the Gaussian distribution, the current non-audio data to be identified is obtained, and the mean, covariance matrix, and mixing coefficient of each non-audio feature in the corresponding Gaussian distribution are calculated. All probability densities are then weighted and summed to obtain the comprehensive density of the non-audio features to be identified. The non-audio recognition result is determined based on the comprehensive density.

[0067] For example, it can be determined whether the overall density is lower than a preset density threshold. If the overall density is lower than the density threshold, it is determined that the non-audio data indicates an abnormal elevator operation; if the overall density is higher than the density threshold, it is determined that the elevator operation status of the non-audio data is normal.

[0068] Step S150: Based on the audio recognition results and non-audio recognition results, a voting method is used to obtain the final detection result of elevator operation.

[0069] In this embodiment, an adversarial network model is used as the voiceprint recognition model to identify audio data, and a Gaussian mixture model is used as the baseline Gaussian distribution to identify non-audio data. After obtaining the audio and non-audio recognition results, a model ensemble voting method is used to vote and obtain the final detection result of elevator operation. For example, the output label of the voiceprint recognition model is set to 1 to indicate anomaly and 0 to indicate normal; the output threshold of the Gaussian mixture model is set to 0.5, where greater than 0.5 indicates normal and less than 0.5 indicates anomaly. The current voiceprint recognition model output is [0,1,1,0,0,1], and the Gaussian mixture model output is [0.3,0.7,0.9,0.4,0.6,0.8]. Therefore, according to the output and definition of the voiceprint recognition model, the abnormal label sequence of the voiceprint recognition model is [0,1,1,0,0,1]; according to the output threshold of the Gaussian mixture model, the abnormal label sequence of the Gaussian mixture model is [1,0,0,1,0,0]. Next, a vote is held. The voiceprint recognition model is given a higher weight; let's say the voiceprint recognition model has a weight of 2, and the Gaussian mixture model has a weight of 1. A vote of 2 represents an elevator malfunction. The voting results are: [1,2,2,1,0,2]. This shows that the data identified in groups 2, 3, and 6 were voted as malfunctioning, while the data identified in groups 1, 4, and 5 were voted as normal. It's important to note that users can adjust the definition of the voting results as needed, such as setting a vote of 0 for normal and a vote greater than 0 for malfunctioning. If the voting results are: [1,2,2,1,0,2], it can be seen that only the data identified in group 5 represents a normal elevator, while the rest represent malfunctioning elevators.

[0070] Furthermore, in one embodiment, after step S150, the following steps are included:

[0071] Step S310: Obtain elevator detection data from each participant and store it in a pre-built group learning platform. The elevator detection data includes audio and non-audio data of elevator operation, as well as the corresponding final detection results.

[0072] Step S320: Anonymize the elevator detection data of each participant through the group learning platform and share it with other participants in the group learning process.

[0073] In this embodiment, the coverage and sensitivity of the anomaly detection system are further improved by collaboratively analyzing elevator detection data from different locations. Through group learning, elevator detection data from different locations is shared. Each location can act as a participant in the group learning process, constructing a group learning platform. Each participant's elevator detection data is stored on the group learning platform. The shared learning platform anonymizes each participant's elevator detection data and shares it with other participants in the group learning process. Each participant can pull elevator detection data from other participants from the shared platform, learn from their data, and iteratively optimize its local recognition model (voiceprint recognition model and Gaussian mixture model), thereby increasing the accuracy of its local recognition model.

[0074] Based on the audio and non-audio data of elevator operation, this method identifies the audio and non-audio data separately, obtaining audio recognition results and non-audio recognition results respectively. Then, based on the audio and non-audio recognition results, a model-based voting method is used to obtain the final detection result. Compared with traditional elevator fault detection methods, this method, based on adversarial network models and Gaussian distribution calculations, can more flexibly adapt to different elevator data characteristics, greatly improving the accuracy of elevator anomaly detection. In addition, considering the interrelationship and temporal information between the audio and non-audio data during elevator operation, it can also effectively detect anomalies in new audio and non-audio data.

[0075] This invention also provides an elevator anomaly detection device, which is used to execute any of the aforementioned elevator anomaly detection methods. Specifically, please refer to... Figure 2 , Figure 2 This is a schematic block diagram of the elevator anomaly detection device provided in an embodiment of the present invention.

[0076] like Figure 2 As shown, the elevator anomaly detection device 100 includes a data acquisition module, a feature extraction module 120, an audio recognition module 130, a non-audio recognition module 140, and a detection module 150.

[0077] The acquisition module 110 is used to acquire audio data and non-audio data of the elevator operation, respectively. The non-audio data includes speed information, acceleration information and position information of the elevator during operation.

[0078] The feature extraction module 120 is used to preprocess the audio data and non-audio data to obtain audio processed data and non-audio processed data, and to extract features from the audio processed data and non-audio processed data respectively to obtain audio features and corresponding non-audio features.

[0079] The audio recognition module 130 is used to embed the audio features to obtain the target voiceprint vector, and input the target voiceprint vector into a pre-trained adversarial network model for voiceprint recognition to obtain the audio recognition result, which is either "elevator normal" or "elevator abnormal".

[0080] The non-audio recognition module 140 is used to calculate the probability density of the mean, covariance matrix, and mixing coefficient of each non-audio feature in a preset benchmark Gaussian distribution, and to perform a weighted summation of all probability densities to obtain a comprehensive density; based on the comprehensive density, the non-audio recognition result is determined, which is either the elevator is normal or the elevator is abnormal. The benchmark Gaussian distribution is obtained by constructing a Gaussian mixture model from the non-audio features of the sample when the elevator is normal, and the mean, covariance matrix, and mixing coefficient of each benchmark Gaussian distribution are estimated by the EM algorithm.

[0081] The detection module 150 is used to obtain the final detection result of elevator operation by voting through model integration voting based on the audio recognition result and the non-audio recognition result.

[0082] In one embodiment, the feature extraction module 120 includes:

[0083] The Fourier transform unit is used to divide the audio processing data into short-time windows and perform a short-time Fourier transform on each short-time window to obtain spectral features, wherein the spectral features include at least frequency and amplitude.

[0084] The zero-crossing rate calculation unit is used to calculate the zero-crossing rate for each short-time window to obtain the signal change characteristics of the audio processing data;

[0085] The audio features are obtained based on the spectral features and signal variation features.

[0086] A statistical feature calculation unit is used to calculate the statistical features of the non-audio processing data, wherein the statistical features include at least the mean, variance, skewness, and kurtosis.

[0087] A transform domain feature calculation unit is used to calculate the transform domain features of the non-audio processing data, wherein the transform domain features include at least discrete cosine transform features and principal component features.

[0088] The non-audio features are obtained based on the statistical features and transform domain features.

[0089] In one embodiment, the feature extraction module 120 further includes:

[0090] The vectorization unit is used to perform vectorization processing on the audio data to obtain audio vector data;

[0091] A noise reduction processing unit is used to perform noise reduction processing on the audio vector data and non-audio data;

[0092] A normalization processing unit is used to normalize the non-audio data;

[0093] A smoothing processing unit is used to smooth the non-audio data;

[0094] The correction processing unit is used to correct the audio vector data and non-audio data.

[0095] In one embodiment, the acquisition module 110 includes:

[0096] An audio acquisition unit is used to record the audio generated by the elevator operation through an audio acquisition device, and to acquire the audio using a data acquisition card to obtain the audio data;

[0097] The non-audio data acquisition unit is used to monitor the sensor data of elevator operation through sensors, and save the sensor data using a sensor interface to obtain the non-audio data.

[0098] In one embodiment, the elevator anomaly detection device 100 further includes:

[0099] The storage module is used to acquire elevator detection data from each participant and store it in a pre-built group learning platform. The elevator detection data includes audio and non-audio data of elevator operation, as well as the corresponding final detection results.

[0100] The sharing module is used to anonymize the elevator detection data of each participant through the group learning platform and share it with other participants in the group learning process.

[0101] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the elevator anomaly detection method as described above.

[0102] In another embodiment of the invention, a computer-readable storage medium is provided. This computer-readable storage medium may be a non-volatile computer-readable storage medium. The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the elevator anomaly detection method as described above.

[0103] Those skilled in the art will readily understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0104] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Units with the same function may be grouped into one unit. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, or it may be an electrical, mechanical, or other form of connection.

[0105] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0106] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0107] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks.

[0108] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting elevator anomalies, characterized in that, include: Audio data and non-audio data of the elevator operation are collected separately. The non-audio data includes the elevator's speed information, acceleration information and position information during operation. The audio data and non-audio data are preprocessed to obtain audio processed data and non-audio processed data, and features are extracted from the audio processed data and non-audio processed data respectively to obtain audio features and corresponding non-audio features; The audio features are embedded to obtain the target voiceprint vector, and the target voiceprint vector is input into a pre-trained adversarial network model for voiceprint recognition to obtain the audio recognition result, which is either "elevator normal" or "elevator abnormal". Calculate the probability density of the mean, covariance matrix, and mixing coefficient of each non-audio feature in a preset benchmark Gaussian distribution, and sum all probability densities by weight to obtain a comprehensive density; determine the non-audio recognition result based on the comprehensive density, wherein the non-audio recognition result is elevator normal or elevator abnormal; the benchmark Gaussian distribution is obtained by constructing a Gaussian mixture model from the non-audio features of the sample when the elevator is normal, and estimate the mean, covariance matrix, and mixing coefficient of each benchmark Gaussian distribution using the EM algorithm; Based on the audio and non-audio recognition results, a model ensemble voting method is used to obtain the final detection result of elevator operation; specifically, this includes: acquiring elevator detection data from each participant and storing it in a pre-built group learning platform, wherein the elevator detection data includes audio and non-audio data of elevator operation, as well as the corresponding final detection result; and anonymizing the elevator detection data of each participant through the group learning platform and sharing it with other participants in the group learning process. The step of extracting features from the audio processed data and the non-audio processed data to obtain audio features and non-audio features respectively includes: The audio processing data is divided into short-time windows, and a short-time Fourier transform is performed on each short-time window to obtain spectral features, which include at least frequency and amplitude. Calculate the zero crossover rate for each short-time window to obtain the signal variation characteristics of the audio processed data; The audio features are obtained based on the spectral features and signal variation features. Calculate the statistical characteristics of the non-audio processing data, wherein the statistical characteristics include at least the mean, variance, skewness, and kurtosis. Calculate the transform domain features of the non-audio processed data, wherein the transform domain features include at least discrete cosine transform features and principal component features; The non-audio features are obtained based on the statistical features and transform domain features.

2. The elevator anomaly detection method according to claim 1, characterized in that, The adversarial network model includes a generator and a discriminator, and the training process of the adversarial network model includes: The sample voiceprint vector of the elevator is obtained, and the sample voiceprint vector is clustered using the DBSCAN algorithm to obtain voiceprint vectors of different categories. The voiceprint vectors of different categories are then input into the generator to generate simulated voiceprint vectors of the corresponding categories. The categories of the voiceprint vectors include voiceprint vectors of elevators operating normally and voiceprint vectors of elevators operating abnormally. Based on the sample voiceprint vector and the corresponding simulated voiceprint vector, the model loss of the adversarial network model is calculated according to a preset loss function, and the model parameters of the initial adversarial network model are iteratively optimized based on the model loss to obtain the adversarial network model.

3. The elevator anomaly detection method according to claim 2, characterized in that, The model loss includes generator loss, discriminator loss, and adversarial loss. Calculating the model loss of the adversarial network model based on a preset loss function includes: The generator loss is calculated using the following formula. : ， Where E represents the expected value of the probability. This represents the simulated samples generated by generator G. Discriminator D believes The expected value of the inverse logarithm of the probability of a false sample; The discriminator loss is calculated using the following formula. : ， in, Indicates for real samples Discriminator D believes The negative log-expected value of the probability of the true sample. This represents the simulated samples generated by generator G. The discriminator believes It is the negative log-expected value of the probability of a false sample; The countermeasure loss is calculated using the following formula. : 。 4. The elevator anomaly detection method according to claim 1, characterized in that, The preprocessing of the audio data and non-audio data includes: The audio data is vectorized to obtain audio vector data; The audio vector data and non-audio data are subjected to noise reduction processing; The non-audio data is normalized. The non-audio data is smoothed. Data correction is performed on the audio vector data and non-audio data.

5. The elevator anomaly detection method according to claim 1, characterized in that, The separate collection of audio and non-audio data from elevator operation includes: The audio generated by the elevator operation is recorded by an audio acquisition device, and the audio is acquired by a data acquisition card to obtain the audio data; The non-audio data is obtained by monitoring the elevator's operation data through sensors and saving the data using a sensor interface.

6. An elevator anomaly detection device, characterized in that, include: The acquisition module is used to acquire audio data and non-audio data of the elevator operation, including speed information, acceleration information and position information of the elevator during operation. The feature extraction module is used to preprocess the audio data and non-audio data to obtain audio processed data and non-audio processed data, and to extract features from the audio processed data and non-audio processed data respectively to obtain audio features and corresponding non-audio features; An audio recognition module is used to embed the audio features to obtain a target voiceprint vector, and input the target voiceprint vector into a pre-trained adversarial network model for voiceprint recognition to obtain an audio recognition result, wherein the audio recognition result is that the elevator is normal or the elevator is abnormal. The non-audio recognition module is used to calculate the probability density of the mean, covariance matrix, and mixing coefficients of each non-audio feature in a preset benchmark Gaussian distribution, and to perform a weighted summation of all probability densities to obtain a comprehensive density; based on the comprehensive density, the non-audio recognition result is determined, which is either the elevator is normal or the elevator is abnormal. The benchmark Gaussian distribution is obtained by constructing a Gaussian mixture model from the non-audio features of the sample when the elevator is normal, and the mean, covariance matrix, and mixing coefficients of each benchmark Gaussian distribution are estimated by the EM algorithm. The detection module is used to obtain the final detection result of elevator operation by voting through model integration voting based on the audio recognition result and the non-audio recognition result; Specifically, it is used to: acquire elevator detection data from each participant and store it in a pre-built group learning platform. The elevator detection data includes audio and non-audio data of elevator operation, as well as the corresponding final detection results; and anonymize the elevator detection data of each participant through the group learning platform and share it with other participants in the group learning process. The feature extraction module includes: The Fourier transform unit is used to divide the audio processing data into short-time windows and perform a short-time Fourier transform on each short-time window to obtain spectral features, wherein the spectral features include at least frequency and amplitude. The zero-crossing rate calculation unit is used to calculate the zero-crossing rate for each short-time window to obtain the signal change characteristics of the audio processing data; The audio features are obtained based on the spectral features and signal variation features. A statistical feature calculation unit is used to calculate the statistical features of the non-audio processing data, wherein the statistical features include at least the mean, variance, skewness, and kurtosis. A transform domain feature calculation unit is used to calculate the transform domain features of the non-audio processing data, wherein the transform domain features include at least discrete cosine transform features and principal component features. The non-audio features are obtained based on the statistical features and transform domain features.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the elevator anomaly detection method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the elevator anomaly detection method as described in any one of claims 1 to 5.

Images