A fault detection method, device, electronic equipment and storage medium

Abstract

This invention discloses a fault detection method, device, electronic equipment, and storage medium, comprising: first acquiring the initial audio signal of the elevator during operation, and recording the elevator's operating state and position corresponding to the initial audio signal; extracting audio signals with loudness exceeding a preset loudness threshold from the initial audio signal and converting them into a spectrum; extracting semantic features from the spectrum to obtain semantic features; performing incremental clustering on the semantic features based on the operating state and position corresponding to the semantic features to obtain the semantic category to which the semantic features belong; recording the number of occurrences of each semantic category within a preset time period; and determining that the elevator has a fault when the number of occurrences of a semantic category exceeds a set threshold. This method eliminates the need to pre-collect a large number of elevator fault audio signal samples, is applicable to most elevators, does not require configuring standard fault audio signals for each elevator, automates the detection process, saves manpower and resources, and improves the efficiency of elevator fault detection.

Description

Technical Field

[0001] This invention relates to the field of elevator technology, and in particular to a fault detection method, device, electronic equipment, and storage medium. Background Technology

[0002] During operation, elevators may produce abnormal noises due to aging or failure of components. Elevator accidents mainly involve malfunctions in the door opening and closing system, loose or broken cables, and other issues. Therefore, fault detection in elevators is of paramount importance.

[0003] Elevators are generally equipped with sound acquisition modules (such as intercom modules). Therefore, considering the real-time nature and convenience of sound acquisition, existing technologies can detect elevator malfunctions based on sound. Specifically, audio signals under elevator malfunction conditions are collected in advance to obtain standard malfunction audio signals. When detecting elevator malfunctions, audio signals during elevator operation are collected and compared with the standard malfunction audio signals to determine whether the elevator has a malfunction.

[0004] On the one hand, elevators are mature and reliable special equipment, and the probability of them malfunctioning is generally low. Therefore, it is difficult to collect a large number of audio signal samples of elevator malfunctions, and it is difficult to determine the standard malfunction audio signal.

[0005] On the other hand, there are many types and numbers of elevators, and each elevator has a different installation environment and usage conditions. Therefore, the fault audio signals of each elevator may be different. The above method of detecting elevators based on sound requires setting a standard fault audio signal for each elevator, which requires a lot of manpower and resources, resulting in low efficiency of elevator fault detection. Summary of the Invention

[0006] This invention provides a fault detection method to solve the problems in the prior art of fault detection in elevators, such as the difficulty in determining standard fault audio signals and the high manpower and material resources required to set standard sound signals for each elevator.

[0007] In a first aspect, the present invention provides a fault detection method, comprising:

[0008] Acquire the initial audio signal during elevator operation and record the elevator's operating status and position corresponding to the initial audio signal;

[0009] Audio signals with loudness exceeding a preset loudness threshold are extracted from the initial audio signal and converted into a spectrum.

[0010] Semantic features are extracted from the spectrogram to obtain semantic features;

[0011] Based on the running state and position corresponding to the semantic features, incremental clustering is performed on the semantic features to obtain the semantic category to which the semantic features belong;

[0012] Record the number of times each semantic category appears within a preset time period;

[0013] When the number of occurrences of the semantic category exceeds a set threshold, it is determined that the elevator is malfunctioning.

[0014] Secondly, the present invention provides a fault detection device, comprising:

[0015] An initial audio acquisition module is used to acquire the initial audio signal during elevator operation and record the elevator's operating status and position corresponding to the initial audio signal.

[0016] The spectrum conversion module is used to extract audio signals with loudness exceeding a preset loudness threshold from the initial audio signal and convert them into a spectrum graph;

[0017] A semantic feature extraction module is used to extract semantic features from the spectrogram to obtain semantic features;

[0018] The incremental clustering module is used to perform incremental clustering on the semantic features based on the running state and position corresponding to the semantic features, so as to obtain the semantic category to which the semantic features belong;

[0019] The occurrence count recording module is used to record the occurrence count of each semantic category within a preset time period;

[0020] The fault determination module is used to determine that the elevator is faulty when the number of occurrences of the semantic category exceeds a set threshold.

[0021] Thirdly, the present invention provides an electronic device, the electronic device comprising:

[0022] At least one processor; and

[0023] A memory communicatively connected to the at least one processor; wherein,

[0024] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the fault detection method described in the first aspect of the present invention.

[0025] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the fault detection method described in the first aspect of the present invention.

[0026] This invention provides a fault detection method. First, the initial audio signal of the elevator during operation is acquired, and the elevator's operating state and position corresponding to the initial audio signal are recorded. Then, audio signals with loudness exceeding a preset loudness threshold are extracted from the initial audio signal and converted into a spectrogram. Semantic features are extracted from the spectrogram to obtain semantic features. Based on the operating state and position corresponding to the semantic features, incremental clustering is performed on the semantic features to obtain the semantic category to which the semantic features belong. The number of occurrences of each semantic category within a preset time period is recorded. When the number of occurrences of a semantic category exceeds a set threshold, it is determined that the elevator has a fault.

[0027] In this invention, a preset loudness threshold is used to filter out audio signals that may be faulty, thus obtaining the target audio signal. In subsequent steps, only the audio signals that may be faulty are analyzed. When an elevator malfunctions at a certain movement state or location, roughly the same faulty audio signal can usually be obtained at that state and location. Therefore, during elevator malfunction operation, a certain number of audio signals of the same type (fault) can accumulate. Thus, by classifying the semantic features of the target audio signal based on the elevator's operating state and location, the target audio signal can be compared with audio signals that may be faulty in the same scene, forming an initial cluster. Based on this, incremental clustering of the semantic features in the same scene yields the semantic category of the latest obtained semantic features. Since a certain number of target audio signals of the same category can only be collected when the elevator malfunctions, allowing for incremental clustering, a malfunction can be determined when the frequency of occurrence of a semantic category exceeds a set threshold.

[0028] On the one hand, in the fault detection method of the present invention, there may be no initial semantic feature data when clustering begins, but the data can be accumulated through the elevator operation process, without the need to collect a large number of elevator fault audio signal samples in advance.

[0029] On the other hand, it can perform fault detection for most elevators without requiring a standard fault audio signal for each elevator. The detection process is automated, saving manpower and resources and improving the efficiency of elevator fault detection.

[0030] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a flowchart of a fault detection method provided in Embodiment 1 of the present invention;

[0033] Figure 2 This is a flowchart of a fault detection method provided in Embodiment 2 of the present invention;

[0034] Figure 3 This is a schematic diagram of the structure of a fault detection device provided in Embodiment 3 of the present invention;

[0035] Figure 4 This is a schematic diagram of the structure of the electronic device provided in Embodiment 4 of the present invention. Detailed Implementation

[0036] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0037] Example 1

[0038] Figure 1 This is a flowchart of a fault detection method provided in Embodiment 1 of the present invention. This embodiment is applicable to fault detection situations. The method can be executed by a fault detection device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the fault detection method includes:

[0039] S101. Obtain the initial audio signal during elevator operation and record the elevator's operating status and position corresponding to the initial audio signal.

[0040] Elevators are typically equipped with accelerometers and sound acquisition modules. Accelerometers are used to acquire the elevator's operating status, including states such as stationary, accelerating, constant speed, and decelerating. Sound acquisition modules are used to collect audio signals. One such module can be an intercom within the elevator, primarily used for passenger calls and requests for assistance; thus, it can also be used to acquire audio signals.

[0041] In addition, position sensors are installed in elevators. Position sensors are the "eyes" of elevators and can determine the position of the elevator. Commonly used position sensors include grating encoders and Hall sensors.

[0042] S102. Extract audio signals with loudness exceeding a preset loudness threshold from the initial audio signal and convert them into a spectrum.

[0043] Specifically, this can be done by: extracting audio signals with loudness exceeding a preset loudness threshold from the initial audio signal to obtain the target audio signal, and then converting the target audio signal into a spectrum.

[0044] The initial audio signal is the audio signal during elevator operation, and the target audio signal is the audio signal whose loudness exceeds the preset loudness threshold. It may be a fault audio signal (abnormal noise of elevator malfunction) or noise.

[0045] Audio signals are continuous-time signals, i.e., time-domain signals, typically digitized using discrete sampling. The elements included are time and amplitude (intensity). The spectrum in a spectral signal refers to the representation of a time-domain signal in the frequency domain, including both frequency and amplitude. Converting a target audio signal into a spectrum can be achieved through Fourier transform, Mel-ray diffraction, or energy density methods. For example, Fourier transform can be used for frequency domain analysis, decomposing a time-domain signal into a series of sine and cosine functions to obtain the signal's spectral information.

[0046] S103. Extract semantic features from the spectrogram to obtain semantic features.

[0047] Semantic features refer to features used in fields such as natural language processing or computer vision to represent the meaning of text or images. These features capture the semantic information in the data, that is, the meaning or significance of the data, not just its surface form. In computer vision, semantic features may be representations of objects, scenes, or attributes in an image. The essence of semantic features is a feature vector, which can be extracted using convolutional neural networks (CNNs) or pre-trained image classification models (such as ResNet, Inception, etc.). These models are trained to recognize different objects and scenes in images and represent them as vectors for further processing and analysis. It is known that semantic features are mainly processed and analyzed based on images. Therefore, in step S102, the audio signal is first converted into a spectrogram to analyze the semantic features of the spectrogram.

[0048] Semantic feature extraction can be performed using deep learning models. Deep learning combines low-level features to form more abstract high-level representations of attribute categories or features, thereby discovering distributed feature representations of data.

[0049] Specifically, the spectrogram is input into a preset semantic feature extraction model to output the semantic features of the sound in the spectrogram; wherein, the training data of the semantic feature extraction model includes audio signals other than elevator operation.

[0050] The semantic feature extraction model is based on a deep learning model. This deep learning model doesn't directly identify sound categories; its output is the semantic features of the sound. The distance or density between semantic features of different categories in the feature space should be greater than that between semantic features of the same category. This is a characteristic of the deep learning model's output and also its training objective. During model training, a deep learning model capable of outputting semantic features of different categories is first trained. This is the output characteristic of the deep learning model. In the usage phase, the semantic feature classification layer at the end of the deep learning model is removed, resulting in the semantic feature extraction model.

[0051] Deep learning models differ from audio recognition models that directly identify and output sound categories. Audio recognition models require training with elevator malfunction audio datasets and can only recognize trained categories. In contrast, deep learning models can be trained using spectral data from other categories with semantic features. Therefore, their training data can include audio signals that are not related to elevator operation, such as different sounds of object collisions or birdsong. The deep learning model only needs to have the ability to output semantic features based on the spectrogram.

[0052] It should be noted that the semantic features mentioned above refer to the features of sounds that are perceived as similar by human hearing (such as the sound of two specific objects colliding with each other). Multiple sounds with the same semantic features may not necessarily show a simple linear relationship between the amplitudes of each frequency point in their respective spectrograms. Therefore, it is necessary to use deep learning algorithm models with high-dimensional and non-linear expressive capabilities for fitting.

[0053] S104. Based on the running state and position corresponding to the semantic features, perform incremental clustering on the semantic features to obtain the semantic category to which the semantic features belong.

[0054] Incremental clustering addresses the problem of maintaining or altering the structure of k clusters. For example, a new data point in a specific sequence might be assigned to one of the existing k clusters, or it might be assigned to a new cluster, thus increasing the number of clusters. When a new data point might be assigned to one of the existing k clusters, the result of the previous clustering is used to assign the data point to an existing cluster; that is, the newly added data point is assigned to the cluster whose center is closest to it, and the center is moved towards the new data point.

[0055] The above spectrograms could be either the spectrograms corresponding to fault audio or the spectrograms corresponding to noise. The operating state and location corresponding to the semantic features represent the scene in which the elevator operates. When comparing spectrograms under the same scene, it is easier to determine whether the spectrogram corresponds to fault audio or is noise that occurs by chance.

[0056] For noise, firstly, from a physics perspective, noise vibrations are irregular, and its sound waveform is chaotic and irregular, differing from the semantic features of the spectrum of faulty audio. Secondly, noise is sporadic, with very few corresponding audio frequencies. Based on these two factors, the semantic features of noise are difficult to categorize into a specific type of semantic feature during incremental clustering; the semantic features of noise are typically scattered throughout the feature space.

[0057] For the spectrogram of fault audio, when the elevator is faulty, the same or similar fault audio spectrograms can be frequently collected during operation. The semantic features corresponding to these spectrograms are also the same or similar. Therefore, when performing incremental clustering, they are more likely to be classified into a certain type of semantic feature.

[0058] When performing incremental clustering, a certain clustering range can be set. When the number of semantic features within a clustering range reaches a preset number of features, the semantic features within this range can be considered as a semantic category. For semantic features with a small number of features, a separate semantic category is not listed. Therefore, for noisy semantic features with a small number of features, they usually cannot be clustered, and their semantic categories cannot be output.

[0059] It should be noted that initial semantic feature data may not be available at the start of clustering; instead, it can be accumulated during elevator operation, without the need to collect a large number of audio signal samples of elevator malfunctions beforehand. In one scenario, for elevators without initial clustering data, semantic feature data can be accumulated during operation using the method described in this embodiment to obtain semantic categories. In another scenario, if a new malfunction occurs, a new semantic category can be obtained by accumulating semantic feature data.

[0060] S105. Record the number of times each semantic category appears within a preset time period.

[0061] The above semantic features may originate from noise or from abnormal audio sounds caused by elevator malfunctions.

[0062] Noise corresponds to very few points and usually has no definite semantic category. Therefore, the semantic category corresponding to noise is difficult to repeat, and its semantic category usually appears less frequently within a preset time period.

[0063] When an elevator malfunctions at a certain state of motion or at a certain position, the same fault audio signal can usually be repeatedly obtained at that state of motion or at that position. Therefore, during the operation of the elevator malfunctioning, a certain number of the same or similar fault audio signals can be accumulated, and thus a certain number of the same or similar semantic features of the spectrum can be obtained. In this case, the semantic type to which the semantic feature belongs will appear more often.

[0064] S106. When the number of occurrences of a semantic category exceeds a set threshold, it is determined that the elevator is faulty.

[0065] As can be seen from the above, when an elevator malfunctions, there will be a situation where one or more semantic categories appear more frequently. Therefore, when the number of occurrences of a semantic category exceeds a set threshold within a preset time period, it can be determined that the elevator is malfunctioning.

[0066] Each semantic category represents a fault. In this invention, the fault type corresponding to each semantic category is not limited, but it is clear that different semantic categories correspond to different faults. When distinguishing semantic categories, they can be named according to the output order of the semantic categories. For example, the first semantic category obtained is named category 1, and the subsequent semantic categories are named category 2, category 3, and so on.

[0067] In an optional embodiment, after incrementally clustering the semantic features based on their corresponding operating states and locations to obtain the semantic categories to which the semantic features belong, the method further includes storing the semantic features and semantic categories as clustering data. Therefore, after outputting semantic categories for the operating data of an elevator, the previous clustering data can be used for incremental clustering in subsequent fault detection processes. For example, if it is determined during a fault detection process that the occurrence frequency of semantic category 2 exceeds a set threshold, it can be determined that the elevator has a fault, and that the fault is related to semantic category 2. In some cases, if a new fault occurs, a new semantic category can be obtained by accumulating semantic feature data, and then named based on the existing semantic categories. For example, existing semantic categories are named categories 1-13, and when a new semantic category appears, it is named category 14.

[0068] In an optional embodiment, the threshold is set as the product of the number of elevator arrivals within a preset time period and a preset proportion. When the number of occurrences of a semantic category exceeds the set threshold, an elevator malfunction is determined. That is, when the ratio of the number of occurrences N of a certain semantic category to the number of elevator arrivals M exceeds a preset proportion a, an elevator malfunction is determined, where N ≤ M. The preset proportion a can be set according to actual needs, for example, it can be 0.6.

[0069] Elevators are pre-programmed with a destination floor in mind; stopping at that floor signifies arrival. In other words, each elevator trip corresponds to one arrival cycle. This is equivalent to monitoring the frequency of elevator malfunctions during operation; if the frequency is too high, the elevator can be considered to be malfunctioning.

[0070] In this invention, a preset loudness threshold is used to filter out audio signals that may be faulty, thus obtaining the target audio signal. In subsequent steps, only the audio signals that may be faulty are analyzed. When an elevator malfunctions at a certain movement state or location, roughly the same faulty audio signal can usually be obtained at that state and location. Therefore, during elevator malfunction operation, a certain number of audio signals of the same type (fault) can accumulate. Thus, by classifying the semantic features of the target audio signal based on the elevator's operating state and location, the target audio signal can be compared with audio signals that may be faulty in the same scene, forming an initial cluster. Based on this, incremental clustering of the semantic features in the same scene yields the semantic category of the latest obtained semantic features. Since a certain number of target audio signals of the same category can only be collected when the elevator malfunctions, allowing for incremental clustering, a malfunction can be determined when the frequency of occurrence of a semantic category exceeds a set threshold. On the one hand, in the fault detection method of this invention, initial semantic feature data is not required at the start of clustering; instead, it is accumulated through the elevator operation process, eliminating the need to collect a large number of elevator fault audio signal samples beforehand. On the other hand, fault detection can be performed on most elevators (existing elevators generally have sound acquisition devices, acceleration sensors, and position sensors), eliminating the need to configure standard fault audio signals for each elevator. The detection process is automated, saving manpower and resources and improving the efficiency of elevator fault detection.

[0071] Example 2

[0072] Figure 2 This is a flowchart of a fault detection method provided in Embodiment 2 of the present invention. This embodiment is an optimization based on Embodiment 1 described above, such as... Figure 2 As shown, the fault detection method includes:

[0073] S201. Obtain the initial audio signal during elevator operation and record the elevator's operating status and position corresponding to the initial audio signal.

[0074] S202. Extract audio signals with loudness exceeding a preset loudness threshold from the initial audio signal and convert them into a spectrum.

[0075] S203. Extract semantic features from the spectrogram to obtain semantic features.

[0076] S201-S203 are similar to S101-S103 in Embodiment 1, and you can refer to the relevant descriptions of S101-S103 for details.

[0077] S204. Based on the running state and position corresponding to the semantic features, insert the semantic features into the corresponding vector set.

[0078] Semantic features are feature vectors. The same running state and the same position can be considered the same scene. Placing the semantic features of the same scene into the same set yields the vector set corresponding to that scene. For example, vector set A1 (scene A1) represents a constant speed state and the first floor, vector set A2 (scene A2) represents a constant speed state and the second floor, scene B1 represents a decelerating state and the first floor, and scene B2 represents a decelerating state and the second floor. Taking vector set A1 as an example, A1 = {A...} 11 A 12 A 13 A 14 ….}, where the semantic features (feature vectors) are all feature data obtained under the uniform speed state and the first floor scene.

[0079] S205. Feature space is constructed based on vector sets.

[0080] A set of vectors can form a feature space, that is, spatial coordinates.

[0081] S206. Determine whether the semantic feature is located in an existing cluster in the feature space.

[0082] Each cluster corresponds to a semantic category.

[0083] If yes, then execute S207; otherwise, execute S208-S209.

[0084] Since semantic features in this invention are incrementally clustered, there may be situations where there is no feature data or the amount of feature data is too small to form clusters temporarily. Therefore, the purpose of this step is to detect the environment in which the newly added semantic features exist.

[0085] S207. Determine the semantic category of the cluster as the semantic category to which the semantic feature belongs.

[0086] If a semantic feature is located in an existing cluster in the feature space, it means that the semantic feature belongs to that cluster, and the semantic category of the cluster can be determined as the semantic category to which the semantic feature belongs.

[0087] S212-S213 are executed after S207.

[0088] S208. Determine the number of all semantic features within a first target area at a preset distance around the semantic features.

[0089] Cluster analysis itself is not an automatic task. It usually requires modification of data preprocessing and model parameters until the results achieve the desired attributes. The preset distance is a parameter in the incremental clustering algorithm, which depends on the individual dataset and the intended use of the results. Therefore, it can be set according to actual needs.

[0090] S209. Determine whether the number of all semantic features is greater than the preset clustering threshold.

[0091] If yes, then execute S210. If no, then execute S211.

[0092] If a semantic feature is not located in an existing cluster in the feature space, it could be due to two reasons. First, the total feature data (clustering data) in the feature space may be too small to form a cluster before the current feature is added. Second, the current semantic feature may differ significantly from the semantic features in existing clusters; for example, the current semantic feature might be noise or a semantic feature corresponding to a new faulty audio. Therefore, it is necessary to detect the quantity (density) of semantic features within a certain range around the current semantic feature to determine its attributes.

[0093] The first target region is obtained by centering on the newly added semantic features. It is then determined whether the number of all semantic features in the first target region is greater than the preset clustering threshold, that is, whether the newly added semantic features meet the clustering mechanism. The clustering mechanism is that the number of semantic features within a certain range must be greater than the preset clustering threshold.

[0094] S210. Group all semantic features within the first target region into the same cluster and generate the semantic category of the cluster as the semantic category of the semantic feature.

[0095] S212-S213 are executed after S210.

[0096] If the number of all semantic features in the first target region is greater than the preset clustering threshold, it can be determined that the newly added semantic feature satisfies the clustering mechanism. Then, all semantic features in the first target region can be grouped into the same cluster, and the semantic category of the cluster can be generated as the semantic category of the newly added semantic feature.

[0097] This step is applicable when: before the current semantic feature is added to the feature space, the total feature data (clustering data) in the first target region is insufficient to form a cluster, but the threshold has been reached. After the current semantic feature is added to the feature space, the first target region satisfies the clustering mechanism and can form a cluster, thus obtaining the semantic type of the newly added semantic feature.

[0098] S211. The semantic feature is determined to be noise.

[0099] S212-S213 are executed after S211.

[0100] If the number of all semantic features within the first target region is less than or equal to the preset clustering threshold, it indicates that the first target region has not yet met the clustering mechanism. Therefore, the newly added semantic feature cannot be classified into any cluster, and is thus determined to be noise. However, it should be noted that classifying a newly added semantic feature as noise may be temporary. For ease of description, the newly added semantic feature is referred to as a labeled feature. If, during subsequent data accumulation, the first target region containing the labeled feature meets the clustering mechanism, the attributes of the labeled feature will change, and it can be classified into a cluster and assigned a semantic type.

[0101] The above method allows for the reasonable localization of the semantic type attributes of newly added semantic features in the feature space, improving the accuracy of elevator fault detection. In practice, an incremental clustering approach is employed, enabling clustering even without initial clustering data. As the clustering data increases, the semantic type attributes of the semantic features can be output. This eliminates the need for pre-collecting elevator data, is suitable for various elevator types due to resource constraints, and does not require configuring standard fault audio signals for each elevator. The detection process is automated, saving manpower and resources and improving elevator fault detection efficiency.

[0102] In an optional embodiment, after determining the semantic category of the cluster to which the semantic feature belongs in S207, the method further includes: acquiring other semantic features that are within a preset distance from the semantic feature as neighboring features; for each neighboring feature, determining a second target region within a preset distance around the neighboring feature; determining whether there is a third target region in the second target region where the number of all semantic features in the region is greater than a preset clustering threshold; if so, when the third target region does not belong to any cluster, classifying all semantic features in the third target region into the same cluster and generating the semantic category of the cluster.

[0103] In the process of S208-S211, when the newly added semantic features cannot be clustered into clusters, the clustering characteristics of the newly added semantic features within a certain range are detected. In this embodiment, when the newly added semantic features have been clustered into clusters, the influence of the newly added semantic features on the semantic types of adjacent features is further analyzed.

[0104] It should be noted that the aforementioned adjacent features can be semantic features belonging to a certain cluster, i.e., clustered semantic features, or semantic features originally defined as noise, i.e., unclustered semantic features.

[0105] In this invention, a preset distance is set for the clusters to limit the size of each cluster. The radii of the first, second, and third target regions mentioned above are all within this preset distance. For the second target region, its semantic type may change after adding a new semantic feature. When a third target region exists within the second target region where the number of all semantic features is greater than a preset clustering threshold, the third target region satisfies the clustering characteristic. At this point, it is determined whether the third target region is located in an existing cluster. If so, no new cluster needs to be generated; otherwise, all semantic features of the third target region are grouped into the same cluster, and a semantic category for the cluster is generated. Semantic features in the third target region that originally belonged to a certain cluster can be grouped into multiple clusters, thus possessing a "dual" or "multiple" identity, corresponding to two or more semantic types. Semantic features in the third target region that were originally defined as noise can be grouped into a new cluster and assigned a new semantic type. In this embodiment, when adding new semantic features, the impact of semantic feature changes on the semantic types of adjacent semantic features can be considered, ensuring the real-time and accurate detection results of elevator faults.

[0106] S212. Record the number of times each semantic category appears within a preset time period.

[0107] S213. When the number of occurrences of a semantic category exceeds a set threshold, it is determined that the elevator is faulty.

[0108] It should be noted that, as described above, some semantic features may have multiple identities. When a new semantic feature has two or more semantic types, it may output two or more semantic types at the same time. However, as the elevator data continues to increase, the semantic types will form a clustering trend based on the actual fault situation, which will cause the frequency of different semantic types to vary. Only when the frequency of a semantic type exceeds the set threshold can it be determined as an elevator fault. Of course, it is also possible that the elevator has multiple faults at the same time and the frequency of different semantic types exceeds the set threshold.

[0109] In an optional embodiment, after determining that the elevator has a fault, the method further includes: identifying semantic categories whose occurrence frequency exceeds a set threshold as target categories; and using the elevator's operating status and location corresponding to the target category as elevator fault information. The fault information can provide guidance for maintenance personnel, who can locate the elevator based on the location information and combine it with the operating status to perform maintenance work, thereby improving maintenance efficiency.

[0110] In this embodiment of the invention, on the one hand, based on the operating state and position corresponding to the semantic features, the semantic features are inserted into the corresponding vector set; a feature space is constructed based on the vector set; it is determined whether the semantic features are located in an existing cluster in the feature space, with each cluster corresponding to a semantic category; if so, the semantic category of the cluster is determined as the semantic category to which the semantic features belong; if not, the number of all semantic features within a first target area at a preset distance around the semantic features is determined; it is determined whether the number of all semantic features is greater than a preset clustering threshold; if so, all semantic features within the first target area are grouped into the same cluster, and the semantic category of the cluster is generated as the semantic category of the semantic features; if not, the semantic features are determined to be noise. Through the above method, the semantic type attributes of newly added semantic features in the feature space can be reasonably located, improving the accuracy of detecting elevator malfunctions. In the specific process, an incremental clustering method is adopted, which can still perform clustering even without initial clustering data. As the clustering data increases, the semantic type attributes of semantic features can be output. There is no need to collect elevator data in advance. Due to the limited manpower and material resources, it is applicable to various elevators. There is no need to configure standard fault audio signals for each elevator. The detection process is automated, saving manpower and material resources and improving the efficiency of elevator fault detection.

[0111] On the other hand, when adding new semantic features, the impact of changes in semantic features on the semantic types of adjacent semantic features can be taken into account, so that the detection results of elevator faults have real-time performance and accuracy.

[0112] Example 3

[0113] Figure 3 This is a schematic diagram of a fault detection device provided in Embodiment 3 of the present invention. Figure 3 As shown, the fault detection device includes:

[0114] The initial audio acquisition module 301 is used to acquire the initial audio signal during elevator operation and record the elevator's operating status and position corresponding to the initial audio signal.

[0115] The spectrum conversion module 302 is used to extract audio signals with loudness exceeding a preset loudness threshold from the initial audio signal and convert them into a spectrum diagram;

[0116] The semantic feature extraction module 303 is used to extract semantic features from the spectrogram to obtain semantic features;

[0117] The incremental clustering module 304 is used to perform incremental clustering on the semantic features based on the running state and position corresponding to the semantic features, so as to obtain the semantic category to which the semantic features belong;

[0118] The occurrence count recording module 305 is used to record the occurrence count of each semantic category within a preset time period;

[0119] The fault determination module 306 is used to determine that the elevator has a fault when the number of occurrences of the semantic category exceeds a set threshold.

[0120] In an optional embodiment, the semantic feature extraction module 303 includes:

[0121] The semantic feature extraction submodule is used to input the spectrogram into a preset semantic feature extraction model to output the semantic features of the sound in the spectrogram; wherein, the training data of the semantic feature extraction model includes audio signals other than elevator operation.

[0122] In an optional embodiment, the incremental clustering module 304 includes:

[0123] The set acquisition submodule is used to insert the semantic features into the corresponding vector set based on the running state and position corresponding to the semantic features;

[0124] A feature space construction submodule is used to construct a feature space based on the vector set;

[0125] The first judgment submodule is used to determine whether the semantic feature is located in an existing cluster in the feature space, and each cluster corresponds to a semantic category; if yes, the semantic category determination submodule is executed; if no, the quantity determination submodule is executed.

[0126] The semantic category determination first submodule is used to determine the semantic category of the cluster to which the semantic feature belongs;

[0127] The quantity determination submodule is used to determine the quantity of all semantic features within a first target region at a preset distance around the semantic feature;

[0128] The quantity judgment submodule is used to determine whether the quantity of all semantic features is greater than a preset clustering threshold; if so, the content of the semantic category determination submodule is executed; otherwise, the content of the noise determination submodule is executed.

[0129] The semantic category determination second submodule is used to group all semantic features within the first target region into the same cluster and generate the semantic category of the cluster as the semantic category of the semantic feature;

[0130] The noise determination submodule is used to determine that the semantic feature is noise.

[0131] In an optional embodiment, the incremental clustering module 304 further includes:

[0132] The adjacent feature determination submodule is used to obtain other semantic features whose distance from the semantic feature is within the preset distance, as adjacent features;

[0133] The second target region determination submodule is used to determine a second target region at a preset distance around each of the adjacent features;

[0134] The region determination submodule is used to determine whether there is a third target region in the second target region where the number of all semantic features in the region is greater than a preset clustering threshold; if so, the content of the semantic category determination third submodule is executed.

[0135] The semantic category determination third submodule is used to classify all semantic features within the third target region into the same cluster and generate the semantic category of the cluster when the third target region does not belong to any cluster.

[0136] In an optional embodiment, the fault detection device further includes:

[0137] The storage module is used to store the semantic features and semantic categories as clustering data.

[0138] In an optional embodiment, the set threshold is the product of the number of elevator arrivals within the preset time period and a preset proportion.

[0139] In an optional embodiment, the fault detection device further includes:

[0140] The target category determination module is used to determine the semantic category whose occurrence frequency exceeds a set threshold as the target category;

[0141] The fault information determination module is used to determine the operating status and location of the elevator corresponding to the target category as the elevator's fault information.

[0142] The fault detection device provided in the embodiments of the present invention can execute the fault detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0143] Example 4

[0144] Figure 4A schematic diagram of an electronic device 40 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0145] like Figure 4 As shown, the electronic device 40 includes at least one processor 41 and a memory, such as a read-only memory (ROM) 42 or a random access memory (RAM) 43, communicatively connected to the at least one processor 41. The memory stores computer programs executable by the at least one processor. The processor 41 can perform various appropriate actions and processes based on the computer program stored in the ROM 42 or loaded into the RAM 43 from storage unit 48. The RAM 43 may also store various programs and data required for the operation of the electronic device 40. The processor 41, ROM 42, and RAM 43 are interconnected via a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0146] Multiple components in electronic device 40 are connected to I / O interface 45, including: input unit 46, such as keyboard, mouse, etc.; output unit 47, such as various types of monitors, speakers, etc.; storage unit 48, such as disk, optical disk, etc.; and communication unit 49, such as network card, modem, wireless transceiver, etc. Communication unit 49 allows electronic device 40 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0147] Processor 41 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 41 performs the various methods and processes described above, such as fault detection methods.

[0148] In some embodiments, the fault detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 40 via ROM 42 and / or communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, one or more steps of the fault detection method described above may be performed. Alternatively, in other embodiments, processor 41 may be configured to perform the fault detection method by any other suitable means (e.g., by means of firmware).

[0149] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0150] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0151] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0152] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0153] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0154] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0155] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0156] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A fault detection method characterized by, include: Acquire the initial audio signal during elevator operation and record the elevator's operating status and position corresponding to the initial audio signal; Audio signals with loudness exceeding a preset loudness threshold are extracted from the initial audio signal and converted into a spectrum. Semantic features are extracted from the spectrogram to obtain semantic features; Based on the running state and position corresponding to the semantic features, incremental clustering is performed on the semantic features to obtain the semantic category to which the semantic features belong; Record the number of times each semantic category appears within a preset time period; When the number of occurrences of the semantic category exceeds a set threshold, it is determined that the elevator is malfunctioning. The step of incrementally clustering the semantic features based on their corresponding running states and positions to obtain the semantic category to which the semantic features belong includes: Based on the running state and position corresponding to the semantic features, the semantic features are inserted into the corresponding vector set; The feature space is constructed based on the aforementioned vector set; Determine whether the semantic feature is located in an existing cluster in the feature space, with each cluster corresponding to a semantic category; If so, the semantic category of the cluster to which the semantic feature belongs is determined; If not, determine the number of all semantic features within a first target region at a preset distance around the semantic feature; Determine whether the number of all semantic features is greater than a preset clustering threshold; If so, all semantic features within the first target region are grouped into the same cluster, and the semantic category of the cluster is generated as the semantic category of the semantic feature; If not, the semantic feature is determined to be noise.

2. The fault detection method of claim 1, wherein, The semantic feature extraction from the spectrogram to obtain semantic features includes: The spectrogram is input into a preset semantic feature extraction model to output the semantic features of the sound in the spectrogram; wherein, the training data of the semantic feature extraction model includes audio signals other than elevator operation.

3. The fault detection method of claim 1, wherein, After determining the semantic category of the cluster to which the semantic feature belongs, the method further includes: Obtain other semantic features whose distance from the semantic feature is within the preset distance, and use them as adjacent features; For each of the adjacent features, a second target region at a preset distance is determined around the adjacent feature; Determine whether there exists a third target region in the second target region where the number of all semantic features in the region is greater than a preset clustering threshold; If so, when the third target region does not belong to any cluster, all semantic features within the third target region are grouped into the same cluster, and a semantic category of the cluster is generated.

4. The fault detection method of claim 1, wherein, After performing incremental clustering on the semantic features based on their corresponding running states and positions to obtain the semantic categories to which the semantic features belong, the method further includes: The semantic features and semantic categories are used as clustering data and stored.

5. The fault detection method according to any one of claims 1 to 4, characterized in that, The set threshold is the product of the number of elevator arrivals within the preset time period and a preset ratio.

6. The fault detection method of any one of claims 1-4, wherein, After confirming that the elevator is malfunctioning, the following steps are also included: The semantic category whose occurrence frequency exceeds a set threshold is identified as the target category. The operating status and location of the elevator corresponding to the target category are used as elevator fault information.

7. A fault detection apparatus characterized by comprising: include: An initial audio acquisition module is used to acquire the initial audio signal during elevator operation and record the elevator's operating status and position corresponding to the initial audio signal. The spectrum conversion module is used to extract audio signals with loudness exceeding a preset loudness threshold from the initial audio signal and convert them into a spectrum graph; A semantic feature extraction module is used to extract semantic features from the spectrogram to obtain semantic features; The incremental clustering module is used to perform incremental clustering on the semantic features based on the running state and position corresponding to the semantic features, so as to obtain the semantic category to which the semantic features belong; The occurrence count recording module is used to record the occurrence count of each semantic category within a preset time period; The fault determination module is used to determine that the elevator is faulty when the number of occurrences of the semantic category exceeds a set threshold. The incremental clustering module includes: The set acquisition submodule is used to insert the semantic features into the corresponding vector set based on the running state and position corresponding to the semantic features; A feature space construction submodule is used to construct a feature space based on the vector set; The first judgment submodule is used to determine whether the semantic feature is located in an existing cluster in the feature space, and each cluster corresponds to a semantic category; if yes, the semantic category determination submodule is executed; if no, the quantity determination submodule is executed. The semantic category determination first submodule is used to determine the semantic category of the cluster to which the semantic feature belongs; The quantity determination submodule is used to determine the quantity of all semantic features within a first target region at a preset distance around the semantic feature; The quantity judgment submodule is used to determine whether the quantity of all semantic features is greater than the preset clustering threshold; if so, the content of the semantic category determination submodule is executed; if not, the content of the noise determination submodule is executed. The semantic category determination second submodule is used to group all semantic features within the first target region into the same cluster and generate the semantic category of the cluster as the semantic category of the semantic feature; The noise determination submodule is used to determine that the semantic feature is noise.

8. An electronic device, comprising: The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the fault detection method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the fault detection method according to any one of claims 1-7.

Images