Fault type determination method and device, computer device, and readable storage medium

Abstract

Embodiments of the present application provide a fault type determination method and device, computer equipment and a readable storage medium, relating to the technical field of fault voiceprint monitoring. The method comprises: determining the similarity between sample voiceprint vectors of different devices of the same type; dividing different devices of the same type into multiple subtypes according to the similarity; obtaining a device voiceprint vector of a target device in the different devices of the same type; and if the device voiceprint vector of the target device is determined to be a fault voiceprint vector, determining the fault type of the target device according to the corresponding relationship in a target fault sample library. The method provided in the embodiments of the present application first classifies devices of the same type according to the similarity between sample voiceprint vectors of different devices of the same type, and the classified devices share a fault sample library, which can reduce the influence of the differences between different devices on the process of determining the fault type, and further improve the accuracy of determining the fault type of the device.

Description

Technical Field

[0001] This invention relates to the field of fault voiceprint monitoring technology, and in particular to a method, apparatus, computer device, and readable storage medium for determining fault types. Background Technology

[0002] During operation, vibrations occur between the device body and its components, or even between the components themselves, generating audio data that characterizes the device's state. This audio data is unique to each device and can be measured and analyzed using electroacoustic instruments. Therefore, we call the characteristic of the device's operating state carried by this audio data a voiceprint. Voiceprints contain a wealth of device state information; thus, they change as the device's operating state changes. Voiceprints can serve as an important feature parameter for anomaly detection and fault analysis.

[0003] In existing technologies, using device acoustic signatures for anomaly detection and fault analysis requires the prior establishment of a fault sample library. Based on the principle of fault sharing, devices of the same type share a fault sample library, and the collected device acoustic signature is compared with the fault sample information in the same library to determine the fault type. However, even for devices of the same type, due to differences in component size, operating power, etc., their acoustic signatures during operation can vary significantly, resulting in low accuracy in determining the fault type.

[0004] Therefore, improving the accuracy of fault type identification has become an urgent technical problem to be solved. Summary of the Invention

[0005] This application provides a method, apparatus, computer device, and readable storage medium for determining fault types, which improves the accuracy of fault type determination.

[0006] The first aspect of this application provides a method for determining a fault type, comprising: determining the similarity between sample voiceprint vectors of different devices of the same type, wherein the sample voiceprint vectors are voiceprint vectors when the devices are operating normally; dividing the different devices of the same type into multiple subtypes based on the similarity; wherein the similarity between different devices in a subtype is greater than or equal to a first preset threshold; each subtype corresponds to a fault sample library, which includes the correspondence between fault voiceprint vectors and fault types; obtaining the device voiceprint vector of a target device among the different devices of the same type; if the device voiceprint vector of the target device is determined to be a fault voiceprint vector, then determining the fault type of the target device based on the correspondence in the target fault sample library; the target fault sample library is the fault sample library corresponding to the subtype to which the target device belongs.

[0007] The fault type determination method provided in this application embodiment classifies devices of the same type with a similarity greater than or equal to a first preset threshold into a subtype based on the similarity between sample voiceprint vectors of devices of the same type. It can be seen that after classification, the differences between devices within each subtype are small, meaning their sound characteristics are also relatively similar. This allows each subtype to correspond to a fault sample library. Thus, when determining the fault type of a target device, the method relies on the correspondence in the target fault sample library, leading to more accurate fault type determination. The method provided in this application embodiment can reduce the impact of differences between different devices on the fault type determination process, thereby significantly improving the accuracy of fault type determination.

[0008] In conjunction with the first implementation of the first aspect, the steps for obtaining the device voiceprint vector of a target device among different devices of the same type include: obtaining device audio data of the target device; determining the device voiceprint vector corresponding to the device audio data; and determining the device voiceprint vector of the target device as a fault voiceprint vector. The steps for determining the device voiceprint vector of the target device as a fault voiceprint vector include: obtaining a first similarity set based on the similarity between the device voiceprint vector and the sample voiceprint vector of each device in the subtype to which the target device belongs; and if the maximum similarity in the first similarity set is less than a first preset threshold, then the device voiceprint vector of the target device is determined to be a fault voiceprint vector.

[0009] Combining the second implementation method of the first aspect, the steps for determining the device voiceprint vector corresponding to the device audio data include: extracting feature data of the device audio data; obtaining device state parameters of the target device, which include K groups of fundamental frequencies and target formant frequencies of the target device, where the target formant frequency is the first K resonant frequencies arranged in descending order of the resonant frequencies generated by the target device, and K is a positive integer; and obtaining the device voiceprint vector corresponding to the device audio data based on the feature data and device state parameters.

[0010] Combining the third implementation method of the first aspect, the steps of obtaining the device voiceprint vector corresponding to the device audio data based on feature data and device status parameters include: concatenating the feature data and device status parameters and inputting them into a neural network to obtain the device voiceprint vector corresponding to the device audio data.

[0011] In conjunction with the fourth implementation method of the first aspect, the steps of obtaining the device voiceprint vector corresponding to the device audio data based on the feature data and device status parameters include: concatenating the feature data and device status parameters and inputting them into the first part of the neural network to obtain the hidden layer features of the network, the neural network including the first part and the second part; concatenating the device status parameters and the hidden layer features of the network and inputting them into the second part of the neural network to obtain the device voiceprint vector corresponding to the device audio data.

[0012] Combining the fifth implementation method of the first aspect, the steps of determining the fault type of the target device according to the correspondence in the target fault sample library include: obtaining a second similarity set based on the similarity between the fault voiceprint vector and each fault voiceprint vector in the target fault sample library; if the maximum similarity in the second similarity set is greater than or equal to a second preset threshold, then the target fault type corresponding to the fault voiceprint vector with the maximum similarity in the target fault sample library is determined according to the correspondence in the target fault sample library, and the target fault type is the fault type of the target device.

[0013] In conjunction with the sixth implementation method of the first aspect, the step of determining the fault type of the target device based on the correspondence in the target fault sample library further includes: if the maximum similarity in the second similarity set is less than the second preset threshold, then output prompt information, which is used to prompt the input fault type; in response to obtaining the input fault type, determine the fault type of the target device as the input fault type; the method further includes: writing the correspondence between the fault voiceprint vector and the input fault type into the target fault sample library.

[0014] A second aspect of this application provides a fault type determination apparatus, comprising: a first determination module, configured to determine the similarity between sample voiceprint vectors of different devices of the same type, wherein the sample voiceprint vectors are voiceprint vectors when the devices are operating normally; a classification module, configured to divide different devices of the same type into multiple subtypes based on the similarity; wherein the similarity between different devices in a subtype is greater than or equal to a first preset threshold; each subtype corresponds to a fault sample library, the fault sample library including the correspondence between fault voiceprint vectors and fault types; an acquisition module, configured to acquire the device voiceprint vector of a target device among different devices of the same type; and a second determination module, configured to determine that the device voiceprint vector of the target device is a fault voiceprint vector, and to determine the fault type of the target device based on the correspondence in the target fault sample library; the target fault sample library is the fault sample library corresponding to the subtype to which the target device belongs.

[0015] In conjunction with the first implementation of the second aspect, the acquisition module is further configured to: acquire device audio data of the target device; determine the device voiceprint vector corresponding to the device audio data; the second determination module is further configured to: obtain a first similarity set based on the similarity between the device voiceprint vector and the sample voiceprint vector of each device in the subtype to which the target device belongs; if the maximum similarity in the first similarity set is less than a first preset threshold, then determine that the device voiceprint vector of the target device is a fault voiceprint vector.

[0016] In conjunction with the second implementation method of the second aspect, the acquisition module is also configured to: extract feature data of device audio data; acquire device status parameters of the target device, the device status parameters including K groups of fundamental frequencies and target formant frequencies of the target device, the target formant frequencies being the first K resonant frequencies arranged in descending order of the resonant frequencies generated by the target device, where K is a positive integer; and obtain the device voiceprint vector corresponding to the device audio data based on the feature data and device status parameters.

[0017] In conjunction with the third implementation method of the second aspect, the acquisition module is also configured to: concatenate the feature data and device status parameters and input them into the neural network to obtain the device voiceprint vector corresponding to the device audio data.

[0018] In conjunction with the fourth implementation method of the second aspect, the acquisition module is also configured to: concatenate the feature data and device status parameters and input them into the first part of the neural network to obtain the hidden layer features of the network, the neural network including the first part and the second part; concatenate the device status parameters and the hidden layer features of the network and input them into the second part of the neural network to obtain the device voiceprint vector corresponding to the device audio data.

[0019] In conjunction with the fifth implementation of the second aspect, the second determining module is further configured to: obtain a second similarity set based on the similarity between the fault acoustic vector and each fault acoustic vector in the target fault sample library; if the maximum similarity in the second similarity set is greater than or equal to a second preset threshold, then determine the target fault type corresponding to the fault acoustic vector with the maximum similarity in the target fault sample library based on the correspondence in the target fault sample library, and the target fault type is the fault type of the target device.

[0020] In conjunction with the sixth implementation of the second aspect, the second determining module is further configured to: if the maximum similarity in the second similarity set is less than the second preset threshold, output a prompt message, the prompt message being used to prompt the input fault type; in response to obtaining the input fault type, determine the fault type of the target device as the input fault type; the fault type determining device further includes a writing module, used to write the correspondence between the fault voiceprint vector and the input fault type into the target fault sample library.

[0021] A third aspect of this application provides a computer device, including: a processor and a memory; the memory is used to store program code and transmit the computer program code to the processor; the processor is used to execute the above-described fault type determination method according to the instructions in the computer program code.

[0022] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when run on a computer device, causes the device to perform the aforementioned fault type determination method.

[0023] The beneficial effects described in aspects two through four can be referred to the analysis of the beneficial effects in aspect one, and will not be repeated here. Attached Figure Description

[0024] The accompanying drawings are provided to further understand the technical solutions of the present invention and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of the present invention and do not constitute a limitation on the technical solutions of the present invention.

[0025] Figure 1 This is a schematic diagram of the structure of a fault type determination system provided in an embodiment of this application;

[0026] Figure 2 A flowchart illustrating a method for determining a fault type provided in an embodiment of this application;

[0027] Figure 3 A flowchart illustrating a method for obtaining a device's voiceprint vector, provided in an embodiment of this application;

[0028] Figure 4 This is a schematic diagram of a device voiceprint vector generation process provided in an embodiment of this application;

[0029] Figure 5 A flowchart illustrating a method for determining a fault type provided in an embodiment of this application;

[0030] Figure 6 This is a schematic diagram of a fault type determination device provided in an embodiment of this application;

[0031] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0033] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0034] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "connected" and "linked" should be interpreted broadly, for example, as a fixed connection, a detachable connection, or an integral connection. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances. Furthermore, when describing pipelines, the terms "connected" and "linked" as used in this application have the meaning of establishing electrical connection. The specific meaning needs to be understood in conjunction with the context.

[0035] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0036] During operation, vibrations occur between the device body and its components, or even between the components themselves, generating audio data that characterizes the device's state. This audio data is unique to each device and can be measured and analyzed using electroacoustic instruments. Therefore, we call the characteristic of the device's operating state carried by this audio data a "soundprint." Soundprints contain a wealth of device state information; thus, soundprints change as the device's operating state changes. Soundprints can serve as an important feature parameter for anomaly detection and fault analysis.

[0037] In related technologies, the use of device acoustic signatures for anomaly detection and fault analysis requires the prior establishment of a fault sample library. Based on the principle of fault sharing, devices of the same type share a fault sample library. The collected device acoustic signatures are compared with fault sample information in the same library to determine the fault type. However, even for devices of the same type, differences in component size, operating power, and other factors can lead to significant variations in their acoustic signatures during operation, resulting in low accuracy in determining the fault type.

[0038] In this context, the same type of equipment can refer to equipment with the same function or the same physical form. For example, in a feasible embodiment, the same type of equipment is a pump. A pump is a machine that transports or pressurizes liquids. Based on different working principles, it can be divided into types such as positive displacement pumps and vane pumps. However, they all have the same function and can all be called the same type of equipment.

[0039] Since voiceprints are sound wave spectra carrying speech information displayed by electroacoustic instruments, anomaly detection and fault analysis based on device voiceprints essentially involve analyzing the voiceprint features of the device. This requires mathematical operations on the device's audio data to extract a set of feature description vectors {x1, x2, ..., xn} (where n is a positive integer). Therefore, for ease of description, in this embodiment, voiceprints can also be referred to as voiceprint vectors.

[0040] To address the aforementioned issues, this application introduces a method for determining fault types. This method can divide devices of the same type into multiple subtypes based on the similarity between sample voiceprint vectors of devices of the same type. Devices within each subtype share a fault sample library, which can significantly improve the accuracy of fault type determination.

[0041] The fault type determination method provided in this application embodiment can be applied to, for example... Figure 1 The fault type shown indicates the system. For example... Figure 1 As shown, the fault type determination system includes: a processing unit 101, an audio acquisition unit 102, and a device 103. The processing unit 101 can establish a connection with the audio acquisition unit 102 and the device 103 via a wired or wireless network, and the audio acquisition unit 102 and the device 103 can also establish a connection via a wired or wireless network.

[0042] In this context, device 103 refers to a device with a specific function or a specific physical form, which vibrates during operation between its body and components, or between the components themselves, generating device audio data. Examples include power equipment that transmits electrical energy and pumps that transport liquids. This application does not impose any restrictions on the specific form of device 103.

[0043] The audio acquisition device 102 is used to collect device audio data generated during the operation of device 103 and send it to the processing device 101. The processing device 101 then generates a corresponding device voiceprint vector based on the device audio data, and determines whether the device voiceprint vector is a fault voiceprint vector based on the similarity between the device voiceprint vector and pre-stored sample voiceprint vectors. If it is determined to be a fault voiceprint vector, the processing device 101 determines the corresponding fault type of device 103 according to a pre-set fault sample library.

[0044] The audio acquisition device 102 can be deployed separately from the device 103, or the audio acquisition device 102 can be installed on the device 103 to acquire audio data generated during the operation of the device 103. The audio acquisition device 102 can be a microphone acquisition system or a sound array device; this application embodiment does not impose any limitations on this.

[0045] In some embodiments, the processing device 101 can be a server, and the specific implementation of the server is not limited in this application embodiment. The server can be a single server, or a server cluster consisting of multiple servers. In some embodiments, the server cluster can also be a distributed cluster.

[0046] In other embodiments, the processing device 101 may also be a computer device, and the embodiments of this application do not limit the specific form of the computer device. For example, the computer device may specifically be a terminal device. The terminal device may be referred to as: terminal, user equipment (UE), terminal device, access terminal, user unit, user station, mobile station, remote station, remote terminal, mobile device, user terminal, wireless communication device, user agent, or user device, etc. Specifically, the terminal device may be a mobile phone, augmented reality (AR) device, virtual reality (VR) device, tablet computer, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc.

[0047] It should be noted that, Figure 1 This is merely an illustrative example of an application scenario where the fault type determination system contains only one device. In actual applications, the fault type determination system may include multiple devices 103 and their corresponding audio acquisition devices 102. This application embodiment does not impose any limitations on this.

[0048] It should be pointed out that, Figure 1 The structure shown does not constitute a limitation on the fault type determination system. In other embodiments, the system may include fewer or more components than shown, or combine certain components, or have different component arrangements. This application does not impose any limitations on this.

[0049] Figure 2 The diagram shown is a flowchart illustrating a method for determining a fault type according to an embodiment of this application, which can be applied to... Figure 1 The processing unit 101 in the system shown. For example... Figure 2 As shown, the method includes the following steps:

[0050] S201. Determine the similarity between sample voiceprint vectors of different devices of the same type.

[0051] The sample voiceprint vector is the voiceprint vector corresponding to the audio data generated during normal device operation. Since the operating state of a device can be divided into normal operation and fault state, and the types of faults in a fault state are diverse, the corresponding voiceprint vectors for the audio data generated in a fault state are also diverse, and therefore cannot be used as a basis for device classification. Therefore, this embodiment uses the similarity between sample voiceprint vectors of devices as the basis for classification.

[0052] For example, if different devices of the same type are pump A, pump B, and pump C, then it is necessary to determine the similarity of the sample voiceprint vectors of pump A and pump B, the similarity of the sample voiceprint vectors of pump B and pump C, and the similarity of the sample voiceprint vectors of pump A and pump C.

[0053] In the embodiments of this application, the algorithm for calculating similarity may be, but is not limited to, cosine similarity scoring, probabilistic linear discriminant analysis (PLDA) scoring, or cosine distance, etc.

[0054] For example, when using the cosine similarity algorithm to calculate similarity, since the voiceprint vector is an n-dimensional vector, let's assume the voiceprint vector A is {A1, A2, ..., A3}. n The voiceprint vector B is {B1, B2, ..., B}. n The similarity between vector A and vector B is calculated as follows:

[0055]

[0056] The resulting number is the similarity score between vector A and vector B.

[0057] S202. Based on similarity, different devices of the same type are divided into multiple subtypes.

[0058] In this subtype, the similarity between different devices is greater than or equal to a first preset threshold. Each subtype corresponds to a fault sample library, which includes the correspondence between fault voiceprint vectors and fault types.

[0059] Specifically, after obtaining the similarity between the sample voiceprint vectors corresponding to the devices, the devices are classified according to the similarity. Devices whose sample voiceprint vectors have a similarity greater than or equal to a first preset threshold are classified as devices of the same subtype. That is, devices within each subtype have a similarity greater than or equal to the first preset threshold. After classification, devices of each subtype can share a fault sample library, which can greatly improve the accuracy of fault type determination.

[0060] The first preset threshold is pre-set by the system. In actual application, it can be set according to needs, and this application embodiment does not impose any restrictions on it. For example, as a feasible implementation method, the first preset threshold is 75%, then the similarity between devices in each subtype after classification is greater than or equal to 75%.

[0061] As one possible implementation, S202 can be specifically as follows: generate a similarity matrix using the similarity between sample voiceprint vectors of different devices, feed the similarity matrix and a first preset threshold into a clustering algorithm, the clustering algorithm uses the first preset threshold to cluster the similarity within the similarity matrix, and then clusters the devices according to the devices corresponding to the similarity, dividing different devices of the same type into multiple subtypes.

[0062] The clustering algorithm used can be hierarchical clustering (AHC) or spectral clustering, etc. In practical applications, it can be set according to the requirements, and this application embodiment does not impose any restrictions on it.

[0063] S203. Obtain the device voiceprint vector of the target device among different devices of the same type.

[0064] Since the target device emits audio data during operation, anomaly detection and fault analysis based on the target device's audio data essentially involve analyzing the voiceprint features of the target device's audio data. This means performing mathematical operations on the target device's audio data to extract a set of feature description vectors {x1,x2,...,xn}, which is the device's voiceprint vector. Then, the current state of the target device is determined based on the device's voiceprint vector.

[0065] As a feasible implementation method, S203 may include: (11) and (12).

[0066] (11) Obtain the device audio data of the target device.

[0067] (12) Determine the device voiceprint vector corresponding to the device audio data.

[0068] During the operation of the target device, the sound generated by vibrations between the device's body and firmware, components, or between the components themselves is acquired and used as device audio data. Then, based on the device audio data, the corresponding device voiceprint vector is determined. It should be understood that determining the device voiceprint vector corresponding to the device audio data can adopt implementation methods commonly used in the art, and the applicant will not impose any further limitations here.

[0069] In some embodiments, since the state changes are complex during device operation and the target device will be in different states over time, device state parameters can be introduced as a supplement to the device audio data during the process of forming the device voiceprint vector, so that the generated device voiceprint vector is related to the device's operating state.

[0070] In this embodiment, the device state parameters are parameters used to characterize the device state. For example, motion parameters (displacement, velocity, acceleration) and magnitude parameters (peak value, average value, and RMS value) generated by vibration during device operation.

[0071] As a feasible way to achieve this, such as Figure 3 As shown, the steps for determining the device voiceprint vector corresponding to the device audio data may include:

[0072] S301, Extract feature data from device audio data.

[0073] The method for extracting feature data from the device audio data can be a commonly used method in this technical field, such as feature extraction based on filter banks (Fbank) or Mel-frequency cepstral coefficients (MFCC). In practical applications, the appropriate method can be selected according to the requirements, and this application embodiment does not impose any restrictions on it.

[0074] In some embodiments, Fbank features retain more of the original audio data, and therefore have a high correlation with the device audio data of the target device. Thus, as a feasible implementation, Fbank feature extraction is used to extract feature data from the device audio data, resulting in 128-dimensional Fbank feature data.

[0075] S302. Obtain the device status parameters of the target device.

[0076] To ensure that device status parameters can accurately represent device status, this application also discloses a method for obtaining device status parameters of a target device, specifically:

[0077] The operating status of the equipment is characterized by extracting K sets of fundamental frequencies and a target resonant peak frequency. The target resonant peak frequency is the first K resonant frequencies when the resonant frequencies generated by the equipment are arranged in descending order. Here, K is a positive integer. As a feasible implementation, K is set to 4, using 4 sets of fundamental frequencies plus the target resonant peak frequency to characterize the operating status of the equipment. This application embodiment does not impose any restrictions on this.

[0078] In this implementation, the fundamental frequency and the target resonant frequency, which are arranged in the first K positions, are used as the device status parameters. These device status parameters can accurately characterize the device status.

[0079] S303. Obtain the device voiceprint vector corresponding to the device audio data based on the feature data and device status parameters.

[0080] Introducing device status parameters as a supplement to the device audio data allows the generated device voiceprint vector to better represent the device's operating status. This results in more accurate determination of the device's operating status based on the voiceprint vector.

[0081] The neural network is a pre-trained network model, including but not limited to Residual Network (RestNet) and Time Delay Neural Network (TDNN). In practical applications, specific network models can be used according to requirements, and this application embodiment does not impose any limitations on this.

[0082] It is worth noting that the embodiments of this application do not specifically limit the order of S301 and S302. As one feasible implementation, S301 can be executed first and then S302; as another feasible implementation, S302 can be executed first and then S301.

[0083] As a feasible implementation method, S303 can be specifically implemented as follows: after concatenating the feature data and device status parameters, input them into the neural network to obtain the device voiceprint vector corresponding to the device audio data.

[0084] By concatenating feature data and device status parameters and inputting them into a neural network, the generated device voiceprint vector becomes correlated with the device's operating status, making the determination of the device's operating status more accurate.

[0085] In some embodiments, since the feature data of the device audio data has a dimension of 128, while the device state parameters have only a single-digit dimension, and the information contained in the device state parameters represents the device's operating state, it is not easy for the neural network to learn. In other words, the neural network cannot easily model the device state parameters when generating the device voiceprint vector.

[0086] As another feasible way to achieve this, combined with Figure 4 As shown, S303 may include the following steps:

[0087] S3031. The feature data and device status parameters are concatenated and input into the first part of the neural network to obtain the hidden layer features of the network.

[0088] S3032. After concatenating the device status parameters and the hidden layer features of the network, input them into the second part of the neural network to obtain the device voiceprint vector corresponding to the device audio data.

[0089] The neural network consists of a first part and a second part. (Combined) Figure 4 As shown, since the feature data has a 128-dimensional dimension while the device state parameters have only a single-digit dimension, and the information contained in the device state parameters represents the device's operating state, neural networks cannot easily model the device state parameters. Therefore, the feature data and device state parameters are concatenated. After successful concatenation, the concatenated data is fed into the first part of the neural network for calculation to obtain the hidden layer features. The device state parameters and hidden layer features are then concatenated again to improve the modeling ability of the neural network. After successful concatenation, the concatenated data is fed into the second part of the neural network for calculation to obtain the device voiceprint vector. This makes the generated device voiceprint vector correlated with the device's operating state, resulting in more accurate judgments of the device's operating state based on the device voiceprint vector.

[0090] S204. If the device acoustic vector of the target device is determined to be a fault acoustic vector, then the fault type of the target device is determined according to the corresponding relationship in the target fault sample library.

[0091] The target fault sample library is the fault sample library corresponding to the subtype to which the target device belongs. Specifically, since different devices of the same type have been classified in S202, each device has its corresponding target fault sample library. If the device acoustic vector of the target device is determined to be a fault acoustic vector, it means that the target device is in a fault state. Therefore, the correspondence between the fault acoustic vector and the fault type in the target fault sample library is used to determine the fault type of the target device.

[0092] As a feasible implementation method, the steps of determining the device acoustic vector of the target device as the fault acoustic vector include: (21) and (22).

[0093] (21) Based on the similarity between the device's voiceprint vector and the sample voiceprint vector of each device in the subtype to which the target device belongs, the first similarity set is obtained;

[0094] (22) If the maximum similarity in the first similarity set is less than the first preset threshold, then the device voiceprint vector of the target device is determined to be the fault voiceprint vector.

[0095] Specifically, the similarity between the device's voiceprint vector and the sample voiceprint vectors of each device in the subtype to which the target device belongs is calculated to obtain a first similarity set. If any similarity score in the first similarity set is greater than a preset threshold, it means that the device's voiceprint vector is similar to the sample voiceprint vector of a certain device, so the device is judged to be in normal operating condition. If all similarity scores in the first similarity set are less than the preset threshold, it means that the device's voiceprint vector is dissimilar to the sample voiceprint vectors of all devices in the subtype to which the target device belongs, and the device's voiceprint vector is judged to be a fault voiceprint vector, and the target device is in a faulty state.

[0096] Before determining the fault type of the target device, first determine whether the target device is in a fault state. If the device is in a fault state, then further determine the fault type of the target device based on the fault voiceprint vector. By first determining whether the device is in a fault state, and then determining the fault type if it is, a large number of normal voiceprint vectors can be filtered out, eliminating the need for frequent fault type determination and improving the accuracy of fault type determination.

[0097] As a feasible implementation method, the steps to determine the fault type of the target equipment based on the correspondence in the target fault sample library include: (31) and (32).

[0098] (31) Based on the similarity between the fault voiceprint vector and each fault voiceprint vector in the target fault sample library, a second similarity set is obtained.

[0099] (32) If the maximum similarity in the second similarity set is greater than or equal to the second preset threshold, then the target fault type corresponding to the fault voiceprint vector with the maximum similarity in the target fault sample library is determined according to the correspondence in the target fault sample library.

[0100] The second preset threshold is pre-set by the system. In actual application, it can be set according to requirements, and this application embodiment does not impose any restrictions on it. For example, as a feasible implementation method, the second preset threshold is 80%. If the maximum similarity in the second similarity set is greater than or equal to 80%, then the target fault type is the fault type corresponding to the fault voiceprint vector that generates the maximum similarity.

[0101] The target fault type is the fault type of the target device.

[0102] Specifically, the similarity between the fault acoustic vector and all fault acoustic vectors in the target fault sample library is calculated to obtain a second similarity set. If the maximum similarity in the second similarity set is greater than a second preset threshold, it means that the fault acoustic vector is most similar to the fault acoustic vector that generates the maximum similarity. Therefore, the fault type corresponding to the fault acoustic vector with the maximum similarity is taken as the target fault type. The target fault type obtained in this way is also the most consistent fault type in the target fault sample library, that is, the fault type of the target device obtained is the most accurate.

[0103] The fault type determination method provided in this application embodiment classifies devices of the same type with a similarity greater than or equal to a first preset threshold into a subtype based on the similarity between sample voiceprint vectors of devices of the same type. It can be seen that after classification, the differences between devices within each subtype are small, meaning their sound characteristics are also relatively similar. This allows each subtype to correspond to a fault sample library. Thus, when determining the fault type of a target device, the method relies on the correspondence in the target fault sample library, leading to more accurate fault type determination. The method provided in this application embodiment can reduce the impact of differences between different devices on the fault type determination process, thereby significantly improving the accuracy of fault type determination.

[0104] In some embodiments, the maximum similarity in the second similarity set may be less than the second threshold, that is, all similarities in the second similarity set are less than the second threshold, which means that the fault soundprint vector of the target device is not similar to the fault soundprint vector in the fault sample library, and the fault type of the target device cannot be determined at this time.

[0105] As a feasible implementation method, the steps of determining the fault type of the target device based on the correspondence in the target fault sample library also include: (41) and (42).

[0106] (41) If the maximum similarity in the second similarity set is less than the second preset threshold, then output a prompt message.

[0107] (42) In response to the acquisition of the input fault type, determine the fault type of the target device as the input fault type.

[0108] The prompt message is used to suggest the type of fault to be entered. Specifically, the prompt message may indicate that the currently processed fault voiceprint vector is not recorded in the fault sample database, allowing backend personnel to analyze the fault type corresponding to the fault voiceprint vector and input the fault type.

[0109] This application does not impose specific limitations on the output format of the prompt information. As one feasible implementation method, the prompt information can be output as video; as another feasible implementation method, the prompt information can be output as audio.

[0110] If the maximum similarity in the second similarity set is less than the second threshold, that is, if all similarities in the second similarity set are less than the second threshold, it means that the fault soundprint vector of the target device is not similar to the fault soundprint vector in the fault sample library. The target device is in a fault state, but there is no corresponding fault type. The user or on-site expert needs to confirm the fault type of the target device manually.

[0111] As a feasible approach, the method also includes:

[0112] The correspondence between the fault acoustic vector and the input fault type is written into the target fault sample library.

[0113] The correspondence between the fault acoustic vector and the input fault type is written into the target fault sample library, so that when the device in the subtype to which the target device belongs generates a similar fault acoustic vector, its fault type can be identified as the input fault type, without the need for manual confirmation.

[0114] It should be understood that, in the process of identifying fault types using the fault sample library provided in this application embodiment, the target fault sample library may not store the correspondence between fault voiceprint vectors and fault types. Therefore, if the second similarity set obtained by calculating the similarity between the fault voiceprint vector and each fault voiceprint vector in the target fault sample library is empty (i.e., the maximum similarity in the second similarity set is less than the second preset threshold), the above method can also be used to output prompt information to prompt the backend personnel to input the fault type.

[0115] In other words, the method provided in this application supports cold start of the fault sample library. It can start running even when the fault sample library has no fault samples, and during operation, the fault sample library is continuously expanded to include more fault types.

[0116] In some embodiments, a sample database can be set up to store sample voiceprint vectors of different devices of the same type. During execution of S201, devices can be directly classified based on the sample voiceprint vectors in the sample database. This ensures that after classification, devices of the same subtype correspond to one fault sample database, and devices of the same subtype also correspond to a sub-sample database. Thus, when determining whether the device voiceprint vector of a target device is a fault voiceprint vector, it can be determined based on the sample voiceprint vectors in the sub-sample database corresponding to the target device. Because the similarity between classified devices is high, having devices of the same subtype correspond to a sub-sample database allows for a more accurate determination of whether a device voiceprint vector is a fault voiceprint vector.

[0117] Please see Figure 5 , Figure 5 The diagram shown is a flowchart illustrating a method for determining a fault type according to an exemplary embodiment of this application.

[0118] When the device is running, the sample data under normal operating conditions is first fed into the voiceprint extraction model to obtain the sample voiceprint vector corresponding to the normal sample data, which is then stored as sample data in the sample database. Then, the following steps are used for voiceprint recognition:

[0119] S501. Obtain the device voiceprint vector based on the device audio data.

[0120] The device audio data generated during device operation and the device status parameters of the current device status are fed into the voiceprint extraction model, and the model output is the device voiceprint vector.

[0121] It should be understood that the voiceprint extraction model can be the neural network mentioned above, or other pre-trained models, and the embodiments of this application do not limit it in any way.

[0122] S502. Compare the similarity between the device's voiceprint vector and the sample voiceprint vectors in the sample database.

[0123] Specifically, the similarity between the device's voiceprint vector and the sample voiceprint vector is calculated to obtain a first similarity set. Then, all data in the first similarity set are compared with a first system threshold.

[0124] S503. Determine whether the device's acoustic vector is a fault acoustic vector.

[0125] If any similarity score in the first similarity set is greater than a preset threshold, then the voiceprint vector is similar to the sample data in the sample database, and the device is determined to be in normal operating condition. If the similarity scores of the voiceprint vector with all samples in the first similarity set of the sample voiceprint vectors in the sample database are less than the preset threshold, then the voiceprint vector is a fault voiceprint, and the device is in a faulty state.

[0126] S5041. If the device is in normal operating condition, the output device will operate normally.

[0127] S5042. If the device's acoustic vector is a fault acoustic vector, then the fault acoustic vector is compared with the fault acoustic vector in the target fault sample library.

[0128] Specifically, the similarity between the fault voiceprint and the fault voiceprint vector in the target fault sample library can be calculated to obtain a second similarity set. Then, all data in the second similarity set can be compared with the system threshold.

[0129] S505. Determine if it is a known fault.

[0130] If there is no fault sample data in the target fault sample library, the fault voiceprint is determined to be an unknown fault. If the element with the highest similarity in the second similarity set is greater than or equal to a preset threshold, the fault voiceprint is determined to be similar to the fault voiceprint vector that generates the highest similarity, and the fault voiceprint is a known fault. If all similarity elements in the second similarity set are less than the preset threshold, the fault voiceprint is determined to be dissimilar to all fault voiceprint vectors in the target fault sample library, and the fault voiceprint is not a known fault, but an unknown fault.

[0131] S5061. If the fault voiceprint is a known fault, then output the fault type corresponding to the fault voiceprint vector.

[0132] Retrieve the fault type corresponding to the fault voiceprint vector in the target fault sample library, use it as the fault type of the fault voiceprint, and output it.

[0133] S5062. If the fault voiceprint is an unknown fault, then the output is an unknown fault.

[0134] If the fault voiceprint is an unknown fault, the system outputs a prompt message indicating that the fault is unknown and awaits human confirmation and input of the fault type before naming the fault.

[0135] S507. After receiving the fault type, store the fault voiceprint vector and fault type into the target fault sample library.

[0136] After being manually confirmed by users or on-site experts, the unknown fault is named, and this name becomes the fault type corresponding to the fault's acoustic vector. The fault acoustic vector and fault type are then stored in the target fault sample library to expand the library. This allows the system to promptly identify the fault type when similar fault acoustic vectors are subsequently detected, without requiring further manual confirmation.

[0137] This application also provides a device for determining the type of fault, such as... Figure 6 As shown, the determining device 60 includes: a first determining module 61, a classification module 62, an acquisition module 63, and a second determining module 64.

[0138] The first determining module 61 is used to determine the similarity between sample voiceprint vectors of different devices of the same type, wherein the sample voiceprint vector is the voiceprint vector when the device is running normally.

[0139] Classification module 62 is used to divide different devices of the same type into multiple subtypes based on similarity; wherein, the similarity between different devices in a subtype is greater than or equal to a first preset threshold; each subtype corresponds to a fault sample library, which includes the correspondence between fault soundprint vectors and fault types;

[0140] Module 63 is used to acquire the device voiceprint vector of the target device among different devices of the same type;

[0141] The second determining module 64 is used to determine that the device acoustic vector of the target device is a fault acoustic vector, and to determine the fault type of the target device according to the corresponding relationship in the target fault sample library; the target fault sample library is the fault sample library corresponding to the subtype to which the target device belongs.

[0142] As a feasible implementation method, the acquisition module is also configured to: acquire device audio data of the target device; determine the device voiceprint vector corresponding to the device audio data; the second determination module is also configured to: obtain a first similarity set based on the similarity between the device voiceprint vector and the sample voiceprint vector of each device in the subtype to which the target device belongs; if the maximum similarity in the first similarity set is less than a first preset threshold, then determine the device voiceprint vector of the target device as a fault voiceprint vector.

[0143] As a feasible implementation method, the acquisition module is also configured to: extract feature data of device audio data; acquire device status parameters of the target device, including the K groups of fundamental frequencies and target formant frequencies of the target device, where the target formant frequency is the first K resonant frequencies arranged in descending order of the resonant frequencies generated by the target device, and K is a positive integer; and obtain the device voiceprint vector corresponding to the device audio data based on the feature data and device status parameters.

[0144] As a feasible implementation method, the acquisition module is also configured to: concatenate the feature data and device status parameters and input them into the neural network to obtain the device voiceprint vector corresponding to the device audio data.

[0145] As a feasible implementation method, the acquisition module is also configured to: concatenate the feature data and device status parameters and input them into the first part of the neural network to obtain the hidden layer features of the network, the neural network including the first part and the second part; concatenate the device status parameters and the hidden layer features of the network and input them into the second part of the neural network to obtain the device voiceprint vector corresponding to the device audio data.

[0146] As a feasible implementation method, the second determining module is further configured to: obtain a second similarity set based on the similarity between the fault acoustic vector and each fault acoustic vector in the target fault sample library; if the maximum similarity in the second similarity set is greater than or equal to a second preset threshold, then determine the target fault type corresponding to the fault acoustic vector with the maximum similarity in the target fault sample library based on the correspondence in the target fault sample library, and the target fault type is the fault type of the target device.

[0147] As a feasible implementation method, the second determining module is further configured to: if the maximum similarity in the second similarity set is less than the second preset threshold, output a prompt message, the prompt message being used to prompt the input fault type; in response to obtaining the input fault type, determine the fault type of the target device as the input fault type; the fault type determining device further includes a writing module, used to write the correspondence between the fault voiceprint vector and the input fault type into the target fault sample library.

[0148] This application also provides a computer device; please refer to [link / reference]. Figure 7 The device includes a memory 71 and a processor 72. The memory 71 is used to store computer program code and transmit the computer program code to the processor 72. The processor 72 is used to determine the fault type of the audio data generated by the device according to the instructions in the computer program code.

[0149] Optionally, the memory 71 may be a non-transitory computer-readable storage medium, such as read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc., and the embodiments of this application do not impose any limitations on this.

[0150] The processor 72 may be a central processing unit (CPU), a general-purpose processor, a network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller, a programmable logic device (PLD), or any combination thereof, and the embodiments of this application do not impose any limitations on this.

[0151] This application also provides a computer program product including one or more instructions, which are stored in the memory 71 of a computer device and executed by a processor 72 to complete the various processes of the above embodiments.

[0152] This application also provides a computer-readable storage medium, which includes computer-executable instructions that, when executed on a computer, cause the computer to perform the method provided in the above embodiments.

[0153] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0154] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0155] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0156] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0157] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, etc.) or a processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0158] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for determining a fault type, characterized in that, The method includes: Determine the similarity between sample voiceprint vectors of different devices of the same type, wherein the sample voiceprint vectors are the voiceprint vectors when the devices are operating normally; Based on the similarity, different devices of the same type are divided into multiple subtypes; wherein, the similarity between different devices in a subtype is greater than or equal to a first preset threshold; each subtype corresponds to a fault sample library, and the fault sample library includes the correspondence between fault voiceprint vectors and fault types; Obtain the device voiceprint vector of the target device among different devices of the same type; If the device voiceprint vector of the target device is determined to be a fault voiceprint vector, then the fault type of the target device is determined according to the correspondence in the target fault sample library; the target fault sample library is the fault sample library corresponding to the subtype to which the target device belongs.

2. The method according to claim 1, characterized in that, The step of obtaining the device voiceprint vector of the target device among different devices of the same type includes: Obtain the device audio data of the target device; Determine the device voiceprint vector corresponding to the device audio data; The step of determining that the device acoustic vector of the target device is a fault acoustic vector includes: A first similarity set is obtained based on the similarity between the device's voiceprint vector and the sample voiceprint vector of each device in the subtype to which the target device belongs; If the maximum similarity in the first similarity set is less than the first preset threshold, then the device voiceprint vector of the target device is determined to be a fault voiceprint vector.

3. The method according to claim 2, characterized in that, The step of determining the device voiceprint vector corresponding to the device audio data includes: Extract feature data from the device's audio data; Obtain the device status parameters of the target device. The device status parameters include the K sets of fundamental frequencies and the target resonant frequency of the target device. The target resonant frequency is the resonant frequency that is arranged in the first K positions when the resonant frequencies generated by the target device are arranged in descending order, where K is a positive integer. The device voiceprint vector corresponding to the device audio data is obtained based on the feature data and the device status parameters.

4. The method according to claim 3, characterized in that, The step of obtaining the device voiceprint vector corresponding to the device audio data based on the feature data and the device state parameters includes: The feature data and the device status parameters are concatenated and input into a neural network to obtain the device voiceprint vector corresponding to the device audio data.

5. The method according to claim 3, characterized in that, The step of obtaining the device voiceprint vector corresponding to the device audio data based on the feature data and the device state parameters includes: The feature data and the device status parameters are concatenated and input into the first part of the neural network to obtain the hidden layer features of the network. The neural network includes the first part and the second part. The device state parameters and the hidden layer features of the network are concatenated and input into the second part of the neural network to obtain the device voiceprint vector corresponding to the device audio data.

6. The method according to any one of claims 1-5, characterized in that, The step of determining the fault type of the target device based on the correspondence in the target fault sample library includes: A second similarity set is obtained based on the similarity between the fault voiceprint vector and each fault voiceprint vector in the target fault sample library; If the maximum similarity in the second similarity set is greater than or equal to the second preset threshold, then according to the correspondence in the target fault sample library, the target fault type corresponding to the fault voiceprint vector corresponding to the maximum similarity in the target fault sample library is determined, and the target fault type is the fault type of the target device.

7. The method according to claim 6, characterized in that, The step of determining the fault type of the target device based on the correspondence in the target fault sample library further includes: If the maximum similarity in the second similarity set is less than the second preset threshold, a prompt message is output, which is used to prompt the input fault type; In response to receiving the input fault type, the fault type of the target device is determined to be the input fault type; The method further includes: The correspondence between the fault voiceprint vector and the input fault type is written into the target fault sample library.

8. A fault type determination device, characterized in that, include: The first determining module is used to determine the similarity between sample voiceprint vectors of different devices of the same type, wherein the sample voiceprint vectors are voiceprint vectors when the devices are operating normally. The classification module is used to divide different devices of the same type into multiple subtypes based on the similarity; wherein the similarity between different devices in a subtype is greater than or equal to a first preset threshold; each subtype corresponds to a fault sample library, and the fault sample library includes the correspondence between fault voiceprint vectors and fault types; The acquisition module is used to acquire the device voiceprint vector of the target device among different devices of the same type; The second determining module is used to determine that the device voiceprint vector of the target device is a fault voiceprint vector, and to determine the fault type of the target device according to the correspondence in the target fault sample library; the target fault sample library is the fault sample library corresponding to the subtype to which the target device belongs.

9. A computer device, characterized in that, The device includes: a processor and a memory; The memory is used to store computer program code and to transmit the computer program code to the processor; The processor is configured to execute the fault type determination method as described in any one of claims 1-7 according to instructions in the computer program code.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed on the device of claim 9, causes the device to perform the method for determining the type of fault as described in any one of claims 1-7.

Images