A method and system for monitoring and processing abnormal sounds during operation of industrial equipment

Abstract

The present invention relates to the field of audio detection technology, and in particular to a method and system for monitoring and processing sound anomalies during the operation of industrial equipment. The method divides the equipment audio data generated during the operation into data packets, and configures and outputs a state transition diagram associated with the equipment audio data; based on the state transition diagram, the data packet signals between the energy channel audio data packets in the equipment audio data are reorganized, and multiple reference audio data associated with the equipment audio data are output; for each reference audio data, the anomaly detection confidence of each energy channel audio data packet is calculated based on the fundamental frequency data; based on the anomaly detection confidence of each energy channel audio data packet, anomaly detection is performed on the industrial equipment operation audio library, and an anomaly detection result associated with the equipment audio data is output. The present invention fully considers the richness and diversity of the device audio data packet representation caused by the different energy positions in the device audio data, and improves the accuracy of anomaly detection.

Description

Technical Field

[0001] The present invention relates to the field of audio detection technology, and in particular to a method and system for monitoring and processing sound anomalies during the operation of industrial equipment. Background Art

[0002] Industrial equipment operation audio library monitoring refers to the control problem of using computer program equipment to control the audio data of the industrial equipment operation process. In order to detect anomalies in the industrial equipment operation audio library, it is often necessary to generate equipment audio data through energy and its associated properties.

[0003] In the existing method, in response to the equipment audio data generated during the operation process, anomaly detection is performed based on the energy packet overlap between the equipment audio data and the content stored in the industrial equipment operation audio library, and the anomaly detection result of the equipment audio data is output.

[0004] However, existing methods do not take into account the richness and diversity of device audio data packet representations caused by the different energy positions in device audio data, resulting in low accuracy of anomaly detection. Summary of the Invention

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] According to a first aspect of the present invention, the present invention seeks protection for a method for monitoring and processing sound anomalies during the operation of industrial equipment, comprising:

[0007] Segmenting device audio data generated during the operation into data packets, and configuring and outputting a state transition diagram associated with the device audio data, wherein the state transition diagram is used to represent a state relationship associated between audio data packets of each channel in the device audio data;

[0008] Based on the state transition diagram, reorganize data packet signals between energy channel audio data packets in the device audio data, and output a plurality of reference audio data associated with the device audio data;

[0009] For each of the reference audio data, based on the fundamental frequency data of each energy channel audio data packet in the reference audio data, calculate and output an anomaly detection confidence score for each energy channel audio data packet in the reference audio data, where the anomaly detection confidence score is used to indicate the risk of the energy channel audio data packet during an anomaly detection process;

[0010] Based on the anomaly detection confidence of each energy channel audio data packet in each reference audio data, anomaly detection is performed on the industrial equipment operation audio library to output an anomaly detection result associated with the equipment audio data.

[0011] Furthermore, the device audio data generated during the operation is divided into data packets, and a state transition diagram associated with the device audio data is configured and output, including:

[0012] Acquiring device audio data generated during the operation;

[0013] Using a recurrent neural network, the device audio data is divided into data packets, and audio data packets of each channel associated with the device audio data are output;

[0014] Based on the state relationship between the audio data packets of each channel associated with the device audio data, a state transition diagram associated with the device audio data is configured and output.

[0015] Furthermore, the device audio data is divided into data packets by a recurrent neural network, and the audio data packets of each channel associated with the device audio data are output, including:

[0016] Generate the device audio data into the encoder of the recurrent neural network for encoding, and output an audio vector associated with the device audio data;

[0017] The audio vector is generated and sent to the decoder of the recurrent neural network for decoding, and the audio data packets of each channel associated with the device audio data are output.

[0018] Furthermore, for each of the reference audio data, based on the fundamental frequency data of each energy channel audio data packet in the reference audio data, calculating and outputting the abnormality detection confidence of each energy channel audio data packet in the reference audio data includes:

[0019] Acquire a first energy signal of the reference audio data, where the first energy signal is used to represent fluctuations between energy channel audio data packets in the reference audio data;

[0020] Merging the energy channel audio data packets located adjacent to the reference audio data in the first energy signal to form a second energy signal of the reference audio data;

[0021] For each energy channel audio data packet in the second energy signal, sequentially obtain non-energy channel audio data packets on both sides of the energy channel audio data packet until reaching other energy channel audio data packets, and output a confidence list of the energy channel audio data packets;

[0022] Based on the confidence list of each of the energy channel audio data packets in the reference audio data, an abnormality detection confidence of each of the energy channel audio data packets in the reference audio data is calculated and output.

[0023] Furthermore, the calculating and outputting the anomaly detection confidence of each of the energy channel audio data packets in the reference audio data based on the confidence list of each of the energy channel audio data packets in the reference audio data includes:

[0024] Dividing the confidence list of the energy channel audio data packet into a first list and a second list according to the position of the energy channel audio data packet in the confidence list;

[0025] outputting a ratio of the non-energy channel audio data packets in the confidence list based on the number of the non-energy channel audio data packets in the first list and the second list;

[0026] Based on the candidate overlap of each of the non-energy channel audio data packets in the confidence list and the ratio of the non-energy channel audio data packets, the abnormality detection confidence of the energy channel audio data packet is calculated and output, the candidate overlap is used to represent the maximum value of the overlap between the non-energy channel audio data packet and each overlapping non-energy channel audio data packet, and the overlapping non-energy channel audio data packet is used to represent each of the non-energy channel audio data packets included in the confidence list associated with the energy channel audio data packet in each of the other reference audio data.

[0027] Furthermore, the calculating and outputting the abnormality detection confidence of the energy channel audio data packet based on the candidate overlap of each of the non-energy channel audio data packets in the confidence list and the ratio of the non-energy channel audio data packets includes:

[0028] Determining an overlap average and an overlap variance based on candidate overlaps of each of the non-energy channel audio data packets in the confidence list;

[0029] For each of the non-energy channel audio data packets in the confidence list, divide the difference between the candidate overlap of the non-energy channel audio data packet and the overlap average by the overlap variance, then square the difference and output the calculated overlap value of the non-energy channel audio data packet;

[0030] After multiplying the cumulative value of the overlap calculation values ​​of each non-energy channel audio data packet by the ratio of the non-energy channel audio data packet, the result is divided by the number of non-energy channel audio data packets in the confidence list, and the abnormality detection confidence of the energy channel audio data packet is output.

[0031] Furthermore, before calculating and outputting the abnormality detection confidence of each energy channel audio data packet in the reference audio data based on the fundamental frequency data of each energy channel audio data packet in the reference audio data, the method further includes:

[0032] determining an audio anomaly weight of each of the reference audio data based on the first energy signal associated with each of the reference audio data and the third energy signal associated with the device audio data;

[0033] performing linear normalization processing on the audio anomaly weights of the reference audio data, and outputting the normalized audio anomaly weights of the reference audio data;

[0034] Screening out the reference audio data whose normalized audio abnormality weight is greater than a preset weight threshold, and outputting optimized reference audio data;

[0035] The step of calculating and outputting, for each reference audio data item, an abnormality detection confidence level of each energy channel audio data item in the reference audio data item based on the fundamental frequency data of each energy channel audio data item in the reference audio data item, including:

[0036] For each of the optimized reference audio data, based on the fundamental frequency data of each energy channel audio data packet in the optimized reference audio data, the abnormality detection confidence of each energy channel audio data packet in the optimized reference audio data is calculated and output.

[0037] Furthermore, determining the audio anomaly weight of each reference audio data based on the first energy signal associated with each reference audio data and the third energy signal associated with the device audio data includes:

[0038] comparing a first energy signal associated with the reference audio data with a third energy signal associated with the device audio data, and outputting a number of candidate channel audio data packets having different bit sequences between the first energy signal and the third energy signal;

[0039] Divide the number of candidate channel audio data by the total number of energy channel audio data packets of the reference audio data, and output a first ratio;

[0040] dividing the number of channel audio data packets in the reference audio data by the number of channel audio data packets in the device audio data, and outputting a second ratio;

[0041] The first ratio is multiplied by the second ratio to output an audio anomaly weight of the reference audio data.

[0042] Furthermore, the abnormality detection of the industrial equipment operation audio library is performed based on the abnormality detection confidence of each energy channel audio data packet in each reference audio data, and the abnormality detection result associated with the equipment audio data is output, including:

[0043] Based on the abnormality detection confidence of each energy channel audio data packet in the reference audio data, respectively set the channel audio weight of each energy channel audio data packet in the reference audio data;

[0044] Calculating, based on the channel audio weights of the energy channel audio data packets in the reference audio data, a correlation score of each initial anomaly detection record output by the reference audio data for anomaly detection on the industrial equipment operation audio library;

[0045] Each of the initial anomaly detection records in the reference audio data, in which the correlation score is greater than a preset correlation threshold, is determined as an anomaly detection result associated with the device audio data.

[0046] According to the second aspect of the present invention, the present invention claims protection

[0047] A system for monitoring and processing abnormal sounds during the operation of industrial equipment, characterized by comprising:

[0048] a transition graph configuration unit, configured to divide the device audio data generated during the operation into data packets, and to configure and output a state transition graph associated with the device audio data, wherein the state transition graph is used to represent the state relationship associated between the audio data packets of each channel in the device audio data;

[0049] an audio association unit, configured to reorganize data packet signals between energy channel audio data packets in the device audio data based on the state transition diagram, and output a plurality of reference audio data associated with the device audio data;

[0050] a confidence calculation unit, configured to calculate and output, for each reference audio data, an anomaly detection confidence of each energy channel audio data packet in the reference audio data based on the fundamental frequency data of each energy channel audio data packet in the reference audio data, wherein the anomaly detection confidence is used to indicate the risk of the energy channel audio data packet in an anomaly detection process;

[0051] An industrial equipment operation audio library anomaly detection unit is used to perform anomaly detection on the industrial equipment operation audio library based on the anomaly detection confidence of each energy channel audio data packet in each reference audio data, and output an anomaly detection result associated with the equipment audio data;

[0052] The system for monitoring and processing sound anomalies during the operation of industrial equipment is used to execute the method for monitoring and processing sound anomalies during the operation of industrial equipment.

[0053] The present invention relates to the field of audio detection technology, and in particular to a method and system for monitoring and processing sound anomalies during the operation of industrial equipment. The method divides the equipment audio data generated during the operation into data packets, and configures and outputs a state transition diagram associated with the equipment audio data; based on the state transition diagram, the data packet signals between the energy channel audio data packets in the equipment audio data are reorganized, and multiple reference audio data associated with the equipment audio data are output; for each reference audio data, the anomaly detection confidence of each energy channel audio data packet is calculated based on the fundamental frequency data; based on the anomaly detection confidence of each energy channel audio data packet, anomaly detection is performed on the industrial equipment operation audio library, and an anomaly detection result associated with the equipment audio data is output. The present invention fully considers the richness and diversity of the device audio data packet representation caused by the different energy positions in the device audio data, and improves the accuracy of anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A first workflow diagram of a method for monitoring and processing abnormal sounds during operation of industrial equipment, as claimed in an embodiment of the present invention;

[0055] Figure 2 A second workflow diagram of a method for monitoring and processing abnormal sounds during operation of industrial equipment, as claimed in an embodiment of the present invention;

[0056] Figure 3 A third workflow diagram of a method for monitoring and processing abnormal sounds during operation of industrial equipment, as claimed in an embodiment of the present invention;

[0057] Figure 4 A fourth workflow diagram of a method for monitoring and processing abnormal sounds during operation of industrial equipment, as claimed in an embodiment of the present invention;

[0058] Figure 5 A fifth workflow diagram of a method for monitoring and processing abnormal sounds during operation of industrial equipment, as claimed in an embodiment of the present invention;

[0059] Figure 6 This is a structural module diagram of a system for monitoring and processing sound anomalies during the operation of industrial equipment, as claimed in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0061] The terms "first," "second," and "third" in this disclosure are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features identified. Therefore, features identified as "first," "second," or "third" may explicitly or implicitly include multiple such features. In the description of this disclosure, "multiple" means at least two, for example, two, three, etc., unless otherwise specifically defined. All directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of this disclosure are intended only to illustrate the relative positional relationships and movement of components in a specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications will also change accordingly. Furthermore, the terms "including," "having," and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements and may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to such process, method, product, or apparatus.

[0062] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in multiple embodiments of the invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0063] The following describes a specific embodiment of a method and system for monitoring and processing sound anomalies during the operation of industrial equipment, provided by an embodiment of the present invention.

[0064] Figure 1 A first workflow diagram of a method for monitoring and processing sound anomalies during the operation of industrial equipment is provided. The method for monitoring and processing sound anomalies during the operation of industrial equipment can be applied to a server. The method for monitoring and processing sound anomalies during the operation of industrial equipment can include the following S101 to S104.

[0065] S101, dividing the device audio data generated during the operation into data packets, and configuring and outputting a state transition diagram associated with the device audio data, the state transition diagram being used to represent the state relationship associated between the audio data packets of each channel in the device audio data.

[0066] In this embodiment, a state transition diagram is used to represent the state relationships between channel audio data packets in device audio data. Specifically, the state transition diagram generally includes nodes and edges, where nodes represent channel audio data packets of the device audio data, and edges represent the state relationships between channel audio data packets.

[0067] As an example, the server first splits the device audio data generated during the operation process and splits it into independent channel audio data packets.

[0068] S102 : Based on the state transition diagram, reorganize data packet signals between energy channel audio data packets in the device audio data, and output a plurality of reference audio data associated with the device audio data.

[0069] In this embodiment, the energy channel audio data packet is used to represent the audio body in the device audio data, which may specifically include an actor, an affected party, a tool, and attributes.

[0070] As an example, the server identifies energy channel audio data packets from the state transition diagram, then performs packet signal reassembly between the energy channel audio data packets, and outputs multiple reference audio data associated with the device audio data.

[0071] S103, for each reference audio data, based on the fundamental frequency data of each energy channel audio data packet in the reference audio data, calculate and output the anomaly detection confidence of each energy channel audio data packet in the reference audio data. The anomaly detection confidence is used to represent the risk of the energy channel audio data packet in the anomaly detection process.

[0072] In this embodiment, the pitch frequency data is used to represent the distribution position of the energy channel audio data packets in the reference audio data, and specifically may include their position in the sentence and the non-energy channel audio data packets surrounding them. The non-energy channel audio data packets are used to represent audio data packets of other channels except the energy channel audio data packets.

[0073] The anomaly detection confidence is a numerical metric used to indicate the risk of an energy channel audio packet in the anomaly detection process. Generally, energy channel audio packets with more important locations and closer relationships with non-energy channel audio packets have higher anomaly detection confidence.

[0074] As an example, the server calculates the anomaly detection confidence of each energy channel audio data packet based on the fundamental frequency data of the energy channel audio data packet based on a machine learning algorithm or a rule-based method.

[0075] S104: Based on the anomaly detection confidence of each energy channel audio data packet in each reference audio data, perform anomaly detection on the industrial equipment operation audio library and output an anomaly detection result associated with the equipment audio data.

[0076] In this embodiment, the industrial equipment operation audio library is a database containing a large amount of information, which can be structured data (such as a relational database) or unstructured data (such as a data packet file).

[0077] As an example, the server performs anomaly detection on an industrial equipment operation audio library based on the anomaly detection confidence level of each energy channel audio packet in the reference audio data. During anomaly detection, energy channel audio packets with higher anomaly detection confidence levels in the anomaly detection reference audio data are prioritized as anomaly detection criteria to improve the accuracy and efficiency of anomaly detection.

[0078] In the method for monitoring and processing sound anomalies during industrial equipment operation provided in this embodiment, the device audio data generated during the operation is first segmented into packets, and a state transition diagram is configured and outputted that represents the state relationships between the audio packets of each channel in the device audio data. Based on the state transition diagram, the device audio data is correlated and output as multiple reference audio data. Then, based on the fundamental frequency data of the audio packets of each energy channel in the reference audio data, the anomaly detection confidence level of each energy channel audio packet in the reference audio data is calculated. Based on the anomaly detection confidence level of each energy channel audio packet in the reference audio data, anomaly detection is performed on the industrial equipment operation audio library, and an anomaly detection result associated with the device audio data is output. In this manner, the embodiment of the present invention correlates the device audio data and outputs multiple reference audio data. Based on the fundamental frequency data of each energy channel audio packet in the reference audio data, an associated anomaly detection confidence level is set for each energy channel audio packet. This fully accounts for the richness and diversity of the device audio data packet representations resulting from the different energy positions in the device audio data, thereby improving the accuracy of anomaly detection.

[0079] As an optional embodiment, Figure 2 As shown, S101 may specifically include the following S201 to S203.

[0080] S201, obtaining device audio data generated during the operation process;

[0081] S202, dividing the device audio data into data packets using a recurrent neural network, and outputting audio data packets of each channel associated with the device audio data;

[0082] S203: Based on the state relationship between the audio data packets of each channel associated with the device audio data, configure and output a state transition diagram associated with the device audio data.

[0083] Through this embodiment, the device audio data is divided into data packets based on a recurrent neural network, and a state transition diagram is configured based on state analysis technology, so as to achieve in-depth understanding and processing of the device audio data generated during the operation process, thereby improving the accuracy and efficiency of query processing.

[0084] As an optional embodiment, S202 may specifically include:

[0085] Generate device audio data into the encoder of the recurrent neural network for encoding, and output the audio vector associated with the device audio data;

[0086] The audio vector is generated and sent to the decoder of the recurrent neural network for decoding, and the audio data packets of each channel associated with the device audio data are output.

[0087] The server first converts the device audio data into a machine-readable vector format.

[0088] As an optional embodiment, Figure 3 As shown, S103 may specifically include the following S301 to S304.

[0089] S301, obtaining a first energy signal of reference audio data, where the first energy signal is used to represent fluctuations between energy channel audio data packets in the reference audio data;

[0090] S302: Merge energy channel audio data packets located adjacent to the reference audio data in the first energy signal to form a second energy signal of the reference audio data;

[0091] S303: For each energy channel audio data packet in the second energy signal, sequentially obtain non-energy channel audio data packets on both sides of the energy channel audio data packet until reaching other energy channel audio data packets, and output a confidence list of the energy channel audio data packets;

[0092] S304 : Based on the confidence list of each energy channel audio data packet in the reference audio data, calculate and output the abnormality detection confidence of each energy channel audio data packet in the reference audio data.

[0093] In this embodiment, the first energy signal is a signal indicating fluctuations between energy channel audio data packets in the reference audio data.

[0094] The second energy signal is used to represent a new energy signal output after merging energy channel audio data packets located at adjacent positions of the reference audio data.

[0095] The confidence list includes the energy channel audio data packet and the non-energy channel audio data packets on both sides of the energy channel audio data packet.

[0096] As an example, the server identifies energy channel audio data packets in the reference audio data, and arranges the identified energy channel audio data packets in order of their positions in the audio to form a signal table, namely, a first energy signal.

[0097] Then, the first energy signal is traversed to determine whether there is an energy channel audio data packet located adjacent to the reference audio data, and the adjacent energy channel audio data packets that can be merged are merged into a new energy unit to form a second energy signal.

[0098] Then, for each energy channel audio data packet in the second energy signal, starting from its position in the reference audio data, expand in both directions until encountering another energy channel audio data packet. Collect all channel audio data packets in this range to form a confidence list for the energy channel audio data packet.

[0099] Finally, a confidence mechanism or algorithm is used to process the channel audio packets in the confidence list, and the anomaly detection confidence score for the energy channel audio packet associated with the confidence list is output. The calculation of the anomaly detection confidence score can involve various factors, such as the position of the energy channel audio packet in the confidence list and its interaction with surrounding non-energy channel audio packets.

[0100] This embodiment analyzes and processes the energy channel audio packets in the reference audio data, ultimately calculating and outputting their anomaly detection confidence levels. This facilitates subsequent anomaly detection of the industrial equipment operation audio library based on the anomaly detection confidence levels of the energy channel audio packets in the reference audio data, outputting associated anomaly detection results for the device audio data, and improving anomaly detection accuracy.

[0101] As an optional embodiment, Figure 4 As shown, S304 may specifically include the following S401 to S403.

[0102] S401, dividing the confidence list of the energy channel audio data packet into a first list and a second list according to the position of the energy channel audio data packet in the confidence list;

[0103] S402: Outputting a ratio of non-energy channel audio data packets in a confidence list based on the number of non-energy channel audio data packets in the first list and the number of non-energy channel audio data packets in the second list;

[0104] S403, based on the candidate overlap of each non-energy channel audio data packet in the confidence list and the ratio of the non-energy channel audio data packets, calculate the abnormality detection confidence of the output energy channel audio data packet, the candidate overlap is used to represent the maximum value of the overlap between the non-energy channel audio data packet and each overlapping non-energy channel audio data packet, and the overlapping non-energy channel audio data packet is used to represent each non-energy channel audio data packet included in the confidence list associated with the energy channel audio data packet in other reference audio data.

[0105] In this embodiment, the first list and the second list each include all non-energy channel audio data packets located to one side of the energy channel audio data packet in the confidence list. For example, the first list may include all non-energy channel audio data packets located before the energy channel audio data packet in the confidence list, and the second list may include all non-energy channel audio data packets located after the energy channel audio data packet in the confidence list.

[0106] The candidate overlap is the maximum overlap between a non-energy channel audio data packet in the first list or the second list and each non-energy channel audio data packet included in the confidence list associated with the energy channel audio data packet in each other reference audio data. For example, if there are four reference audio data items a, b, c, and d, then the candidate overlap of non-energy channel audio data packet x in the confidence list of energy channel audio data packet a1 associated with reference audio data a is the maximum overlap between non-energy channel audio data packet x and each non-energy channel audio data packet y in the confidence list of energy channel audio data packet a1 in reference audio data items b, c, and d.

[0107] The ratio of the non-energy channel audio data packets is used to represent the distribution of the non-energy channel audio data packets in the first list and the second list, so as to evaluate the distribution stability of the non-energy channel audio data packets around the energy channel audio data packets.

[0108] As an example, the server first determines the specific position of the energy channel audio data packet in the confidence list, and then divides the confidence list into two parts, a first list and a second list, based on the position.

[0109] Then, the number of non-energy channel audio data packets in the first list and the second list is counted respectively, and based on the number of non-energy channel audio data packets in the first list and the second list, the ratio of non-energy channel audio data packets in the confidence list is calculated.

[0110] The candidate overlap of each non-energy channel audio packet in the confidence list is then calculated. Combined with the non-energy channel audio packet ratio in the confidence list, a preset algorithm or model is used to calculate the anomaly detection confidence of the energy channel audio packet. The preset algorithm or model may consider multiple factors, such as the distribution of non-energy channel audio packets (reflected by the non-energy channel audio packet ratio) and the overlap between non-energy channel audio packets and overlapping non-energy channel audio packets (reflected by the candidate overlap). Specific calculation methods may include weighted summation, product operations, and exponential operations.

[0111] This embodiment accurately calculates the anomaly detection confidence level for each energy channel audio packet in the reference audio data based on the candidate overlap of each non-energy channel audio packet in the confidence list and the ratio of non-energy channel audio packets in the confidence list. This facilitates subsequent anomaly detection of the industrial equipment operation audio library based on the anomaly detection confidence level of each energy channel audio packet in the reference audio data, and outputs anomaly detection results associated with the equipment audio data, thereby improving the accuracy of anomaly detection.

[0112] As an optional embodiment, S403 may specifically include:

[0113] Determining an overlap average and an overlap variance based on candidate overlaps of each non-energy channel audio data packet in the confidence list;

[0114] For each non-energy channel audio data packet in the confidence list, divide the difference between the candidate overlap of the non-energy channel audio data packet and the average overlap by the overlap variance and then square it, and output the calculated overlap value of the non-energy channel audio data packet;

[0115] Multiply the cumulative value of the overlap calculation values ​​of each non-energy channel audio data packet by the ratio of the non-energy channel audio data packet, divide it by the number of non-energy channel audio data packets in the confidence list, and output the abnormality detection confidence of the energy channel audio data packet.

[0116] The larger the ratio of non-energy channel audio data packets, the more stable the distribution of non-energy channel audio data packets and the greater the confidence of anomaly detection; the greater the candidate overlap of non-energy channel audio data packets, the more overlap there is between non-energy channel audio data packets and other overlapping non-energy channel audio data packets, and the greater the confidence of anomaly detection.

[0117] This embodiment accurately calculates the anomaly detection confidence level for each energy channel audio packet in the reference audio data based on the candidate overlap of each non-energy channel audio packet in the confidence list and the ratio of non-energy channel audio packets in the confidence list. This facilitates subsequent anomaly detection of the industrial equipment operation audio library based on the anomaly detection confidence level of each energy channel audio packet in the reference audio data, and outputs anomaly detection results associated with the equipment audio data, thereby improving the accuracy of anomaly detection.

[0118] As an optional embodiment, Figure 5 As shown, before S103, the method for monitoring and processing sound anomalies during the operation of industrial equipment may further include the following S501 to S503.

[0119] S501, determining an audio anomaly weight of each reference audio data based on a first energy signal associated with each reference audio data and a third energy signal associated with device audio data;

[0120] S502, performing linear normalization processing on the audio anomaly weights of each reference audio data, and outputting the normalized audio anomaly weights of each reference audio data;

[0121] S503, filtering out each reference audio data whose normalized audio abnormality weight is greater than a preset weight threshold, and outputting optimized reference audio data;

[0122] S103 may specifically include:

[0123] For each optimized reference audio data, based on the fundamental frequency data of each energy channel audio data packet in the optimized reference audio data, an abnormality detection confidence of each energy channel audio data packet in the optimized reference audio data is calculated and output.

[0124] In this embodiment, the third energy signal is a signal indicating fluctuations between audio data packets of various energy channels in the device audio data, and the audio anomaly weight is used to indicate the possibility of generating state anomaly detection errors in the reference audio data.

[0125] The optimized reference audio data is used to represent the reference audio data remaining after removing the reference audio data that is prone to causing state abnormality detection errors.

[0126] For example, the server compares the overlap, correlation, or state matching between the first energy signal of each reference audio data point and the third energy signal of the device audio data point, and assigns an audio anomaly weight to each reference audio data point. This audio anomaly weight reflects the degree of state matching between the reference audio data point and the device audio data point, as well as the potential degree of anomaly.

[0127] Then, the audio anomaly weights of all reference audio data are linearly normalized so that all weight values ​​fall within the range of 0 to 1. This facilitates subsequent comparison and screening operations.

[0128] Then, based on the needs of the application scenario and the characteristics of the data, a preset weight threshold is set. The normalized audio anomaly weight is compared with the preset weight threshold. If the normalized weight of a reference audio data item is greater than this threshold, it is considered to have a high audio anomaly, does not meet the optimization requirements, and should be screened out. Finally, the reference audio data items whose normalized weight is less than or equal to the preset weight threshold are retained as the optimized reference audio data set.

[0129] This embodiment calculates the audio anomaly weight for each reference audio data point based on the first energy signal associated with the reference audio data and the third energy signal associated with the device audio data. This uses the audio anomaly weight to filter out reference audio data that may cause errors in state anomaly detection. This effectively reduces the impact of audio anomalies on query results and improves the efficiency and accuracy of information anomaly detection and processing.

[0130] As an optional embodiment, S501 may specifically include:

[0131] Comparing a first energy signal associated with the reference audio data with a third energy signal associated with the device audio data, and outputting the number of candidate channel audio data packets having different bit sequences of energy channel audio data packets in the first energy signal and the third energy signal;

[0132] Divide the number of candidate channel audio data by the total number of energy channel audio data packets of the reference audio data, and output a first ratio;

[0133] Divide the number of channel audio data packets in the reference audio data by the number of channel audio data packets in the device audio data, and output a second ratio;

[0134] The first ratio is multiplied by the second ratio to output an audio anomaly weight of the reference audio data.

[0135] This embodiment calculates the audio anomaly weight of each reference audio data item based on the number of candidate channel audio items with different bit orders of energy channel audio data items in the first energy signal associated with the reference audio data and the third energy signal associated with the device audio data, the total number of energy channel audio data items in the reference audio data, the number of channel audio data items in the reference audio data, and the number of channel audio data items in the device audio data. This allows the audio anomaly weight to be used to filter out reference audio data items that may cause state anomaly detection errors. This can effectively reduce the impact of audio anomalies on query results and improve the efficiency and accuracy of information anomaly detection and processing.

[0136] As an optional embodiment, S104 may specifically include:

[0137] Based on the anomaly detection confidence of each energy channel audio data packet in the reference audio data, respectively set the channel audio weight of each energy channel audio data packet in the reference audio data;

[0138] Calculate, based on the channel audio weights of the energy channel audio data packets in the reference audio data, the correlation scores of the initial anomaly detection records output by the reference audio data for anomaly detection of the industrial equipment operation audio library;

[0139] Each of the initial anomaly detection records in each reference audio data whose relevance score is greater than a preset relevance threshold is determined as an anomaly detection result associated with the device audio data.

[0140] In this embodiment, the server assigns a channel audio weight to each energy channel audio packet based on the anomaly detection confidence level of each energy channel audio packet in the reference audio data. A larger channel audio weight indicates a higher importance of the energy channel audio packet in the anomaly detection process.

[0141] The reference audio data with channel audio weights is then generated into the industrial equipment operation audio library for anomaly detection, outputting a series of initial anomaly detection records. For each initial anomaly detection record, a relevance score is determined based on the matching of the energy channel audio packets contained therein with the energy channel audio packets in the reference audio data, as well as the channel audio weights of these energy channel audio packets. Specifically, the scoring algorithm may involve weighted summation, cosine overlap calculation, and other methods.

[0142] Then, a correlation threshold is preset to filter out highly correlated anomaly detection records. The calculated output of the initial anomaly detection records are sorted from high to low packet signals based on their correlation scores. Records with correlation scores greater than the preset correlation threshold are selected as the final anomaly detection results for device audio data association. The filtered anomaly detection results are presented to the operation process in a list, summary, or other format, providing a visual and interactive interface for query results.

[0143] Through this embodiment, an anomaly detection method based on energy channel audio data packet anomaly detection confidence and channel audio weight is provided, which can more accurately capture the query intention of the operation process, thereby improving the accuracy and efficiency of information anomaly detection.

[0144] Based on the method for monitoring and processing abnormal sounds during the operation of industrial equipment provided by the present invention, the present invention also provides a specific embodiment of a system for monitoring and processing abnormal sounds during the operation of industrial equipment.

[0145] Figure 6 A structural schematic diagram of a sound anomaly monitoring and processing system for the operation of industrial equipment provided by an embodiment of the present invention is shown. The sound anomaly monitoring and processing system for the operation of industrial equipment may include a conversion graph configuration unit 610, an audio association unit 620, a confidence calculation unit 630, and an industrial equipment operation audio library anomaly detection unit 640.

[0146] A transition graph configuration unit 610 is configured to divide the device audio data generated during the operation into data packets and configure and output a state transition graph associated with the device audio data. The state transition graph is used to represent the state relationship between the audio data packets of each channel in the device audio data.

[0147] The audio association unit 620 is configured to reorganize data packet signals between energy channel audio data packets in the device audio data based on the state transition diagram, and output a plurality of reference audio data associated with the device audio data;

[0148] A confidence calculation unit 630 is configured to calculate, for each reference audio data item, an anomaly detection confidence level for each energy channel audio data item in the reference audio data item based on the fundamental frequency data of each energy channel audio data item in the reference audio data item, where the anomaly detection confidence level is used to indicate the risk of the energy channel audio data item during anomaly detection.

[0149] The industrial equipment operation audio library anomaly detection unit 640 is used to perform anomaly detection on the industrial equipment operation audio library based on the anomaly detection confidence of each energy channel audio data packet in each reference audio data, and output an anomaly detection result associated with the equipment audio data.

[0150] In a sound anomaly monitoring and processing system for industrial equipment operation, provided by an embodiment of the present invention, the device audio data generated during the operation is first segmented into data packets, and a state transition diagram is configured and outputted that represents the state relationships between audio packets in each channel of the device audio data. Based on the state transition diagram, the device audio data is correlated and output as multiple reference audio data. Then, based on the fundamental frequency data of the audio packets in each energy channel of the reference audio data, the anomaly detection confidence level of each audio packet in the reference audio data is calculated. Based on the anomaly detection confidence level of each audio packet in the reference audio data, anomaly detection is performed on the industrial equipment operation audio library, and an anomaly detection result associated with the device audio data is output. In this manner, the embodiment of the present invention correlates the device audio data and outputs multiple reference audio data. Based on the fundamental frequency data of each audio packet in the reference audio data, an associated anomaly detection confidence level is assigned to each audio packet in the energy channel of the reference audio data. This fully accounts for the richness and diversity of the device audio data packet representations resulting from the different energy positions in the device audio data, thereby improving the accuracy of anomaly detection.

[0151] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0152] In addition, the functional units in the various embodiments of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software functional units. The above is only an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation based on the content of the present invention specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present invention.

[0153] The above detailed description of the specific embodiments of the invention is intended to be illustrative only, and the present invention is not limited to the specific embodiments described above. For those skilled in the art, any equivalent modifications or substitutions to the invention are also within the scope of the present invention. Therefore, equivalent changes, modifications, and improvements made without departing from the spirit and scope of the present invention are also encompassed within the scope of the present invention.

Claims

1. A method for monitoring and processing abnormal sounds during the operation of industrial equipment, characterized in that: include: Segmenting device audio data generated during the operation into data packets, and configuring and outputting a state transition diagram associated with the device audio data, wherein the state transition diagram is used to represent a state relationship associated between audio data packets of each channel in the device audio data; Based on the state transition diagram, reorganize data packet signals between energy channel audio data packets in the device audio data, and output a plurality of reference audio data associated with the device audio data; Acquire a first energy signal of the reference audio data, where the first energy signal is used to represent fluctuations between energy channel audio data packets in the reference audio data; Merging the energy channel audio data packets located adjacent to the reference audio data in the first energy signal to form a second energy signal of the reference audio data; For each energy channel audio data packet in the second energy signal, sequentially obtain non-energy channel audio data packets on both sides of the energy channel audio data packet until reaching other energy channel audio data packets, and output a confidence list of the energy channel audio data packets; Based on the confidence list of each energy channel audio data packet in the reference audio data, calculate and output an anomaly detection confidence of each energy channel audio data packet in the reference audio data, where the anomaly detection confidence is used to represent the risk of the energy channel audio data packet in the anomaly detection process; Based on the anomaly detection confidence of each energy channel audio data packet in each reference audio data, anomaly detection is performed on the industrial equipment operation audio library to output an anomaly detection result associated with the equipment audio data.

2. The method for monitoring and processing abnormal sounds during operation of industrial equipment according to claim 1, characterized in that: The device audio data generated during the operation is divided into data packets, and a state transition diagram associated with the device audio data is configured and output, including: Acquiring device audio data generated during the operation; Using a recurrent neural network, the device audio data is divided into data packets, and audio data packets of each channel associated with the device audio data are output; Based on the state relationship between the audio data packets of each channel associated with the device audio data, a state transition diagram associated with the device audio data is configured and output.

3. The method for monitoring and processing abnormal sounds during the operation of industrial equipment according to claim 2, characterized in that: The method of dividing the device audio data into data packets by a recurrent neural network and outputting audio data packets of each channel associated with the device audio data comprises: Generate the device audio data into the encoder of the recurrent neural network for encoding, and output an audio vector associated with the device audio data; The audio vector is generated and sent to the decoder of the recurrent neural network for decoding, and the audio data packets of each channel associated with the device audio data are output.

4. The method for monitoring and processing abnormal sounds during operation of industrial equipment according to claim 3, characterized in that: The calculating and outputting the abnormality detection confidence of each of the energy channel audio data packets in the reference audio data based on the confidence list of each of the energy channel audio data packets in the reference audio data includes: Dividing the confidence list of the energy channel audio data packet into a first list and a second list according to the position of the energy channel audio data packet in the confidence list; outputting a ratio of the non-energy channel audio data packets in the confidence list based on the number of the non-energy channel audio data packets in the first list and the second list; Based on the candidate overlap of each of the non-energy channel audio data packets in the confidence list and the ratio of the non-energy channel audio data packets, the abnormality detection confidence of the energy channel audio data packet is calculated and output, the candidate overlap is used to represent the maximum value of the overlap between the non-energy channel audio data packet and each overlapping non-energy channel audio data packet, and the overlapping non-energy channel audio data packet is used to represent each of the non-energy channel audio data packets included in the confidence list associated with the energy channel audio data packet in each of the other reference audio data.

5. The method for monitoring and processing abnormal sounds during operation of industrial equipment according to claim 4, characterized in that: The calculating and outputting the abnormality detection confidence of the energy channel audio data packet based on the candidate overlap of each of the non-energy channel audio data packets in the confidence list and the ratio of the non-energy channel audio data packets includes: Determining an overlap average and an overlap variance based on candidate overlaps of each of the non-energy channel audio data packets in the confidence list; For each of the non-energy channel audio data packets in the confidence list, divide the difference between the candidate overlap of the non-energy channel audio data packet and the overlap average by the overlap variance, then square the difference and output the calculated overlap value of the non-energy channel audio data packet; After multiplying the cumulative value of the overlap calculation values ​​of each non-energy channel audio data packet by the ratio of the non-energy channel audio data packet, the result is divided by the number of non-energy channel audio data packets in the confidence list, and the abnormality detection confidence of the energy channel audio data packet is output.

6. The method for monitoring and processing abnormal sounds during operation of industrial equipment according to claim 1, wherein: Before calculating and outputting anomaly detection confidence of each energy channel audio data packet in the reference audio data based on the fundamental frequency data of each energy channel audio data packet in the reference audio data, the method further includes: determining an audio anomaly weight of each of the reference audio data based on the first energy signal associated with each of the reference audio data and the third energy signal associated with the device audio data; performing linear normalization processing on the audio anomaly weights of the reference audio data, and outputting the normalized audio anomaly weights of the reference audio data; Screening out the reference audio data whose normalized audio abnormality weight is greater than a preset weight threshold, and outputting optimized reference audio data; The step of calculating and outputting, for each reference audio data item, an abnormality detection confidence level of each energy channel audio data item in the reference audio data item based on the fundamental frequency data of each energy channel audio data item in the reference audio data item, including: For each of the optimized reference audio data, based on the fundamental frequency data of each energy channel audio data packet in the optimized reference audio data, the abnormality detection confidence of each energy channel audio data packet in the optimized reference audio data is calculated and output.

7. The method for monitoring and processing abnormal sounds during operation of industrial equipment according to claim 6, characterized in that: The determining, based on the first energy signal associated with each reference audio data and the third energy signal associated with the device audio data, the audio anomaly weight of each reference audio data includes: comparing a first energy signal associated with the reference audio data with a third energy signal associated with the device audio data, and outputting a number of candidate channel audio data packets having different bit sequences between the first energy signal and the third energy signal; Divide the number of candidate channel audio data by the total number of energy channel audio data packets of the reference audio data, and output a first ratio; dividing the number of channel audio data packets in the reference audio data by the number of channel audio data packets in the device audio data, and outputting a second ratio; The first ratio is multiplied by the second ratio to output an audio anomaly weight of the reference audio data.

8. The method for monitoring and processing abnormal sounds during operation of industrial equipment according to claim 1, wherein: The method of performing anomaly detection on the industrial equipment operation audio library based on the anomaly detection confidence of each energy channel audio data packet in each reference audio data and outputting an anomaly detection result associated with the equipment audio data includes: Based on the abnormality detection confidence of each energy channel audio data packet in the reference audio data, respectively set the channel audio weight of each energy channel audio data packet in the reference audio data; Calculating, based on the channel audio weights of the energy channel audio data packets in the reference audio data, a correlation score of each initial anomaly detection record output by the reference audio data for anomaly detection on the industrial equipment operation audio library; Each of the initial anomaly detection records in the reference audio data, in which the correlation score is greater than a preset correlation threshold, is determined as an anomaly detection result associated with the device audio data.

9. A system for monitoring and processing abnormal sounds during the operation of industrial equipment, characterized in that: include: a transition graph configuration unit, configured to divide the device audio data generated during the operation into data packets, and to configure and output a state transition graph associated with the device audio data, wherein the state transition graph is used to represent the state relationship associated between the audio data packets of each channel in the device audio data; an audio association unit, configured to reorganize data packet signals between energy channel audio data packets in the device audio data based on the state transition diagram, and output a plurality of reference audio data associated with the device audio data; a confidence calculation unit, configured to calculate and output, for each reference audio data, an anomaly detection confidence of each energy channel audio data packet in the reference audio data based on the fundamental frequency data of each energy channel audio data packet in the reference audio data, wherein the anomaly detection confidence is used to indicate the risk of the energy channel audio data packet in an anomaly detection process; An industrial equipment operation audio library anomaly detection unit is used to perform anomaly detection on the industrial equipment operation audio library based on the anomaly detection confidence of each energy channel audio data packet in each reference audio data, and output an anomaly detection result associated with the equipment audio data; The system for monitoring and processing sound anomalies during the operation of industrial equipment is used to execute the method for monitoring and processing sound anomalies during the operation of industrial equipment as described in any one of claims 1 to 8.

Images