Sound abnormity monitoring processing method and system for industrial equipment operation process

Abstract

The invention relates to the technical field of audio detection, in particular to a sound abnormity monitoring processing method and system in the operation process of industrial equipment, and the method comprises the steps: carrying out the data packet segmentation of equipment audio data generated in the operation process, and configuring and outputting a state conversion diagram associated with the equipment audio data; based on the state transition diagram, performing data packet signal recombination between energy sound channel audio data packets in the equipment audio data, and outputting a plurality of reference audio data associated with the equipment audio data; for each reference audio data, based on the fundamental tone frequency data, calculating the anomaly detection confidence of each energy sound channel audio data packet; and performing anomaly detection on the industrial equipment operation audio library based on the anomaly detection confidence of each energy sound channel audio data packet, and outputting an anomaly detection result associated with the equipment audio data. According to the invention, richness and diversity of data packet expression of the equipment audio data caused by different energy positions in the equipment audio data are fully considered, and the accuracy of anomaly detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of audio detection, and particularly to a method and system for monitoring and processing abnormal sounds during the operation of industrial equipment. Background Art

[0002] Monitoring of the audio library during the operation of industrial equipment refers to the control problem of using computer program equipment for the audio data during the operation of industrial equipment. In order to obtain abnormal detection results in the audio library of industrial equipment operation, it is often necessary to generate device audio data through energy and its associated attributes.

[0003] In the existing methods, in response to the device audio data generated during the operation process, abnormal detection is performed based on the packet overlap of energy between the device audio data and the stored content in the industrial equipment operation audio library, and the abnormal detection result of the device audio data is output.

[0004] However, the existing methods do not consider the richness and diversity of the expression of the device audio data packets caused by the different energy positions in the device audio data, resulting in low accuracy of abnormal detection. Summary of the Invention

[0005] To achieve the above object, the present invention provides the following technical solutions: According to the first aspect of the present invention, the present invention claims a method for monitoring and processing abnormal sounds during the operation of industrial equipment, including: Performing packet segmentation on the device audio data generated during the operation process, and configuring and outputting a state transition diagram associated with the device audio data, where the state transition diagram is used to represent the state relationship between the audio data packets of each channel in the device audio data; Based on the state transition diagram, performing packet signal recombination between the energy channel audio data packets in the device audio data, and outputting a plurality of reference audio data associated with the device audio data; For each of the reference audio data, based on the pitch frequency data of each energy channel audio data packet in the reference audio data, calculating and outputting the abnormal detection confidence of each energy channel audio data packet in the reference audio data, where the abnormal detection confidence is used to represent the risk of the energy channel audio data packet during the abnormal detection process; Based on the abnormal detection confidence of each energy channel audio data packet in each of the reference audio data, performing abnormal detection on the industrial equipment operation audio library and outputting the abnormal detection result associated with the device audio data.

[0006] Further, the performing packet segmentation on the device audio data generated during the operation process, and configuring and outputting a state transition diagram associated with the device audio data, includes: Obtain the device audio data generated during the operation process; Perform packet segmentation on the device audio data through a recurrent neural network, and output each channel audio packet associated with the device audio data; Configure and output a state transition diagram associated with the device audio data based on the state relationship between each channel audio packet associated with the device audio data.

[0007] Further, the performing packet segmentation on the device audio data through a recurrent neural network and outputting each channel audio packet associated with the device audio data includes: Encode the device audio data into the encoder of the recurrent neural network to output an audio vector associated with the device audio data; Generate the audio vector into the decoder of the recurrent neural network for decoding, and output each channel audio packet associated with the device audio data.

[0008] Further, for each of the reference audio data, calculating and outputting the anomaly detection confidence of each energy channel audio packet in the reference audio data based on the pitch frequency data of each energy channel audio packet in the reference audio data includes: Obtain a first energy signal of the reference audio data, where the first energy signal is used to represent the fluctuation situation between each energy channel audio packet in the reference audio data; Merge the energy channel audio packets located at adjacent positions of the reference audio data in the first energy signal to form a second energy signal of the reference audio data; For each energy channel audio packet in the second energy signal, sequentially obtain the non-energy channel audio packets on both sides of the energy channel audio packet until reaching other energy channel audio packets, and output a confidence list of the energy channel audio packet; Calculate and output the anomaly detection confidence of each energy channel audio packet in the reference audio data based on the confidence list of each energy channel audio packet in the reference audio data.

[0009] Further, the calculating and outputting the anomaly detection confidence of each energy channel audio packet in the reference audio data based on the confidence list of each energy channel audio packet in the reference audio data includes: Divide the confidence list of the energy channel audio packet into a first list and a second list according to the position of the energy channel audio packet in the confidence list; Output the ratio of non - energy channel audio data packets in the confidence list based on the number of non - energy channel audio data packets in the first list and the second list; Calculate and output the anomaly detection confidence of the energy channel audio data packets based on the candidate overlap of each non - energy channel audio data packet in the confidence list and the ratio of non - energy channel audio data packets. The candidate overlap is used to represent the maximum value of the overlap between the non - energy channel audio data packet and each overlapping non - energy channel audio data packet, and the overlapping non - energy channel audio data packet is used to represent each non - energy channel audio data packet included in the confidence list associated with the energy channel audio data packet in each of the other reference audio data.

[0010] Further, the calculating and outputting the anomaly detection confidence of the energy channel audio data packets based on the candidate overlap of each non - energy channel audio data packet in the confidence list and the ratio of non - energy channel audio data packets includes: Determine the average overlap and the variance of overlap based on the candidate overlap of each non - energy channel audio data packet in the confidence list; For each non - energy channel audio data packet in the confidence list, divide the difference between the candidate overlap of the non - energy channel audio data packet and the average overlap by the variance of overlap, and then perform a square calculation to output the overlap calculation value of the non - energy channel audio data packet; Multiply the accumulated value of the overlap calculation values of each non - energy channel audio data packet by the ratio of non - energy channel audio data packets, and then divide by the number of non - energy channel audio data packets in the confidence list to output the anomaly detection confidence of the energy channel audio data packets.

[0011] Further, before calculating and outputting the anomaly detection confidence of each energy channel audio data packet in the reference audio data based on the pitch frequency data of each energy channel audio data packet in the reference audio data, the method further includes: Determine the audio anomaly weight of each reference audio data based on the first energy signal associated with each reference audio data and the third energy signal associated with the device audio data; Perform linear normalization processing on the audio anomaly weights of each reference audio data to output the normalized audio anomaly weights of each reference audio data; Screen out the reference audio data with the normalized audio anomaly weight greater than the preset weight threshold, and output the optimized reference audio data; For each of the reference audio data, based on the pitch frequency data of each energy channel audio data packet in the reference audio data, calculating and outputting the anomaly detection confidence of each energy channel audio data packet in the reference audio data, includes: For each of the optimized reference audio data, based on the pitch frequency data of each energy channel audio data packet in the optimized reference audio data, calculating and outputting the anomaly detection confidence of each energy channel audio data packet in the optimized reference audio data.

[0012] Further, determining the audio anomaly weight of each of the reference audio data based on the first energy signal associated with each of the reference audio data and the third energy signal associated with the device audio data, includes: Comparing the first energy signal associated with the reference audio data with the third energy signal associated with the device audio data, and outputting the number of candidate channel audios with different bit sequences of the energy channel audio data packets in the first energy signal and the third energy signal; Dividing the number of candidate channel audios by the total number of energy channel audio data packets of the reference audio data, and outputting a first ratio; Dividing the number of channel audio data packets in the reference audio data by the number of channel audio data packets in the device audio data, and outputting a second ratio; Multiplying the first ratio by the second ratio, and outputting the audio anomaly weight of the reference audio data.

[0013] Further, based on the anomaly detection confidence of each energy channel audio data packet in each of the reference audio data, performing anomaly detection on the industrial equipment operation audio library and outputting the anomaly detection result associated with the device audio data, includes: Based on the anomaly detection confidence of each energy channel audio data packet in the reference audio data, respectively setting the channel audio weights of each energy channel audio data packet in the reference audio data; Based on the channel audio weights of each energy channel audio data packet in the reference audio data, calculating the relevance scores of each initial anomaly detection record output by performing anomaly detection on the industrial equipment operation audio library by the reference audio data; Determining, as the anomaly detection result associated with the device audio data, each of the initial anomaly detection records in each of the reference audio data with a relevance score greater than a preset relevance threshold.

[0014] According to the second aspect of the present invention, the present invention requests protection A sound anomaly monitoring and processing system for an industrial equipment operation process, characterized by including: A conversion graph configuration unit is used to perform data packet segmentation on the device audio data generated during the operation process, and configure and output a state transition graph associated with the device audio data. The state transition graph is used to represent the state relationship associated between the audio data packets of each channel in the device audio data; An audio association unit is used to perform data packet signal recombination between the energy channel audio data packets in the device audio data based on the state transition graph, and output a plurality of reference audio data associated with the device audio data; A confidence calculation unit is used to calculate and output the anomaly detection confidence of each energy channel audio data packet in the reference audio data for each of the reference audio data based on the pitch frequency data of each energy channel audio data packet in the reference audio data. The anomaly detection confidence is used to represent the risk of the energy channel audio data packet during the anomaly detection process; An industrial equipment operation audio library anomaly detection unit is used to perform anomaly detection on the industrial equipment operation audio library based on the anomaly detection confidence of each energy channel audio data packet in each of the reference audio data, and output the anomaly detection result associated with the device audio data; The sound anomaly monitoring and processing system for industrial equipment operation is used to execute the sound anomaly monitoring and processing method for industrial equipment operation.

[0015] The present invention relates to the technical field of audio detection, and in particular, to a sound anomaly monitoring and processing method and system for industrial equipment operation. It performs data packet segmentation on the device audio data generated during the operation process, configures and outputs a state transition graph associated with the device audio data; based on the state transition graph, performs data packet signal recombination between the energy channel audio data packets in the device audio data, and outputs a plurality of reference audio data associated with the device audio data; for each reference audio data, calculates the anomaly detection confidence of each energy channel audio data packet based on the pitch frequency data; based on the anomaly detection confidence of each energy channel audio data packet, performs anomaly detection on the industrial equipment operation audio library and outputs the anomaly detection result associated with the device audio data. The present invention fully considers the richness and diversity of the device audio data packet representation caused by the different energy positions in the device audio data, and improves the accuracy of anomaly detection. Description of the Drawings

[0016] Figure 1 It is the first working flowchart of a sound anomaly monitoring and processing method for industrial equipment operation requested to be protected by an embodiment of the present invention; Figure 2 It is the second working flowchart of a sound anomaly monitoring and processing method for industrial equipment operation requested to be protected by an embodiment of the present invention; Figure 3This is the third working flowchart of a method for monitoring and processing abnormal sounds during the operation of industrial equipment claimed in the embodiments of the present invention; Figure 4 This is the fourth working flowchart of a method for monitoring and processing abnormal sounds during the operation of industrial equipment claimed in the embodiments of the present invention; Figure 5 This is the fifth working flowchart of a method for monitoring and processing abnormal sounds during the operation of industrial equipment claimed in the embodiments of the present invention; Figure 6 This is the structural module diagram of a system for monitoring and processing abnormal sounds during the operation of industrial equipment claimed in the embodiments of the present invention. Detailed implementation manners

[0017] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0018] The terms "first", "second", and "third" in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first", "second", and "third" may explicitly or implicitly include multiple such features. In the description of the present invention, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise specifically defined. All directional indications (such as up, down, left, right, front, back...) in the embodiments of the present invention are only used to explain the relative positional relationship and movement conditions between components in a specific posture (as shown in the drawings). If the specific posture changes, the directional indications will also change accordingly. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally further include unlisted steps or units, or may optionally further include other steps or units inherent to these processes, methods, products, or devices.

[0019] Referring to "embodiments" herein means that the specific features, structures, or characteristics described in conjunction with the embodiments can be included in multiple embodiments of the present invention. The phrase appears at various positions in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0020] The following introduces a specific embodiment of a method and system for monitoring and processing sound anomalies during the operation of industrial equipment provided by the embodiments of the present invention.

[0021] Figure 1 A first working flowchart of a method for monitoring and processing sound anomalies during the operation of industrial equipment is provided. This method for monitoring and processing sound anomalies during the operation of industrial equipment can be applied to the server side, and this method for monitoring and processing sound anomalies during the operation of industrial equipment may include the following S101 to S104.

[0022] S101, perform packet segmentation on the device audio data generated during the operation process, and configure a state transition diagram associated with the device audio data. The state transition diagram is used to represent the state relationship between the audio packets of each channel in the device audio data.

[0023] In this embodiment, the state transition diagram is used to represent the state relationship between the audio packets of each channel in the device audio data. Specifically, the state transition diagram generally includes nodes and edges, where the nodes represent the audio packets of the channels in the device audio data, and the edges represent the state relationship between the audio packets of the channels.

[0024] As an example, the server first performs segmentation processing on the device audio data generated during the operation process, and cuts it into independent audio packets of each channel.

[0025] S102, based on the state transition diagram, perform packet signal recombination between the energy channel audio packets in the device audio data, and output multiple reference audio data associated with the device audio data.

[0026] In this embodiment, the energy channel audio packets are used to represent the audio main body in the device audio data, and specifically may include an actor, a recipient, a tool, and attributes, etc.

[0027] As an example, the server identifies the energy channel audio packets from the state transition diagram, and then performs packet signal and recombination between the energy channel audio packets, and outputs multiple reference audio data associated with the device audio data.

[0028] S103, for each reference audio data, based on the fundamental frequency data of each energy channel audio packet in the reference audio data, calculate and output the anomaly detection confidence of each energy channel audio packet in the reference audio data. The anomaly detection confidence is used to represent the risk of the energy channel audio packet during the anomaly detection process.

[0029] In this embodiment, the pitch frequency data is used to represent the distribution position of the energy channel audio data packets in the reference audio data, and specifically may include their positions in the sentence and the non-energy channel audio data packets around them, etc. Among them, the non-energy channel audio data packets are used to represent the other channel audio data packets except the energy channel audio data packets.

[0030] The anomaly detection confidence level is a numerical index used to represent the risk of the energy channel audio data packets during the anomaly detection process. Generally, the energy channel audio data packets with more important positions and closer relationships with the non-energy channel audio data packets have higher anomaly detection confidence levels.

[0031] As an example, the server calculates the anomaly detection confidence level of each energy channel audio data packet based on the pitch frequency data of the energy channel audio data packets by using a machine learning algorithm or a rule-based method.

[0032] S104. Based on the anomaly detection confidence levels of the energy channel audio data packets in each reference audio data, perform anomaly detection on the industrial equipment operation audio library and output the anomaly detection results associated with the device audio data.

[0033] In this embodiment, the industrial equipment operation audio library is a database containing a large amount of information, which can be structured data (such as a relational database) or unstructured data (such as a data packet file).

[0034] As an example, the server performs anomaly detection on the industrial equipment operation audio library based on the anomaly detection confidence levels of the energy channel audio data packets in the reference audio data. Among them, during anomaly detection, the energy channel audio data packets with higher anomaly detection confidence levels in the anomaly detection reference audio data are preferentially used as the anomaly detection conditions to improve the accuracy and efficiency of anomaly detection.

[0035] In the method for monitoring and processing abnormal sounds during the operation of industrial equipment provided in this embodiment, first, the device audio data generated during the operation process is segmented into data packets, and a state transition diagram representing the state relationship between the audio data packets of each channel in the device audio data is configured for output. Then, based on the state transition diagram, multiple reference audio data are associated and output for the device audio data. Next, based on the pitch frequency data of the audio data packets of each energy channel in the reference audio data, the abnormal detection confidence of the audio data packets of each energy channel in the reference audio data is calculated. Thus, based on the abnormal detection confidence of the audio data packets of each energy channel in the reference audio data, the abnormal detection of the industrial equipment operation audio library can be performed to output the abnormal detection result associated with the device audio data. In this way, in the embodiment of the present invention, multiple reference audio data are associated and output for the device audio data, and based on the pitch frequency data of the audio data packets of each energy channel in the reference audio data, the associated abnormal detection confidence is set for the audio data packets of each energy channel. Considering fully the richness and diversity of the expression of the device audio data packets caused by the different energy positions in the device audio data, the accuracy of abnormal detection can be improved.

[0036] As an alternative embodiment, as Figure 2 shown, S101 may specifically include the following S201 to S203.

[0037] S201, obtain the device audio data generated during the operation process; S202, through a recurrent neural network, segment the device audio data into data packets, and output the audio data packets of each channel associated with the device audio data; S203, based on the state relationship between the audio data packets of each channel associated with the device audio data, configure and output the state transition diagram associated with the device audio data.

[0038] Through this embodiment, the device audio data generated during the operation process is segmented into data packets based on a recurrent neural network, and a state transition diagram is configured based on state analysis technology, so as to achieve in-depth understanding and processing of the device audio data generated during the operation process, and further improve the accuracy and efficiency of query processing.

[0039] As an alternative embodiment, S202 may specifically include: Encode the device audio data into the encoder of the recurrent neural network to output the audio vector associated with the device audio data; Generate the audio vector into the decoder of the recurrent neural network to decode and output the audio data packets of each channel associated with the device audio data.

[0040] The server first converts the device audio data into a machine-readable vector form.

[0041] As an alternative embodiment, asFigure 3 As shown in Figure 3 , S103 may specifically include S301 to S304 as follows.

[0042] S301, obtaining a first energy signal of the reference audio data, where the first energy signal is used to represent the fluctuation situation between the audio data packets of each energy channel in the reference audio data; S302, merging the audio data packets of the energy channels located at adjacent positions in the reference audio data in the first energy signal to form a second energy signal of the reference audio data; S303, for each audio data packet of the energy channel in the second energy signal, sequentially obtaining the audio data packets of the non - energy channels on both sides of the audio data packet of the energy channel until reaching other audio data packets of the energy channel, and outputting a confidence list of the audio data packets of the energy channel; S304, calculating and outputting the anomaly detection confidence of each audio data packet of the energy channel in the reference audio data based on the confidence list of each audio data packet of the energy channel in the reference audio data.

[0043] In this embodiment, the first energy signal is a signal representing the fluctuation situation between the audio data packets of each energy channel in the reference audio data.

[0044] The second energy signal is used to represent the new energy signal output after merging the audio data packets of the energy channels located at adjacent positions in the reference audio data.

[0045] The confidence list includes the audio data packets of the energy channel and the audio data packets of the non - energy channels on both sides of the audio data packet of the energy channel.

[0046] As an example, the server identifies the audio data packets of the energy channels in the reference audio data, and arranges the identified audio data packets of the energy channels in the order of their positions in the audio to form a signal table, that is, the first energy signal.

[0047] Then, traverse the first energy signal to determine whether there are audio data packets of the energy channels located at adjacent positions in the reference audio data, and merge the adjacent energy channel audio data packets that can be merged into a new energy unit to form the second energy signal.

[0048] Then, for each audio data packet of the energy channel in the second energy signal, starting from its position in the reference audio data, expand to both sides until encountering another audio data packet of the energy channel. Collect all the audio data packets within this range to form the confidence list of this audio data packet of the energy channel.

[0049] Finally, a confidence mechanism or algorithm is used to process the channel audio data packets in the confidence list, and the anomaly detection confidence of the energy channel audio data packets associated with the confidence list is output. Among them, the calculation of the anomaly detection confidence can involve various factors, such as the position of the energy channel audio data packet in the confidence list, its interaction with the surrounding non-energy channel audio data packets, etc.

[0050] Through this embodiment, the energy channel audio data packets in the reference audio data are analyzed and processed, and finally their anomaly detection confidence is calculated and output. Thus, it helps to perform anomaly detection on the industrial equipment operation audio library based on the anomaly detection confidence of each energy channel audio data packet in the reference audio data, and output the anomaly detection results associated with the device audio data, which can improve the accuracy of anomaly detection.

[0051] As an alternative embodiment, as Figure 4 shown, S304 may specifically include the following S401 to S403.

[0052] S401, divide the confidence list of the energy channel audio data packets into a first list and a second list according to the position of the energy channel audio data packets in the confidence list; S402, output the ratio of the non-energy channel audio data packets in the confidence list based on the number of non-energy channel audio data packets in the first list and the second list; S403, calculate and output the anomaly detection confidence of the energy channel audio data packets based on the candidate overlap of each non-energy channel audio data packet in the confidence list and the ratio of the non-energy channel audio data packets. The candidate overlap is used to represent the maximum value of the overlap between the non-energy channel audio data packet and each overlapping non-energy channel audio data packet, and the overlapping non-energy channel audio data packet is used to represent each non-energy channel audio data packet included in the confidence list associated with the energy channel audio data packet in each other reference audio data.

[0053] In this embodiment, the first list and the second list respectively include all the non-energy channel audio data packets located on one side of the energy channel audio data packet in the confidence list. Exemplarily, the first list may be the non-energy channel audio data packets located in front of the energy channel audio data packet in the confidence list, and the second list may be the non-energy channel audio data packets located behind the energy channel audio data packet in the confidence list.

[0054] The candidate overlap is the maximum value of the overlap between the non-energy channel audio data packets in the first list or the second list and each non-energy channel audio data packet included in the confidence list associated with the energy channel audio data packet in each of the other reference audio data. For example, if there are four reference audio data, namely a, b, c, and d, then the candidate overlap of the non-energy channel audio data packet x in the confidence list of the energy channel audio data packet a1 associated with the reference audio data a is the maximum value of the overlap between the non-energy channel audio data packet x and each non-energy channel audio data packet y in the confidence list of the energy channel audio data packet a1 in the reference audio data b, c, and d.

[0055] The non-energy channel audio data packet ratio is used to represent the distribution of non-energy channel audio data packets in the first list and the second list, so as to evaluate the distribution stability of non-energy channel audio data packets around the energy channel audio data packet.

[0056] As an example, the server first determines the specific position of the energy channel audio data packet in the confidence list, and then based on this position, divides the confidence list into two parts, namely the first list and the second list.

[0057] Then, count the number of non-energy channel audio data packets in the first list and the second list respectively. And based on the number of non-energy channel audio data packets in the first list and the second list, calculate the non-energy channel audio data packet ratio of the confidence list.

[0058] Then, calculate the candidate overlap of each non-energy channel audio data packet in the confidence list, and combine the non-energy channel audio data packet ratio of the confidence list, and use a preset algorithm or model to calculate the anomaly detection confidence of the energy channel audio data packet. Among them, the preset algorithm or model may need to consider multiple factors, such as the distribution of non-energy channel audio data packets (reflected by the non-energy channel audio data packet ratio), the overlap between non-energy channel audio data packets and overlapping non-energy channel audio data packets (reflected by the candidate overlap), etc. Among them, the specific calculation methods may include weighted summation, product operation, and exponential operation, etc.

[0059] Through this embodiment, based on the candidate overlap of each non-energy channel audio data packet in the confidence list and the non-energy channel audio data packet ratio of the confidence list, accurately calculate the anomaly detection confidence of each energy channel audio data packet in the reference audio data. Thus, it helps to subsequently perform anomaly detection on the industrial equipment operation audio library based on the anomaly detection confidence of each energy channel audio data packet in the reference audio data, and output the anomaly detection result associated with the device audio data, which can improve the accuracy of anomaly detection.

[0060] As an alternative embodiment, S403 may specifically include: Determine the average overlap and the variance of overlap based on the candidate overlap of each non - energy channel audio data packet in the confidence list; For each non - energy channel audio data packet in the confidence list, divide the difference between the candidate overlap of the non - energy channel audio data packet and the average overlap by the variance of overlap, and then perform a square calculation to output the overlap calculation value of the non - energy channel audio data packet; Multiply the accumulated value of the overlap calculation values of each non - energy channel audio data packet by the ratio of non - energy channel audio data packets, and then divide by the number of non - energy channel audio data packets in the confidence list to output the anomaly detection confidence of the energy channel audio data packet.

[0061] The larger the ratio of non - energy channel audio data packets, the more stable the distribution of non - energy channel audio data packets, and the greater the anomaly detection confidence; the larger the candidate overlap of non - energy channel audio data packets, the more overlapping the non - energy channel audio data packets are with other overlapping non - energy channel audio data packets, and the greater the anomaly detection confidence.

[0062] Through this embodiment, based on the candidate overlap of each non - energy channel audio data packet in the confidence list and the ratio of non - energy channel audio data packets in the confidence list, accurately calculate the anomaly detection confidence of each energy channel audio data packet in the reference audio data. Thus, it helps to subsequently perform anomaly detection on the industrial equipment operation audio library based on the anomaly detection confidence of each energy channel audio data packet in the reference audio data and output the anomaly detection result associated with the device audio data, which can improve the accuracy of anomaly detection.

[0063] As an alternative embodiment, as Figure 5 shown, before S103, the method for monitoring and processing sound anomalies during the operation of industrial equipment may further include the following S501 to S503.

[0064] S501, determine the audio anomaly weight of each reference audio data based on the first energy signal associated with each reference audio data and the third energy signal associated with the device audio data; S502, perform linear normalization processing on the audio anomaly weights of each reference audio data to output the normalized audio anomaly weights of each reference audio data; S503, screen out each reference audio data with a normalized audio anomaly weight greater than the preset weight threshold to output optimized reference audio data; S103 may specifically include: For each optimized reference audio data, calculate and output the anomaly detection confidence of each energy channel audio data packet in the optimized reference audio data based on the fundamental frequency data of each energy channel audio data packet in the optimized reference audio data.

[0065] In this embodiment, the third energy signal is a signal representing the fluctuation situation between the audio data packets of each energy channel in the device audio data, and the audio anomaly weight is used to represent the possibility of generating a state anomaly detection error in the reference audio data.

[0066] The optimized reference audio data is used to represent the remaining reference audio data after removing the reference audio data that is likely to generate a state anomaly detection error.

[0067] As an example, the server assigns an audio anomaly weight to each reference audio data by comparing the overlap degree, correlation degree or state matching degree between the first energy signal of each reference audio data and the third energy signal of the device audio data. This audio anomaly weight reflects the matching degree of the reference audio data with the device audio data in terms of state and the possible degree of anomaly.

[0068] Then, linear normalization processing is performed on the audio anomaly weights of all reference audio data so that all weight values fall within the range of 0 to 1. This can facilitate subsequent comparison and screening operations.

[0069] Then, based on the requirements of the application scenario and the characteristics of the data, a preset weight threshold is set. The normalized audio anomaly weight is compared with the preset weight threshold. If the normalized weight of a certain reference audio data is greater than this threshold, it is considered that its audio anomaly is relatively high and does not meet the optimization requirements, and it should be screened out. Finally, the reference audio data with normalized weights less than or equal to the preset weight threshold is retained as the optimized set of reference audio data.

[0070] Through this embodiment, based on the first energy signal associated with the reference audio data and the third energy signal associated with the device audio data, the audio anomaly weight of each reference audio data is calculated, so as to screen out the reference audio data that may generate a state anomaly detection error through the audio anomaly weight. It can effectively reduce the impact of audio anomalies on the query results and improve the efficiency and accuracy of information anomaly detection and processing.

[0071] As an alternative embodiment, S501 may specifically include: Compare the first energy signal associated with the reference audio data with the third energy signal associated with the device audio data, and output the candidate channel audio numbers with different bit orders of the audio data packets of the energy channels in the first energy signal and the third energy signal; Divide the candidate channel audio numbers by the total number of audio data packets of the energy channels of the reference audio data, and output the first ratio; Divide the number of audio data packets of the channels in the reference audio data by the number of audio data packets of the channels in the device audio data, and output the second ratio; Multiply the first ratio by the second ratio and output the audio anomaly weight of the reference audio data.

[0072] In this embodiment, based on the candidate channel audio numbers with different bit orders of the energy channel audio data packets in the first energy signal associated with the reference audio data and the third energy signal associated with the device audio data, the total number of energy channel audio data packets in the reference audio data, the number of channel audio data packets in the reference audio data, and the number of channel audio data packets in the device audio data, calculate the audio anomaly weight of each reference audio data, so that the reference audio data that may cause state anomaly detection errors can be screened out by the audio anomaly weight. It can effectively reduce the impact of audio anomalies on the query results and improve the efficiency and accuracy of information anomaly detection and processing.

[0073] As an alternative embodiment, S104 may specifically include: Based on the anomaly detection confidence of each energy channel audio data packet in the reference audio data, set the channel audio weight of each energy channel audio data packet in the reference audio data respectively; Based on the channel audio weights of each energy channel audio data packet in the reference audio data, calculate the relevance scores of each initial anomaly detection record output by the anomaly detection of the industrial equipment operation audio library by the reference audio data; Determine each of the initial anomaly detection records with a relevance score greater than a preset relevance threshold in each reference audio data as the anomaly detection result associated with the device audio data.

[0074] In this embodiment, the server assigns a channel audio weight to each energy channel audio data packet based on the anomaly detection confidence of each energy channel audio data packet in the reference audio data. The greater the channel audio weight, the higher the importance of the energy channel audio data packet in the anomaly detection process.

[0075] Then, generate the reference audio data with channel audio weights into the industrial equipment operation audio library for anomaly detection and output a series of initial anomaly detection records. For each initial anomaly detection record, determine the relevance score based on the matching situation between the energy channel audio data packets it contains and the energy channel audio data packets in the reference audio data, as well as the channel audio weights of these energy channel audio data packets. Specifically, the scoring algorithm may involve weighted summation, cosine overlap calculation, etc.

[0076] Then, a relevance threshold is preset to screen high-relevance anomaly detection records. The initially calculated anomaly detection records are sorted from high to low according to the relevance score of the data packet signals, and the records with a relevance score greater than the preset relevance threshold are selected as the final anomaly detection results associated with the device audio data. The selected anomaly detection results are presented to the operation process in the form of a list, summary, or other forms, providing a visualization and interaction interface for the query results.

[0077] Through this embodiment, an anomaly detection method based on the anomaly detection confidence of energy channel audio data packets and the channel audio weights is provided, which can more accurately capture the query intention of the operation process, thereby improving the accuracy and efficiency of information anomaly detection.

[0078] Based on the method for monitoring and processing sound anomalies in the operation process of industrial equipment provided by the present invention. Correspondingly, the present invention also provides a specific embodiment of a system for monitoring and processing sound anomalies in the operation process of industrial equipment.

[0079] Figure 6 FIG. shows a schematic structural diagram of a system for monitoring and processing sound anomalies in the operation process of industrial equipment provided by an embodiment of the present invention. The system for monitoring and processing sound anomalies in the operation process of industrial equipment may include a conversion graph configuration unit 610, an audio association unit 620, a confidence calculation unit 630, and an industrial equipment operation audio library anomaly detection unit 640.

[0080] The conversion graph configuration unit 610 is used to perform data packet segmentation on the device audio data generated during the operation process, and configure and output a state transition graph associated with the device audio data. The state transition graph is used to represent the state relationship between the channel audio data packets in the device audio data; The audio association unit 620 is used to perform data packet signal recombination between the energy channel audio data packets in the device audio data based on the state transition graph, and output multiple reference audio data associated with the device audio data; The confidence calculation unit 630 is used to calculate and output the anomaly detection confidence of each energy channel audio data packet in the reference audio data for each reference audio data based on the pitch frequency data of each energy channel audio data packet in the reference audio data. The anomaly detection confidence is used to represent the risk of the energy channel audio data packet during the anomaly detection process; The industrial equipment operation audio library anomaly detection unit 640 is used to perform anomaly detection on the industrial equipment operation audio library based on the anomaly detection confidence of each energy channel audio data packet in each reference audio data, and output the anomaly detection results associated with the device audio data.

[0081] In the sound anomaly monitoring and processing system for industrial equipment operation provided by the embodiments of the present invention, first, the device audio data generated during the operation process is packetized, and a state transition diagram representing the state relationship between the audio data packets of each channel in the device audio data is configured. Then, based on the state transition diagram, multiple reference audio data are associated and output for the device audio data. Next, based on the pitch frequency data of each energy channel audio data packet in the reference audio data, the anomaly detection confidence level of each energy channel audio data packet in the reference audio data is calculated. Thus, based on the anomaly detection confidence level of each energy channel audio data packet in the reference audio data, an anomaly detection can be performed on the industrial equipment operation audio library to output the anomaly detection result associated with the device audio data. In this way, the embodiments of the present invention associate and output multiple reference audio data for the device audio data, and based on the pitch frequency data of each energy channel audio data packet in the reference audio data, set the associated anomaly detection confidence level for each energy channel audio data packet. Fully considering the richness and diversity of the expression of the device audio data packets caused by the different energy positions in the device audio data, the accuracy of anomaly detection can be improved.

[0082] In several embodiments provided by the present invention, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, indirect couplings or communication connections of devices or units, and can be in electrical, mechanical, or other forms.

[0083] In addition, each functional unit in various embodiments of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units. The above is only the implementation manner of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or equivalent process transformation based on the content of the specification and drawings of the present invention, or direct or indirect application in other related technical fields, is equally included in the patent protection scope of the present invention.

[0084] The specific embodiments of the invention have been described in detail above, but they are only examples, and the invention is not limited to the specific embodiments described above. For those skilled in the art, any equivalent modifications or substitutions to the invention are also within the scope of the invention. Therefore, all equivalent transformations, modifications, improvements, etc. made without departing from the spirit and principles of the invention should be covered within the scope of the invention.

Claims

1. A method for monitoring and processing abnormal sounds during the operation of industrial equipment, characterized in that, Including: Performing packet segmentation on the device audio data generated during the operation process, and configuring and outputting a state transition diagram associated with the device audio data, where the state transition diagram is used to represent the state relationship associated between the audio data packets of each channel in the device audio data; Based on the state transition diagram, performing packet signal recombination between the energy channel audio data packets in the device audio data, and outputting a plurality of reference audio data associated with the device audio data; For each of the reference audio data, based on the pitch frequency data of each energy channel audio data packet in the reference audio data, calculating and outputting the anomaly detection confidence of each energy channel audio data packet in the reference audio data, where the anomaly detection confidence is used to represent the risk of the energy channel audio data packet during the anomaly detection process; Based on the anomaly detection confidence of each energy channel audio data packet in each of the reference audio data, performing anomaly detection on the industrial equipment operation audio library and outputting the anomaly detection result associated with the device audio data.

2. The method for monitoring and processing abnormal sounds during the operation of industrial equipment according to claim 1, wherein, The performing packet segmentation on the device audio data generated during the operation process and configuring and outputting a state transition diagram associated with the device audio data includes: Obtaining the device audio data generated during the operation process; Performing packet segmentation on the device audio data through a recurrent neural network, and outputting the audio data packets of each channel associated with the device audio data; Based on the state relationship between the audio data packets of each channel associated with the device audio data, configuring and outputting a state transition diagram associated with the device audio data.

3. The method for monitoring and processing abnormal sound during the operation of industrial equipment according to claim 2, characterized in that, The performing packet segmentation on the device audio data through a recurrent neural network and outputting the audio data packets of each channel associated with the device audio data includes: Encoding the device audio data into the encoder of the recurrent neural network, and outputting an audio vector associated with the device audio data; Generating the audio vector into the decoder of the recurrent neural network for decoding, and outputting the audio data packets of each channel associated with the device audio data.

4. The method for monitoring and processing abnormal sounds during the operation of industrial equipment according to claim 1, wherein, The calculating and outputting the anomaly detection confidence of each energy channel audio data packet in each of the reference audio data based on the pitch frequency data of each energy channel audio data packet in the reference audio data includes: Obtaining a first energy signal of the reference audio data, where the first energy signal is used to represent the fluctuation condition between the energy channel audio data packets in the reference audio data; Merging the energy channel audio data packets located at the adjacent positions in the reference audio data in the first energy signal to form a second energy signal of the reference audio data; For each energy channel audio data packet in the second energy signal, sequentially obtaining the non-energy channel audio data packets on both sides of the energy channel audio data packet until reaching other energy channel audio data packets, and outputting a confidence list of the energy channel audio data packet; Based on the confidence list of each energy channel audio data packet in the reference audio data, calculating and outputting the anomaly detection confidence of each energy channel audio data packet in the reference audio data.

5. The method for monitoring and processing abnormal sounds during the operation of industrial equipment according to claim 4, wherein, Calculating and outputting the anomaly detection confidence of each energy channel audio data packet in the reference audio data based on the confidence list of each energy channel audio data packet in the reference audio data includes: Dividing the confidence list of the energy channel audio data packets into a first list and a second list according to the positions of the energy channel audio data packets in the confidence list; Outputting the ratio of non-energy channel audio data packets in the confidence list based on the number of non-energy channel audio data packets in the first list and the second list; Calculating and outputting the anomaly detection confidence of the energy channel audio data packets based on the candidate overlap of each non-energy channel audio data packet in the confidence list and the ratio of non-energy channel audio data packets, where the candidate overlap is used to represent the maximum value of the overlap between the non-energy channel audio data packet and each overlapping non-energy channel audio data packet, and the overlapping non-energy channel audio data packet is used to represent each non-energy channel audio data packet included in the confidence lists associated with the energy channel audio data packets in each of the other reference audio data; 6. The method for monitoring and processing abnormal sound during the operation of industrial equipment according to claim 5, wherein The calculating and outputting the anomaly detection confidence of the energy channel audio data packets based on the candidate overlap of each non-energy channel audio data packet in the confidence list and the ratio of non-energy channel audio data packets includes: Determining the average overlap and the variance of overlap based on the candidate overlap of each non-energy channel audio data packet in the confidence list; For each non-energy channel audio data packet in the confidence list, calculating the square of the result of dividing the difference between the candidate overlap of the non-energy channel audio data packet and the average overlap by the variance of overlap, and outputting the overlap calculation value of the non-energy channel audio data packet; Multiplying the accumulated value of the overlap calculation values of each non-energy channel audio data packet by the ratio of non-energy channel audio data packets and then dividing by the number of non-energy channel audio data packets in the confidence list, and outputting the anomaly detection confidence of the energy channel audio data packet; 7. The method for monitoring and processing abnormal sounds during the operation of industrial equipment according to claim 1, characterized in that, Before calculating and outputting the anomaly detection confidence of each energy channel audio data packet in the reference audio data for each reference audio data based on the pitch frequency data of each energy channel audio data packet in the reference audio data, the method further includes: Determining the audio anomaly weight of each reference audio data based on the first energy signal associated with each reference audio data and the third energy signal associated with the device audio data; Performing linear normalization processing on the audio anomaly weights of each reference audio data, and outputting the normalized audio anomaly weights of each reference audio data; Screening out the reference audio data with the normalized audio anomaly weights greater than the preset weight threshold, and outputting the optimized reference audio data; For each reference audio data, calculating and outputting the anomaly detection confidence of each energy channel audio data packet in the reference audio data based on the pitch frequency data of each energy channel audio data packet in the reference audio data includes: For each of the optimized reference audio data, based on the pitch frequency data of each energy channel audio data packet in the optimized reference audio data, calculate and output the anomaly detection confidence of each energy channel audio data packet in the optimized reference audio data.

8. The method for monitoring and processing abnormal sounds during the operation of industrial equipment according to claim 7, characterized in that, The determining of the audio anomaly weights of each of the reference audio data based on the first energy signal associated with each of the reference audio data and the third energy signal associated with the device audio data includes: Compare the first energy signal associated with the reference audio data with the third energy signal associated with the device audio data, and output the number of candidate channel audio data with different bit orders of the energy channel audio data packets in the first energy signal and the third energy signal; Divide the number of candidate channel audio data by the total number of energy channel audio data packets of the reference audio data, and output a first ratio; Divide the number of channel audio data packets in the reference audio data by the number of channel audio data packets in the device audio data, and output a second ratio; Multiply the first ratio by the second ratio, and output the audio anomaly weight of the reference audio data.

9. The method for monitoring and processing abnormal sounds during the operation of industrial equipment according to claim 1, characterized in that, The performing of anomaly detection on the industrial equipment operation audio library based on the anomaly detection confidence of each energy channel audio data packet in each of the reference audio data and outputting the anomaly detection result associated with the device audio data includes: Based on the anomaly detection confidence of each energy channel audio data packet in the reference audio data, respectively set the channel audio weights of each energy channel audio data packet in the reference audio data; Based on the channel audio weights of each energy channel audio data packet in the reference audio data, calculate the relevance scores of each initial anomaly detection record output by performing anomaly detection on the industrial equipment operation audio library by the reference audio data; Determine each initial anomaly detection record with a relevance score greater than a preset relevance threshold in each of the reference audio data as the anomaly detection result associated with the device audio data.

10. A sound anomaly monitoring and processing system for the operation process of industrial equipment, characterized in that, Including: A conversion graph configuration unit, configured to perform data packet segmentation on the device audio data generated during the operation process, and configure and output a state transition graph associated with the device audio data, where the state transition graph is used to represent the state relationship associated between each channel audio data packet in the device audio data; An audio association unit, configured to perform data packet signal recombination between the energy channel audio data packets in the device audio data based on the state transition graph, and output a plurality of reference audio data associated with the device audio data; A confidence calculation unit, configured to, for each of the reference audio data, based on the pitch frequency data of each energy channel audio data packet in the reference audio data, calculate and output the anomaly detection confidence of each energy channel audio data packet in the reference audio data, where the anomaly detection confidence is used to represent the risk of the energy channel audio data packet during the anomaly detection process; An industrial equipment operation audio library anomaly detection unit, configured to perform anomaly detection on the industrial equipment operation audio library based on the anomaly detection confidence of each energy channel audio data packet in each of the reference audio data, and output the anomaly detection result associated with the device audio data; The sound anomaly monitoring and processing system for industrial equipment operation processes is used to execute a method for sound anomaly monitoring and processing in industrial equipment operation processes as described in any one of claims 1-9.

Images