Audio processing method, device, equipment, system and computer program product

Abstract

The invention provides an audio processing method, device, equipment and system and a computer program product, and the audio processing equipment comprises a processing assembly, and a pickup assembly and a communication assembly which are electrically connected with the processing assembly. Wherein the pickup assembly can pick up underground audio data, and the audio data comprises working audio data generated in the working process of underground equipment. The processor can process the audio data and determine the working state of the underground equipment, and the working state comprises normal working of the underground equipment or fault of the underground equipment. The communication assembly can send the working state of the underground equipment to a monitoring platform of the underground equipment. Therefore, the audio processing equipment can be used for analyzing the underground audio data so as to determine whether the underground equipment fails or not, and the result is sent to the monitoring platform of the underground equipment, so that a large number of inspection personnel do not need to be configured, and the operation cost of an enterprise is reduced.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to an audio processing method, apparatus, device, system and computer program product. Background Art

[0002] Downhole equipment failure is a major cause of underground accidents. Currently, equipment inspections rely primarily on scheduled manual inspections to promptly detect and resolve equipment failures and ensure safe underground operations. However, manual inspections require a large number of inspectors, increasing operating costs. Summary of the Invention

[0003] In view of this, the present application proposes an audio processing method, device, equipment, system and computer program product to solve the problem that the manual inspection method in the prior art requires a large number of inspection personnel, which increases the operating costs of the enterprise.

[0004] The technical solutions proposed in this application are as follows:

[0005] In a first aspect, an embodiment of the present application provides an audio processing device, comprising:

[0006] a processing component, and a sound pickup component and a communication component electrically connected to the processing component respectively;

[0007] The sound pickup component is used to pick up audio data downhole; the audio data includes working audio data generated by downhole equipment during operation;

[0008] The processing component is used to process the audio data to determine the working status of the downhole equipment; the working status includes the downhole equipment being normally operated or the downhole equipment being faulty;

[0009] The communication component is used to send the working status of the downhole equipment to the monitoring platform of the downhole equipment.

[0010] Furthermore, the communication component is connected to the monitoring platform by wire through one or more cascaded audio processing devices; and / or, the communication component is wirelessly bridged to the monitoring platform through one or more bridged audio processing devices.

[0011] Furthermore, the communication component is wirelessly bridged to the monitoring platform via one or more bridged audio processing devices; the communication component is also used to:

[0012] Determine the network nodes that the communication component needs to pass through for data communication with the monitoring platform based on the remaining bandwidth and the remaining number of hops of each network node in the wireless bridge network where the communication component is located;

[0013] The network node includes other audio processing devices in the wireless bridging network; the remaining number of hops represents the number of network nodes between the network node and the monitoring platform.

[0014] Furthermore, the processing component processes the audio data, including:

[0015] The audio data is input into a pre-trained noise reduction model to obtain processed audio data output by the noise reduction model; the noise reduction model is trained using sample audio data from downhole as training samples and with the goal of separating the working audio of downhole equipment from the sample audio data.

[0016] Furthermore, the processing component processes the audio data to determine the working status of the downhole equipment, including:

[0017] The audio data is input into a pre-trained fault classification model to obtain a fault classification result output by the fault classification model; the fault classification model is obtained by fine-tuning the pre-trained model using the sample audio data downhole as training samples with the goal of identifying the fault type corresponding to the sample audio data; the pre-trained model is obtained by learning the fault audio of the downhole equipment.

[0018] In a second aspect, an embodiment of the present application provides an audio processing system, comprising: a plurality of audio processing devices and a monitoring platform for downhole equipment;

[0019] The multiple audio processing devices communicate data with the monitoring platform of the downhole equipment through wired cascading and / or wireless bridging;

[0020] Among them, any one of the audio processing devices is used to pick up audio data underground, and the audio data includes the working audio data generated by the underground equipment during operation; the audio data is processed to determine the working status of the underground equipment, and the working status includes the normal operation of the underground equipment or the failure of the underground equipment; the working status of the underground equipment is sent to the monitoring platform of the underground equipment.

[0021] In a third aspect, an embodiment of the present application provides an audio processing method, applied to an audio processing device, the method comprising:

[0022] Acquiring downhole audio data; the audio data includes working audio data generated by downhole equipment during operation;

[0023] Processing the audio data to determine a working state of the downhole equipment; the working state includes a normal working state of the downhole equipment or a fault of the downhole equipment;

[0024] The working status of the downhole equipment is sent to a monitoring platform of the downhole equipment.

[0025] In a fourth aspect, an embodiment of the present application provides an audio processing apparatus, applied to an audio processing device, the apparatus comprising:

[0026] An acquisition unit, configured to acquire audio data downhole; the audio data includes working audio data generated by downhole equipment during operation;

[0027] A determination unit, configured to process the audio data to determine a working state of the downhole equipment; the working state includes a normal working state of the downhole equipment or a fault of the downhole equipment;

[0028] The sending unit is used to send the working status of the downhole equipment to the monitoring platform of the downhole equipment.

[0029] In a fifth aspect, an embodiment of the present application provides an electronic device, including:

[0030] A memory and a processor; wherein the memory is used to store a program; and the processor is used to implement any of the above methods by running the program in the memory.

[0031] In a sixth aspect, embodiments of the present application provide a computer program product, comprising a computer program. When executed by a processor, the computer program implements any of the above methods. Optionally, the computer program may be stored on a computer-readable storage medium or in the cloud; the processor of the computer device reads the computer program from the computer-readable storage medium or the cloud.

[0032] The audio processing device proposed in this application includes a processing component, and a sound pickup component and a communication component electrically connected to the processing component. The sound pickup component is capable of picking up audio data from downhole equipment, including operating audio data generated by downhole equipment during operation. The processor is capable of processing the audio data to determine the operating status of the downhole equipment, which may include normal operation or failure of the downhole equipment. The communication component is capable of transmitting the operating status of the downhole equipment to a downhole equipment monitoring platform. In this way, the audio processing device can be used to analyze the downhole audio data to determine whether the downhole equipment is faulty, and the results can be transmitted to the downhole equipment monitoring platform. This eliminates the need for a large number of inspection personnel and reduces the company's operating costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0034] Figure 1 It is a structural diagram of an audio processing device provided in an embodiment of the present application.

[0035] Figure 2 This is a structural diagram of a cascade circuit provided in an embodiment of the present application.

[0036] Figure 3 This is a structural diagram of a wireless bridging line provided in an embodiment of the present application.

[0037] Figure 4 It is a structural diagram of another audio processing device provided in an embodiment of the present application.

[0038] Figure 5 It is a structural diagram of the hardware layer of the audio processing device provided in an embodiment of the present application.

[0039] Figure 6 This is a schematic diagram of the pin structure of the aviation plug provided in an embodiment of the present application.

[0040] Figure 7 This is a flowchart of an audio processing method provided in an embodiment of the present application.

[0041] Figure 8 Schematic diagram of the structure of an audio processing device provided in an embodiment of the present application.

[0042] Figure 9 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0044] Downhole equipment failure is one of the important causes of downhole accidents, which may not only lead to production interruption, but also pose a serious threat to the life safety of underground workers.

[0045] Currently, equipment inspections rely primarily on scheduled manual inspections to promptly detect equipment failures and eliminate potential hazards, ensuring the safety and continuity of underground operations. However, manual inspections have numerous limitations. For example, inspectors must frequently enter the underground environment, facing complex geological conditions and potential safety risks, resulting in a highly intensive and dangerous workload. Furthermore, manual inspections are inefficient and require long inspection cycles, making it difficult to detect subtle anomalies in equipment operation. This can lead to the accumulation of potential failures and ultimately serious accidents. Furthermore, manual inspections require a large number of inspectors, which not only increases operating costs but also wastes human resources. This is particularly true in large mines or deep-well operations, where the inspection scope is extensive and the equipment is diverse, further increasing the difficulty and cost of manual inspections.

[0046] Therefore, although manual inspections still play an important role in current underground safety management, their limitations are becoming increasingly prominent, and there is an urgent need to explore more efficient and safe equipment monitoring methods. Based on this, this application proposes an audio processing method, device, equipment, system and computer program product. This technical solution uses an audio processing device to analyze the audio data in the well to determine whether the underground equipment is faulty, and sends the results to the monitoring platform of the underground equipment, thereby achieving more efficient and safe supervision of the underground equipment.

[0047] Figure 1 A schematic diagram of the structure of an audio processing device is shown in FIG. Figure 1 As shown, the audio processing device includes a processing component 10, and a sound pickup component 11 and a communication component 12 electrically connected to the processing component 10 respectively.

[0048] The sound pickup component 11 is used to capture downhole audio data, including operating audio data generated by downhole equipment and downhole noise. A microphone can be used as this component. More specifically, to address complex downhole noise environments, such as mechanical vibration and electromagnetic interference, a dual-microphone linear array can be used as this component. This dual-microphone spatial filtering technique significantly improves the signal-to-noise ratio compared to traditional single-microphone solutions. In some embodiments, the dual-microphone linear array is fixed at a 35° angle within an explosion-proof enclosure and supports dynamic beamforming, making it suitable for localizing sound sources in confined downhole spaces.

[0049] The processing component 10 is used to process the above-mentioned audio data to determine the working status of the downhole equipment; the working status includes the normal operation of the downhole equipment or the failure of the downhole equipment. In some embodiments, the working audio data of the downhole equipment when it is in different fault types can be collected in advance to construct a working audio database corresponding to different fault types. After obtaining the downhole audio data, the similarity between the audio data and the working audio corresponding to different fault types is calculated. If the similarity between the audio data and the working audio corresponding to each fault type is less than a set threshold, it indicates that the working status of the downhole equipment includes the normal operation of the downhole equipment; if the similarity between the target working audio and the audio data in the working audio corresponding to each fault type is greater than or equal to the set threshold, it indicates that the working status of the downhole equipment includes a downhole equipment failure, and the fault type is the fault type corresponding to the target working audio.

[0050] The above threshold values ​​can be set according to actual conditions and are not limited in this embodiment.

[0051] The communication component 12 is used to transmit the operating status of the downhole equipment to the downhole equipment monitoring platform. In some embodiments, when the operating status of the downhole equipment includes normal operation of the downhole equipment, the communication component 12 may not transmit the operating status of the downhole equipment to the downhole equipment monitoring platform. When the operating status of the downhole equipment includes a downhole equipment failure, the communication component 12 transmits the downhole equipment failure operating status to the monitoring platform, enabling personnel to promptly detect the downhole equipment failure and perform repairs to avoid affecting production or causing an accident.

[0052] With this setup, audio processing equipment can be used to analyze the audio data underground to determine whether the underground equipment is faulty, and the results can be sent to the monitoring platform of the underground equipment. This eliminates the need to deploy a large number of inspection personnel, reducing the company's operating costs.

[0053] As an optional implementation method, another embodiment of the present application discloses that the communication component is connected to the monitoring platform by wire through one or more cascaded audio processing devices; and / or the communication component is wirelessly bridged to the monitoring platform through one or more bridged audio processing devices.

[0054] More specifically, the communication component is connected to the monitoring platform by wire through the cascaded audio processing device. The communication component can also be directly connected to the monitoring platform by wire, which is not limited in this embodiment.

[0055] like Figure 2 As shown in the figure, each cascade line is equipped with 6 audio processing devices. Figure 2The embodiment shown is an example provided for explaining the technical solution, and does not limit each cascade line to only 6 audio processing devices. The number of audio processing devices in each cascade line can be determined according to actual conditions.

[0056] In some embodiments, the audio processing device is powered by a dual-circuit mining intrinsically safe power supply, and 6 audio processing devices are provided in each cascade line, which can realize fault monitoring of underground equipment within a range of up to 100 meters.

[0057] With this configuration, audio processing devices can be connected using a wired cascade method, which can improve signal transmission efficiency.

[0058] The communication component is wirelessly bridged to the monitoring platform through the bridged audio processing device. The communication component can also be directly wirelessly bridged to the monitoring platform, which is not limited in this embodiment.

[0059] like Figure 3 As shown in the figure, each wireless bridge line is equipped with three audio processing devices. Figure 3 The illustrated embodiment is an example provided to illustrate the technical solution, and does not limit each wireless bridge line to only three audio processing devices. The number of audio processing devices in each wireless bridge line can be determined based on actual conditions.

[0060] In some embodiments, by wirelessly bridging between audio processing devices, three audio processing devices can be bridged at a time. Data can be transmitted wirelessly by simply powering the audio processing devices, enabling fault monitoring of underground equipment within a range of up to 400 meters.

[0061] With this setup, audio processing devices can be connected using wireless bridging, avoiding the limitations of data cables and increasing the deployment range of audio processing devices.

[0062] In addition, a hybrid deployment of wired cascading and wireless bridging can be adopted. Wired cascading can cover the core area, while wireless bridging can be used to expand to blind spots, breaking through the distance limitations of traditional single transmission methods and increasing the detection range.

[0063] In some embodiments, the monitoring platform includes a downhole server, an uphole server, a cloud server, and a fault monitoring system. The audio processing device and the downhole server are connected via a wired cascade or wireless bridge. The audio processing device transmits the operating status of the downhole equipment to the downhole server, which in turn transmits the operating status to the uphole server. The uphole server transmits the operating status of the downhole equipment to the monitoring system via the cloud server. This allows personnel at the monitoring system to promptly understand the operating status of the downhole equipment and respond promptly when downhole equipment fails.

[0064] As an optional implementation, another embodiment of the present application discloses a wireless bridge connection between the communication component and the monitoring platform via one or more bridged audio processing devices. The communication component is further configured to determine the network nodes through which data communication between the communication component and the monitoring platform must pass based on the remaining bandwidth and remaining hop count of each network node in the wireless bridge network in which the communication component resides.

[0065] When the communication component is wirelessly bridged to the monitoring platform through one or more bridging audio processing devices, it is necessary to determine the specific path for the audio processing device to transmit signals to the monitoring platform according to the actual network status of the wireless bridging network.

[0066] The remaining bandwidth and number of hops at each network node in the wireless bridge network where the communication component resides can be monitored to determine the network nodes that the communication component must traverse for data communication with the monitoring platform. Network nodes include other audio processing devices in the wireless bridge network; the number of hops remaining represents the number of network nodes between the network node and the monitoring platform.

[0067] Specifically, the weight of each network node is first determined based on its remaining bandwidth and number of remaining hops. The higher the remaining bandwidth, the higher the corresponding network node weight; the fewer the remaining hops, the higher the corresponding network node weight. The network node with the highest weight is selected as the next hop. Using this next hop as a benchmark, the network node with the highest weight is then selected again as the next hop, until the next hop reaches the monitoring platform. All the network nodes that the communication component needs to pass through for data communication with the monitoring platform form the path for the audio processing device to transmit signals to the monitoring platform.

[0068] More specifically, the protocol stack design of the wireless bridging network can be performed based on the Mesh networking enhancement protocol of WiFi6 (802.11ax).

[0069] In this embodiment, the frame structure is optimized, including: a new "cascade status field" is added to the control frame, the format of which is JSON nested TLV, including node load, channel quality, and number of hops; the data frame supports aggregate transmission (A-MPDU), and a single frame can carry a maximum of 4KB of voiceprint data.

[0070] The relay logic includes: the node periodically broadcasts Hello packets and maintains a neighbor table, including the signal strength (RSSI) of neighbor nodes and the remaining power of neighbor nodes. The node cycle can be selected to be 5S.

[0071] The weight of each network node is determined based on the remaining bandwidth and the number of remaining hops of each network node, and the network node with the highest weight is selected as the next hop. In some embodiments, the weight of each network node is determined according to the following formula: ij :

[0072] W ij = Remaining bandwidth / 10+1 / number of hops

[0073] This embodiment also sets a retransmission mechanism, that is, a dynamic backoff based on the contention window. For example, the minimum value CW of the contention window is min Set to 15, the maximum value of the contention window CW max If set to 1023, the maximum number of retries is 3.

[0074] With this setup, the maximum single-hop distance of the bridge network is 130 meters. When there are three Bluetooth interference sources in the 2.4 GHz band, the throughput remains greater than or equal to 80 Mbps.

[0075] As an optional implementation, another embodiment of the present application discloses that the processing component of the above embodiment processes the audio data, specifically including:

[0076] The audio data is input into a pre-trained denoising model to obtain processed audio data output by the denoising model; the denoising model is trained using sample audio data from downhole as training samples and with the goal of separating the working audio of downhole equipment from the sample audio data.

[0077] In this embodiment, a noise reduction model is pre-trained, and the audio data is processed using the noise reduction model, which can adaptively reduce the noise of the audio data and improve the signal-to-noise ratio.

[0078] A large amount of sample audio data from downhole can be obtained as training samples, and the working audio of downhole equipment in the sample audio data can be used as training labels. During training, the training samples are input into the denoising model to obtain the prediction results output by the denoising model. By comparing the prediction results output by the denoising model with the training labels, the loss value of the denoising model is determined. With the goal of reducing the loss value of the denoising model, the parameters of the denoising model are adjusted, and then the above training process is repeated until the various parameters of the denoising model meet the requirements. The above-mentioned denoising model can be obtained by training a model based on the attention mechanism, such as Transformer, etc., which is not limited in this embodiment.

[0079] The audio data is input into a pre-trained noise reduction model. The noise reduction model can separate device abnormal sounds from environmental noise based on the attention mechanism, and obtain the processed audio data output by the noise reduction model.

[0080] With this setting, the noise reduction model can be used for adaptive noise reduction, thereby improving the signal-to-noise ratio and thus improving the reliability of the fault identification results.

[0081] As an optional implementation, another embodiment of the present application discloses that the processing component of the above embodiment processes the audio data to determine the working status of the downhole equipment, specifically including:

[0082] The audio data is input into a pre-trained fault classification model, and the fault classification model outputs a fault classification result; the fault classification model is obtained by fine-tuning the pre-trained model using the sample audio data from underground as training samples, with the goal of identifying the fault type corresponding to the sample audio data; the pre-trained model is obtained by learning the fault audio of the underground equipment.

[0083] In this embodiment, a pre-trained fault classification model is used to process audio data, enabling the determination of the fault type of downhole equipment based on the audio data. In some embodiments, the audio data is first subjected to adaptive noise reduction processing using the noise reduction model described in the above embodiment, and then the noise-reduced audio data is input into the fault classification model for fault classification. This reduces interference caused by noise and improves the reliability of the classification results.

[0084] A large amount of downhole audio sample data can be obtained as training samples, and the fault types corresponding to the sample audio data can be used as training labels. In some embodiments, if fault classification is performed using denoised audio data, a large amount of downhole audio sample data can be obtained, and adaptive denoising processing can be performed on the sample audio data according to the above embodiments, and the processed audio data can be used as training samples.

[0085] In some embodiments, the training samples need to be pre-processed before fine-tuning. Specifically, the training samples can be framed and windowed. For example, the frame length is 32ms, the frame shift is 10ms, and the windowing can use a Hamming window.

[0086] A short-time Fourier transform (STFT) or a fast Fourier transform (FFT) is performed to generate a spectrogram. In some embodiments, a signal of 1024 sampling points is transformed from the time domain to the frequency domain to generate a spectrogram.

[0087] Then, Mel filtering is performed. Specifically, based on the principle of nonlinear distribution of the Mel scale, filters in the low-frequency area (100Hz~4kHz) can be densely set, and filters in the high-frequency area (4kHz~96kHz) can be sparsely set to match the energy concentration characteristics of the equipment fault soundprint. In some embodiments, 25 groups of filters are set in the low-frequency area and 15 groups of filters are set in the high-frequency area. Among them, 40 groups of filters are compared with the traditional 24 groups, and the feature discrimination is improved by 18% in the roller bearing wear identification task.

[0088] Finally, logarithmic compression (Log Compression) and discrete cosine transform (DCT, Discrete Cosine Transform) are performed to achieve data compression and energy conversion.

[0089] During fine-tuning, training samples are input into the fault classification model to obtain the model's output predictions. The model's loss is determined by comparing the model's output predictions with the training labels. The model's parameters are adjusted to minimize the model's loss. The fine-tuning process is then repeated until all parameters meet the requirements. The fault classification model can be trained based on a pre-trained model.

[0090] The pre-trained model is obtained by learning the fault audio of downhole equipment. Specifically, a fault soundprint database covering downhole equipment such as conveyors, rollers, and pumps can be established. The fault soundprint database is used to pre-train the basic model to obtain the above-mentioned pre-trained model, so that the pre-trained model has the ability to classify and identify downhole equipment faults. The pre-trained model is then fine-tuned based on sample audio data from downhole, that is, the model's generalization ability is optimized through transfer learning, thereby improving recognition accuracy. The above-mentioned basic model can be based on any neural network model, and this embodiment does not limit it.

[0091] The audio data is input into a pre-trained fault classification model, which can determine the fault type of the downhole equipment based on the audio data.

[0092] In some embodiments, the aforementioned basic model utilizes a lightweight deep neural network (DNN) model architecture. More specifically, the DNN model architecture of this embodiment includes a three-layer convolutional neural network (CNN), a two-layer long short-term memory (LSTM) network, and a connection layer.

[0093] CNN layer: Conv1D×3, number of filters [64, 128, 256], kernel width [5, 3, 3], stride 2, ReLU activation. CNN layers are used to extract local frequency domain patterns, such as periodic noise from bearings.

[0094] LSTM layer: Bi-LSTM × 2, 128 hidden units, dropout = 0.3. The LSTM layer is used to capture the long-term contextual associations of fault voiceprints, such as the transient characteristics of pump startup.

[0095] Output layer: Softmax classification.

[0096] With this setup, a fault classification model is obtained by fine-tuning the pre-trained model, effectively improving recognition accuracy. At the same time, a lightweight algorithm model is used to reduce overall device power consumption. The low-power design extends the long-term service life of the device. Furthermore, a lightweight model is embedded in the audio processing device itself to achieve real-time extraction and classification of fault features, reducing reliance on cloud servers and shortening response time.

[0097] As an optional implementation, such as Figure 4 As shown, another embodiment of the present application discloses that the audio processing device of the above embodiment may further include an environmental sensor 13 electrically connected to the processing component. Environmental sensor 13 can detect parameters such as underground temperature, humidity, and air pressure, and transmit these parameters to processing component 10. Processing component 10 adjusts the fault determination threshold in real time based on changes in the underground environmental temperature, humidity, and air pressure to avoid false alarms. For example, low-frequency filtering is automatically enhanced when humidity exceeds 90%.

[0098] The system of audio processing equipment can be divided into interaction layer, algorithm layer, driver layer and hardware layer.

[0099] The interaction layer includes an edge gateway and an IP configuration interface. The interaction layer can log in to the configuration interface by accessing the device's default IP address on the WEB interface via wired or wireless means. Among them, the configuration interface can include: audio processing device configuration, network configuration, transmission configuration, audio configuration, system configuration and OTA upgrade interface. The algorithm layer includes the adaptive noise reduction and fault classification algorithms of the above-mentioned embodiments, as well as the model training method. The driver layer uses the Linux operating system, including PDMdriver, SDMMC driver, Flash memory driver, DDR driver, IIC driver, I2S driver, GMACdriver, Wifi driver, etc.

[0100] Hardware layer such as Figure 5 As shown, the Quad-core Corex-A53 architecture is used as the processing component, the sound pickup component uses a dual-microphone linear array, and temperature and humidity sensors and air pressure sensors are used as environmental sensors.

[0101] The dual-microphone linear array interacts with the Quad-core Corex-A53 architecture via clock / PDM signals. The temperature and humidity sensors interact with the Quad-core Corex-A53 architecture via the serial communication protocol (IIC). The pressure sensor interacts with the Quad-core Corex-A53 architecture via the I2S communication protocol. Memory devices use DDR4 x 16-bit, EMMC, and a built-in TF card. Communication components include dual-band Wi-Fi 6 and a SEWITCH PHY chip.

[0102] The audio processing device also includes a power chip. In order to ensure power safety, an isolation network transformer and an isolation power supply are provided. Figure 5 shown.

[0103] A standard aviation plug can also be used to further ensure safety. In some embodiments, the pin structure of the aviation plug is as follows: Figure 6 As shown, the plug includes a power supply port and a network communication port. Cascading between devices can be achieved through specially customized coal safety cables. The definition of each pin can be found in Table 1.

[0104] Connector pins definition 1 TX+ 2 TX- 3 RX+ 4 RX- 5 Negative pole of power supply 6 Positive pole of power supply 7 null 8 null

[0105] Table 1

[0106] The audio processing device uses an explosion-proof housing that meets the IP67 waterproof rating. The audio processing device can be fixed to the location where testing is required using methods such as magnets, adhesives, screws, and clamps.

[0107] This embodiment utilizes a dual-microphone linear array to collect sound sources, combined with an intelligent noise reduction algorithm, to capture abnormal noise from underground equipment. Optionally, it can be equipped with algorithm models specific to the coal mining industry, creating an intelligent monitoring device that integrates data collection, status assessment, and anomaly diagnosis. Compared to other microphone types, this system offers a high level of intelligence and multi-dimensional environmental perception capabilities. Furthermore, its integrated gateway functionality enables 24 / 7 online monitoring, saving enterprises manual inspection costs and fostering a safe, intelligent, and healthy underground environment.

[0108] The embodiment of the present application further discloses an audio processing system, comprising: multiple audio processing devices and a downhole equipment monitoring platform.

[0109] Multiple audio processing devices communicate with the downhole equipment's monitoring platform via wired cascading and / or wireless bridging. Each audio processing device is used to capture downhole audio data, including operating audio data generated by the downhole equipment during operation; process the audio data to determine the downhole equipment's operating status, which may include normal operation or failure; and transmit the downhole equipment's operating status to the downhole equipment's monitoring platform.

[0110] For the specific limitations of each audio processing device in the audio processing system, please refer to the limitations of the audio processing devices above, which will not be repeated here.

[0111] As an optional implementation, another embodiment of the present application discloses that each audio processing device in the audio processing system distributes the load using a load distribution strategy, which specifically includes static distribution and dynamic adjustment.

[0112] Static allocation: Set the maximum power consumption of each audio processing device to less than a set value, for example, less than or equal to 2 W. Static allocation can be achieved through the internal current limiting circuit of the audio processing device, and this implementation does not impose any restrictions.

[0113] Dynamic adjustment: The power chip can monitor the current of each audio processing device in real time. If the power of a certain level of audio processing equipment is greater than the first threshold, the power consumption of non-core functions will be automatically reduced. For example, the wireless module will be turned off to prioritize voiceprint collection. If the power of a certain level of audio processing equipment is greater than the second threshold, the fuse will cut off the power supply and trigger a local alarm.

[0114] The second threshold is greater than the first threshold, wherein the first threshold and the second threshold can be set according to actual conditions, for example, the first threshold is set to 1.8W, and the second threshold is set to 2.1W.

[0115] In addition, the cascade bus uses magnetic coupling isolation, allowing nodes to be added or deleted during operation.

[0116] In a specific example, audio processing equipment was deployed in a 300-meter-long coal mine conveyor tunnel. Four audio processing devices were deployed, three of which were wired cascaded and one wirelessly bridged. Dynamic load distribution was enabled, and the power consumption of each node was stabilized at 1.5 to 1.8 W. The end-to-end latency for the wired cascade segment was less than or equal to 50 ms, and for the wireless bridge segment was less than or equal to 120 ms. The fault identification accuracy rate was 95.3% during a 72-hour continuous operation test.

[0117] Furthermore, the present application embodiment provides an audio processing method, which can be executed by the audio processing device of the above embodiment. Figure 7 As shown, the method includes:

[0118] S101, obtaining audio data underground;

[0119] S102, processing the audio data to determine the working status of the downhole equipment;

[0120] S103: Send the working status of the downhole equipment to the monitoring platform of the downhole equipment.

[0121] The audio data includes working audio data generated by the downhole equipment during operation, and the working status includes the downhole equipment operating normally or failing.

[0122] For the specific limitations of this method, please refer to the limitations of the audio processing device above, which will not be repeated here.

[0123] Corresponding to the above audio processing method, the present application also discloses an audio processing device, see Figure 8 As shown, the device includes:

[0124] The acquisition unit 100 is used to acquire audio data in the well; the audio data includes working audio data generated by the well equipment during operation;

[0125] The determination unit 110 is used to process the audio data and determine the working status of the downhole equipment; the working status includes the downhole equipment being in normal operation or being faulty;

[0126] The sending unit 120 is used to send the working status of the downhole equipment to the monitoring platform of the downhole equipment.

[0127] Specifically, the device provided in this embodiment is based on the same application concept as the method provided in the above embodiments of this application, can execute the method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not fully described in this embodiment, please refer to the specific processing content of the method provided in the above embodiments of this application, and will not be repeated here.

[0128] The functions implemented by the above units can be implemented by the same or different processors respectively, and the embodiments of the present application are not limited thereto.

[0129] It should be understood that the units in the above devices can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and the memory stores instructions. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit of the device. The processor can be a general-purpose processor, such as a CPU or a microprocessor, and the memory can be a memory within the device or a memory outside the device. Alternatively, the units in the device can be implemented in the form of hardware circuits. The functions of some or all units can be realized by designing the hardware circuits. The hardware circuit can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units can be realized by designing the logical relationships between the components within the circuit. For another example, in another implementation, the hardware circuit can be implemented by a PLD. For example, an FPGA can include a large number of logic gate circuits. The connection relationships between the logic gate circuits are configured through a configuration file to realize the functions of some or all of the above units. All units of the above devices can be implemented entirely in the form of a processor calling software, or entirely in the form of hardware circuits, or partially in the form of a processor calling software, with the remaining parts implemented in the form of hardware circuits.

[0130] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and execute instructions, such as a CPU, a microprocessor, a GPU, or a DSP. In another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit may be fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the configuration of the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0131] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0132] In addition, the various units in the above apparatus may be fully or partially integrated together, or may be implemented independently. In one implementation, these units are integrated together and implemented in the form of a system-on-chip (SOC). The SOC may include at least one processor for implementing any of the above methods or implementing the functions of the various units of the apparatus. The at least one processor may be of different types, such as a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0133] An embodiment of the present application further provides a control device, which includes a processor and an interface circuit. The processor in the control device is connected to an input and output component through the interface circuit of the control device.

[0134] The input and output components specifically refer to hardware components that enable users to input information and output information to users, such as microphones, keyboards, handwriting tablets, touch screens, displays, speakers, printers, etc.

[0135] The above-mentioned interface circuit can be any interface circuit that can realize data communication function, for example, it can be a USB interface circuit, a Type-C interface circuit, a serial port circuit, a PCIE circuit, etc.

[0136] The processor in the control device is a circuit with signal processing capabilities. By executing any of the audio processing methods described in the above embodiments, it achieves more efficient and safe monitoring of downhole equipment. The specific implementation of the processor can be found in the above-mentioned processor implementation method, and is not strictly limited in the present embodiment.

[0137] When the control device is applied to a device with human-computer interaction function, the input and output components of the control device can be the input components and output components on the device, such as a microphone, keyboard, handwriting tablet, touch screen, display, audio player, etc. At the same time, the processor of the control device can be the CPU or GPU of the device, etc., and the interface circuit of the control device can be the interface circuit between the information input component of the device and the processor such as the CPU or GPU.

[0138] Corresponding to the above audio processing method, the present application embodiment also discloses an electronic device, see Figure 9 As shown, the electronic device includes:

[0139] Memory 200 and processor 210;

[0140] The memory 200 is connected to the processor 210 and is used to store programs;

[0141] The processor 210 is configured to implement the audio processing method disclosed in any of the above embodiments by running the program stored in the memory 200 .

[0142] Specifically, the electronic device may further include: a bus, a communication interface 220 , an input device 230 and an output device 240 .

[0143] The processor 210, the memory 200, the communication interface 220, the input device 230 and the output device 240 are interconnected via a bus.

[0144] A bus may include a pathway that transfers information between components of a computer system.

[0145] Processor 210 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, or the like, or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component.

[0146] The processor 210 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0147] The memory 200 stores a program for executing the technical solution of the present application, and may also store an operating system and other key services. Specifically, the program may include program code, and the program code includes computer operating instructions. More specifically, the memory 200 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, disk storage, flash, etc.

[0148] The input device 230 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0149] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speakers, etc.

[0150] The communication interface 220 may include any device such as a transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0151] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement the various steps of the audio processing method provided in the above embodiments of the present application.

[0152] In addition to the above methods and devices, embodiments of the present application may also be computer program products, which include a computer program. When the computer program is executed by a processor, it can execute the audio processing method provided by any of the above embodiments of the present application. Optionally, the computer program can be stored on a computer device's readable storage medium or in the cloud; the computer device's processor reads the computer program from the readable storage medium or the cloud.

[0153] The computer program product may be written in any combination of one or more programming languages ​​to implement the program code of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0154] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0155] In addition, the embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the processor executes the various steps of the audio processing method provided in the above embodiment.

[0156] The computer-readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0157] Specifically, the specific working contents of each part of the above-mentioned electronic device, computer program product and storage medium, as well as the specific processing contents when the computer program product or the computer program on the above-mentioned storage medium is executed by the processor, can be found in the contents of the various embodiments of the above-mentioned audio processing method, and will not be repeated here.

[0158] For the sake of simplicity, the aforementioned method embodiments are described as a series of action combinations. However, those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required for this application.

[0159] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between the various embodiments can be referred to in conjunction with each other. For device embodiments, since they are generally similar to method embodiments, their description is relatively simplified. For relevant parts, refer to the description of the method embodiments.

[0160] The steps in the methods of each embodiment of the present application can be adjusted in sequence, merged, and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0161] The modules and sub-modules in the devices and terminals in the various embodiments of the present application can be combined, divided, and deleted according to actual needs.

[0162] In the several embodiments provided in this application, it should be understood that the disclosed terminals, devices, and methods can be implemented in other ways. For example, the terminal embodiments described above are merely illustrative. For example, the division of modules or submodules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or module, which can be electrical, mechanical or other forms.

[0163] The modules or submodules described as separate components may or may not be physically separate, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules may be selected to achieve the purpose of this embodiment according to actual needs.

[0164] In addition, each functional module or submodule in each embodiment of the present application may be integrated into a processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into a single module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or software functional modules or submodules.

[0165] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0166] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, software units executed by a processor, or a combination of the two. The software units may be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art.

[0167] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0168] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An audio processing device, characterized in that include: a processing component, and a sound pickup component and a communication component electrically connected to the processing component respectively; The sound pickup component is used to pick up audio data downhole; the audio data includes working audio data generated by downhole equipment during operation; The processing component is used to process the audio data to determine the working status of the downhole equipment; the working status includes the downhole equipment being normally operated or the downhole equipment being faulty; The communication component is used to send the working status of the downhole equipment to the monitoring platform of the downhole equipment.

2. The audio processing device according to claim 1, characterized in that The communication component is connected to the monitoring platform by wire through one or more cascaded audio processing devices; And / or, the communication component is wirelessly bridged to the monitoring platform via one or more bridged audio processing devices.

3. The audio processing device according to claim 2, characterized in that The communication component is wirelessly bridged to the monitoring platform via one or more bridged audio processing devices; The communication component is further configured to: Determine the network nodes that the communication component needs to pass through for data communication with the monitoring platform based on the remaining bandwidth and the remaining number of hops of each network node in the wireless bridge network where the communication component is located; The network node includes other audio processing devices in the wireless bridging network; the remaining number of hops represents the number of network nodes between the network node and the monitoring platform.

4. The audio processing device according to claim 1, wherein The processing component processes the audio data, including: Inputting the audio data into a pre-trained noise reduction model to obtain processed audio data output by the noise reduction model; The noise reduction model is trained using sample audio data from downhole as training samples, with the goal of separating the working audio of downhole equipment from the sample audio data.

5. The audio processing device according to claim 1, characterized in that The processing component processes the audio data to determine the working status of the downhole equipment, including: Inputting the audio data into a pre-trained fault classification model to obtain a fault classification result output by the fault classification model; The fault classification model is obtained by fine-tuning a pre-trained model using sample audio data from underground equipment as training samples and aiming to identify the fault type corresponding to the sample audio data; the pre-trained model is obtained by learning the fault audio of underground equipment.

6. An audio processing system, characterized in that: include: Multiple audio processing devices and monitoring platforms for downhole equipment; The multiple audio processing devices communicate data with the monitoring platform of the downhole equipment through wired cascading and / or wireless bridging; Among them, any one of the audio processing devices is used to pick up audio data underground, and the audio data includes the working audio data generated by the underground equipment during operation; the audio data is processed to determine the working status of the underground equipment, and the working status includes the normal operation of the underground equipment or the failure of the underground equipment; the working status of the underground equipment is sent to the monitoring platform of the underground equipment.

7. An audio processing method, characterized in that: Applied to an audio processing device, the method includes: Acquiring downhole audio data; the audio data includes working audio data generated by downhole equipment during operation; Processing the audio data to determine a working state of the downhole equipment; the working state includes a normal working state of the downhole equipment or a fault of the downhole equipment; The working status of the downhole equipment is sent to a monitoring platform of the downhole equipment.

8. An audio processing device, characterized in that: Applied to audio processing equipment, the device comprises: An acquisition unit, configured to acquire audio data downhole; the audio data includes working audio data generated by downhole equipment during operation; A determination unit, configured to process the audio data to determine a working state of the downhole equipment; the working state includes a normal working state of the downhole equipment or a fault of the downhole equipment; The sending unit is used to send the working status of the downhole equipment to the monitoring platform of the downhole equipment.

9. An electronic device, characterized in that: include: memory and processor; Wherein, the memory is used to store programs; The processor is configured to implement the method according to claim 7 by running the program in the memory.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to claim 7 is implemented.

Images