Audio processing methods, apparatus, devices, systems and computer program products

Abstract

This application discloses an audio processing method, apparatus, device, system, and computer program product. The audio processing device includes a processing component, and a pickup component and a communication component electrically connected to the processing component. The pickup component can pick up audio data from underground, including operational audio data generated by underground equipment during operation. The processor can process the audio data to determine the operating status of the underground equipment, including whether the equipment is operating normally or malfunctioning. The communication component can send the operating status of the underground equipment to a monitoring platform. This allows the audio processing device to analyze the underground audio data to determine if the equipment is malfunctioning and send the results to the monitoring platform, eliminating the need for a large number of inspection personnel and reducing operating costs for the enterprise.

Description

Technical Field

[0001] This application relates to the field of audio processing technology, and in particular to an audio processing method, apparatus, device, system and computer program product. Background Technology

[0002] Equipment malfunctions in underground mines are a major cause of accidents. Currently, equipment is mainly inspected manually at regular intervals to detect and resolve malfunctions promptly, ensuring safe underground operations. However, manual inspections require a large number of personnel, increasing operating costs for companies. Summary of the Invention

[0003] In view of this, this application proposes an audio processing method, apparatus, device, system, and computer program product to solve the problem that the existing manual inspection method requires a large number of inspection personnel, which increases the operating costs of enterprises.

[0004] The technical solution proposed in this application is as follows:

[0005] In a first aspect, embodiments of this application provide an audio processing device, comprising:

[0006] A processing component, and a pickup component and a communication component electrically connected to the processing component respectively;

[0007] The sound pickup component is used to pick up audio data from downhole; the audio data includes working audio data generated by downhole equipment during operation.

[0008] The processing component is used to process the audio data and determine the working status of the downhole equipment; the working status includes whether the downhole equipment is working normally or whether the downhole equipment is malfunctioning.

[0009] The communication component is used to send the working status of the downhole equipment to the monitoring platform of the downhole equipment.

[0010] Furthermore, the communication component is wired to the monitoring platform via one or more cascaded audio processing devices; and / or, the communication component is wirelessly bridged to the monitoring platform via one or more bridged audio processing devices.

[0011] Furthermore, the communication component is wirelessly bridged to the monitoring platform via one or more bridged audio processing devices; the communication component is also used for:

[0012] Based on the remaining bandwidth and remaining hop count of each network node in the wireless bridging network where the communication component is located, determine the network nodes that the communication component needs to pass through for data communication with the monitoring platform;

[0013] The network nodes include other audio processing devices in the wireless bridging network; the remaining hop count represents the number of network nodes between the network nodes and the monitoring platform.

[0014] Furthermore, the processing component processes the audio data, including:

[0015] The audio data is input into a pre-trained noise reduction model to obtain the processed audio data output by the noise reduction model. The noise reduction model is trained using sample audio data from the well as training samples, with the goal of separating the working audio of the well equipment from the sample audio data.

[0016] Furthermore, the processing component processes the audio data to determine the operating status of the downhole equipment, including:

[0017] The audio data is input into a pre-trained fault classification model to obtain the fault classification result output by the fault classification model. The fault classification model is obtained by fine-tuning the pre-trained model with the sample audio data from the well as the sample audio data as the training sample and the fault type corresponding to the sample audio data as the objective. The pre-trained model is obtained by learning the fault audio of the well equipment.

[0018] Secondly, embodiments of this application provide an audio processing system, including: multiple audio processing devices and a monitoring platform for downhole equipment;

[0019] The multiple audio processing devices communicate with the monitoring platform of the downhole equipment via wired cascading and / or wireless bridging.

[0020] Any one of the audio processing devices is used to pick up audio data from downhole, including working audio data generated by the downhole equipment during operation; the audio data is processed to determine the working status of the downhole equipment, including whether the downhole equipment is working normally or malfunctioning; and the working status of the downhole equipment is sent to the monitoring platform of the downhole equipment.

[0021] Thirdly, embodiments of this application provide an audio processing method applied to an audio processing device, the method comprising:

[0022] Acquire downhole audio data; the audio data includes working audio data generated by downhole equipment during operation;

[0023] The audio data is processed to determine the working status of the downhole equipment; the working status includes whether the downhole equipment is working normally or the downhole equipment is malfunctioning.

[0024] The operating status of the downhole equipment is sent to the monitoring platform of the downhole equipment.

[0025] Fourthly, embodiments of this application provide an audio processing apparatus, applied to an audio processing device, the apparatus comprising:

[0026] The acquisition unit is used to acquire downhole audio data; the audio data includes working audio data generated by downhole equipment during operation.

[0027] A determining unit is used to process the audio data and determine the working status of the downhole equipment; the working status includes whether the downhole equipment is working normally or whether the downhole equipment is malfunctioning;

[0028] The sending unit is used to send the working status of the downhole equipment to the monitoring platform of the downhole equipment.

[0029] Fifthly, embodiments of this application provide an electronic device, including:

[0030] A memory and a processor; wherein the memory is used to store a program; and the processor is used to implement the method described in any one of the above by running the program in the memory.

[0031] Sixthly, embodiments of this application provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the method described in any of the above-mentioned methods. Optionally, the computer program may be stored on a readable storage medium of a computer device or in the cloud; the processor of the computer device reads the computer program from the readable storage medium or the cloud.

[0032] The audio processing device proposed in this application includes a processing component, and a pickup component and a communication component electrically connected to the processing component. The pickup component can pick up audio data from underground, including operational audio data generated by the underground equipment during operation. The processor can process the audio data to determine the operating status of the underground equipment, including whether the equipment is operating normally or malfunctioning. The communication component can send the operating status of the underground equipment to the monitoring platform of the underground equipment. This allows the audio processing device to analyze the underground audio data to determine if the equipment is malfunctioning and send the results to the monitoring platform, eliminating the need for a large number of inspection personnel and reducing the company's operating costs. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0034] Figure 1 This is a schematic diagram of the structure of an audio processing device provided in an embodiment of this application.

[0035] Figure 2 This is a schematic diagram of a cascaded circuit provided in an embodiment of this application.

[0036] Figure 3 This is a schematic diagram of the structure of a wireless bridging line provided in an embodiment of this application.

[0037] Figure 4 This is a schematic diagram of another audio processing device provided in an embodiment of this application.

[0038] Figure 5 This is a schematic diagram of the hardware layer structure of the audio processing device provided in the embodiments of this application.

[0039] Figure 6 This is a schematic diagram of the pin structure of the aviation plug provided in the embodiment of this application.

[0040] Figure 7 This is a flowchart illustrating an audio processing method provided in an embodiment of this application.

[0041] Figure 8 This is a schematic diagram of the structure of an audio processing device provided in an embodiment of this application.

[0042] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0043] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0044] Equipment failure in underground mines is one of the major causes of underground accidents. It can not only lead to production interruptions, but also pose a serious threat to the lives of underground workers.

[0045] Currently, equipment inspections are primarily conducted manually on a regular schedule to promptly identify equipment malfunctions and eliminate potential hazards, ensuring the safety and continuity of underground operations. However, manual inspections have several limitations. For example, inspection personnel need to frequently enter the underground environment, facing complex geological conditions and potential safety risks, resulting in high workload and danger. Moreover, manual inspections are inefficient, have long inspection cycles, and struggle to detect subtle anomalies in equipment operation in a timely manner, potentially leading to the accumulation of hidden faults and ultimately serious accidents. Furthermore, manual inspections require a large number of personnel, which not only increases the company's operating costs but may also lead to a waste of human resources. This is especially true in large mines or deep-well operations, where the inspection scope is wide and the types of equipment are diverse, further increasing the difficulty and cost of manual inspections.

[0046] Therefore, although manual inspection still plays an important role in current downhole safety management, its limitations are becoming increasingly apparent, necessitating the exploration of more efficient and safer equipment monitoring methods. Based on this, this application proposes an audio processing method, apparatus, device, system, and computer program product. This technical solution utilizes audio processing equipment to analyze downhole audio data to determine whether downhole equipment is malfunctioning, and sends the results to the downhole equipment monitoring platform, achieving a more efficient and safer monitoring effect for downhole equipment.

[0047] Figure 1 A schematic diagram of the audio processing device is shown, such as... Figure 1 As shown, the audio processing device includes a processing component 10, and a pickup component 11 and a communication component 12, which are electrically connected to the processing component 10, respectively.

[0048] The sound pickup component 11 is used to pick up audio data from downhole, including working audio data generated by downhole equipment during operation and downhole noise. A microphone can be used as the sound pickup component 11; more specifically, for complex downhole noise environments, such as mechanical vibration and electromagnetic interference, a dual-microphone linear array can be used as the sound pickup component 11, employing dual-microphone spatial filtering technology, which significantly improves the signal-to-noise ratio compared to traditional single-microphone solutions. In some embodiments, the dual-microphone linear array is fixed at a 35° angle within an explosion-proof enclosure, supporting dynamic beamforming to adapt to sound source localization in confined downhole spaces.

[0049] Processing component 10 processes the aforementioned audio data to determine the working status of the downhole equipment; the working status includes normal operation of the downhole equipment or downhole equipment malfunction. In some embodiments, working audio data of the downhole equipment under different malfunction types can be collected in advance to construct a working audio database corresponding to different malfunction types. After obtaining the downhole audio data, the similarity between the audio data and the working audio corresponding to different malfunction types is calculated. If the similarity between the audio data and the working audio corresponding to each malfunction type is less than a set threshold, it indicates that the working status of the downhole equipment includes normal operation; if, among the working audio corresponding to each malfunction type, there is a target working audio with a similarity greater than or equal to the set threshold, it indicates that the working status of the downhole equipment includes downhole equipment malfunction, and the malfunction type is the malfunction type corresponding to the target working audio.

[0050] The threshold values ​​mentioned above can be set according to actual conditions, and this embodiment does not impose any limitations.

[0051] The communication component 12 is used to send the operating status of the downhole equipment to the monitoring platform of the downhole equipment. In some embodiments, when the operating status of the downhole equipment includes normal operation, the communication component 12 may not send the operating status of the downhole equipment to the monitoring platform. When the operating status of the downhole equipment includes a downhole equipment malfunction, the communication component 12 then sends the downhole equipment malfunction status to the monitoring platform, so that the staff can promptly detect the downhole equipment malfunction and carry out maintenance to avoid affecting production or causing accidents.

[0052] This setup allows for the analysis of underground audio data using audio processing equipment to determine if the underground equipment is malfunctioning, and the results are sent to the monitoring platform of the underground equipment. This eliminates the need for a large number of inspection personnel, reducing the company's operating costs.

[0053] As an optional implementation, another embodiment of this application discloses that the communication component is wiredly connected to the monitoring platform through one or more cascaded audio processing devices; and / or, the communication component is wirelessly bridged to the monitoring platform through one or more bridged audio processing devices.

[0054] More specifically, the communication component is wired to the monitoring platform through cascaded audio processing equipment. The communication component can also be directly wired to the monitoring platform, but this embodiment does not limit this.

[0055] like Figure 2 As shown, each cascaded line contains six audio processing devices. It should be noted that... Figure 2The embodiments shown are examples provided to explain the technical solutions and are not limited to a maximum of 6 audio processing devices in each cascaded line. The number of audio processing devices in each cascaded line can be determined according to the actual situation.

[0056] In some embodiments, the audio processing equipment is powered by a dual-path intrinsically safe power supply for mining, with six audio processing devices installed in each cascaded line, enabling fault monitoring of underground equipment within a range of up to 100 meters.

[0057] This setup allows for wired cascading of audio processing devices, improving signal transmission efficiency.

[0058] The communication component is wirelessly bridged to the monitoring platform via the bridged audio processing device. The communication component can also be directly wirelessly bridged to the monitoring platform; this embodiment does not limit this.

[0059] like Figure 3 As shown, each wireless bridging line contains three audio processing devices. It should be noted that... Figure 3 The embodiments shown are examples provided to illustrate the technical solutions and do not limit each wireless bridging line to only 3 audio processing devices. The number of audio processing devices in each wireless bridging line can be determined according to the actual situation.

[0060] In some embodiments, by wirelessly bridging the audio processing devices, up to three audio processing devices can be bridged at once. Data can be transmitted wirelessly simply by powering the audio processing devices, enabling fault monitoring of downhole equipment within a range of up to 400 meters.

[0061] This setup allows for wireless bridging of audio processing devices, avoiding the limitations imposed by data cables and increasing the deployment range of audio processing equipment.

[0062] In addition, a hybrid deployment of wired cascading and wireless bridging can be adopted. Wired cascading can cover the core area, while wireless bridging can extend to blind spots, breaking through the distance limitations of traditional single transmission methods and improving the detection range.

[0063] In some embodiments, the monitoring platform includes a downhole server, a surface server, a cloud server, and a fault monitoring system. The audio processing device is wired-cascaded or wirelessly bridged to the downhole server. The audio processing device sends the operating status of the downhole equipment to the downhole server, which in turn sends it to the surface server. The surface server then transmits the operating status of the downhole equipment to the monitoring system via the cloud server. Personnel at the monitoring system can promptly understand the operating status of the downhole equipment and respond quickly when a fault occurs.

[0064] As an optional implementation, another embodiment of this application discloses that the communication component wirelessly bridges with the monitoring platform through one or more bridged audio processing devices. The communication component is further configured to: determine the network nodes that the communication component needs to traverse for data communication with the monitoring platform based on the remaining bandwidth and remaining hop count of each network node in the wireless bridging network where the communication component resides.

[0065] When the communication component is wirelessly bridged to the monitoring platform through one or more bridged audio processing devices, it is necessary to determine the specific path for the audio processing device to transmit signals to the monitoring platform based on the actual network status of the wireless bridging network.

[0066] It can detect the remaining bandwidth and remaining hop count of each network node in the wireless bridging network where the communication component resides, and determine the network nodes that the communication component needs to traverse to communicate with the monitoring platform. Network nodes include other audio processing devices in the wireless bridging network; the remaining hop count indicates the number of network nodes between the network node and the monitoring platform.

[0067] Specifically, the weight of each network node is first determined based on its remaining bandwidth and remaining hop count. Higher remaining bandwidth corresponds to a higher weight, and fewer remaining hop counts correspond to a higher weight. The network node with the highest weight is selected as the next hop. Then, using this next hop as a reference, the network node with the highest weight is selected again as the next hop, until the connection to the monitoring platform is reached. All network nodes traversed by the communication components for data communication with the monitoring platform constitute the path for the audio processing device to transmit signals to the monitoring platform.

[0068] More specifically, the protocol stack design for wireless bridging networks can be based on the WiFi 6 (802.11ax) Mesh networking enhancement protocol.

[0069] In this embodiment, the frame structure is optimized, including: the control frame adds a "cascaded status field", which is in JSON nested TLV format and includes node load, channel quality, and hop count; the data frame supports aggregated transmission (A-MPDU) and a maximum of 4KB of voiceprint data per frame.

[0070] The relay logic includes: nodes periodically broadcasting Hello packets and maintaining a neighbor table, including the signal strength (RSSI) of neighboring nodes and the remaining power of neighboring nodes. The node period can be selected at 5-second intervals.

[0071] Based on the remaining bandwidth and remaining hop count of each network node, the weight of each network node is determined, and the network node with the highest weight is selected as the next hop. In some embodiments, the weight W of each network node is determined according to the following formula. ij :

[0072] W ij = Remaining bandwidth / 10 + 1 / number of hops

[0073] This embodiment also includes a retransmission mechanism, namely dynamic backoff based on a contention window. For example, the minimum value of the contention window, CW... min Set to 15, the maximum value of the competition window CW max Set to 1023, the maximum number of retries is 3.

[0074] With this bridging network setup, the maximum single hop distance is 130m, and the throughput remains greater than or equal to 80Mbps even when there are 3 Bluetooth interference sources in the 2.4GHz band.

[0075] As an optional implementation, another embodiment of this application discloses that the processing component of the above embodiments processes audio data, specifically including:

[0076] The audio data is input into a pre-trained noise reduction model to obtain the processed audio data output by the noise reduction model. The noise reduction model is trained using sample audio data from the well as training samples and with the goal of separating the working audio of the well equipment from the sample audio data.

[0077] In this embodiment, a noise reduction model is pre-trained. The noise reduction model is used to process the audio data, which can adaptively reduce the noise of the audio data and improve the signal-to-noise ratio.

[0078] A large amount of downhole audio data can be acquired as training samples, and the operating audio of downhole equipment in the sample audio data can be used as training labels. During training, the training samples are input into the denoising model to obtain the prediction results output by the denoising model. By comparing the prediction results output by the denoising model with the training labels, the loss value of the denoising model is determined. The parameters of the denoising model are adjusted with the goal of reducing the loss value of the denoising model. Then, the above training process is repeated until all the parameters of the denoising model meet the requirements. The above denoising model can be trained based on an attention mechanism, such as Transformer, etc., and this embodiment is not limited to this.

[0079] The audio data is input into a pre-trained noise reduction model, which can separate device noise from environmental noise based on an attention mechanism, and obtain the processed audio data output by the noise reduction model.

[0080] This setup allows for adaptive noise reduction using a noise reduction model, improving the signal-to-noise ratio and thus enhancing the reliability of fault identification results.

[0081] As an optional implementation, another embodiment of this application discloses that the processing component of the above embodiments processes audio data to determine the working status of the downhole equipment, specifically including:

[0082] Audio data is input into a pre-trained fault classification model to obtain the fault classification result output by the fault classification model. The fault classification model is obtained by fine-tuning the pre-trained model with the sample audio data from the well as the goal of identifying the fault type corresponding to the sample audio data. The pre-trained model is obtained by learning the fault audio of the well equipment.

[0083] In this embodiment, a fault classification model is pre-trained. This model is used to process the audio data, enabling the determination of the fault type in the downhole equipment. In some embodiments, the audio data is first adaptively denoised using the denoising model described above, and then the denoised audio data is input into the fault classification model for fault classification. This reduces noise interference and improves the reliability of the classification results.

[0084] A large amount of downhole sample audio data can be obtained as training samples, and the fault type corresponding to the sample audio data can be used as training labels. In some embodiments, if noise-reduced audio data is used for fault classification, a large amount of downhole sample audio data can be obtained, and adaptive noise reduction processing can be performed on the sample audio data according to the description in the above embodiments, and the processed audio data can be used as training samples.

[0085] In some embodiments, the training samples need to be preprocessed before fine-tuning begins. Specifically, the training samples can be first divided into frames and windowed. For example, the frame length is 32ms and the frame shift is 10ms, and a Hamming window can be used for windowing.

[0086] Perform a Short-Time Fourier Transform (STFT) or a Fast Fourier Transform (FFT) to generate a spectrum. In some embodiments, the signal with 1024 sampling points is transformed from the time domain to the frequency domain to generate a spectrum.

[0087] Then, Mel-ray filtering is performed. Specifically, based on the principle of nonlinear distribution of the Mel-ray scale, filters are densely arranged in the low-frequency region (100Hz–4kHz) and sparsely arranged in the high-frequency region (4kHz–96kHz) to match the energy concentration characteristics of equipment fault sound patterns. In some embodiments, 25 filters are set in the low-frequency region and 15 filters are set in the high-frequency region. Among them, 40 filters, compared with the traditional 24 filters, improve the feature discrimination of roller bearing wear identification task by 18%.

[0088] Finally, log compression and discrete cosine transform (DCT) are performed to achieve data compression and energy conversion.

[0089] During fine-tuning, training samples are input into the fault classification model to obtain the predicted results output by the model. By comparing the predicted results with the training labels, the loss value of the fault classification model is determined. The parameters of the model are adjusted to reduce the loss value, and this fine-tuning process is repeated until all parameters meet the requirements. The fault classification model described above can be trained based on a pre-trained model.

[0090] The pre-trained model is obtained by learning from the fault audio of downhole equipment. Specifically, a fault audioprint database covering downhole equipment such as conveyors, idlers, and pumps can be established. This database is used to pre-train the base model, enabling it to classify and identify downhole equipment faults. The pre-trained model is then fine-tuned based on sample audio data from downhole equipment; that is, its generalization ability is optimized through transfer learning, thereby improving recognition accuracy. The base model can be based on any neural network model; this embodiment does not impose any limitations.

[0091] Audio data is input into a pre-trained fault classification model, which can determine the fault type of downhole equipment based on the audio data.

[0092] In some embodiments, the aforementioned base model employs a lightweight deep neural network (DNN) model architecture. More specifically, the DNN model architecture in this embodiment includes a 3-layer convolutional neural network (CNN), a 2-layer long short-term memory (LSTM) network, and connection layers.

[0093] CNN layer: Conv1D×3, number of filters [64, 128, 256], kernel width [5, 3, 3], stride 2, ReLU activation. The CNN layer is used to extract local frequency domain patterns, such as periodic abnormal noises from bearings.

[0094] LSTM layer: Bi-LSTM×2, 128 hidden units, dropout=0.3. The LSTM layer is used to capture long-term contextual associations of fault sound patterns, such as transient features of pump start-up.

[0095] Output layer: Softmax classification.

[0096] This setup allows for the fine-tuning of the pre-trained model to obtain a fault classification model, effectively improving recognition accuracy. Simultaneously, the use of a lightweight algorithm model reduces overall power consumption, extending the device's lifespan. Furthermore, embedding a lightweight model within the audio processing device enables real-time extraction and classification of fault features, reducing reliance on cloud servers and lowering response time.

[0097] As an optional implementation method, such as Figure 4 As shown in another embodiment of this application, the audio processing device of the above embodiments may further include an environmental sensor 13 electrically connected to the processing component. The environmental sensor 13 can detect parameters such as downhole temperature, humidity, and air pressure, and send these parameters to the processing component 10. The processing component 10 adjusts the fault judgment threshold in real time according to changes in downhole environmental temperature, humidity, and air pressure to avoid false alarms. For example, it automatically enhances low-frequency filtering when the humidity is greater than 90%.

[0098] Audio processing devices can be divided into an interaction layer, an algorithm layer, a driver layer, and a hardware layer.

[0099] The interaction layer includes an edge gateway and an IP configuration interface. The interaction layer accesses the device's default IP address via a web interface through wired or wireless connections, allowing login to the configuration interface. This configuration interface includes settings for audio processing devices, network configuration, transmission configuration, audio configuration, system configuration, and OTA upgrades. The algorithm layer includes the adaptive noise reduction and fault classification algorithms described in the above embodiments, as well as model training methods. The driver layer uses the Linux operating system and includes drivers for PDM, SDMMC, Flash memory, DDR, IIC, I2S, GMAC, and Wi-Fi.

[0100] Hardware layer such as Figure 5 As shown, the Quad-core Corex-A53 architecture is used as the processing component, the microphone component uses a dual-microphone linear array, and temperature and humidity sensors and barometric pressure sensors are used as environmental sensors.

[0101] The dual-microphone linear array interacts with the Quad-core Corex-A53 architecture via CLOCK / PDM signals; the temperature and humidity sensor interacts with the Quad-core Corex-A53 architecture via the IIC serial communication protocol; and the barometric pressure sensor interacts with the Quad-core Corex-A53 architecture via the I2S communication protocol. The memory device uses DDR4×16bit, eMMC, and a built-in TF card. Communication components include dual-band Wi-Fi 6 and a SEWITCH PHY chip.

[0102] Audio processing equipment also includes power chips, and to ensure electrical safety, it is equipped with isolated network transformers and isolated power supplies, such as... Figure 5 As shown.

[0103] Standard aviation connectors can also be used to further ensure safety. In some embodiments, the pin structure of the aviation connector is as follows: Figure 6 As shown, the plug includes a power supply port and a network communication port. The devices can be cascaded through specially customized coal safety cables. The definitions of each pin can be found in Table 1.

[0104] connector pins definition 1 TX+ 2 TX- 3 RX+ 4 RX- 5 negative terminal of power supply 6 Positive power supply 7 null 8 null

[0105] Table 1

[0106] The audio processing equipment features an explosion-proof housing and meets an IP67 waterproof rating. It can be secured to the location requiring testing using magnetic attachments, adhesive tape, screws, or clamps.

[0107] This embodiment utilizes a dual-microphone linear array for sound source acquisition, combined with an intelligent noise reduction algorithm, to capture abnormal sounds from underground equipment. It can also optionally incorporate algorithm models specific to the coal mining industry, forming an integrated intelligent monitoring device encompassing sound acquisition, status assessment, and anomaly diagnosis. Compared to other types of microphones, it boasts a high level of intelligence and multi-dimensional environmental perception capabilities. Furthermore, it integrates gateway functionality, enabling 24 / 7 online monitoring, saving companies on manual inspection costs and creating a safe, intelligent, and healthy underground environment.

[0108] This application also discloses an audio processing system, including: multiple audio processing devices and a downhole equipment monitoring platform.

[0109] Multiple audio processing devices communicate with the monitoring platform of the downhole equipment via wired cascading and / or wireless bridging. Each audio processing device is used to pick up audio data from the downhole equipment, including operational audio data generated during operation; process the audio data to determine the operating status of the downhole equipment, including whether the equipment is operating normally or malfunctioning; and then transmit the operating status of the downhole equipment to the monitoring platform.

[0110] For specific limitations on each audio processing device in this audio processing system, please refer to the limitations on audio processing devices mentioned above, which will not be repeated here.

[0111] As an optional implementation, another embodiment of this application discloses that the audio processing devices in the audio processing system use a load allocation strategy to distribute the load. Specifically, this includes static allocation and dynamic adjustment.

[0112] Static allocation: Set the maximum power consumption of each audio processing device to be less than a set value, for example, less than or equal to 2W. Static allocation can be achieved through the internal current limiting circuit of the audio processing device, which is not limited in this implementation.

[0113] Dynamic adjustment: The power chip can monitor the current of each audio processing device in real time. If the power of a certain audio processing device exceeds the first threshold, the power consumption of non-core functions will be automatically reduced, such as turning off the wireless module and prioritizing voiceprint acquisition. If the power of a certain audio processing device exceeds the second threshold, the fuse will cut off the power supply and trigger a local alarm.

[0114] The second threshold is greater than the first threshold. The size of the first and second thresholds can be set according to the actual situation. For example, the first threshold can be set to 1.8W and the second threshold can be set to 2.1W.

[0115] In addition, the cascaded bus uses magnetic coupling isolation, allowing nodes to be added or removed during operation.

[0116] In one specific embodiment, audio processing equipment was deployed in a 300-meter-long coal mine conveyor roadway. The deployment consisted of four audio processing devices: three wired cascaded and one wirelessly bridged, with dynamic load distribution enabled. The power consumption of each node remained stable between 1.5 and 1.8W. The end-to-end latency was less than or equal to 50ms for the wired cascaded segment and less than or equal to 120ms for the wireless bridged segment. The fault identification accuracy rate was 95.3% after 72 hours of continuous operation testing.

[0117] Furthermore, this application provides an audio processing method, which can be executed by the audio processing device described in the above embodiments. See also... Figure 7 As shown, the method includes:

[0118] S101. Obtain audio data from downhole;

[0119] S102. Process the audio data to determine the working status of the downhole equipment;

[0120] S103. Send the working status of the downhole equipment to the monitoring platform of the downhole equipment.

[0121] The audio data includes the working audio data generated by the downhole equipment during operation, and the working status includes whether the downhole equipment is working normally or malfunctioning.

[0122] For specific limitations of this method, please refer to the limitations on audio processing devices mentioned above, which will not be repeated here.

[0123] Corresponding to the above-described audio processing method, this application also discloses an audio processing apparatus, see [link to relevant documentation]. Figure 8 As shown, the device includes:

[0124] The acquisition unit 100 is used to acquire downhole audio data; the audio data includes working audio data generated by downhole equipment during operation;

[0125] The determination unit 110 is used to process audio data and determine the working status of the downhole equipment; the working status includes whether the downhole equipment is working normally or malfunctioning.

[0126] The sending unit 120 is used to send the working status of the downhole equipment to the monitoring platform of the downhole equipment.

[0127] Specifically, the apparatus provided in this embodiment belongs to the same concept as the method provided in the above embodiments of this application, and can execute the method provided in any of the above embodiments of this application, possessing the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in the specific processing content of the method provided in the above embodiments of this application, and will not be repeated here.

[0128] The functions implemented by the above units can be implemented by the same or different processors, and this application embodiment does not limit this.

[0129] It should be understood that the units in the above device can be implemented by a processor calling software. For example, the device includes a processor connected to a memory containing instructions. The processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of each unit in the device. The processor can be a general-purpose processor, such as a CPU or microprocessor, and the memory can be internal or external to the device. Alternatively, the units in the device can be implemented as hardware circuits. By designing the hardware circuits, some or all of the unit functions can be implemented. The hardware circuits can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are implemented by designing the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a PLD, such as an FPGA, which can include a large number of logic gates. The connection relationships between the logic gates are configured through configuration files to implement the functions of some or all of the above units. All units in the above device can be implemented entirely by a processor calling software, entirely by hardware circuits, or partially by a processor calling software with the remaining parts implemented by hardware circuits.

[0130] In this application embodiment, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU, or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the above units. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.

[0131] As can be seen, each unit in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0132] Furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a System-on-Chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.

[0133] This application also proposes a control device, which includes a processor and an interface circuit. The processor in the control device is connected to an input / output component through the interface circuit of the control device.

[0134] The input / output component specifically refers to the hardware component that enables the user to input information and output information to the user, such as a microphone, keyboard, handwriting tablet, touch screen, monitor, speaker, printer, etc.

[0135] The aforementioned interface circuit can be any interface circuit capable of implementing data communication functions, such as a USB interface circuit, a Type-C interface circuit, a serial port circuit, a PCIe circuit, etc.

[0136] The processor in this control device is a circuit with signal processing capabilities. By executing any of the audio processing methods described in the above embodiments, it achieves a more efficient and safer monitoring effect on the downhole equipment. The specific implementation of this processor can be found in the processor implementation methods described above; this application does not impose strict limitations on the embodiments.

[0137] When the control device is applied to a device with human-computer interaction function, the input and output components of the control device can be input components and output components on the device, such as microphone, keyboard, handwriting tablet, touch screen, display, audio player, etc. Meanwhile, the processor of the control device can be the device's built-in CPU or GPU, etc., and the interface circuit of the control device can be the interface circuit between the device's information input components and the CPU or GPU processor.

[0138] Corresponding to the above audio processing method, this application also discloses an electronic device, see [link to relevant documentation]. Figure 9 As shown, the electronic device includes:

[0139] Memory 200 and processor 210;

[0140] The memory 200 is connected to the processor 210 and is used to store programs;

[0141] The processor 210 is configured to implement the audio processing method disclosed in any of the above embodiments by running a program stored in the memory 200.

[0142] Specifically, the aforementioned electronic device may further include: a bus, a communication interface 220, an input device 230, and an output device 240.

[0143] The processor 210, memory 200, communication interface 220, input device 230, and output device 240 are interconnected via a bus. Among them:

[0144] A bus can include a pathway for transmitting information between various components of a computer system.

[0145] The processor 210 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0146] Processor 210 may include a main processor, as well as a baseband chip, modem, etc.

[0147] The memory 200 stores a program for executing the technical solution of this application, and may also store an operating system and other critical business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 200 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0148] Input device 230 may include a device for receiving user input data and information, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0149] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0150] The communication interface 220 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0151] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement the various steps of the audio processing method provided in the above embodiments of this application.

[0152] In addition to the methods and devices described above, embodiments of this application can also be computer program products, which include computer programs. When the computer programs are run by a processor, they can execute the audio processing methods provided in any of the above embodiments of this application. Optionally, the computer programs can be stored on a readable storage medium of a computer device or in the cloud; the processor of the computer device reads the computer programs from the readable storage medium or the cloud.

[0153] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0154] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0155] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the various steps of the audio processing method provided in the above embodiments.

[0156] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0157] Specifically, the specific working content of each part of the aforementioned electronic device, computer program product, and storage medium, as well as the specific processing content of the computer program product or the computer program on the aforementioned storage medium when run by the processor, can all be found in the various embodiments of the aforementioned audio processing method, and will not be repeated here.

[0158] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0159] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0160] The steps in the methods of the various embodiments of this application can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in each embodiment can be replaced or combined.

[0161] The modules and sub-modules in the apparatus and terminal in the various embodiments of this application can be merged, divided, and deleted according to actual needs.

[0162] It should be understood, in the several embodiments provided in this application, that the disclosed terminals, devices, and methods can be implemented in other ways. For example, the terminal embodiments described above are merely illustrative; for instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0163] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0164] Furthermore, the functional modules or sub-modules in the various embodiments of this application can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or in the form of software functional modules or sub-modules.

[0165] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0166] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software unit executed by a processor, or a combination of both. The software unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0167] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0168] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An audio processing device, characterized in that, include: A processing component, and a pickup component and a communication component electrically connected to the processing component respectively; The sound pickup component is used to pick up audio data from downhole; the audio data includes working audio data generated by downhole equipment during operation. The processing component is used to process the audio data and determine the working status of the downhole equipment; the working status includes whether the downhole equipment is working normally or whether the downhole equipment is malfunctioning. The communication component is wirelessly bridged to the monitoring platform of the downhole equipment via one or more bridged audio processing devices. It is used to determine the path for the audio processing device to transmit signals to the monitoring platform based on the remaining bandwidth and remaining hop count of each network node in the wireless bridging network where the communication component is located, and to send the working status of the downhole equipment to the monitoring platform through the path. The network nodes include other audio processing devices in the wireless bridging network. The remaining hop count represents the number of network nodes between the network nodes and the monitoring platform.

2. The audio processing device according to claim 1, characterized in that, The processing component processes the audio data, including: The audio data is input into a pre-trained noise reduction model to obtain the processed audio data output by the noise reduction model. The noise reduction model is trained using sample audio data from downhole as training samples, with the goal of separating the working audio of downhole equipment from the sample audio data.

3. The audio processing device according to claim 1, characterized in that, The processing component processes the audio data to determine the operating status of the downhole equipment, including: The audio data is input into a pre-trained fault classification model to obtain the fault classification result output by the fault classification model; The fault classification model is obtained by fine-tuning a pre-trained model using sample audio data from downhole as training samples and identifying the fault type corresponding to the sample audio data as the objective; the pre-trained model is obtained by learning from the fault audio of downhole equipment.

4. An audio processing system, characterized in that, include: A monitoring platform for multiple audio processing devices and downhole equipment; The multiple audio processing devices communicate with the monitoring platform of the downhole equipment via wireless bridging. In this system, any one of the audio processing devices is used to pick up audio data from downhole, including operational audio data generated by the downhole equipment during operation. The audio data is processed to determine the operational status of the downhole equipment, which includes whether the equipment is operating normally or malfunctioning. Based on the remaining bandwidth and hop count of each network node in the wireless bridging network, a path for transmitting signals to the monitoring platform is determined, and the operational status of the downhole equipment is sent to the monitoring platform through this path. The network nodes include other audio processing devices in the wireless bridging network. The remaining hop count represents the number of network nodes between the network nodes and the monitoring platform.

5. An audio processing method, characterized in that, The method is applied to an audio processing device, which communicates with a monitoring platform of downhole equipment via wireless bridging; the method includes: Acquire downhole audio data; the audio data includes working audio data generated by the downhole equipment during operation; The audio data is processed to determine the working status of the downhole equipment; the working status includes whether the downhole equipment is working normally or the downhole equipment is malfunctioning. Based on the remaining bandwidth and remaining hop count of each network node in the wireless bridging network, a path for transmitting signals to the monitoring platform is determined, and the working status of the downhole equipment is sent to the monitoring platform through this path; the network nodes include other audio processing devices in the wireless bridging network; the remaining hop count represents the number of network nodes between the network nodes and the monitoring platform.

6. An audio processing apparatus, characterized in that, An audio processing device is used to communicate with a monitoring platform of downhole equipment via wireless bridging; the device includes: An acquisition unit is used to acquire downhole audio data; the audio data includes working audio data generated by the downhole equipment during operation. A determining unit is used to process the audio data and determine the working status of the downhole equipment; the working status includes whether the downhole equipment is working normally or whether the downhole equipment is malfunctioning; The transmitting unit is used to determine the path for transmitting signals to the monitoring platform based on the remaining bandwidth and remaining hop count of each network node in the wireless bridging network, and to send the working status of the downhole equipment to the monitoring platform through the path; the network nodes include other audio processing devices in the wireless bridging network; the remaining hop count represents the number of network nodes between the network nodes and the monitoring platform.

7. An electronic device, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is configured to implement the method of claim 5 by running a program in the memory.

8. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method as described in claim 5.

Images