Coal and rock identification methods and devices

Abstract

This disclosure proposes a method and apparatus for coal and rock identification, relating to the field of computer technology. The method includes: monitoring a coal mining machine using a sound sensor to acquire a test audio signal and multiple reference audio features; extracting features from the test audio signal to acquire a first audio feature; determining a second audio feature from the multiple reference audio features based on the similarity between the first audio feature and the multiple reference audio features; and determining a first predicted category of the coal and rock cut by the coal mining machine based on the category to which the second audio feature belongs, wherein the first predicted category indicates coal or rock. Therefore, automatic identification of the coal and rock cut by the coal mining machine corresponding to the test audio signal can be achieved based on the similarity between the audio features of the test audio signal of the coal and rock cut by the coal mining machine and multiple reference audio features of known categories, i.e., identifying whether the coal mining machine is cutting coal or rock.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method and apparatus for coal and rock identification. Background Technology

[0002] Coal is one of my country's important resources. With the continuous development of coal mining technology, coal mining operations are moving towards less manpower and more intelligent processes.

[0003] Coal and rock identification refers to the process of automatically identifying coal or rock objects using relevant methods. This identification allows for a more accurate assessment of the cutting force of coal mining machines, which is crucial for ensuring the quality of the output coal.

[0004] In order to enable coal mine workers to obtain timely and accurate information about the state of coal and rock cutting by the coal mining machine, that is, to determine whether the coal mining machine is currently cutting coal or rock, it is very important to achieve automatic coal and rock identification. Summary of the Invention

[0005] This disclosure aims to at least partially address one of the technical problems in the related art.

[0006] This disclosure proposes a coal and rock identification method and apparatus, which can acquire the audio features of the audio signal to be tested when a coal mining machine cuts coal and rock based on deep learning technology. Thus, based on the similarity between the audio features of the audio signal to be tested and multiple reference audio features of known categories, the method can automatically identify whether the coal mining machine is cutting coal or rock.

[0007] The first aspect of this disclosure provides a method for coal and rock identification, including:

[0008] The coal mining machine is monitored by a sound sensor to obtain the audio signal to be tested and to acquire multiple reference audio features;

[0009] Feature extraction is performed on the audio signal to be tested to obtain the first audio feature;

[0010] A second audio feature is determined from the plurality of reference audio features based on the similarity between the first audio feature and the plurality of reference audio features;

[0011] Based on the category to which the second audio feature belongs, a first predicted category of the coal and rock cut by the coal mining machine is determined, wherein the first predicted category is used to indicate coal or rock.

[0012] The coal and rock identification method of this disclosure monitors a coal mining machine using a sound sensor to acquire a test audio signal and multiple reference audio features. It then extracts features from the test audio signal to obtain a first audio feature. Based on the similarity between the first audio feature and the multiple reference audio features, a second audio feature is determined from the multiple reference audio features. Finally, based on the category to which the second audio feature belongs, a first predicted category of the coal and rock cut by the coal mining machine is determined, whereby the first predicted category indicates coal or rock. Thus, based on deep learning technology, the audio features of the test audio signal of the coal mining machine cutting coal and rock can be acquired. This allows for automatic identification of the coal and rock cut by the coal mining machine during operation, i.e., identifying whether the coal mining machine is cutting coal or rock, based on the similarity between the audio features of the test audio signal and multiple reference audio features of known categories.

[0013] A second aspect of this disclosure provides a coal and rock identification device, comprising:

[0014] The first processing module is used to monitor the coal mining machine through a sound sensor to obtain the audio signal to be tested and to acquire multiple reference audio features;

[0015] The first extraction module is used to extract features from the audio signal to be tested in order to obtain first audio features;

[0016] The first determining module is configured to determine a second audio feature from the plurality of reference audio features based on the similarity between the first audio feature and the plurality of reference audio features;

[0017] The second determining module is used to determine a first predicted category of the coal and rock cut by the coal mining machine based on the category to which the second audio feature belongs, wherein the first predicted category is used to indicate coal or rock.

[0018] The coal and rock identification device of this embodiment monitors a coal mining machine using a sound sensor to acquire a test audio signal and multiple reference audio features. It then extracts features from the test audio signal to obtain a first audio feature. Based on the similarity between the first audio feature and the multiple reference audio features, a second audio feature is determined from the multiple reference audio features. Finally, based on the category to which the second audio feature belongs, a first predicted category of the coal and rock cut by the coal mining machine is determined, whereby the first predicted category indicates coal or rock. Thus, based on deep learning technology, the audio features of the test audio signal of the coal mining machine cutting coal and rock can be acquired. This allows for automatic identification of the coal and rock cut by the coal mining machine corresponding to the test audio signal, i.e., identifying whether the coal mining machine is cutting coal or rock, based on the similarity between the audio features of the test audio signal and multiple reference audio features of known categories.

[0019] A third aspect of this disclosure provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the coal and rock identification method as proposed in the first aspect of this disclosure.

[0020] The fourth aspect of this disclosure provides a non-transitory computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the coal and rock identification method as proposed in the first aspect of this disclosure.

[0021] The fifth aspect of this disclosure provides a computer program product that, when executed by a processor, performs the coal and rock identification method as described in the first aspect of this disclosure.

[0022] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0023] The above and / or additional aspects and advantages of this disclosure will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which:

[0024] Figure 1 This is a schematic flowchart of the coal and rock identification method provided in Embodiment 1 of this disclosure;

[0025] Figure 2 This is a schematic flowchart of the coal and rock identification method provided in Embodiment 2 of this disclosure;

[0026] Figure 3 This is a schematic flowchart of the coal and rock identification method provided in Embodiment 3 of this disclosure;

[0027] Figure 4 This is a schematic flowchart of the coal and rock identification method provided in Embodiment 4 of this disclosure;

[0028] Figure 5 This is a schematic diagram of the coal and rock identification process provided in this disclosure;

[0029] Figure 6 This is a schematic diagram of the coal and rock identification device provided in Embodiment 5 of this disclosure. Detailed Implementation

[0030] Embodiments of this disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure.

[0031] The audio signals of coal mining equipment (such as coal mining machines) during operation are closely related to the mechanical structure and working state of the equipment. When the condition of parts or components of the equipment changes after operation, their audio signal characteristics will also change accordingly. Therefore, the audio signals of coal mining equipment during operation can serve as an important indicator for analyzing the operating status of the equipment. Monitoring methods based on audio signals are of great significance for coal and rock identification technology in coal mining machine operation. Related technologies involve collecting coal and rock image signals, boom vibration signals, and motor current signals from the coal mining machine. Based on the information collected, the state of coal and rock cutting by the coal mining machine is determined, that is, whether the machine is currently cutting coal or rock.

[0032] However, image signals are easily obscured by dust, affecting the accuracy of coal and rock condition identification results, while mechanical vibration and motor current signals are indirect monitoring methods and are not direct or efficient.

[0033] Considering that the audio signal of the coal mining machine cutting coal and rock is strong and has a high signal-to-noise ratio, which is beneficial to the coal and rock identification task, this disclosure can identify the state of coal and rock cutting by the coal mining machine based on the audio signal of the coal mining machine during operation.

[0034] The coal and rock identification method and apparatus of this disclosure are described below with reference to the accompanying drawings.

[0035] Figure 1 This is a schematic flowchart of the coal and rock identification method provided in Embodiment 1 of this disclosure.

[0036] This disclosure illustrates the example of the coal and rock identification method being configured in a coal and rock identification method device. This coal and rock identification method device can be applied to any electronic device so that the electronic device can perform the coal and rock identification method function.

[0037] Among them, electronic devices can be any device with computing capabilities, such as PCs (Personal Computers), industrial computers, host computers, mobile terminals, servers, etc. Mobile terminals can be hardware devices with various operating systems, touch screens and / or displays, such as mobile phones, tablets, personal digital assistants, wearable devices, etc.

[0038] like Figure 1 As shown, the coal and rock identification method may include the following steps:

[0039] Step 101: Monitor the coal mining machine using a sound sensor to obtain the audio signal to be tested and acquire multiple reference audio features.

[0040] In this embodiment of the disclosure, the audio signal to be measured can be an audio signal obtained by a sound sensor (such as a microphone or microphone array) monitoring the coal mining machine when the coal mining machine is working, that is, when the coal mining machine is cutting coal and / or coal rock.

[0041] There are no restrictions on the location or installation position of the sound sensor. For example, the sound sensor can be set or installed around the coal mining machine, or the sound sensor can be set or installed on the body of the coal mining machine.

[0042] It should be noted that the number of sound sensors may be, but is not limited to, one, and this disclosure does not impose any restrictions on this.

[0043] In this embodiment of the disclosure, the coal mining machine can be monitored by a sound sensor to obtain the audio signal to be tested.

[0044] For example, using a sound sensor as an example, a microphone array can be used to monitor a coal mining machine in order to obtain the audio signal to be tested.

[0045] In one possible implementation of this disclosure, the reference audio features may include a first reference audio feature belonging to coal, and / or a second reference audio feature belonging to rock.

[0046] In this embodiment of the disclosure, the first reference audio feature may be a feature of the audio signal collected when the coal mining machine is cutting coal.

[0047] In this embodiment of the disclosure, the second reference audio feature may be a feature of the audio signal collected when the coal mining machine cuts the rock.

[0048] In order to obtain multiple reference audio features, in one possible implementation of this disclosure embodiment, when the audio signal to be tested is detected, the first environmental information corresponding to the coal mining machine can be obtained; wherein, the audio signal to be tested can be detected under the first environmental information; multiple candidate audio features can also be obtained, wherein the multiple candidate audio features can have corresponding second environmental information; based on the first environmental information and the second environmental information, multiple reference audio features can be determined from the multiple candidate audio features, wherein the second environmental information corresponding to the reference audio features can be matched with the first environmental information.

[0049] In the embodiments of this disclosure, the first environmental information or the second environmental information may include geological information (such as the hardness of coal, the hardness of rock, the proportion of coal-rock mixture, etc.), working condition information (such as the operating power of the coal mining machine, the cutting rate of the coal mining machine, the blade temperature of the coal mining machine, etc.), etc., and this disclosure does not limit it.

[0050] In this embodiment of the disclosure, when the audio signal to be tested is detected, the first environmental information corresponding to the coal mining machine can be obtained; wherein, the audio signal to be tested can be obtained by monitoring under the first environmental information.

[0051] In this embodiment of the disclosure, the candidate audio features may have corresponding second environmental information.

[0052] As one possible implementation, a correspondence between candidate audio features and second environmental information can be established in advance, and this correspondence can be stored so that when determining candidate audio features, the above correspondence can be queried to determine the second environmental information corresponding to the candidate audio features.

[0053] In this embodiment of the disclosure, the first environmental information corresponding to the audio signal to be tested can be matched with the second environmental information corresponding to any one of the multiple candidate audio features. When the first environmental information corresponding to the audio signal to be tested matches the second environmental information corresponding to any one of the candidate audio features, for example, the first environmental information is the same as the second environmental information corresponding to any one of the candidate audio features, or the similarity between the first environmental information and the second environmental information corresponding to any one of the candidate audio features is greater than a preset second similarity threshold (e.g., 80%, 90%), then any one of the candidate audio features can be used as a reference audio feature.

[0054] Therefore, by using the candidate audio features that match the second environmental information of the multiple candidate audio features with the first environmental information corresponding to the audio signal to be tested as reference audio features, the accuracy of subsequent identification of the audio signal to be tested can be improved.

[0055] Step 102: Extract features from the audio signal to be tested to obtain the first audio features.

[0056] In this embodiment of the disclosure, feature extraction can be performed on the audio signal to be tested to obtain first audio features. For example, feature extraction algorithms based on deep learning can be used to extract features from the audio signal to be tested to obtain the first audio features. For instance, a pre-trained deep residual network can be used to extract features from the audio signal to be tested to obtain the first audio features.

[0057] Step 103: Determine the second audio feature from the multiple reference audio features based on the similarity between the first audio feature and the multiple reference audio features.

[0058] In this embodiment of the disclosure, the similarity between the first audio feature and multiple reference audio features can be calculated. For example, the Euclidean Distance (ED) algorithm, the Manhattan Distance (MD) algorithm, the cosine similarity algorithm, etc., can be used to calculate the similarity between the first audio feature and multiple reference audio features.

[0059] In this embodiment of the disclosure, a second audio feature can be determined from multiple reference audio features based on the similarity between the first audio feature and multiple reference audio features.

[0060] As one possible implementation, after calculating the similarity between the first audio feature and multiple reference audio features, the largest similarity among the above similarities can be determined, and the reference audio feature corresponding to the largest similarity can be used as the second audio feature.

[0061] For example, assuming multiple reference audio features are reference audio feature X and reference audio feature Y, after obtaining the similarity between the first audio feature Z and reference audio feature X, and the similarity between the first audio feature Z and reference audio feature Y, the largest similarity among the above similarities can be determined. For example, if the first audio feature Z and reference audio feature X have the largest similarity, then reference audio feature X can be used as the second audio feature.

[0062] As another possible implementation, after calculating the similarity between the first audio feature and multiple reference audio features, reference audio features with similarity higher than a set similarity threshold can be used as the second audio feature.

[0063] The similarity threshold is set to a pre-defined similarity threshold, such as 85%, 90%, or 95%.

[0064] Step 104: Determine the first predicted category of the coal and rock cut by the coal mining machine based on the category to which the second audio feature belongs, wherein the first predicted category is used to indicate coal or rock.

[0065] In this embodiment of the disclosure, the category to which the second audio feature belongs may be, for example, coal or rock.

[0066] In this embodiment of the disclosure, the first prediction category can be used to indicate coal or rock, that is, to indicate whether the coal mining machine is cutting coal or rock.

[0067] In this embodiment of the disclosure, the first predicted category of the coal and rock cut by the coal mining machine can be determined based on the category to which the second audio feature belongs.

[0068] As an example, when the category to which the second audio feature belongs is coal, that is, when the second audio feature is the first reference audio feature belonging to coal, the first predicted category of the coal and rock cut by the coal mining machine can be determined to be coal; when the category to which the second audio feature belongs is rock, that is, when the second audio feature is the second reference audio feature belonging to rock, the first predicted category of the coal and rock cut by the coal mining machine can be determined to be rock.

[0069] The coal and rock identification method of this disclosure monitors a coal mining machine using a sound sensor to acquire a test audio signal and multiple reference audio features. It then extracts features from the test audio signal to obtain a first audio feature. Based on the similarity between the first audio feature and the multiple reference audio features, a second audio feature is determined from the multiple reference audio features. Finally, based on the category to which the second audio feature belongs, a first predicted category of the coal and rock cut by the coal mining machine is determined, where the first predicted category indicates coal or rock. Thus, based on deep learning technology, the audio features of the test audio signal of the coal mining machine cutting coal and rock can be acquired. This allows for automatic identification of the coal and rock cut by the coal mining machine during operation, i.e., identifying whether the coal mining machine is cutting coal or rock, based on the similarity between the audio features of the test audio signal and multiple reference audio features of known categories.

[0070] When multiple reference audio features include a first reference audio feature belonging to coal and a second reference audio feature belonging to rock, in order to clearly explain how the above embodiments of this disclosure obtain multiple reference audio features, this disclosure also proposes a coal and rock identification method.

[0071] Figure 2 This is a schematic flowchart of the coal and rock identification method provided in Embodiment 2 of this disclosure.

[0072] like Figure 2 As shown, based on any embodiment of this disclosure, multiple reference audio features can be obtained through the following steps:

[0073] Step 201: Acquire multiple first reference audio signals, wherein the first reference audio signals are audio signals collected by the sound sensor when the coal mining machine is cutting coal.

[0074] In this embodiment of the disclosure, the first reference audio signal may be an audio signal collected by a sound sensor when the coal mining machine is cutting coal.

[0075] In the embodiments of this disclosure, the first reference audio signal may be obtained from historical data or from the training dataset, and this disclosure does not limit this.

[0076] Step 202: Acquire multiple second reference audio signals, wherein the second reference audio signals are audio signals collected by the sound sensor when the coal mining machine cuts the rock.

[0077] In this embodiment of the disclosure, the second reference audio signal may be an audio signal collected by a sound sensor when the coal mining machine cuts the rock.

[0078] In the embodiments of this disclosure, the second reference audio signal may be obtained from historical data or from the training dataset, and this disclosure does not limit this.

[0079] It should be noted that this disclosure does not restrict the execution order of steps 201 and 202. For example, steps 201 and 202 can be executed in parallel, or step 202 can be executed before step 201. This disclosure only provides an example of step 201 being executed before step 202.

[0080] Step 203: Perform feature extraction on multiple first reference audio signals to obtain the first sub-reference audio features of each first reference audio signal.

[0081] It should be noted that the method for feature extraction of any one of the multiple first reference audio signals in this embodiment is similar to the method for feature extraction of the audio signal to be tested in step 102, and will not be described in detail here.

[0082] In this embodiment of the disclosure, features can be extracted from multiple first reference audio signals to obtain first sub-reference audio features of each first reference audio signal.

[0083] Step 204: Extract features from multiple second reference audio signals to obtain the second sub-reference audio features of each second reference audio signal.

[0084] It should be noted that the method for feature extraction of any one of the multiple second reference audio signals in this embodiment is similar to the method for feature extraction of the audio signal to be tested in step 102, and will not be described in detail here.

[0085] In this embodiment of the disclosure, features can be extracted from multiple second reference audio signals to obtain second sub-reference audio features of each second reference audio signal.

[0086] It should be noted that this disclosure does not restrict the execution order of steps 203 and 204. For example, steps 203 and 204 can be executed in parallel, or step 204 can be executed before step 203. This disclosure only provides an example of step 203 being executed before step 204.

[0087] Step 205: Determine the first reference audio feature based on each first sub-reference audio feature, and determine the second reference audio feature based on each second sub-reference audio feature.

[0088] In this embodiment of the disclosure, the first reference audio feature can be determined based on each first sub-reference audio feature.

[0089] As an example, at least one first sub-reference audio feature can be selected from multiple first sub-reference audio features as the first reference audio feature.

[0090] As another example, the weighted sum of the first sub-reference audio features can be used to obtain the fused first reference audio features.

[0091] Similarly, in this disclosure, the second reference audio features can be determined based on each second sub-reference audio feature.

[0092] The coal and rock identification method of this disclosure acquires multiple first reference audio signals, wherein the first reference audio signals are audio signals collected by a sound sensor when a coal mining machine cuts coal; acquires multiple second reference audio signals, wherein the second reference audio signals are audio signals collected by a sound sensor when a coal mining machine cuts rock; performs feature extraction on the multiple first reference audio signals respectively to obtain first sub-reference audio features for each first reference audio signal; performs feature extraction on the multiple second reference audio signals respectively to obtain second sub-reference audio features for each second reference audio signal; determines first reference audio features based on each first sub-reference audio feature, and determines second reference audio features based on each second sub-reference audio feature. Therefore, multiple reference audio features containing different categories can be effectively and accurately acquired.

[0093] As one possible implementation, when multiple reference audio features include a first reference audio feature belonging to coal and a second reference audio feature belonging to rock, in order to clearly illustrate how the first reference audio feature is determined based on each first sub-reference audio feature and the second reference audio feature is determined based on each second sub-reference audio feature in the above embodiments of this disclosure, this disclosure also proposes a coal and rock identification method.

[0094] Figure 3 This is a schematic flowchart of the coal and rock identification method provided in Embodiment 3 of this disclosure.

[0095] like Figure 3 As shown, based on any embodiment of this disclosure, multiple reference audio features can be obtained through the following steps:

[0096] Step 301: Acquire multiple first reference audio signals, wherein the first reference audio signals are audio signals collected by the sound sensor when the coal mining machine is cutting coal.

[0097] Step 302: Acquire multiple second reference audio signals, wherein the second reference audio signals are audio signals collected by the sound sensor when the coal mining machine cuts the rock.

[0098] Step 303: Extract features from multiple first reference audio signals to obtain the first sub-reference audio features of each first reference audio signal.

[0099] Step 304: Extract features from multiple second reference audio signals to obtain the second sub-reference audio features of each second reference audio signal.

[0100] The execution process of steps 301 to 304 can be found in the execution process of any embodiment of this disclosure, and will not be described in detail here.

[0101] Step 305: Obtain the first weight of each first sub-reference audio feature and the second weight of each second sub-reference audio feature.

[0102] In this embodiment of the disclosure, a first weight of each first sub-reference audio feature and a second weight of each second sub-reference audio feature can be obtained.

[0103] It should be noted that the first weight of each first sub-reference audio feature and the second weight of each second sub-reference audio feature may be the same or different, and this disclosure does not impose any restrictions on this.

[0104] It should also be noted that the first weight of each first sub-reference audio feature and the second weight of each second sub-reference audio feature can be set according to the actual application requirements.

[0105] Step 306: Based on the first weight of each first sub-reference audio feature, perform a weighted summation of each first sub-reference audio feature to obtain the first reference audio feature.

[0106] In this embodiment of the disclosure, the first reference audio features can be obtained by weighted summation of each first sub-reference audio feature according to the first weight of each first sub-reference audio feature.

[0107] Step 307: Based on the second weight of each second sub-reference audio feature, perform a weighted summation of each second sub-reference audio feature to obtain the second reference audio feature.

[0108] In this embodiment of the disclosure, the second reference audio features can be obtained by weighted summation of each second sub-reference audio feature according to the second weight of each second sub-reference audio feature.

[0109] To improve the accuracy of acquiring multiple reference audio features and thus enhance the accuracy of coal and rock identification, in one possible implementation of this disclosure, a sound sensor can be used to monitor the sample coal mining machine to obtain multiple training audio signals. These training audio signals can then be categorized to obtain their labeled categories. Feature extraction is performed on each of the training audio signals to obtain training audio features for each signal. For any given training audio feature, a second predicted category of the coal and rock cut by the sample coal mining machine can be determined based on the similarity between that feature and the multiple reference audio features. Finally, the multiple reference audio features can be updated based on the second predicted category and the labeled categories of the training audio signals.

[0110] In this embodiment of the disclosure, the training audio signal can be the audio signal obtained by the sound sensor monitoring the coal mining machine when the sample coal mining machine is working, that is, when the sample coal mining machine is cutting coal or coal rock.

[0111] In this embodiment of the disclosure, multiple training audio signals can also be categorized to obtain categorized classes for the training audio signals. For example, the categorized class for a training audio signal obtained when the sample coal mining machine is cutting coal can be coal, and the categorized class for a training audio signal obtained when the sample coal mining machine is cutting rock can be rock.

[0112] It should be noted that when labeling multiple training audio signals by category, manual labeling can be used, or a pre-trained labeling model can be used to automatically label the categories of multiple training audio signals. This disclosure does not impose any restrictions on this.

[0113] In this embodiment of the disclosure, feature extraction can be performed on multiple training audio signals to obtain the training audio features of each training audio signal.

[0114] It should be noted that the method for feature extraction of any training audio signal from multiple training audio signals is similar to the method for feature extraction of the test audio signal in step 102, and will not be elaborated here.

[0115] In this embodiment of the disclosure, for any training audio feature, a second predicted category of the coal and rock cut by the sample coal mining machine can be determined based on the similarity between the training audio feature and multiple reference audio features, wherein the second predicted category can indicate coal or rock.

[0116] As an example, for any training audio feature, the similarity between that training audio feature and multiple reference audio features can be calculated. The maximum similarity can then be determined from these similarities, and the category of the reference audio feature corresponding to this maximum similarity can be used as the second predicted category of the coal and rock cut by the sample mining machine. The category of the reference audio feature can be, for example, coal or rock.

[0117] In this embodiment of the disclosure, multiple reference audio features can be updated based on a second predicted category and the labeled category of the training audio signal.

[0118] As one possible implementation, it can be determined whether the difference between the second predicted category and the labeled category of the training audio signal is greater than a set difference threshold; in response to the difference being greater than the set difference threshold, the first weight of each first sub-reference audio feature can be updated, and / or the second weight of each second sub-reference audio feature can be updated; and the first reference audio feature among multiple reference audio features can be updated according to the updated first weight of each first sub-reference audio feature, and / or the second reference audio feature among multiple reference audio features can be updated according to the updated second weight of each second reference audio feature.

[0119] As an example, when the second predicted category indicates coal and the labeled type of the training audio signal indicates rock, it can be determined whether the difference between the second predicted category and the labeled category of the training audio signal is greater than a set difference threshold (such as 0, 0.1, etc.). When the difference is greater than the set difference threshold, it indicates that the reference audio features belonging to rock among the multiple reference audio features are not accurate enough. The second weights of each second sub-reference audio feature can be updated, and the second reference audio features among the multiple reference audio features can be updated according to the updated second weights of each second reference audio feature.

[0120] As another example, when the second predicted category indicates rock and the labeled type of the training audio signal indicates coal, it can be determined whether the difference between the second predicted category and the labeled category of the training audio signal is greater than a set difference threshold (e.g., 0, 0.1, etc.). When the difference is greater than the set difference threshold, it indicates that the reference audio features belonging to coal among the multiple reference audio features are not accurate enough. The first weights of each first sub-reference audio feature can be updated, and the first reference audio feature among the multiple reference audio features can be updated according to the updated first weights of each first sub-reference audio feature.

[0121] As another possible implementation, when the difference is less than or equal to a set difference threshold and the second prediction category indicates coal, the first sub-reference audio features can be updated using the training audio features corresponding to the training audio signal; the third weights of each updated first sub-reference audio feature can be obtained; and the updated first sub-reference audio features can be weighted and summed according to the third weights of each updated first sub-reference audio feature to obtain the updated first reference audio features.

[0122] For example, when there is no difference between the second predicted category and the labeled category of the training audio signal, and the second predicted category indicates coal, the training audio feature corresponding to the training audio signal can also be used as the first sub-reference audio feature. That is, the training audio feature is added to the set to which multiple first sub-reference audio features belong, and the third weight of each updated first sub-reference audio feature can be obtained. Thus, based on the third weight of each updated first sub-reference audio feature, the updated first sub-reference audio features can be weighted and summed to obtain the updated first reference audio feature, which improves the accuracy of the original first reference audio feature.

[0123] As another possible implementation, when the difference is less than or equal to a set difference threshold and the second predicted category indicates rock, the training audio features of the training audio signal can be used to update the second sub-reference audio features; the fourth weight of each updated second sub-reference audio feature can be obtained; and the updated second sub-reference audio features can be weighted and summed according to the fourth weight of each updated second sub-reference audio feature to obtain the updated second reference audio features.

[0124] For example, when there is no difference between the second predicted category and the labeled category of the training audio signal, and the second predicted category indicates coal, the training audio feature corresponding to the training audio signal can also be used as the second sub-reference audio feature. That is, the training audio feature is added to the set to which multiple second sub-reference audio features belong, and the fourth weight of each updated second sub-reference audio feature can be obtained. Then, based on the fourth weight of each updated second sub-reference audio feature, the updated second sub-reference audio features can be weighted and summed to obtain the updated second reference audio feature. That is, the original second reference audio feature is improved to enhance the accuracy of the second reference audio feature.

[0125] The coal and rock identification method of this disclosure obtains a first weight for each first sub-reference audio feature and a second weight for each second sub-reference audio feature; it then performs a weighted summation of the first sub-reference audio features based on their first weights to obtain a first reference audio feature; and finally, it performs a weighted summation of the second sub-reference audio features based on their second weights to obtain a second reference audio feature. This allows for the effective and accurate acquisition of both the first and second reference audio features.

[0126] To clearly illustrate how a sound sensor monitors a coal mining machine to obtain the audio signal to be measured in any embodiment of this disclosure, this disclosure also proposes a coal and rock identification method.

[0127] Figure 4 This is a schematic flowchart of the coal and rock identification method provided in Embodiment 4 of this disclosure.

[0128] like Figure 4 As shown, the coal and rock identification method may include the following steps:

[0129] Step 401: Monitor the coal mining machine using a sound sensor to obtain the initial audio signal.

[0130] In this embodiment of the disclosure, the initial audio signal may be an audio signal obtained by a sound sensor monitoring the coal mining machine when it is working.

[0131] Step 402: Detect the initial audio signal to determine whether the initial audio signal meets the set conditions.

[0132] In this embodiment of the disclosure, the setting conditions can be preset.

[0133] As one possible implementation, the conditions can include at least one of the following four conditions:

[0134] I. The acquisition times corresponding to each sub-audio signal in the initial audio signal are continuous. 。

[0135] In this embodiment of the disclosure, the initial audio signal may include multiple sub-audio signals. A sub-audio signal may be a segment of audio signal or a frame of audio signal from the initial audio signal.

[0136] Understandably, after acquiring the initial audio signal, multiple consecutive sub-audio signals can be sequentially encapsulated and packaged, with each data packet having a corresponding timestamp. After acquiring the data packets corresponding to multiple sub-audio signals, the acquisition times of each sub-audio signal in the initial audio signal can be determined based on the timestamps corresponding to the multiple data packets.

[0137] 2. The measured value of the sub-audio signal with the set proportion in the initial audio signal does not exceed the set measurement threshold.

[0138] In this embodiment of the disclosure, the set measurement threshold can be preset, for example, the set measurement threshold can be the maximum value that the digital sensor can measure.

[0139] In the embodiments disclosed herein, the percentage can be preset, such as 98%, 95%, etc., and this disclosure does not limit it.

[0140] It should be understood that the above example uses the setting condition that the measured value of a sub-audio signal representing a set percentage of the initial audio signal does not exceed a set measurement threshold. In practical applications, the setting condition may also include: the measured value of a sub-audio signal representing a set percentage of the initial audio signal exceeds a set value, where the set value can be a pre-set value, such as the minimum value that a digital sensor can measure. Alternatively, the setting condition may also include: the measured value of a sub-audio signal representing a set percentage of the initial audio signal exceeds a set value but does not exceed a measurement threshold.

[0141] 3. The number of target sub-audio signals contained in the initial audio signal is greater than the set number threshold. The target sub-audio signals can be audio signals in the initial audio signal that are continuously acquired at the acquisition time and whose measured values ​​do not exceed the measurement threshold.

[0142] In this embodiment of the disclosure, the number threshold can be preset.

[0143] In practical applications, the set conditions may also include: the number of target sub-audio signals contained in the initial audio signal is greater than the set number threshold, wherein the target sub-audio signal can be an audio signal in the initial audio signal that is continuously acquired at the acquisition time, whose measured value exceeds the set value, but does not exceed the measurement threshold.

[0144] Fourth, the similarity between the third audio feature of the initial audio signal and the coal-rock fusion feature is greater than the set first similarity threshold. The third audio feature is obtained by feature extraction from the initial audio signal; the coal-rock fusion feature is obtained by fusing multiple reference audio features.

[0145] In the embodiments of this disclosure, the first similarity threshold can be preset, such as 70%, 85%, etc., and this disclosure does not limit it.

[0146] In this embodiment of the disclosure, features can be extracted from the initial audio signal to obtain a third audio feature.

[0147] It should be noted that the method for extracting features from the initial audio signal is similar to the method for extracting features from the audio signal to be tested in step 102, and will not be elaborated here.

[0148] In this embodiment of the disclosure, the coal-rock fusion feature can be obtained by fusing multiple reference audio features. For example, multiple reference audio features can be weighted and summed to obtain the coal-rock fusion feature.

[0149] In this embodiment of the disclosure, the similarity between the third audio feature of the initial audio signal and the coal-rock fusion feature can be calculated. For example, Euclidean distance algorithm, Manhattan distance algorithm, cosine similarity algorithm, etc. can be used to calculate the similarity between the third audio feature of the initial audio signal and the coal-rock fusion feature.

[0150] In this embodiment of the disclosure, after obtaining the similarity between the coal-rock fusion feature and the third audio feature of the initial audio signal, it can be determined whether the similarity between the coal-rock fusion feature and the third audio feature of the initial audio signal is greater than a set first similarity threshold.

[0151] In this embodiment of the disclosure, when the similarity between the third audio feature of the initial audio signal and the coal-rock fusion feature is greater than the set first similarity threshold, it indicates that the coal mining machine is in the working state of cutting coal or rock when the initial audio signal is collected.

[0152] Step 403: In response to the initial audio signal meeting the set conditions, the initial audio signal is used as the audio signal to be tested.

[0153] In this embodiment of the disclosure, when the initial audio signal meets the set conditions, the initial audio signal can be used as the audio signal to be tested.

[0154] Step 404: Obtain multiple reference audio features.

[0155] Step 405: Perform feature extraction on the audio signal to be tested to obtain the first audio feature.

[0156] Step 406: Determine the second audio feature from the multiple reference audio features based on the similarity between the first audio feature and the multiple reference audio features.

[0157] Step 407: Determine the first predicted category of the coal and rock cut by the coal mining machine based on the category to which the second audio feature belongs, wherein the first predicted category is used to indicate coal or rock.

[0158] The execution process of steps 404 to 407 can be found in the execution process of any embodiment of this disclosure, and will not be described in detail here.

[0159] As an example application scenario, the coal and rock identification method disclosed herein is applied to a server for illustrative purposes. The coal and rock identification process can be described as follows: Figure 5 As shown, the acquisition module may include a sound sensor, a built-in network card, etc., and can be configured on the coal mining machine body. The sound sensor can be a sensor array. After acquiring the initial audio signal, the sensor array uses Pulse Code Modulation (PCM) to sample, quantize, and encode the initial audio signal into a discrete digital signal. The initial audio signal is then losslessly stored and transmitted using PCM encoding. Simultaneously, the built-in network card enables the initial audio signal acquired by a hardware terminal deployed near the coal mining machine cutter head to be transmitted via TCP / IP protocol to a terminal communication interface deployed on a server. The transmitted data may include the PCM-encoded binary initial audio signal and metadata such as sampling frequency, number of channels, sampling accuracy, and acquisition time.

[0160] The server may include the following modules:

[0161] 1. Detection module, used to detect the initial audio signal in real time.

[0162] The detection module can receive and process the initial audio signal received by the terminal communication interface in real time, and can detect the initial audio signal in the following ways: (1) It can detect whether the received audio signal (i.e., the sound data stream) has missing or discontinuous data blocks due to network instability, etc., that is, determine whether the acquisition time corresponding to each sub-audio signal in the initial audio signal is continuous; (2) It can detect whether the audio signal of each channel exceeds the range of the digital sensor and produces digital clipping distortion, that is, determine whether the measured value of the sub-audio signal with a set proportion in the initial audio signal does not exceed the set measurement threshold; (3) It uses a neural network-based sound event detection algorithm to detect whether the currently received audio signal is the audio signal generated when the coal mining machine cuts coal and rock, that is, determine whether the similarity between the third audio feature of the initial audio signal and the coal and rock fusion feature is greater than the set first similarity threshold, wherein the third audio feature is obtained by feature extraction of the initial audio signal; the coal and rock fusion feature is obtained by fusing multiple reference audio features. Only when the initial audio signal is continuous, at least one channel's initial audio signal does not exceed the range of the digital sensor, and the coal mining machine is in the working state of cutting coal or rock, can the current initial audio signal be used as the audio signal to be measured, and the audio signal to be measured can be transmitted to other modules for further processing or storage.

[0163] 2. Coal and rock condition identification module, used to identify the coal and rock cut by the coal mining machine based on the audio signal to be tested.

[0164] The coal and rock condition identification module can receive the audio signal to be tested transmitted by the detection module, and can use a pre-trained deep residual network to extract features from the audio signal to obtain the first audio feature. Furthermore, after acquiring the audio signal to be tested, the pre-loaded coal cutting acoustic features (denoted as the first reference audio feature in this disclosure) and rock cutting acoustic features (denoted as the second reference audio feature in this disclosure) can be used to calculate the distance between the first audio feature of the real-time acquired audio signal and acoustic features of different categories (denoted as reference audio features in this disclosure). The category corresponding to the nearest reference audio feature can be used as the predicted category of the coal and rock condition identification module, whereby the predicted category can indicate whether the coal mining machine is cutting coal or rock.

[0165] As one possible implementation, after identifying the coal and rock cut by the mining machine, the coal and rock condition recognition module can update the corresponding acoustic signature features. For example, if the predicted category indicates that the mining machine is cutting coal, the first audio feature acquired this time can be used as the acoustic signature feature for coal cutting in the next recognition process; or, if the predicted category indicates that the mining machine is cutting rock, the first audio feature acquired this time can be used as the acoustic signature feature for rock cutting in the next recognition process. As sound signals are continuously acquired and the coal and rock cut by the mining machine are identified, the initial values ​​of the pre-loaded acoustic signature features are gradually "forgotten". Simultaneously, the audio signal to be tested, the first audio feature of the audio signal to be tested, and the corresponding predicted category used by the coal and rock condition recognition module can be transmitted and stored in the data storage module.

[0166] It should be noted that the preloaded voiceprint features can be the first sub-reference audio features corresponding to the audio signals collected by the coal mining machine when cutting coal (referred to as the first reference audio signal in this disclosure) or the second sub-reference audio features corresponding to the audio signals collected by the coal mining machine when cutting rock (referred to as the second reference audio signal in this disclosure) stored in the training dataset or historical data of the data storage module. Alternatively, the first sub-reference audio features can be weighted and summed to obtain the coal cutting voiceprint features, or the second sub-reference audio features can be weighted and summed to obtain the rock cutting voiceprint features.

[0167] Alternatively, it is possible to obtain first environmental information containing information such as geological conditions and coal mining machine operating conditions corresponding to the current initial audio signal, and obtain multiple candidate audio features from the database of the data storage module. Among them, multiple candidate audio features have second environmental information containing information such as corresponding geological conditions and coal mining machine operating conditions. The candidate audio features whose second environmental information is similar to the first environmental information are used as the initial values ​​of the voiceprint features.

[0168] It should be noted that in the initial stage of the coal and rock condition identification module's identification of the coal and rock cut by the coal mining machine, the voiceprint features may not be accurate. To improve the accuracy of the voiceprint features, the coal and rock condition identification module can have bidirectional data transmission with the manual annotation module. When the category prediction is incorrect, the coal cutting voiceprint features and / or rock cutting voiceprint features can be modified in real time based on the correct category labeled by the manual annotation. For example, the first weight of each first sub-reference audio feature can be updated, and / or the second weight of each second sub-reference audio feature can be updated, and based on the updated first weight of each first sub-reference audio feature, the first reference audio feature among multiple reference audio features can be updated, and / or the second reference audio feature among multiple reference audio features can be updated based on the updated second weight of each second reference audio feature.

[0169] 3. The manual labeling module can receive the predicted category transmitted by the coal and rock condition recognition module and manually label the category to which the audio signal to be tested belongs. When there is a difference between the predicted category and the manually labeled category to which the audio signal to be tested belongs, i.e., when the prediction is incorrect, the correct category label can be fed back to the coal and rock condition recognition module to modify the voiceprint features, and then transmitted to the data storage module to obtain the correct category label.

[0170] 4. The data storage module can store the training dataset and the audio signals, voiceprint features, and category labels (which can be manually labeled or predicted) collected during each test of the coal mining machine, using object storage. The data storage module can receive data from the coal and rock condition recognition module and the manual labeling module, and store the category labels corrected by manual feedback. Furthermore, the data storage module supports adding annotations (such as geological conditions, coal mining machine operating conditions, and other environmental information) to each initial audio signal collected, providing a reference for the selection of voiceprint features during subsequent audio signal collection. The data storage module also supports adding reliable manual annotations to the initial audio signals offline. The labeled dataset can be used for further training and fine-tuning of the feature extraction network, and can also be used to obtain prototypes of coal cutting voiceprint features and rock cutting voiceprint features.

[0171] In summary, the advantages can be seen in the following aspects:

[0172] 1. Audio signals from the drum cutting teeth during coal or rock breaking operations can be directly extracted using a sensor array deployed on the coal mining machine.

[0173] Second: A few-sample learning method can be used to update the sound signature features of coal cutting or rock cutting in real time, and the update strategy can be adjusted.

[0174] Third: The data storage module has a knowledge base addition function, which can simultaneously store audio signals, voiceprint features, and environmental information when the audio signals are collected (such as coal and rock hardness, temperature, humidity, etc.). This prior knowledge can be used for the initialization of the model for new tests.

[0175] The coal and rock identification method disclosed herein has the following two advantages compared with the prior art:

[0176] First, since the real-time acoustic signature of the coal mining machine is relatively obvious during the real-time cutting of coal and rock, the coal and rock identification method disclosed herein can identify coal and rock more directly and efficiently. Moreover, the coal and rock identification method disclosed herein is flexible, low-cost, and not easily restricted by the environment.

[0177] Second: Train a dedicated neural network for coal and rock identification tasks. A few-shot learning method can be used to update voiceprint features in real time and adjust the update strategy to obtain a more adaptable and robust coal and rock identification algorithm for use in coal and rock identification tasks.

[0178] The coal and rock identification method disclosed herein monitors the coal mining machine using a sound sensor to acquire an initial audio signal; the initial audio signal is then detected to determine whether it meets set conditions; and if the initial audio signal meets the set conditions, it is used as the audio signal to be tested. Therefore, by detecting the initial audio signal and acquiring only the audio signal that meets the set conditions, the accuracy of subsequent identification results can be improved.

[0179] With the above Figures 1 to 4 Corresponding to the coal and rock identification method provided in the embodiments, this disclosure also provides a coal and rock identification device. Because the coal and rock identification device provided in the embodiments of this disclosure is similar to the one described above... Figures 1 to 4 The coal and rock identification method provided in the embodiments corresponds to the coal and rock identification device provided in the embodiments of this disclosure, and will not be described in detail in the embodiments of this disclosure.

[0180] Figure 6 This is a schematic diagram of the coal and rock identification device provided in Embodiment 5 of this disclosure.

[0181] like Figure 6 As shown, the coal and rock identification device 600 may include: a first processing module 601, a first extraction module 602, a first determination module 603, and a second determination module 604.

[0182] The first processing module 601 is used to monitor the coal mining machine through a sound sensor to obtain the audio signal to be tested and to acquire multiple reference audio features.

[0183] The first extraction module 602 is used to extract features from the audio signal to be tested in order to obtain the first audio features.

[0184] The first determining module 603 is used to determine a second audio feature from multiple reference audio features based on the similarity between the first audio feature and multiple reference audio features.

[0185] The second determining module 604 is used to determine a first predicted category of the coal and rock cut by the coal mining machine based on the category to which the second audio feature belongs, wherein the first predicted category is used to indicate coal or rock.

[0186] In one possible implementation of this disclosure, the plurality of reference audio features include a first reference audio feature belonging to coal and a second reference audio feature belonging to rock; the first processing module 601 can be used to: acquire a plurality of first reference audio signals, wherein the first reference audio signals are audio signals collected by a sound sensor when a coal mining machine cuts coal; acquire a plurality of second reference audio signals, wherein the second reference audio signals are audio signals collected by a sound sensor when a coal mining machine cuts rock; perform feature extraction on the plurality of first reference audio signals respectively to obtain a first sub-reference audio feature of each first reference audio signal; perform feature extraction on the plurality of second reference audio signals respectively to obtain a second sub-reference audio feature of each second reference audio signal; determine a first reference audio feature based on each first sub-reference audio feature, and determine a second reference audio feature based on each second sub-reference audio feature.

[0187] In one possible implementation of this disclosure, the first processing module 601 may be used to: obtain a first weight of each first sub-reference audio feature and obtain a second weight of each second sub-reference audio feature; perform a weighted summation of each first sub-reference audio feature according to the first weight of each first sub-reference audio feature to obtain a first reference audio feature; and perform a weighted summation of each second sub-reference audio feature according to the second weight of each second sub-reference audio feature to obtain a second reference audio feature.

[0188] In one possible implementation of this disclosure, the coal and rock identification device 600 may further include:

[0189] The second processing module is used to monitor the sample coal mining machine through a sound sensor to obtain multiple training audio signals, and to label the multiple training audio signals to obtain the labeled categories of the multiple training audio signals.

[0190] The second extraction module is used to extract features from multiple training audio signals to obtain the training audio features of each training audio signal.

[0191] The third determination module is used to determine the second predicted category of coal and rock cut by the sample coal mining machine based on the similarity between any training audio feature and multiple reference audio features.

[0192] The update module is used to update multiple reference audio features based on the second predicted category and the labeled category of the training audio signal.

[0193] In one possible implementation of this disclosure, the update module may be specifically configured to: determine whether the difference between the second predicted category and the labeled category of the training audio signal is greater than a set difference threshold; in response to the difference being greater than the set difference threshold, update the first weight of each first sub-reference audio feature, and / or update the second weight of each second sub-reference audio feature; update the first reference audio feature among the plurality of reference audio features according to the updated first weight of each first sub-reference audio feature, and / or update the second reference audio feature among the plurality of reference audio features according to the updated second weight of each second reference audio feature.

[0194] In one possible implementation of this disclosure, the update module may specifically be used to: update the first sub-reference audio features using training audio features in response to a difference less than or equal to a set difference threshold and a second predicted category indicating coal; obtain the third weights of each updated first sub-reference audio feature; perform a weighted summation of each updated first sub-reference audio feature according to the third weights of each updated first sub-reference audio feature to obtain the updated first reference audio feature; update the second sub-reference audio features using training audio features in response to a difference less than or equal to a set difference threshold and a second predicted category indicating rock; obtain the fourth weights of each updated second sub-reference audio feature; and perform a weighted summation of each updated second sub-reference audio feature according to the fourth weights of each updated second sub-reference audio feature to obtain the updated second reference audio feature.

[0195] In one possible implementation of this disclosure, the first processing module 601 may be specifically used to: in response to detecting the audio signal to be tested, acquire first environmental information corresponding to the coal mining machine; wherein the audio signal to be tested is detected under the first environmental information; acquire multiple candidate audio features, wherein the multiple candidate audio features have corresponding second environmental information; determine multiple reference audio features from the multiple candidate audio features according to the first environmental information and the second environmental information, wherein the second environmental information corresponding to the reference audio features matches the first environmental information.

[0196] In one possible implementation of this disclosure, the first processing module 601 may be used to: monitor the coal mining machine through a sound sensor to obtain an initial audio signal; detect the initial audio signal to determine whether the initial audio signal meets the set conditions; and, in response to the initial audio signal meeting the set conditions, use the initial audio signal as the audio signal to be tested.

[0197] In one possible implementation of this disclosure, the set conditions may include at least one of the following: the acquisition times corresponding to each sub-audio signal in the initial audio signal are continuous; the measured values ​​of the sub-audio signals with a set proportion in the initial audio signal do not exceed a set measurement threshold; the number of target sub-audio signals included in the initial audio signal is greater than a set number threshold, wherein the target sub-audio signals are audio signals in the initial audio signal whose acquisition times are continuous and whose measured values ​​do not exceed the measurement threshold; the similarity between the third audio feature of the initial audio signal and the coal-rock fusion feature is less than or equal to a set first similarity threshold, wherein the third audio feature is obtained by feature extraction from the initial audio signal; and the coal-rock fusion feature is obtained by fusing multiple reference audio features.

[0198] The coal and rock identification device of this embodiment monitors a coal mining machine using a sound sensor to acquire a test audio signal and multiple reference audio features. It then extracts features from the test audio signal to obtain a first audio feature. Based on the similarity between the first audio feature and the multiple reference audio features, a second audio feature is determined from the multiple reference audio features. Finally, based on the category to which the second audio feature belongs, a first predicted category of the coal and rock cut by the coal mining machine is determined, whereby the first predicted category indicates coal or rock. Thus, based on deep learning technology, the audio features of the test audio signal of the coal mining machine cutting coal and rock can be acquired. This allows for automatic identification of the coal and rock cut by the coal mining machine corresponding to the test audio signal, i.e., identifying whether the coal mining machine is cutting coal or rock, based on the similarity between the audio features of the test audio signal and multiple reference audio features of known categories.

[0199] To implement the above embodiments, this disclosure also proposes an electronic device, wherein the electronic device may be a server or detection device as described in the foregoing embodiments; it includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the coal and rock identification method as proposed in any of the foregoing embodiments of this disclosure.

[0200] To implement the above embodiments, this disclosure also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the coal and rock identification method as proposed in any of the foregoing embodiments of this disclosure.

[0201] To implement the above embodiments, this disclosure also proposes a computer program product that, when the instructions in the computer program product are executed by a processor, performs the coal and rock identification method as proposed in any of the foregoing embodiments of this disclosure.

[0202] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0203] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0204] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.

[0205] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0206] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0207] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0208] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0209] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A method for coal and rock identification, characterized in that, The method includes: The coal mining machine is monitored by a sound sensor to obtain the audio signal to be tested and to acquire multiple reference audio features; The plurality of reference audio features include a first reference audio feature belonging to coal and a second reference audio feature belonging to rock. Acquiring the plurality of reference audio features includes: acquiring a plurality of first reference audio signals, wherein the first reference audio signals are audio signals collected by the sound sensor when the coal mining machine cuts coal; acquiring a plurality of second reference audio signals, wherein the second reference audio signals are audio signals collected by the sound sensor when the coal mining machine cuts rock; performing feature extraction on the plurality of first reference audio signals respectively to obtain a first sub-reference audio feature for each first reference audio signal; performing feature extraction on the plurality of second reference audio signals respectively to obtain a second sub-reference audio feature for each second reference audio signal; determining the first reference audio feature based on each first sub-reference audio feature, and determining the second reference audio feature based on each second sub-reference audio feature. Feature extraction is performed on the audio signal to be tested to obtain the first audio feature; A second audio feature is determined from the plurality of reference audio features based on the similarity between the first audio feature and the plurality of reference audio features; Based on the category to which the second audio feature belongs, a first predicted category of the coal and rock cut by the coal mining machine is determined, wherein the first predicted category is used to indicate coal or rock; The acquisition of multiple reference audio features includes: In response to the detection of the audio signal to be tested, first environmental information corresponding to the coal mining machine is acquired; wherein the audio signal to be tested is obtained by monitoring under the first environmental information; Multiple candidate audio features are obtained, wherein the multiple candidate audio features have corresponding second environmental information; Based on the first environmental information and the second environmental information, a plurality of reference audio features are determined from the plurality of candidate audio features, wherein the second environmental information corresponding to the reference audio features matches the first environmental information, and the first environmental information or the second environmental information includes the hardness of coal, the hardness of rock, the coal-rock mixture ratio, the operating power of the coal mining machine, the cutting rate of the coal mining machine, and the blade temperature of the coal mining machine. The method further includes: The sample coal mining machine is monitored by a sound sensor to obtain multiple training audio signals, and the multiple training audio signals are labeled with categories to obtain the labeled categories of the multiple training audio signals. Feature extraction is performed on the multiple training audio signals respectively to obtain the training audio features of each training audio signal; For any of the training audio features, a second predicted category of the coal and rock cut by the sample coal mining machine is determined based on the similarity between the training audio features and the plurality of reference audio features. Determine whether the difference between the second predicted category and the labeled category of the training audio signal is greater than a set difference threshold; In response to the difference being greater than the set difference threshold, the first weight of each of the first sub-reference audio features is updated, and / or the second weight of each of the second sub-reference audio features is updated; The first reference audio features among the plurality of reference audio features are updated according to the first weight of each of the updated first sub-reference audio features, and / or the second reference audio features among the plurality of reference audio features are updated according to the second weight of each of the updated second sub-reference audio features.

2. The method according to claim 1, characterized in that, The step of determining the first reference audio feature based on each of the first sub-reference audio features, and determining the second reference audio feature based on each of the second sub-reference audio features, includes: Obtain the first weight of each of the first sub-reference audio features, and obtain the second weight of each of the second sub-reference audio features; Based on the first weight of each first sub-reference audio feature, the first sub-reference audio features are weighted and summed to obtain the first reference audio features. Based on the second weight of each second sub-reference audio feature, the second sub-reference audio features are weighted and summed to obtain the second reference audio features.

3. The method according to claim 1, characterized in that, The step of updating the plurality of reference audio features based on the second predicted category and the labeled category of the training audio signal further includes: In response to the difference being less than or equal to the set difference threshold, and the second predicted category indicating coal, the first sub-reference audio features are updated using the trained audio features; Obtain the updated third weights for each of the first sub-reference audio features; Based on the third weight of each of the updated first sub-reference audio features, the updated first sub-reference audio features are weighted and summed to obtain the updated first reference audio features. In response to the difference being less than or equal to the set difference threshold, and the second predicted category indicating rock, the second sub-reference audio features are updated using the trained audio features; Obtain the updated fourth weights for each of the second sub-reference audio features; Based on the fourth weight of each of the updated second sub-reference audio features, the updated second sub-reference audio features are weighted and summed to obtain the updated second reference audio features.

4. The method according to any one of claims 1-3, characterized in that, The monitoring of the coal mining machine using a sound sensor to obtain the audio signal to be measured includes: The coal mining machine is monitored by the sound sensor to obtain the initial audio signal; The initial audio signal is detected to determine whether the initial audio signal meets the set conditions; In response to the initial audio signal meeting the set conditions, the initial audio signal is used as the audio signal to be tested.

5. The method according to claim 4, characterized in that, The setting conditions include at least one of the following: The acquisition times corresponding to each sub-audio signal in the initial audio signal are continuous; The measured value of the sub-audio signal with a set proportion in the initial audio signal does not exceed the set measurement threshold; The number of target sub-audio signals included in the initial audio signal is greater than a set number threshold, wherein the target sub-audio signal is an audio signal in the initial audio signal whose acquisition time is continuous and whose measured value does not exceed the measurement threshold; The similarity between the third audio feature of the initial audio signal and the coal-rock fusion feature is greater than a set first similarity threshold, wherein the third audio feature is obtained by feature extraction of the initial audio signal; and the coal-rock fusion feature is obtained by fusing the multiple reference audio features.

6. A coal and rock identification device, employing the coal and rock identification method according to any one of claims 1-5, characterized in that, The device includes: The first processing module is used to monitor the coal mining machine through a sound sensor to obtain the audio signal to be tested and to acquire multiple reference audio features; The extraction module is used to extract features from the audio signal to be tested in order to obtain first audio features; The first determining module is configured to determine a second audio feature from the plurality of reference audio features based on the similarity between the first audio feature and the plurality of reference audio features; The second determining module is used to determine a first predicted category of the coal and rock cut by the coal mining machine based on the category to which the second audio feature belongs, wherein the first predicted category is used to indicate coal or rock.

Images