A method, device, equipment, medium and product for labeling a sound sample

Abstract

The present disclosure relates to a method and device for labeling sound samples, equipment, media and products, the method comprising: obtaining a set of sound samples to be labeled; wherein the set of sound samples to be labeled comprises a plurality of sound samples to be labeled; extracting the mel-spectral spectrogram information and the first sound statistical indicators of each sound sample to be labeled; determining the class of the first sound statistical indicators of each sound sample to be labeled by a target determination rule to obtain an initial label set corresponding to the set of sound samples to be labeled; wherein the target determination rule is a rule generated according to the statistical characteristics corresponding to each sound class in the labeled sound samples; and calibrating the initial label set based on the mel-spectral spectrogram information of each sound sample to be labeled to obtain a target label set. The present disclosure can realize batch labeling of the set of sound samples to be labeled, improve the labeling efficiency of sound samples, and improve the accuracy, consistency and reliability of the labeling results.

Description

Technical Field

[0001] This disclosure relates to the field of sound sample annotation technology, and in particular to a method, apparatus, device, medium and product for annotating sound samples. Background Technology

[0002] In the application of industrial environmental sound data processing and intelligent recognition in coal mines, accurate annotation of sound samples is of great significance. Due to the complex production environment, strong background noise, and diverse sound types in coal mines, and the strong overlap and uncertainty of different sounds in the time and frequency domains, there is an urgent need to provide a sound sample annotation method with high accuracy and high efficiency to achieve effective annotation of sound samples in complex industrial environments.

[0003] In related technologies, sound samples are usually processed by manual annotation, semi-automatic annotation, or automatic annotation. However, such annotation methods have problems such as high cost of manual participation, low annotation efficiency, insufficient adaptability of the model in complex environments, and difficulty in uniformly controlling the quality of labels. As a result, the accuracy and consistency of sound sample annotation results are poor, making it difficult to meet the actual application needs of sound sample annotation under complex working conditions in coal mines. Summary of the Invention

[0004] This disclosure provides a method, apparatus, device, medium, and product for annotating sound samples.

[0005] According to a first aspect of this disclosure, a method for annotating sound samples is provided, the method comprising: Obtain a set of audio samples to be labeled; wherein, the set of audio samples to be labeled includes multiple audio samples to be labeled; Extract the Mel-spectral information and first sound statistical index of each of the aforementioned sound samples to be labeled; The target determination rule is used to classify the first sound statistical index of each of the sound samples to be labeled by the target determination rule, so as to obtain the initial label set corresponding to the sound sample set to be labeled; wherein, the target determination rule is a rule generated based on the statistical features corresponding to each sound category in the labeled sound samples; The initial tag set is calibrated based on the Mel-spectral information of each of the sound samples to be labeled, to obtain the target tag set.

[0006] Further, the calibration of the initial tag set based on the Mel-spectral information of each of the audio samples to be labeled, to obtain the target tag set, includes: A subset of labels in the initial label set is randomly sampled to determine the label accuracy rate of the initial label set; wherein, the label accuracy rate is used to characterize the overall accuracy of the initial label set; A Mel spectrum image is generated based on the Mel spectrum information of each of the aforementioned sound samples to be labeled; The target label set is determined based on the label accuracy and the Mel spectrum image.

[0007] Further, determining the target label set based on the label accuracy and the Mel spectrum image includes: If the label accuracy rate reaches the preset accuracy requirement, the initial label set is determined as the target label set.

[0008] Further, determining the target label set based on the label accuracy and the Mel spectrum image includes: If the label accuracy does not meet the preset accuracy requirement, the Mel spectrum image is output, and a manual calibration instruction based on the Mel spectrum image is received. The initial tag set is calibrated and fine-tuned based on the manual calibration instructions to obtain the target tag set.

[0009] Furthermore, before classifying each of the unlabeled sound samples according to the target determination rule to obtain the initial label set corresponding to the unlabeled sound sample set, the method further includes: Feature extraction is performed on the second sound statistical index of the labeled sound samples to obtain the statistical features corresponding to each sound category in the labeled sound samples; wherein, the statistical features are used to reflect the change features of the corresponding sound category in the time dimension and frequency dimension; The classification machine learning model is trained based on the statistical features to obtain the trained model; The target determination rule is generated based on the trained model.

[0010] Further, the step of extracting features from the second sound statistical index of the labeled sound samples to obtain the statistical features corresponding to each sound category in the labeled sound samples includes: The second sound statistical index is filtered to obtain the target statistical index; wherein, the target statistical index includes short-time average energy, entropy value and amplitude spectrum in Fourier transform spectrum; The short-time average energy and entropy values ​​in the target statistical indicators are used as time series data, and the amplitude spectrum in the target statistical indicators is used as frequency series data. The statistical features are obtained by extracting features from the time series data and the frequency series data.

[0011] Further, the step of extracting features from the time series data and the frequency series data to obtain the statistical features includes: Based on a preset time window, statistical features are extracted from the time series data and the frequency series data to obtain multiple individual statistical features. The statistical features are obtained by concatenating multiple individual statistical features; wherein the statistical features are a one-dimensional feature array.

[0012] According to a second aspect of this disclosure, an annotation apparatus for sound samples is provided, the apparatus comprising: An acquisition module is used to acquire a set of audio samples to be labeled; wherein, the set of audio samples to be labeled includes multiple audio samples to be labeled; The first extraction module is used to extract the Mel-spectrum information and the first sound statistical index of each of the sound samples to be labeled; The determination module is used to determine the category of the first sound statistical index of each of the sound samples to be labeled by means of a target determination rule, so as to obtain an initial label set corresponding to the sound sample set to be labeled; wherein, the target determination rule is a rule generated based on the statistical features corresponding to each sound category in the labeled sound samples; The calibration module is used to calibrate the initial tag set based on the Mel spectrum information of each of the sound samples to be labeled, so as to obtain the target tag set.

[0013] According to a third aspect of this disclosure, an electronic device is provided. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described above.

[0014] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the methods described above.

[0015] According to a fifth aspect of this disclosure, a computer program product is provided. The computer program product includes a computer program that, when executed by a processor, implements the methods described above in this disclosure.

[0016] This disclosure provides a method, apparatus, device, medium, and product for labeling sound samples. First, a set of sound samples to be labeled is obtained; wherein the set of sound samples to be labeled includes multiple sound samples to be labeled. Then, the Mel-spectrum information and a first sound statistical index of each sound sample to be labeled are extracted. Next, the first sound statistical index of each sound sample to be labeled is classified according to a target determination rule to obtain an initial label set corresponding to the set of sound samples to be labeled; wherein the target determination rule is a rule generated based on the statistical characteristics corresponding to each sound category in the labeled sound samples. Finally, the initial label set is calibrated based on the Mel-spectrum information of each sound sample to be labeled to obtain a target label set.

[0017] As described above, this embodiment of the present disclosure first extracts the Mel-spectrum information and first sound statistical index of each sound sample in the set of sound samples to be labeled, and then generates target determination rules based on the statistical features corresponding to each sound category in the labeled sound samples to determine the category of each sound sample to be labeled, so as to obtain an initial label set. Furthermore, the initial label set is calibrated by combining the Mel-spectrum information of each sound sample to be labeled. This not only enables batch labeling of sound sample sets, improves the efficiency of sound sample labeling, and reduces the time cost of manual labeling of each sample, but also further improves the accuracy and consistency of the target label set based on the initial label determination, thereby enhancing the reliability of the sound sample labeling results. Moreover, this embodiment of the present disclosure can also generate target determination rules based on the statistical features corresponding to each sound category in the labeled sound samples, which can improve the standardization and reusability of the sound sample labeling process, thus being more conducive to high-quality labeling of sound samples in complex sound scenarios. Attached Figure Description

[0018] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0019] Figure 1 A flowchart illustrating a method for annotating sound samples provided as an exemplary embodiment of this disclosure; Figure 2 This is a schematic diagram of a sound data preprocessing flow provided in an exemplary embodiment of the present disclosure; Figure 3 A schematic diagram illustrating the framing process of an audio signal provided as an exemplary embodiment of this disclosure; Figure 4 A schematic diagram of the Mel spectrum matrix and image generation flowchart provided for an exemplary embodiment of this disclosure; Figure 5 A schematic diagram of a Mel spectrum image provided for an exemplary embodiment of this disclosure; Figure 6 One of the flowcharts for a method of annotating sound samples provided as another exemplary embodiment of this disclosure; Figure 7 A schematic diagram comparing the Mel spectrum of faulty and normal samples provided for an exemplary embodiment of this disclosure; Figure 8 A second flowchart illustrating a method for annotating sound samples as provided in another exemplary embodiment of this disclosure; Figure 9A third flowchart of a method for annotating sound samples provided as another exemplary embodiment of this disclosure; Figure 10 A schematic block diagram of the functional modules of a sound sample annotation device provided as an exemplary embodiment of the present disclosure; Figure 11 A structural block diagram of an electronic device provided as an exemplary embodiment of this disclosure; Figure 12 A structural block diagram of a computer system provided as an exemplary embodiment of this disclosure; Figure 13 A structural block diagram of a computer program product provided for an exemplary embodiment of this disclosure. Detailed Implementation

[0020] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0021] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0022] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc., used in this disclosure are only used to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0023] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0024] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0025] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0026] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0027] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device. It is understood that the above notification and user authorization process is merely illustrative and does not constitute a limitation on the implementation of this disclosure; other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0028] With the rapid development of intelligent coal mining and sensor technology, the amount of industrial environmental sound data generated during production is increasing. Effective separation, identification, and utilization of this sound data has become a crucial foundation for improving the level of intelligent coal mining. However, the underground coal mine environment is complex, with strong background noise, often containing multiple sound components such as machine operation noise and rock friction noise, leading to blurred sound sample boundaries and significant annotation difficulties. Existing sound sample annotation methods mainly include manual annotation, semi-automatic annotation, automatic annotation, crowdsourcing annotation, and tool- or platform-based annotation methods. While manual annotation offers high accuracy, it suffers from high labor costs and low efficiency. Semi-automatic and automatic annotation, although more efficient, are limited by model performance and lack accuracy in complex industrial environments. Crowdsourcing annotation faces challenges in quality control and ensuring label accuracy. Tool- or platform-based annotation, while improving operational convenience, is easily influenced by subjective factors, making it difficult to establish unified and reliable annotation standards. Therefore, there is an urgent need for a sound sample annotation method suitable for complex industrial environments that improves annotation efficiency while ensuring the accuracy and standardization of annotation results.

[0029] In one embodiment, such as Figure 1 As shown, a method for annotating sound samples is provided, including the following steps: Step 101: Obtain the set of audio samples to be labeled.

[0030] Here, the executing entity can obtain a set of audio samples to be labeled, which includes multiple audio samples to be labeled. The audio samples to be labeled can be audio segments that have not yet been categorized. The set of audio samples to be labeled can be a collection of samples to be processed, consisting of multiple audio samples to be labeled. The executing entity can be a server, terminal device, labeling platform, or other electronic device with data processing capabilities.

[0031] In one possible embodiment, the set of sound samples to be labeled can be derived from continuous sound data collected under actual working conditions in the mine. The executing entity can extract, segment, or organize the collected continuous sound data to obtain multiple sound samples to be labeled, and combine the multiple sound samples to be labeled into a set of sound samples to be labeled. The sound samples to be labeled can correspond to normal working condition sounds, fault working condition sounds, or other industrial environmental sound samples to be distinguished. By organizing multiple sound samples to be labeled into a set of sound samples to be labeled in a unified manner, the executing entity can facilitate subsequent batch labeling based on unified rules.

[0032] Step 102: Extract the Mel spectrum information and first sound statistical index of each sound sample to be labeled.

[0033] Here, after obtaining the set of sound samples to be labeled, the executing entity can extract the Mel spectrum information and the first sound statistical index of each sound sample to be labeled. The Mel spectrum information can be the Mel spectrum matrix and the corresponding Mel spectrum image obtained by dividing the audio signal into frames, performing frequency domain transformation and Mel filtering, which are used to characterize the energy distribution of the sound sample to be labeled on the Mel frequency scale. The first sound statistical index can be a statistical quantity used to reflect the time domain and frequency domain characteristics of the sound sample, such as one or more of the following: short-time average energy, short-time zero-crossing rate, Fourier transform spectrum, energy, entropy, contrast, sharpness, and spectral bandwidth.

[0034] In one possible embodiment, the specific process by which the executing entity extracts the Mel-spectrum information and the first sound statistical index of each sound sample to be labeled may include: First, the executing entity preprocesses each audio sample to be labeled, such as... Figure 2 As shown, Figure 2 An exemplary schematic diagram of the audio data preprocessing workflow is shown. Preprocessing mainly includes pre-emphasis, framing, and windowing. Pre-emphasis is used to enhance the high-frequency components, making the signal spectrum smoother and facilitating subsequent feature extraction. Pre-emphasis can be implemented using a first-order FIR digital high-pass filter, whose transfer function can be expressed as:

[0035] Where α is the pre-weighting coefficient, which is close to 1 and represents the range of change in pre-weighting.

[0036] After pre-emphasis is completed, the executing entity can segment the audio signal into frames. Since the audio signal is generally non-stationary, but can be approximated as stationary within a short time range, the executing entity can divide the audio signal into multiple short time frames along the time axis. Each frame can be, for example, 10ms to 30ms in length. An overlap can be set between adjacent time frames as a frame shift to achieve a smooth transition between adjacent time frames. Figure 3 As shown, Figure 3 An exemplary schematic diagram of the framing process of an audio signal is shown. The executing entity can divide the original audio signal into multiple consecutive time frames according to a preset frame length, and retain a certain overlapping area between adjacent time frames by setting a preset frame shift, thereby improving the continuity and stability of subsequent feature extraction while ensuring the integrity of local temporal features.

[0037] After framing, the execution entity applies a window to each frame of audio signal to reduce spectral leakage caused by frame truncation and to mitigate the Gibbs effect. Commonly used window functions include rectangular windows, Hanning windows, and Hamming windows, which can be expressed as follows: Rectangular window:

[0038] Hanning Window:

[0039] Hanming Window:

[0040] Where L represents the window length, for the audio signal of the i-th frame. (n) Output after windowing for: (n) After completing preprocessing, the executing entity can extract Mel-spectrum information, such as... Figure 4 As shown, Figure 4 An exemplary diagram of the Mel spectrum matrix and image generation flowchart is shown. Specifically, the execution entity can first perform a short-time Fourier transform on the preprocessed short-time frames to convert the time-domain signal into a frequency-domain signal, obtaining the spectral data of each frame. Then, the amplitude spectrum of each frame is calculated, which can be expressed as:

[0041] Where X[k] is the result of the Fourier transform, and Re and Im represent the real and imaginary parts of the complex number, respectively.

[0042] Next, the executing entity applies a Mel filter bank to the frequency domain signal, transforming the signal to a Mel frequency scale, thus obtaining the Mel spectrum, which can be expressed as:

[0043] in, It is the response of the m-th Mel filter.

[0044] Furthermore, the Mel spectrum data of each frame can form a row of a Mel spectrum matrix, thus obtaining a Mel spectrum matrix of dimension T×F, where T represents the number of time frames and F represents the number of Mel frequency bands. In addition to the Mel spectrum matrix, a corresponding Mel spectrum image can also be generated, such as... Figure 5 As shown, Figure 5 An exemplary schematic diagram of a Mel spectrum image is shown, wherein the horizontal axis of the Mel spectrum image represents time, the vertical axis represents Mel frequency, and the color or brightness represents the energy intensity of the corresponding frequency component.

[0045] In addition, the executing entity can also extract first sound statistical indicators. Specifically, the first sound statistical indicators may include one or more of the following: short-time average energy, short-time zero-crossing rate, Fourier transform spectrum, energy, entropy, contrast, sharpness, and spectral bandwidth.

[0046] Short-time average energy refers to the value obtained by averaging the energy of an audio signal over a short time window. The executing entity can use short-time average energy to describe the energy intensity of the signal at a certain moment, thereby helping to capture the changes in the signal over different time periods and obtain information on the dynamic changes in the signal's energy.

[0047] The short-time zero-crossing rate refers to the number of times an audio signal crosses zero within a short time window. The executing entity can use the short-time zero-crossing rate to describe the frequency characteristics and waveform changes of the signal, thereby obtaining information on the dynamic changes of the signal.

[0048] A Fourier transform spectrum is a graph that shows the frequency distribution of a signal and the amplitude of its corresponding frequency components after the signal has been transformed to the frequency domain. The amplitude spectrum and phase spectrum of the spectrum show the intensity and phase information of the signal at different frequencies, respectively.

[0049] Energy refers to the total energy of an audio signal over a specific time period. It can usually be expressed as the average or sum of the squares of the signal amplitude. The executing entity can use energy to reflect the strength or loudness of the signal and detect changes in the volume of the audio and the intensity of audio events. For example, the climax of music or an explosion usually has high energy, while background noise or whispers usually have low energy.

[0050] Entropy is used to measure the uncertainty or complexity of an audio signal. It usually represents the randomness or complexity of information in an audio signal. It is used to describe the complexity of the statistical distribution of a signal within a frame. High entropy indicates that the signal is more complex and has more information in that frame, while low entropy indicates that the signal is more stable or simple. For example, complex passages in music usually have high entropy, while static background noise or white noise usually has low entropy.

[0051] In audio processing, contrast usually refers to the dynamic range of a signal, that is, the difference in intensity or loudness of different components in the signal. It can also be used to describe the relative intensity difference of different frequency components in the signal. High contrast means that the intensity or frequency components in the audio signal have a large difference, making some parts of the signal more prominent. For example, the climax of music usually has a high contrast and presents more obvious changes in intensity. Low contrast means that the intensity or frequency components in the signal have a small difference, making the signal more uniform. For example, stable background noise usually has a low contrast.

[0052] Clarity is used to measure the clarity of an audio signal. It usually involves the intelligibility of speech and the cleanliness of audio. High clarity means that the audio signal has richer details and less background noise, making the speech or music easier to understand. For example, high-quality audio messages usually have high clarity. Low clarity means that there may be more noise or distortion in the audio signal, making the speech or music difficult to understand. For example, low-quality recordings or speech in noisy environments usually have low clarity.

[0053] Spectral bandwidth refers to the range of frequencies contained in the spectrum of an audio signal. It is usually expressed as the span between the lowest and highest frequencies. Wide spectral bandwidth means that the audio signal covers a wide range of frequencies and usually has richer audio details and complexity. For example, music or natural sounds usually have a wide spectral bandwidth. Narrow spectral bandwidth means that the audio signal covers only a narrow range of frequencies and is usually more monotonous or lacks rich details. The signal is relatively simple. For example, some synthesized sounds or simple noise signals may have a narrow spectral bandwidth.

[0054] Through the above processing, the executing entity can extract the corresponding Mel-spectrum information and the first sound statistical index for each sound sample to be labeled.

[0055] In one possible embodiment, the executing entity can filter out target statistical indicators for generating target determination rules by comparing multiple sound statistical indicators of normal samples and faulty samples. The target statistical indicators include short-time average energy, entropy value, and amplitude spectrum in Fourier transform spectrum. The short-time average energy and entropy value can be used as time series data, and the amplitude spectrum can be used as frequency series data. The executing entity can perform feature extraction based on time series data and frequency series data, and generate target determination rules based on the extracted statistical features, which are then used to classify the first sound statistical indicator of the unlabeled sound sample.

[0056] Step 103: Classify the first sound statistical index of each sound sample to be labeled by using the target determination rule to obtain the initial label set corresponding to the sound sample set to be labeled.

[0057] Here, after the executing entity extracts the Mel spectrum information and the first sound statistical index of each sound sample to be labeled, it can use the target determination rule to determine the category of the first sound statistical index of each sound sample to be labeled, and obtain the initial label set corresponding to the sound sample set to be labeled. The target determination rule is a rule generated based on the statistical features corresponding to each sound category in the labeled sound samples, and the initial label set is the label set formed after the initial category determination of each sound sample to be labeled in the sound sample set to be labeled.

[0058] In one possible embodiment, the executing entity can input the first sound statistical index of each sound sample to be labeled into the target determination rule to determine the sound category corresponding to each sound sample to be labeled, and use the determined sound category as the initial label corresponding to each sound sample to be labeled, thereby obtaining the initial label set corresponding to the set of sound samples to be labeled.

[0059] For example, when the target determination rule indicates that the first sound statistical index of a sound sample to be labeled matches the determination condition corresponding to the fault sound category, the executing entity can determine the initial label of the sound sample to be labeled as the fault category label; when the target determination rule indicates that the first sound statistical index of a sound sample to be labeled matches the determination condition corresponding to the normal sound category, the executing entity can determine the initial label of the sound sample to be labeled as the normal category label.

[0060] Step 104: The initial tag set is calibrated based on the Mel spectrum information of each sound sample to be labeled, to obtain the target tag set.

[0061] Here, after the executing entity classifies the first sound statistical index of each sound sample to be labeled according to the target determination rule and obtains the initial label set corresponding to the sound sample set to be labeled, it can calibrate the initial label set based on the Mel spectrum information of each sound sample to be labeled to obtain the target label set. The target label set is the final label result set after further calibration, correction and fine-tuning of the initial label set.

[0062] In one possible embodiment, such as Figure 6 As shown, the initial tag set is calibrated based on the Mel-spectral information of each sound sample to be labeled, resulting in the target tag set. This process includes the following steps: Step 601: Randomly sample a portion of the tags in the initial tag set to determine the tag accuracy of the initial tag set.

[0063] Here, after the executing entity classifies the first sound statistical index of each sound sample to be labeled according to the target determination rule and obtains the initial label set corresponding to the sound sample set to be labeled, it can randomly sample some labels in the initial label set to determine the label accuracy of the initial label set. The label accuracy is used to characterize the overall accuracy of the initial label set.

[0064] In one possible embodiment, the executing entity can select a portion of the labels from the initial label set as inspection objects according to a preset sampling method, and manually verify the inspection objects to determine the proportion of the number of correct labels in the inspection objects to the total number of inspection objects. This proportion is used as the label accuracy rate of the initial label set. The label accuracy rate can be used to measure whether the current batch of the initial label set meets the preset accuracy requirements, thereby providing a basis for judging whether to directly output the target label set or further perform manual auxiliary calibration.

[0065] For example, if the initial label set includes 1000 labels, the executing entity can randomly select 100 labels as inspection objects according to a sampling ratio of 10%, and manually review the 100 labels. If the manual review results show that 93 labels are correct and 7 labels are incorrect, the executing entity can determine that the label accuracy rate of the current initial label set is 93%. If the preset accuracy requirement is 90%, the executing entity can consider that the initial label set has met the preset accuracy requirement and can directly determine the initial label set as the target label set. If the preset accuracy requirement is 95%, the executing entity can consider that the initial label set has not yet met the preset accuracy requirement, and further manual auxiliary calibration can be performed based on the Mel spectrum image.

[0066] Step 602: Generate a Mel spectrum image based on the Mel spectrum information of each sound sample to be labeled.

[0067] Here, after determining the label accuracy of the initial label set, the executing entity can generate a Mel spectrum image based on the Mel spectrum information of each sound sample to be labeled. The Mel spectrum image is a visualization image that corresponds one-to-one with each sound sample to be labeled. The horizontal axis represents time, the vertical axis represents Mel frequency, and the color or brightness represents the energy intensity of the corresponding frequency component.

[0068] In one possible embodiment, the executing entity can generate a Mel spectrum image corresponding to each sound sample to be labeled based on the Mel spectrum matrix corresponding to each sound sample to be labeled, for subsequent manual verification and auxiliary judgment.

[0069] For example, such as Figure 7 As shown, Figure 7 An exemplary diagram comparing the Mel spectra of faulty and normal samples is shown. In the Mel spectra corresponding to the faulty sample, obvious and striped spectral intensity changes can be observed, and a certain continuity is shown in the part with higher spectral intensity, which reflects the continuity of the scraper conveyor jamming fault. In contrast, in the Mel spectra corresponding to the normal sample, there is no obvious striped change in spectral intensity. In other words, the above differences between faulty and normal samples can be used as a reference for manual verification of sample labels.

[0070] It should be noted that under certain complex operating conditions, there may be a small number of samples whose Mel spectra are quite similar to those of faulty samples. In such cases, the executing entity can combine actual downhole human experience with actual sound listening results to make a comprehensive judgment and calibration of the sample labels in order to improve the accuracy and reliability of the target label set.

[0071] Step 603: Determine the target label set based on the label accuracy and the Mel spectrum image.

[0072] Here, after generating a Mel spectrum image based on the Mel spectrum information of each sound sample to be labeled, the executing entity can determine the target label set based on the label accuracy and the Mel spectrum image.

[0073] In one possible embodiment, determining the target label set based on label accuracy and Mel spectrum image includes the following steps: If the label accuracy reaches the preset accuracy requirement, the initial label set will be determined as the target label set.

[0074] Specifically, after the execution entity generates the Mel spectrum image, if the label accuracy reaches the preset accuracy requirement, it can be considered that the initial label set of the current batch has met the labeling accuracy requirement. At this time, there is no need to continue to adjust, and the initial label set can be directly determined as the target label set, and the target label set can be used as the final labeling result of the batch of sound samples to be labeled.

[0075] In one possible embodiment, the preset accuracy requirement can be pre-set according to the actual business scenario, annotation quality requirements, or the training requirements of the subsequent sound classification model. For example, the executing entity can set the corresponding accuracy threshold according to the requirements of the model training for label accuracy in order to control the output quality of the target label set.

[0076] In another possible embodiment, determining the target label set based on label accuracy and Mel spectrum image further includes the following steps: If the label accuracy does not meet the preset accuracy requirements, output the Mel spectrum image and receive the manual calibration instruction returned based on the Mel spectrum image; The initial tag set is calibrated and fine-tuned based on manual calibration instructions to obtain the target tag set.

[0077] Specifically, when the label accuracy does not meet the preset precision requirements, the executing entity can output the Mel spectrum image corresponding to the sound sample to be labeled for manual review and receive manual calibration instructions for the Mel spectrum image. The manual calibration instructions can be modification instructions, confirmation instructions, or reclassification instructions given by humans for some or all of the labels in the initial label set. The executing entity can calibrate and fine-tune the initial label set according to the manual calibration instructions to obtain the target label set. In other words, during the calibration process, the executing entity can make manual assistance judgments based on obvious spectral intensity change features in the Mel spectrum image. For samples under complex working conditions, it can further combine actual downhole human experience and actual sound listening results for comprehensive calibration to improve the accuracy and reliability of the target label set.

[0078] In this embodiment, firstly, the executing entity randomly samples a portion of the labels in the initial label set to determine the label accuracy of the initial label set; then, the executing entity generates a Mel spectrum image based on the Mel spectrum information of each sound sample to be labeled; finally, the executing entity determines the target label set based on the label accuracy and the Mel spectrum image.

[0079] As described above, this embodiment can effectively measure the overall accuracy of the initial label set by randomly sampling some labels in the initial label set. Based on this, the initial label set is calibrated by combining Mel spectrum images. This not only helps to improve the accuracy and reliability of the sound sample annotation results, but also helps to reduce the cost of manual full verification and improve the efficiency of sound sample annotation.

[0080] This disclosure provides a method, apparatus, device, medium, and product for labeling sound samples. First, a set of sound samples to be labeled is obtained; wherein the set of sound samples to be labeled includes multiple sound samples to be labeled. Then, the Mel-spectrum information and a first sound statistical index of each sound sample to be labeled are extracted. Next, the first sound statistical index of each sound sample to be labeled is classified according to a target determination rule to obtain an initial label set corresponding to the set of sound samples to be labeled; wherein the target determination rule is a rule generated based on the statistical characteristics corresponding to each sound category in the labeled sound samples. Finally, the initial label set is calibrated based on the Mel-spectrum information of each sound sample to be labeled to obtain a target label set.

[0081] As described above, this embodiment of the present disclosure first extracts the Mel-spectrum information and first sound statistical index of each sound sample in the set of sound samples to be labeled, and then generates target determination rules based on the statistical features corresponding to each sound category in the labeled sound samples to determine the category of each sound sample to be labeled, so as to obtain an initial label set. Furthermore, the initial label set is calibrated by combining the Mel-spectrum information of each sound sample to be labeled. This not only enables batch labeling of sound sample sets, improves the efficiency of sound sample labeling, and reduces the time cost of manual labeling of each sample, but also further improves the accuracy and consistency of the target label set based on the initial label determination, thereby enhancing the reliability of the sound sample labeling results. Moreover, this embodiment of the present disclosure can also generate target determination rules based on the statistical features corresponding to each sound category in the labeled sound samples, which can improve the standardization and reusability of the sound sample labeling process, thus being more conducive to high-quality labeling of sound samples in complex sound scenarios.

[0082] In one possible embodiment, such as Figure 8 As shown, before classifying each audio sample by its first audio statistical index according to the target determination rule to obtain the initial label set corresponding to the audio sample set to be labeled, the following steps are also included: Step 801: Extract features from the second sound statistical index of the labeled sound samples to obtain the statistical features corresponding to each sound category in the labeled sound samples.

[0083] Here, before the executing entity determines the category of each unlabeled sound sample by using the target determination rule to obtain the initial label set corresponding to the unlabeled sound sample set, it can extract features from the second sound statistical indicators of the labeled sound samples to obtain the statistical features corresponding to each sound category in the labeled sound samples. The second sound statistical indicator can be a set of statistical indicators corresponding to the labeled sound samples, and the statistical features can be feature data extracted based on the second sound statistical indicator to reflect the change features of the corresponding sound category in the time dimension and frequency dimension.

[0084] In one possible embodiment, such as Figure 9 As shown, feature extraction is performed on the second sound statistical index of the labeled sound samples to obtain the statistical features corresponding to each sound category in the labeled sound samples, including the following steps: Step 901: Filter the second sound statistical indicators to obtain the target statistical indicators.

[0085] Here, before the executing entity determines the category of each sound sample to be labeled by using the target determination rule to obtain the initial label set corresponding to the sound sample set, it can screen the second sound statistical indicators to obtain the target statistical indicators. The target statistical indicators can be statistical indicators that have a strong distinguishing effect on different sound categories, selected from multiple candidate statistical indicators. The target statistical indicators can include short-time average energy, entropy value, and amplitude spectrum in the Fourier transform spectrum.

[0086] In one possible embodiment, the executing entity can compare multiple sound statistical indicators of normal samples and faulty samples to select target statistical indicators for subsequent feature extraction and rule generation. Since short-time average energy and entropy value can reflect the characteristics of signal changes over time, and the amplitude spectrum in the Fourier transform spectrum can reflect the distribution characteristics of the signal in the frequency dimension, the executing entity can use short-time average energy, entropy value, and amplitude spectrum as target statistical indicators so that statistical features that can characterize the differences between different sound categories can be extracted in the future.

[0087] Step 902: Use the short-time average energy and entropy values ​​in the target statistical indicators as time series data, and use the amplitude spectrum in the target statistical indicators as frequency series data.

[0088] Here, after obtaining the target statistical indicators, the executing entity can use the short-time average energy and entropy values ​​in the target statistical indicators as time series data, and the amplitude spectrum in the target statistical indicators as frequency series data. The time series data can be data with time as the horizontal axis, reflecting the change of the signal over time, and the frequency series data can be data with frequency as the horizontal axis, reflecting the distribution of the signal in different frequency dimensions.

[0089] In one possible embodiment, both the short-time average energy and entropy value can change over time, with time as the horizontal axis. Therefore, the executing entity can use the short-time average energy and entropy value as time series data. The amplitude spectrum reflects the amplitude distribution corresponding to different frequency components, with frequency as the horizontal axis. Therefore, the executing entity can use the amplitude spectrum as frequency series data. By dividing the target statistical indicators into time series data and frequency series data, the executing entity can extract features from the time dimension and the frequency dimension respectively to improve the expressive power of subsequent statistical features.

[0090] Step 903: Extract features from the time series data and frequency series data to obtain statistical features.

[0091] Here, after the executing entity uses the short-time average energy and entropy value in the target statistical indicators as time series data and the amplitude spectrum in the target statistical indicators as frequency series data, it can extract features from the time series data and frequency series data to obtain statistical features. The statistical features can be feature data extracted based on the time series data and frequency series data, which are used to characterize the change patterns of the corresponding sound category in the time dimension and frequency dimension.

[0092] In one possible embodiment, feature extraction is performed on time series data and frequency series data to obtain statistical features, including the following steps: Based on a preset time window, statistical features are extracted from time series data and frequency series data to obtain multiple individual statistical features; Multiple individual statistical features are concatenated to obtain statistical features.

[0093] Specifically, after taking the short-time average energy and entropy values ​​in the target statistical indicators as time series data and the amplitude spectrum in the target statistical indicators as frequency series data, the executing entity can extract statistical features from the time series data and frequency series data based on a preset time window to obtain multiple individual statistical features. These multiple individual statistical features are then concatenated to obtain statistical features, where the statistical features can be a one-dimensional feature array.

[0094] In one possible embodiment, the executing entity can employ the tsfresh method based on statistical feature extraction to extract features from time series and frequency series data. Specifically, the executing entity can input the series data within a preset time window into the feature extraction process to extract multiple statistically significant feature indicators, thereby reflecting the changing characteristics of the data within that time window. Since each dimension of series data can generate multiple statistical features, the executing entity can concatenate these statistical features to form a one-dimensional feature array.

[0095] In one possible embodiment, after obtaining statistical features, the executing entity can perform correlation analysis on these features and filter out highly correlated features to reduce the time and space complexity of subsequent model training and rule generation. By extracting features from time-series and frequency-series data and obtaining statistical features, the executing entity can obtain effective feature representations for different sound categories, thereby providing support for the subsequent generation of target determination rules based on statistical features.

[0096] Step 802: Train the classification machine learning model based on statistical features to obtain the trained model.

[0097] Here, after obtaining the statistical features corresponding to each sound category in the labeled sound samples, the executing entity can train the classification machine learning model based on the statistical features to obtain the trained model. The classification machine learning model is a machine learning model used to distinguish different sound categories, and the trained model is a model that has the ability to determine sound categories after being trained on samples.

[0098] In one possible embodiment, executorically, the executing entity can first collect actual downhole working condition data to construct a model dataset, then perform preliminary analysis on the acquired data to filter out noisy and invalid data. Afterward, the executing entity can manually label the data and divide the labeled dataset into training, testing, and validation sets according to a preset ratio, for example, a 7:2:1 ratio. Specifically, the executing entity can use the training set to train and tune the hyperparameters of multiple classification machine learning models. These models can include one or more of random forest, support vector machine, and AdaBoost models. The executing entity can adjust parameters such as the number of iterations, learning rate, and batch size of multiple candidate models and select the model that performs best on the test set as the final model; for example, a random forest model can be chosen as the final model.

[0099] Furthermore, the executing entity can test the trained model on the test set and validation set. By comparing the model's accuracy, recall, F1 score, and AUC score, the optimal model hyperparameters and weight coefficients can be selected and validated on actual abnormal state time series data. Through the above training process, the executing entity can obtain the trained model so that it can generate target determination rules in the future.

[0100] Step 803: Generate target determination rules based on the trained model.

[0101] Here, after obtaining the trained model, the executing entity can generate target determination rules based on the trained model. The target determination rules can be rules used to indicate the determination conditions corresponding to different sound categories, which are used to determine the category of the first sound statistical index of the unlabeled sound sample.

[0102] In one possible embodiment, the executing entity can use the trained model to predict validation samples and obtain the corresponding labels, predicted probability values, and features that significantly influence the classification results. Based on these features, the executing entity can further extract corresponding judgment conditions to form target determination rules. Specifically, the executing entity can use the judgment rules output by the trained model as initial time-series data statistical rules, and analyze and adjust these rules in conjunction with actual validation results. For example, the executing entity can analyze the model's misclassified and missed samples, identify the causes of misclassification, and optimize the initial rules based on actual business scenarios to form target determination rules more suitable for actual working conditions.

[0103] Furthermore, after generating the target determination rule, the executing entity can use the target determination rule to classify the first sound statistical index of each sound sample to be labeled, thereby obtaining the initial label set corresponding to the sound sample set to be labeled. The specific steps can be found in steps 101-104, and will not be repeated here.

[0104] In this embodiment, firstly, the executing entity extracts features from the second sound statistical index of the labeled sound samples to obtain the statistical features corresponding to each sound category in the labeled sound samples; then, the executing entity trains the classification machine learning model based on the statistical features to obtain the trained model; finally, the executing entity generates target determination rules based on the trained model.

[0105] As described above, this embodiment first extracts features from the second sound statistical index of the labeled sound samples to obtain the statistical features corresponding to each sound category. Then, it trains the classification machine learning model based on the statistical features and further generates target determination rules based on the trained model. This can transform the category difference information in the labeled sound samples into reusable determination rules, which not only helps to improve the accuracy and standardization of subsequent sound sample category determination, but also helps to reduce reliance on human experience and improve the efficiency and stability of batch labeling of sound samples.

[0106] As described above, the technical solution of this disclosure obtains a set of sound samples to be labeled, extracts the Mel spectrum information and first sound statistical indicators of each sound sample, and generates target determination rules based on the statistical features corresponding to each sound category in the labeled sound samples to achieve initial label determination of the sound sample set to be labeled. Then, the initial label set is calibrated by combining the Mel spectrum information to obtain the target label set. This not only enables batch labeling of sound samples, improves the efficiency of sound sample labeling, and reduces the time cost of manual labeling of each sample, but also improves the accuracy, consistency, and reliability of the target label set by combining random sampling with Mel spectrum-assisted calibration. In addition, by extracting the statistical features of the labeled sound samples and generating reusable target determination rules, the technical solution of this disclosure can also enhance the standardization, normalization, and transferability of the sound sample labeling process, which is more conducive to high-quality labeling of sound samples in complex industrial environments.

[0107] By dividing each functional module according to its corresponding function, this disclosure provides a sound sample annotation device, which can be a server or a chip applied to a server. Figure 10 A schematic block diagram of the functional modules of a sound sample annotation device provided as an exemplary embodiment of this disclosure. Figure 10 As shown, the annotation device for the sound sample includes: The acquisition module 1001 is used to acquire a set of sound samples to be labeled; wherein, the set of sound samples to be labeled includes multiple sound samples to be labeled; The first extraction module 1002 is used to extract the Mel spectrum information and the first sound statistical index of each of the sound samples to be labeled; The determination module 1003 is used to determine the category of the first sound statistical index of each of the sound samples to be labeled by means of a target determination rule, so as to obtain an initial label set corresponding to the sound sample set to be labeled; wherein, the target determination rule is a rule generated based on the statistical features corresponding to each sound category in the labeled sound samples. The calibration module 1004 is used to calibrate the initial tag set based on the Mel spectrum information of each of the sound samples to be labeled, so as to obtain the target tag set.

[0108] In one embodiment, the calibration module 1004 includes: A sampling inspection unit is used to randomly sample a portion of the tags in the initial tag set to determine the tag accuracy rate of the initial tag set; wherein, the tag accuracy rate is used to characterize the overall accuracy of the initial tag set; The generation unit is used to generate a Mel spectrum image based on the Mel spectrum information of each of the sound samples to be labeled; The first determining unit is used to determine the target label set based on the label accuracy and the Mel spectrum image.

[0109] In one embodiment, the calibration module 1004 includes: The second determining unit is used to determine the initial tag set as the target tag set when the tag accuracy reaches the preset accuracy requirement.

[0110] In one embodiment, the calibration module 1004 includes: The receiving unit is configured to output the Mel spectrum image when the label accuracy does not meet the preset accuracy requirement, and to receive the manual calibration instruction returned based on the Mel spectrum image. The calibration unit is used to calibrate and fine-tune the initial tag set based on the manual calibration instructions to obtain the target tag set.

[0111] In one embodiment, the apparatus further includes: The second extraction module is used to extract features from the second sound statistical index of the labeled sound samples to obtain the statistical features corresponding to each sound category in the labeled sound samples; wherein, the statistical features are used to reflect the change features of the corresponding sound category in the time dimension and frequency dimension. The training module is used to train the classification machine learning model based on the statistical features to obtain the trained model; The generation module is used to generate the target determination rule based on the trained model.

[0112] In one embodiment, the second extraction module includes: A filtering unit is used to filter the second sound statistical indicators to obtain target statistical indicators; wherein, the target statistical indicators include short-time average energy, entropy value, and amplitude spectrum in Fourier transform spectrum diagram; The third determining unit is used to take the short-time average energy and entropy value in the target statistical indicators as time series data, and the amplitude spectrum in the target statistical indicators as frequency series data. The first extraction unit is used to extract features from the time series data and the frequency series data to obtain the statistical features.

[0113] In one embodiment, the second extraction module includes: The second extraction unit is used to extract statistical features from the time series data and the frequency series data based on a preset time window to obtain multiple individual statistical features; The splicing unit is used to splice multiple individual statistical features to obtain the statistical features; wherein the statistical features are a one-dimensional feature array.

[0114] Figure 11 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this disclosure. For example... Figure 11 As shown, the electronic device 1100 includes at least one processor 1101 and a memory 1102 coupled to the processor 1101. The processor 1101 can perform the corresponding steps in the methods disclosed in the embodiments of this disclosure.

[0115] The processor 1101 described above can also be called a central processing unit (CPU), which can be an integrated circuit chip with signal processing capabilities. Each step in the method disclosed in this embodiment can be implemented by the integrated logic circuitry in the processor 1101 or by software instructions. The processor 1101 can be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this embodiment can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can be located in the memory 1102, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor 1101 reads information from the memory 1102 and, in conjunction with its hardware, completes the steps of the method described above.

[0116] Furthermore, various operations / processes according to this disclosure, implemented via software and / or firmware, can be transmitted from a storage medium or network to a computer system with a dedicated hardware architecture, such as... Figure 12 The computer system 1200 shown is equipped with the programs that constitute the software. When various programs are installed, the computer system is able to perform various functions, including functions such as those described above. Figure 12 A block diagram of a computer system provided for an exemplary embodiment of this disclosure.

[0117] Computer system 1200 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0118] like Figure 12 As shown, the computer system 1200 includes a computing unit 1201, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 1202 or a computer program loaded from a storage unit 1208 into a random access memory (RAM) 1203. The RAM 1203 may also store various programs and data required for the operation of the computer system 1200. The computing unit 1201, ROM 1202, and RAM 1203 are interconnected via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0119] Multiple components in the computer system 1200 are connected to the I / O interface 1205, including: an input unit 1206, an output unit 1207, a storage unit 1208, and a communication unit 1209. The input unit 1206 can be any type of device capable of inputting information into the computer system 1200. The input unit 1206 can receive input numerical or character information and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 1207 can be any type of device capable of presenting information and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. The storage unit 1208 may include, but is not limited to, a hard disk and an optical disk. The communication unit 1209 allows the computer system 1200 to exchange information / data with other devices via a network such as the Internet, and may include, but is not limited to, a modem, network card, infrared communication device, wireless communication transceiver, and / or chipset, such as Bluetooth™ device, WiFi device, WiMax device, cellular communication device, and / or the like.

[0120] The computing unit 1201 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1201 performs the various methods and processes described above. For example, in some embodiments, the methods disclosed in this disclosure can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1208. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1100 via ROM 1202 and / or communication unit 1209. In some embodiments, the computing unit 1201 can be configured to perform the methods disclosed in this disclosure by any other suitable means (e.g., by means of firmware).

[0121] This disclosure also provides a computer-readable storage medium, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform the methods disclosed in this disclosure.

[0122] The computer-readable storage medium in this disclosure can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. The aforementioned computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specifically, the aforementioned computer-readable storage medium may include electrical connections based on one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0123] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0124] Figure 13 A computer program product 1300 is provided as an exemplary embodiment of the present disclosure. The computer program product 1300 includes a computer program 1301, wherein the computer program 1301, when executed by a processor, implements the methods disclosed in the embodiments of the present disclosure.

[0125] In embodiments of this disclosure, computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer.

[0126] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0127] The modules, components, or units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules, components, or units do not necessarily constitute a limitation on the module, component, or unit itself.

[0128] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that can be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0129] The above description is merely an embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0130] While specific embodiments of this disclosure have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. A method for labeling sound samples, characterized in that, The method includes: Obtain a set of audio samples to be labeled; wherein, the set of audio samples to be labeled includes multiple audio samples to be labeled; Extract the Mel-spectral information and first sound statistical index of each of the aforementioned sound samples to be labeled; The target determination rule is used to classify the first sound statistical index of each of the sound samples to be labeled by the target determination rule, so as to obtain the initial label set corresponding to the sound sample set to be labeled; wherein, the target determination rule is a rule generated based on the statistical features corresponding to each sound category in the labeled sound samples; The initial tag set is calibrated based on the Mel-spectral information of each of the sound samples to be labeled, to obtain the target tag set.

2. The method according to claim 1, characterized in that, The initial tag set is calibrated based on the Mel-spectral information of each of the audio samples to be labeled to obtain the target tag set, including: A subset of labels in the initial label set is randomly sampled to determine the label accuracy rate of the initial label set; wherein, the label accuracy rate is used to characterize the overall accuracy of the initial label set; A Mel spectrum image is generated based on the Mel spectrum information of each of the aforementioned sound samples to be labeled; The target label set is determined based on the label accuracy and the Mel spectrum image.

3. The method according to claim 2, characterized in that, Determining the target label set based on the label accuracy and the Mel spectrum image includes: If the label accuracy rate reaches the preset accuracy requirement, the initial label set is determined as the target label set.

4. The method according to claim 2, characterized in that, Determining the target label set based on the label accuracy and the Mel spectrum image includes: If the label accuracy does not meet the preset accuracy requirement, the Mel spectrum image is output, and a manual calibration instruction based on the Mel spectrum image is received. The initial tag set is calibrated and fine-tuned based on the manual calibration instructions to obtain the target tag set.

5. The method according to claim 1, characterized in that, Before classifying the first sound statistical index of each of the unlabeled sound samples according to the target determination rule to obtain the initial label set corresponding to the unlabeled sound sample set, the method further includes: Feature extraction is performed on the second sound statistical index of the labeled sound samples to obtain the statistical features corresponding to each sound category in the labeled sound samples; wherein, the statistical features are used to reflect the change features of the corresponding sound category in the time dimension and frequency dimension; The classification machine learning model is trained based on the statistical features to obtain the trained model; The target determination rule is generated based on the trained model.

6. The method according to claim 5, characterized in that, The step of extracting features from the second sound statistical index of the labeled sound samples to obtain the statistical features corresponding to each sound category in the labeled sound samples includes: The second sound statistical index is filtered to obtain the target statistical index; wherein, the target statistical index includes short-time average energy, entropy value and amplitude spectrum in Fourier transform spectrum; The short-time average energy and entropy values ​​in the target statistical indicators are used as time series data, and the amplitude spectrum in the target statistical indicators is used as frequency series data. The statistical features are obtained by extracting features from the time series data and the frequency series data.

7. The method according to claim 6, characterized in that, The step of extracting features from the time series data and the frequency series data to obtain the statistical features includes: Based on a preset time window, statistical features are extracted from the time series data and the frequency series data to obtain multiple individual statistical features. The statistical features are obtained by concatenating multiple individual statistical features; wherein the statistical features are a one-dimensional feature array.

8. A device for labeling sound samples, characterized in that, include: An acquisition module is used to acquire a set of audio samples to be labeled; wherein, the set of audio samples to be labeled includes multiple audio samples to be labeled; The first extraction module is used to extract the Mel-spectrum information and the first sound statistical index of each of the sound samples to be labeled; The determination module is used to determine the category of the first sound statistical index of each of the sound samples to be labeled by means of a target determination rule, so as to obtain an initial label set corresponding to the sound sample set to be labeled; wherein, the target determination rule is a rule generated based on the statistical features corresponding to each sound category in the labeled sound samples; The calibration module is used to calibrate the initial tag set based on the Mel spectrum information of each of the sound samples to be labeled, so as to obtain the target tag set.

9. An electronic device, characterized in that, include: At least one processor; Memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the method as described in any one of claims 1-7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images