Audio detection method, device, computer equipment and storage medium

Abstract

This application relates to an audio detection method, apparatus, computer device, and storage medium. The method includes: acquiring multiple audio segments from an audio file to be detected; determining a first detection model and a second detection model; extracting sub-audio features for each audio segment across multiple different audio dimensions; performing audio detection processing on a first feature set consisting of at least one sub-audio feature of the audio segment using the first detection model to obtain a first detection result corresponding to the audio segment; performing audio detection processing on a second feature set consisting of at least one sub-audio feature of the audio segment using the second detection model to obtain a second detection result corresponding to the audio segment; and determining the audio detection result corresponding to the audio file to be detected based on the first and second detection results corresponding to each audio segment. This method can improve the detection efficiency of audio detection.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an audio detection method, apparatus, computer device, and storage medium. Background Technology

[0002] With the development of science and technology, voice-activated devices are being used in various industries. For example, facial recognition payment devices can use speakers or other voice-activated devices to announce payment results. Due to the wide range of applications of voice-activated devices, manufacturers of these devices conduct quality inspections during production to ensure that the audio played by the devices is normal.

[0003] Currently, the main approach involves selecting a certain number of audio devices proportionally, collecting the audio to be tested played by these devices, and then using hardware testing equipment in an audio laboratory to test the audio, obtaining the test results, and determining the quality inspection result for the corresponding audio device based on the test results. However, the experimental environment required for testing the audio using hardware testing equipment is quite complex, resulting in low efficiency in audio testing. Summary of the Invention

[0004] Therefore, it is necessary to provide an audio detection method, apparatus, computer equipment, and storage medium that can improve the efficiency of audio detection in response to the above-mentioned technical problems.

[0005] An audio detection method, the method comprising:

[0006] Multiple audio segments are acquired from the audio to be detected, and a first detection model and a second detection model are determined, wherein the first detection model and the second detection model are models obtained through ensemble learning;

[0007] Extract sub-audio features of each audio segment under multiple different audio dimensions;

[0008] Using the first detection model, audio detection processing is performed on a first feature set consisting of at least one sub-audio feature of the audio segment to obtain a first detection result corresponding to the audio segment;

[0009] Using the second detection model, audio detection processing is performed on a second feature set consisting of at least one sub-audio feature of the audio segment to obtain a second detection result corresponding to the audio segment; wherein, the second feature set contains sub-audio features that are not present in the first feature set:

[0010] Based on the first and second detection results corresponding to each audio segment, the audio detection result corresponding to the audio to be detected is determined.

[0011] An audio detection device, the device comprising:

[0012] The feature acquisition module is used to acquire multiple audio segments in the audio to be detected, determine a first detection model and a second detection model, wherein the first detection model and the second detection model are models obtained through ensemble learning; and extract sub-audio features of each audio segment under multiple different audio dimensions.

[0013] The detection processing module is configured to perform audio detection processing on a first feature set consisting of at least one sub-audio feature of the audio segment using the first detection model, to obtain a first detection result corresponding to the audio segment; and to perform audio detection processing on a second feature set consisting of at least one sub-audio feature of the audio segment using the second detection model, to obtain a second detection result corresponding to the audio segment; wherein the second feature set contains sub-audio features that are not present in the first feature set.

[0014] The result determination module is used to determine the audio detection result corresponding to the audio to be detected based on the first detection result and the second detection result corresponding to each audio segment.

[0015] In one embodiment, the audio detection device is further configured to acquire the audio to be detected played by the sound-producing device to be inspected, and to segment the audio to be detected by means of a preset segment duration and segment step size to obtain multiple audio segments; and to determine the quality inspection result of the sound-producing device based on the audio detection result of the audio to be inspected.

[0016] In one embodiment, the audio detection device is further configured to extract sub-audio features under different audio dimensions; the audio dimensions include at least a timbre dimension, a pitch dimension, and a saturation dimension; the sub-audio features of the timbre dimension include Mel cepstrum features and spectral centroid features; the sub-audio features of the pitch dimension include Mel spectral features; the sub-audio features of the saturation dimension include zero-crossing rate features and spectral flatness features; the first feature set includes Mel spectral features and Mel cepstrum features; the second feature set includes Mel spectral features, Mel cepstrum features, spectral centroid features, zero-crossing rate features, and spectral flatness features.

[0017] In one embodiment, the feature acquisition module is further configured to determine the correspondence between the number of sampling points of the fast Fourier transform and the sub-audio features under different audio dimensions; perform fast Fourier transform on the audio segment using the multiple sampling points included in the correspondence to obtain multiple spectrograms; and extract the corresponding sub-audio features from each spectrogram according to the correspondence.

[0018] In one embodiment, the detection processing module further includes a first detection module, configured to perform feature concatenation processing on the spectrogram corresponding to the audio segment and each sub-audio feature in the first feature set to obtain a plurality of first features to be processed; perform feature processing on the plurality of first features to be processed by the first detection model to obtain feature processing results, and obtain a first detection result of the audio segment based on the feature processing results.

[0019] In one embodiment, the first detection model includes multiple first processing layers; the first detection module is further configured to perform feature processing on the multiple first features to be processed through the first first processing layer among the multiple first processing layers to obtain the output result of the first first processing layer; for each first processing layer other than the first first processing layer, the output result of the preceding first processing layer is used as the input data of the corresponding first processing layer, and each first processing layer performs feature processing on its own input data to obtain the output result of each first processing layer; based on the output result of the last first processing layer and the multiple first features to be processed, a feature processing result corresponding to the first feature set is obtained.

[0020] In one embodiment, the detection processing module further includes a second detection module, configured to perform feature concatenation processing on the spectrogram corresponding to the audio segment and each sub-audio feature in the second feature set to obtain multiple second features to be processed; perform feature processing on the multiple second features to be processed by the second detection model to obtain feature processing results, and obtain a second detection result of the audio segment based on the feature processing results.

[0021] In one embodiment, the second detection model includes multiple second processing layers; the second detection module is further configured to perform feature processing on the multiple second features to be processed through the first second processing layer among the multiple second processing layers to obtain the output result of the first second processing layer; for each second processing layer other than the first second processing layer, the multiple second features to be processed and the output result of the preceding second processing layer are used as the input data of the corresponding second processing layer, and each second processing layer performs feature processing on its own input data to obtain the output result of each second processing layer; based on the output result of the last second processing layer, the feature processing result corresponding to the second feature set is obtained.

[0022] In one embodiment, the second processing layer includes a pooling layer and a convolutional layer; the second detection module is further configured to, for each second processing layer other than the first second processing layer, perform convolution processing on the output of the preceding second processing layer and multiple second features to be processed through the convolutional layer in the current second processing layer to obtain a convolution result; perform pooling processing on the convolution result and multiple second features to be processed through the pooling layer in the current second processing layer to obtain a pooling result, and use the pooling result as the output of the current second processing layer.

[0023] In one embodiment, the result determination module further includes a correction module, configured to determine the model weights corresponding to the first detection model and the second detection model respectively; determine a first ratio between normal audio segments and total audio segments in the audio to be detected based on the first detection results corresponding to each audio segment, and correct the first ratio based on the model weights corresponding to the first detection model to obtain a first correction value; determine a second ratio between normal audio segments and total audio segments in the audio to be detected based on the second detection results corresponding to each audio segment, and correct the second ratio based on the model weights corresponding to the second detection model to obtain a second correction value; and determine the audio detection result corresponding to the audio to be detected based on the first correction value and the second correction value.

[0024] In one embodiment, the correction module is further configured to obtain a first quantity value of normal audio segments and a second quantity value of abnormal audio segments included in the corrected audio to be detected based on the first correction value and the second correction value; when the first quantity value is greater than the second quantity value, the audio to be detected is determined to be normal audio; when the first quantity value is less than or equal to the second quantity value, the audio to be detected is determined to be abnormal audio.

[0025] In one embodiment, the audio detection device further includes a training module, configured to acquire a first sample speech and a first sample label corresponding to the first sample speech, acquire a second sample speech and a second sample label corresponding to the second sample speech; extract sub-audio features of each audio segment in the first sample speech under multiple different audio dimensions to obtain a first sample feature set; extract sub-audio features of the second sample speech under multiple different audio dimensions to obtain a second sample feature set; the second sample feature set contains sub-audio features not present in the first sample feature set; perform audio prediction processing on the first sample feature set using a first detection model to be trained to obtain a first prediction result corresponding to the first sample speech; perform audio prediction processing on the second sample feature set using a second detection model to be trained to obtain a second prediction result corresponding to the second sample speech; perform first training on the first detection model using the first prediction result and the first sample label until a first training stopping condition is met, to obtain a trained first detection model; perform second training on the second detection model using the second prediction result and the second sample label until a second training stopping condition is met, to obtain a trained second detection model; and combine the trained first detection model and the second detection model to obtain a trained audio detection model.

[0026] In one embodiment, the training module is further configured to: acquire test speech and test labels corresponding to the test speech; perform a first speech test on the test speech using a trained first detection model to obtain a first test result, and determine a first difference between the first test result and the test label; perform a second speech test on the test speech using a trained second detection model to obtain a second test result, and determine a second difference between the second test result and the test label; determine the model weights corresponding to the first detection model and the second detection model based on the first difference and the second difference; and obtain a trained audio detection model based on the trained first detection model, the model weights corresponding to the first detection model, the trained second detection model, and the model weights corresponding to the second detection model.

[0027] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0028] Multiple audio segments are acquired from the audio to be detected, and a first detection model and a second detection model are determined, wherein the first detection model and the second detection model are models obtained through ensemble learning;

[0029] Extract sub-audio features of each audio segment under multiple different audio dimensions;

[0030] Using the first detection model, audio detection processing is performed on a first feature set consisting of at least one sub-audio feature of the audio segment to obtain a first detection result corresponding to the audio segment;

[0031] Using the second detection model, audio detection processing is performed on a second feature set consisting of at least one sub-audio feature of the audio segment to obtain a second detection result corresponding to the audio segment; wherein, the second feature set contains sub-audio features that are not present in the first feature set:

[0032] Based on the first and second detection results corresponding to each audio segment, the audio detection result corresponding to the audio to be detected is determined.

[0033] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0034] Multiple audio segments are acquired from the audio to be detected, and a first detection model and a second detection model are determined, wherein the first detection model and the second detection model are models obtained through ensemble learning;

[0035] Extract sub-audio features of each audio segment under multiple different audio dimensions;

[0036] Using the first detection model, audio detection processing is performed on a first feature set consisting of at least one sub-audio feature of the audio segment to obtain a first detection result corresponding to the audio segment;

[0037] Using the second detection model, audio detection processing is performed on a second feature set consisting of at least one sub-audio feature of the audio segment to obtain a second detection result corresponding to the audio segment; wherein, the second feature set contains sub-audio features that are not present in the first feature set:

[0038] Based on the first and second detection results corresponding to each audio segment, the audio detection result corresponding to the audio to be detected is determined.

[0039] A computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the following steps: acquiring multiple audio segments from an audio to be detected; determining a first detection model and a second detection model, wherein the first and second detection models are models obtained through ensemble learning; extracting sub-audio features of each audio segment under multiple different audio dimensions; performing audio detection processing on a first feature set consisting of at least one sub-audio feature of the audio segment using the first detection model to obtain a first detection result corresponding to the audio segment; performing audio detection processing on a second feature set consisting of at least one sub-audio feature of the audio segment using the second detection model to obtain a second detection result corresponding to the audio segment; wherein the second feature set contains sub-audio features not present in the first feature set; and determining an audio detection result corresponding to the audio to be detected based on the first and second detection results corresponding to each audio segment.

[0040] The aforementioned audio detection method, apparatus, computer equipment, storage medium, and computer program, by acquiring audio segments of the audio to be detected, can extract features from the audio segments to obtain sub-audio features under multiple different audio dimensions. By obtaining these sub-audio features under multiple different audio dimensions, each sub-audio feature can be divided into a first feature set and a second feature set. Audio detection processing can then be performed on the first feature set based on a first detection model to obtain a first detection result. Similarly, audio detection processing can be performed on the second feature set based on a second detection model to obtain a second detection result. By determining the first and second detection results corresponding to each audio segment, the first and second detection results can be combined to automatically generate an audio detection result corresponding to the audio to be detected, thereby improving the accuracy of the audio detection result. Since the detection model detects the audio to be detected, compared to traditional methods using hardware detection equipment, this application does not require the deployment of a complex experimental environment, thus improving the efficiency of audio detection.

[0041] In addition, since hardware testing equipment is generally expensive, testing the audio to be tested using a testing model can greatly reduce testing costs. Attached Figure Description

[0042] Figure 1 This is a diagram illustrating the application environment of an audio detection method in one embodiment;

[0043] Figure 2 This is a flowchart illustrating an audio detection method in one embodiment;

[0044] Figure 3 This is a flowchart illustrating the model training steps in one embodiment;

[0045] Figure 4 This is a schematic diagram of sample speech in one embodiment;

[0046] Figure 5 This is a schematic diagram of the model training process in one embodiment;

[0047] Figure 6 This is a schematic diagram of the overall framework for model training in one embodiment;

[0048] Figure 7 This is a flowchart illustrating the quality inspection process for a sound-producing device in one embodiment.

[0049] Figure 8 This is a flowchart illustrating the audio detection steps in a specific embodiment;

[0050] Figure 9 This is a flowchart illustrating the audio detection step in another specific embodiment;

[0051] Figure 10 This is a structural block diagram of an audio detection device in one embodiment;

[0052] Figure 11 This is a structural block diagram of an audio detection device in one embodiment;

[0053] Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] Figure 1 This is a diagram illustrating the application environment of an audio detection method in one embodiment. (Refer to...) Figure 1This audio detection method is applied to an audio detection system 100. The audio detection system 100 includes a sound-producing device 102, a terminal 104, and a server 106. Both the terminal 104 and the server 106 can be used independently to execute the audio detection method, or they can execute it collaboratively. The following description uses the collaborative execution of the audio detection method by the terminal 104 and the server 106 as an example. The terminal 104 is equipped with an audio acquisition device, which can acquire the audio played by the sound-producing device 102 to obtain the audio to be detected, and send the audio to be detected to the server 106. The server 106 can also run an audio detection model, which can perform audio detection on the audio to be detected, obtain the audio detection result, and return the audio detection result to the terminal 104 so that the terminal 104 can display the detection result accordingly.

[0056] The sound-producing device 102 can be a speaker, audio system, loudspeaker, or other device used for sound playback. The terminal 104 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, in-vehicle terminal, smart TV, etc., but is not limited to these. The server 106 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal 104 and server 106 can be connected via a network. This application does not impose any limitations on these aspects.

[0057] In one embodiment, such as Figure 2 As shown, an audio detection method is provided, which is applied to... Figure 1 Taking a computer device as an example, the method includes a terminal and a server. The audio detection method includes the following steps:

[0058] Step S202: Obtain multiple audio segments from the audio to be detected, and determine the first detection model and the second detection model, wherein the first detection model and the second detection model are models obtained through ensemble learning.

[0059] Both the first and second detection models are pre-trained machine learning models with audio detection capabilities. The computer device can obtain the model structure and parameters of a general machine learning model, import these parameters into the first or second detection model to obtain a first or second detection model with the model parameters, and then train the first or second detection model with the model parameters using training samples to obtain a fully trained first or second detection model.

[0060] The first and second detection models are obtained through ensemble learning. Ensemble learning refers to a machine learning model that utilizes multiple learners to achieve a learning task. Its general structure involves first generating a set of "individual learners" and then combining them using a certain strategy. These "individual learners" can be either the first or second detection model in this application. By combining the first and second detection models, audio detection can be achieved.

[0061] Specifically, when audio detection is required, since the duration of the audio to be detected is generally long, it can be segmented to obtain multiple audio segments for easier detection. In one embodiment, an audio acquisition device is installed in the computer device to acquire the audio to be detected. The audio acquisition device can be a microphone, earpiece, or other device used for sound acquisition.

[0062] In one embodiment, when the audio to be detected is divided into multiple audio segments, the computer device can identify silent segments among the multiple audio segments and delete the corresponding silent segments to obtain non-silent audio segments. Further, the computer device extracts sub-audio features of the non-silent segments under different audio dimensions. Since silent segments contain fewer audio features, they reduce the accuracy of audio detection for that audio segment by the first or second detection model. Therefore, deleting the corresponding silent segments can improve the accuracy of audio detection.

[0063] In one embodiment, obtaining multiple audio segments from the audio to be detected includes: obtaining the audio to be detected played by the sound-producing device to be inspected, and segmenting the audio to be detected into multiple audio segments by using a preset segment duration and segment step size; the method further includes: determining the quality inspection result of the sound-producing device based on the audio detection result of the audio to be detected.

[0064] Specifically, before using a sound-producing device, its quality can be inspected to determine whether the audio played by the device is normal, that is, to determine whether the device is silent and whether the audio played by the device has popping sounds or noise.

[0065] Once the audio device to be inspected is obtained, the user can control the device to play audio and control the computer to capture the played audio to obtain the audio to be tested. To improve the accuracy of the audio device quality inspection, the user can control the computer to capture the audio to be tested for a certain duration. For example, the user can control the computer to capture audio to be tested for more than one minute.

[0066] Furthermore, the computer device determines the preset segment duration and segment step size, and generates a sliding window based on the segment duration. It then controls the sliding window to slide within the audio to be detected according to the segment step size, treating the audio segment selected by the sliding window as a single audio segment. For example, when the segment duration is 3 seconds and the segment step size is 2 seconds, the computer device treats the audio from second 0 to second 3 as one audio segment, the audio from second 2 to fifth as another audio segment, and so on, until the last audio segment is obtained.

[0067] Furthermore, when multiple audio segments are acquired, for each audio segment, the computer device can perform audio detection processing using a first detection model and a second detection model to obtain the detection result for that audio segment. The detection results of all audio segments are then combined to obtain the audio detection result for the audio to be detected. When the audio detection result indicates that the audio to be detected has sound and no plosives or noise, the computer device determines that the device playing the audio to be detected is a normal device. When the audio detection result indicates that the audio to be detected has no sound, plosives, or noise, the computer device determines that the device playing the audio to be detected is an abnormal device.

[0068] In one embodiment, the computer device can determine the frequency of plosive sounds or noise in the audio to be tested based on the first detection result and the second detection result corresponding to each audio segment, and determine the quality inspection result of the sound-producing device to be inspected based on the frequency of plosive sounds or noise. When the frequency of plosive sounds or noise in the audio to be tested is less than a preset frequency threshold, the sound-producing device to be inspected is determined to be a normal sound-producing device. When the frequency of plosive sounds or noise in the audio to be tested is greater than or equal to the preset threshold, the sound-producing device to be inspected is determined to be an abnormal sound-producing device.

[0069] In the above embodiments, by segmenting the audio to be detected, a longer audio to be detected can be divided into shorter audio segments, which facilitates the subsequent audio detection of the audio segments.

[0070] Step S204: Extract sub-audio features of each audio segment under multiple different audio dimensions.

[0071] The audio dimension includes at least one of the following: timbre dimension, pitch dimension, saturation dimension, perception dimension, and energy dimension. Sub-audio features under the timbre dimension can be used to characterize the timbre characteristics of audio; sub-audio features under the pitch dimension can be used to characterize the pitch characteristics of audio; sub-audio features under the saturation dimension can be used to characterize the saturation characteristics of audio; sub-audio features under the perception dimension can be used to characterize the human ear's perception of audio; and sub-audio features under the energy dimension can be used to characterize the energy contained in audio within a certain time range.

[0072] Timbre, also known as tone color, is one of the attributes of sound. It primarily reflects the unique quality of the sound emitted by each object. For example, timbre reflects the distinctive waveform characteristics of different sounds. Pitch reflects the highness or lowness of a sound's frequency. Pitch is mainly determined by the frequency of the sound, but it is also related to the sound intensity. For a pure tone of a certain intensity, the pitch rises and falls with the frequency. Saturation reflects the thickness of the mid-frequency range. The thicker the mid-frequency range, the higher the saturation. For example, when a sound is uniform across all frequency bands within the range of human hearing and reaches a certain loudness requirement, the sound can be considered to have high saturation. Sub-audio frequency features in the perceptual dimension can be used to characterize the human ear's perception of audio. For example, sub-audio frequency features in the perceptual dimension can be used to characterize the intensity of the sound's loudness and the sharpness of the sound as subjectively perceived by the human ear.

[0073] Specifically, for each of the multiple audio segments, the computer device can perform a Fourier transform on the current audio segment, converting it from the time domain to the frequency domain, and obtaining the spectrogram of the current audio segment. Furthermore, the computer device processes the spectrogram using various preset feature extraction strategies to extract sub-audio features of the current audio segment in different audio dimensions.

[0074] The preset feature extraction strategy can be a preset feature extraction algorithm or a pre-trained feature extraction machine learning model, etc. The feature extraction machine learning model can acquire audio feature extraction capabilities through sample learning. The machine learning model can employ neural network models, dual-path network models (DPN), support vector machines, or logistic regression models, etc.

[0075] In one embodiment, the feature extraction machine learning model can be a pre-trained, general-purpose machine learning model with feature extraction capabilities. However, applying this general-purpose model to a specific scenario yields poor results; therefore, further training and optimization of the general-purpose model using scenario-specific samples are necessary. The feature extraction machine learning model can be a complex network model composed of multiple interconnected layers. It may include multiple feature extraction layers, each with corresponding model parameters. Each layer may have multiple model parameters, which linearly or non-linearly transform the input audio segment to obtain a feature map as the computation result.

[0076] In one embodiment, when the current audio segment is obtained, the computer device can perform frame segmentation on the current audio segment to obtain multiple audio frames, and perform a Fourier transform on each of the multiple audio frames to obtain a corresponding spectrogram sub-map for each audio frame. Further, the computer device concatenates the spectrogram sub-maps corresponding to each audio frame according to the time dimension to obtain the spectrogram corresponding to the current audio segment, thereby enabling the feature extraction machine learning model to extract the corresponding sub-audio features from the spectrogram.

[0077] Step S206: Using the first detection model, perform audio detection processing on the first feature set consisting of at least one sub-audio feature of the audio segment to obtain the first detection result corresponding to the audio segment.

[0078] Specifically, when extracting sub-audio features of an audio segment under different audio dimensions, the computer device can divide multiple sub-audio features into a first feature set and a second feature set. The second feature set contains sub-audio features not present in the first feature set. Further, the computer device inputs the first feature set into a pre-trained first detection model, which performs feature processing on the first feature set to achieve audio detection processing for the corresponding audio segment, obtaining the corresponding first detection result. In simple terms, the computer device can use the first detection model to perform audio detection processing on each audio segment in the audio to be detected, obtaining the first detection result corresponding to each audio segment.

[0079] Step S208: Using the second detection model, perform audio detection processing on the second feature set consisting of at least one sub-audio feature of the audio segment to obtain the second detection result corresponding to the audio segment; wherein, the second feature set contains sub-audio features that are not present in the first feature set.

[0080] Specifically, when the second feature set is obtained, the computer device can input the second feature set into a pre-trained second detection model. The second detection model performs feature processing on the second feature set to achieve audio detection processing of the corresponding audio segment and obtain the corresponding second detection result. In simple terms, the computer device can use the second detection model to perform audio detection processing on each audio segment in the audio to be detected, obtaining the corresponding second detection result for each audio segment.

[0081] In one embodiment, the computer device can divide multiple sub-audio features according to the characteristics of a first detection model and a second detection model to obtain a first feature set and a second feature set. For example, when the first detection model can better process sub-audio features in the timbre and pitch dimensions, the computer device will allocate the sub-audio features in the timbre and pitch dimensions to the first feature set. When the second detection model can better process sub-audio features in the timbre, pitch, and saturation dimensions, the computer device will allocate the sub-audio features in the timbre, pitch, and saturation dimensions to the second feature set. Furthermore, if the second detection model can handle more types of sub-audio features than the first detection model, the computer device can allocate most of the extracted sub-audio features to the second feature set and allocate a small portion of the extracted sub-audio features to the first feature set.

[0082] In one embodiment, since the first detection model and the second detection model are complementary models obtained through ensemble learning, the extracted sub-audio features can be divided according to the audio dimension to obtain a first feature set and a second feature set. This allows the first detection model and the second detection model to perform audio detection processing on the same audio segment from different audio dimensions, obtaining a first detection result and a second detection result. Consequently, the audio detection result obtained by combining the first detection result and the second detection result is more accurate.

[0083] Step S210: Determine the audio detection result corresponding to the audio to be detected based on the first detection result and the second detection result corresponding to each audio segment.

[0084] Specifically, for each of the multiple audio segments, the computer device can determine a comprehensive detection result corresponding to the current audio segment based on the first and second detection results corresponding to the current audio segment. Based on the comprehensive detection results corresponding to each audio segment, the device can determine the audio detection result corresponding to the audio to be detected. The comprehensive detection result can indicate that the audio segment is a normal audio segment or an abnormal audio segment. When the comprehensive detection result determines that an audio segment is a normal audio segment, it can be determined that the audio segment has sound and no popping sounds or noise. When the comprehensive detection result determines that an audio segment is an abnormal audio segment, it can be determined that the audio segment has no sound, popping sounds, or noise.

[0085] In one embodiment, the computer device can combine the first detection result and the second detection result corresponding to each audio segment to vote on the audio segments, and determine the audio detection result corresponding to the audio to be detected based on the voting result. For each audio segment among multiple audio segments, when the comprehensive detection result of the current audio segment is determined to be a normal audio segment based on the first detection result and the second detection result corresponding to the current audio segment, the computer device will cast its vote for the normal audio segment; when the comprehensive detection result of the current audio segment is determined to be an abnormal audio segment based on the first detection result and the second detection result corresponding to the current audio segment, the computer device will cast its vote for the abnormal audio segment. After the voting is completed, the computer device will count the number of votes obtained by the normal audio segment and the abnormal audio segment respectively. When the number of votes obtained by the normal audio segment is higher than the number of votes obtained by the abnormal audio segment, the audio detection result is determined to be normal audio; when the number of votes obtained by the normal audio segment is equal to or lower than the number of votes obtained by the abnormal audio segment, the audio detection result is determined to be abnormal audio.

[0086] In one embodiment, both the first detection result and the second detection result can be a single numerical value. For example, both the first detection result and the second detection result can be probability values ​​indicating that the audio segment is a normal audio segment. Further, the computer device can determine the weights corresponding to the first detection model and the second detection model, and combine the weights, the first detection result, and the second detection result to obtain the final probability value (also called the comprehensive detection result) that the audio segment is a normal audio segment. For example, the computer device can perform a weighted summation operation on the first detection result and the second detection result to obtain the final probability value that the audio segment is a normal audio segment. When the probability value is greater than a preset probability threshold, the audio segment is determined to be a normal audio segment; when the probability value is less than the preset probability threshold, the audio segment is determined to be an abnormal audio segment. By combining the first detection result, the second detection result, and the weights to determine the probability value of an audio segment as a normal audio segment, the accuracy of the determined probability value can be greatly improved, thereby improving the accuracy of the audio detection result.

[0087] In the aforementioned audio detection method, by acquiring audio segments of the audio to be detected, features can be extracted from the audio segments to obtain sub-audio features under multiple different audio dimensions. These sub-audio features can be divided into a first feature set and a second feature set. Audio detection processing can then be performed on the first feature set based on a first detection model to obtain a first detection result. Similarly, audio detection processing can be performed on the second feature set based on a second detection model to obtain a second detection result. By determining the first and second detection results corresponding to each audio segment, the first and second detection results can be combined to automatically generate an audio detection result corresponding to the audio to be detected, thereby improving the accuracy of the audio detection result. Since the detection model detects the audio to be detected, compared to the traditional method of detecting the audio to be detected using hardware detection equipment, this application does not require the deployment of a complex experimental environment, thus improving the efficiency of audio detection.

[0088] In addition, since hardware testing equipment is generally expensive, testing the audio to be tested using a testing model can greatly reduce testing costs.

[0089] In one embodiment, the audio dimension includes at least a timbre dimension, a pitch dimension, and a saturation dimension; the sub-audio features of the timbre dimension include Mel cepstrum features and spectral centroid features; the sub-audio features of the pitch dimension include Mel spectral features; the sub-audio features of the saturation dimension include zero-crossing rate features and spectral flatness features; the first feature set includes Mel spectral features and Mel cepstrum features; the second feature set includes Mel spectral features, Mel cepstrum features, spectral centroid features, zero-crossing rate features, and spectral flatness features.

[0090] Specifically, computer equipment can extract Mel-frequency cepstral features and spectral centroid features from audio clips to determine timbre. Mel-frequency cepstral features are a set of feature vectors obtained by encoding the physical information of speech (spectral envelope and details). Spectral centroid features refer to the energy concentration points in the signal spectrum, reflecting the brightness of the timbre; brighter signals have more concentrated energy in the high-frequency range, resulting in a larger centroid value. Furthermore, computer equipment can filter the spectrum of the audio clip using a Mel filter to obtain Mel-frequency spectral features to determine pitch. Mel-frequency spectral features reflect the sensitivity of human hearing to sound frequencies; higher frequencies correspond to higher pitches. Computer equipment can also extract zero-crossing rate features and spectral flatness from audio clips to determine saturation. Zero-crossing rate features reflect the number of times a sound signal crosses zero points per unit time, while spectral flatness reflects the similarity between the sound signal and noise; a larger value indicates a higher likelihood of noise. It is easy to understand that a sub-audio feature may also belong to multiple audio dimensions. For example, the Mel cepstral feature takes into account the degree of human ear's perception of different frequencies, so this feature can also be classified as a perception dimension.

[0091] When multiple sub-audio features across multiple audio dimensions are obtained, since the first detection model is suitable for processing sub-audio features of timbre and pitch dimensions, and the second detection model is suitable for processing sub-audio features of timbre, pitch, and saturation dimensions, the computer device can use Mel-spectral features and Mel-cepstrum features as the first feature set, and Mel-spectral features, Mel-cepstrum features, spectral centroid features, zero-crossing rate features, and spectral flatness features as the second feature set. Of course, the computer device can also divide the sub-audio features according to user-specified operations and user requirements to obtain the first and second feature sets. This embodiment does not impose limitations here.

[0092] In this embodiment, by extracting different sub-audio features under different audio dimensions, subsequent comprehensive audio detection of audio segments can be performed based on the sub-audio features of different audio dimensions, thereby improving the detection accuracy of audio detection.

[0093] In one implementation, sub-audio features of each audio segment are extracted under multiple different audio dimensions, including: determining the correspondence between the number of sampling points of the Fast Fourier Transform and each sub-audio feature under different audio dimensions; performing Fast Fourier Transform on the audio segment using the multiple sampling points included in the correspondence to obtain multiple spectrograms; and extracting the corresponding sub-audio features from each spectrogram according to the correspondence.

[0094] The Fast Fourier Transform (FFT) is a fast algorithm for the Discrete Fourier Transform (DFT). All DFTs can be understood as sampling a continuous spectrum. Therefore, the number of sampling points in the FFT refers to the number of times the spectrum from zero to the target sampling rate is uniformly sampled. The more sampling points, the higher the computational accuracy of the FFT.

[0095] Specifically, since different sub-audio features can be extracted from spectrograms of varying precision, and the number of sampling points corresponding to different precision spectrograms is different, a correspondence between the number of sampling points in the Fast Fourier Transform (FFT) and the sub-audio features under different audio dimensions can be pre-established. Therefore, when it is necessary to extract sub-audio features under multiple audio dimensions from an audio segment, the computer device can determine the number of sampling points required to extract each sub-audio feature based on this correspondence, and perform a FFT on the audio segment using each determined number of sampling points to obtain multiple spectrograms. Furthermore, the computer device extracts the corresponding sub-audio features from each spectrogram according to the correspondence between the number of sampling points in the FFT and the sub-audio features under different audio dimensions.

[0096] For example, when the number of sampling points is 4096, the computer can extract Mel spectral features from the spectrum determined by the number of sampling points 4096; when the number of sampling points is 2048, the computer can extract Mel cepstrum features from the spectrum determined by the number of sampling points 2048; and when the number of sampling points is 1024, the computer can extract spectral centroid features, zero-crossing rate features, and spectral flatness features from the spectrum determined by the number of sampling points 1024.

[0097] In this embodiment, by determining the correspondence between the number of sampling points of the Fast Fourier Transform and the sub-audio features under different audio dimensions, a corresponding spectrogram can be generated based on this correspondence. Thus, the corresponding sub-audio features can be accurately extracted based on the corresponding spectrogram, thereby greatly improving the accuracy of the extracted sub-audio features.

[0098] In one embodiment, an audio detection process is performed on a first feature set consisting of at least one sub-audio feature of an audio segment using a first detection model to obtain a first detection result corresponding to the audio segment. This process includes: performing feature concatenation processing on the spectrogram corresponding to the audio segment and each sub-audio feature in the first feature set to obtain multiple first features to be processed; performing feature processing on the multiple first features to be processed by the first detection model to obtain feature processing results; and obtaining the first detection result of the audio segment based on the feature processing results.

[0099] Specifically, after obtaining the first feature set and the spectrogram corresponding to the audio segment, the computer device can concatenate the spectrogram with each sub-audio feature in the first feature set to obtain a first feature set to be processed. The first feature set to be processed includes multiple first features to be processed. Further, the first detection model can perform feature processing on the first feature set to be processed, that is, perform feature processing on multiple first features to be processed in the first feature set to obtain a feature processing result corresponding to the first feature set. Specifically, the first detection model can sequentially perform feature processing on each first feature to be processed in the first feature set to obtain a processing sub-result, and combine the processing sub-results to obtain a feature processing result corresponding to the first feature set. The first detection model can also simultaneously perform feature processing on multiple first features to be processed in the first feature set to obtain a feature processing result corresponding to the first feature set. This embodiment is not limited to this.

[0100] Furthermore, the first detection model may include a fully connected layer, a normalization layer, a max pooling layer, and an output layer. When generating feature processing results, the first detection model sequentially inputs the feature processing results into the fully connected layer, the normalization layer, the max pooling layer, and the output layer, and outputs the first detection result of the audio segment through the output layer. The output layer may include a sigmoid activation function, which outputs the probability value of the audio segment being a normal audio segment.

[0101] In one embodiment, the computer device can concatenate the spectral data corresponding to the spectrogram to the sub-audio features to obtain the corresponding first feature to be processed.

[0102] In the above embodiments, by concatenating the spectrogram with the sub-audio features, the concatenated first feature to be processed can contain more sound information, thereby making the first detection result obtained based on the first feature to be processed containing more sound information more accurate.

[0103] In one embodiment, the first detection model includes multiple first processing layers; the first detection model performs feature processing on multiple first features to be processed to obtain feature processing results, including: performing feature processing on the multiple first features to be processed through the first first processing layer among the multiple first processing layers to obtain the output result of the first first processing layer; for each first processing layer other than the first first processing layer, the output result of the preceding first processing layer is used as the input data of the corresponding first processing layer, and each first processing layer performs feature processing on its own input data to obtain the output result of each first processing layer; based on the output result of the last first processing layer and the multiple first features to be processed, a feature processing result corresponding to the first feature set is obtained.

[0104] Here, the preceding first processing layer refers to the first processing layer that is adjacent to the current first processing layer and is located before the current first processing layer.

[0105] Specifically, the first detection model may include multiple first processing layers. The computer device can input multiple first features to be processed into the first first processing layer, which performs convolutional pooling on the features to obtain the output of the first first processing layer. Further, the first detection model uses the output of the first first processing layer as input data for the second first processing layer, which performs feature processing on the input data to obtain the output of the second first processing layer. The first detection model continues to use the output of the second first processing layer as input data for the third first processing layer, which performs feature processing on the input data to obtain the output of the third first processing layer. This process continues until the output of the last first processing layer is obtained.

[0106] Furthermore, in order to reduce the probability of gradient vanishing, the first detection model inputs the output of the last first processing layer and multiple first features to be processed into a fully connected layer. The fully connected layer processes the output of the last first processing layer and multiple first features to be processed to obtain the feature processing result corresponding to the first feature set. Then, the feature processing result is normalized, pooled and activated to obtain the first detection result corresponding to the audio segment.

[0107] In one embodiment, the first detection model can also transmit the input data of each first processing layer to a fully connected layer, and process the output of the last first processing layer, multiple first features to be processed, and each input data through the fully connected layer to obtain the feature processing result corresponding to the first feature set.

[0108] In one embodiment, the first processing layer includes a convolutional layer and a pooling layer. The first processing layer can perform convolution processing on the input data through the convolutional layer to obtain a convolution result, and then input the convolution result into the pooling layer. The pooling layer processes the convolution result to obtain a pooling result, which is then used as the output of the first processing layer. The convolutional kernel of the convolutional layer can be 3*3, the stride can be 1, and the padding can be 1. The pooling layer can be a max-pooling layer, with a window size of 2*2 and a stride of 2.

[0109] In the above embodiments, only feature processing needs to be performed on the first feature to be processed to obtain the audio detection result of the audio segment, thus greatly improving the detection efficiency of audio detection.

[0110] In one embodiment, an audio detection process is performed on a second feature set consisting of at least one sub-audio feature of an audio segment using a second detection model to obtain a second detection result corresponding to the audio segment. This process includes: performing feature concatenation processing on the spectrogram corresponding to the audio segment and each sub-audio feature in the second feature set to obtain multiple second features to be processed; performing feature processing on the multiple second features to be processed by the second detection model to obtain feature processing results; and obtaining the second detection result of the audio segment based on the feature processing results.

[0111] Specifically, when the second feature set is obtained, the computer device concatenates the spectrogram of the audio segment with each sub-audio feature in the second feature set to obtain a second feature set to be processed. This second feature set includes multiple second features to be processed. Further, the computer device inputs the second feature set to be processed into a second detection model. The second detection model performs feature processing on the second features in the second feature set to be processed, obtaining a feature processing result. This result is then sequentially input into the fully connected layer, normalization layer, max pooling layer, and output layer of the second detection model. The output layer outputs the second detection result of the audio segment. The output layer may include a sigmoid activation function, which outputs the probability value of the audio segment being a normal audio segment.

[0112] In this embodiment, the second feature to be processed is fully processed by multiple model layers in the second detection model, so that the final feature processing result can be more accurate.

[0113] In one embodiment, the second detection model includes multiple second processing layers; the second detection model performs feature processing on multiple second features to be processed to obtain feature processing results, including: performing feature processing on multiple second features to be processed through the first second processing layer to obtain the output result of the first second processing layer; for each second processing layer other than the first second processing layer, the multiple second features to be processed and the output result of the preceding second processing layer are used as the input data of the corresponding second processing layer, and each second processing layer performs feature processing on its own input data to obtain the output result of each second processing layer; based on the output result of the last second processing layer, the feature processing result corresponding to the second feature set is obtained.

[0114] Here, the preceding second processing layer refers to a second processing layer that is adjacent to the current second processing layer and is located before the current second processing layer.

[0115] Specifically, to address the overfitting problem, the second processing model employs a denser connection mechanism. That is, in addition to inputting the output of the preceding second processing layer into the current second processing layer, the second detection model also inputs multiple second processing features into each second processing layer. When the set of second processing features is obtained, the second detection model inputs this set into the first of the multiple second processing layers. The first second processing layer processes the features to obtain its output. Further, the second detection model uses the second processing feature set and the output of the first second processing layer as input data for the second layer, which then processes the input data to obtain its output. The second detection model continues to use the output of the second layer and the second processing feature set as input data for the third layer, which processes the input data to obtain its output. This process continues until the output of the final second processing layer is obtained.

[0116] Furthermore, the second detection model inputs the output of the last second processing layer into the fully connected layer, outputs the feature processing result corresponding to the second feature set through the fully connected layer, and performs normalization, pooling and activation processing on the feature processing result to obtain the second detection result of the audio segment.

[0117] In this embodiment, the audio detection result of the audio segment can be obtained simply by performing feature processing on the second feature to be processed, thus greatly improving the detection efficiency of audio detection.

[0118] In one embodiment, the second processing layer includes a pooling layer and a convolutional layer; each second processing layer performs feature processing on its respective input data to obtain the output results of each second processing layer, including: for each second processing layer other than the first second processing layer, the output results of the preceding second processing layer and multiple second features to be processed are convolved by the convolutional layer in the current second processing layer to obtain a convolution result; the convolution result and multiple second features to be processed are pooled by the pooling layer in the current second processing layer to obtain a pooling result, and the pooling result is used as the output result of the current second processing layer.

[0119] Specifically, the second processing layer includes pooling layers and convolutional layers. For each second processing layer except the first one, the second detection model uses the output of the previous second processing layer and multiple second features to be processed as input data for the convolutional layer in the current second processing layer. The convolutional layer performs convolution processing on the input data to obtain the convolution result. Further, the second detection model uses multiple second features to be processed and the convolution result output by the convolutional layer in the current second processing layer as input data for the pooling layer in the current second processing layer. The pooling layer performs pooling processing on the input data to obtain the pooling result, and the pooling result is used as the output of the current second processing layer.

[0120] In one embodiment, for the first of a plurality of second processing layers, the second detection model inputs a plurality of second features to be processed into the convolutional layer of the first second processing layer, performs convolution processing on the plurality of second features to be processed through the convolutional layer to obtain the convolution result, and inputs the convolution result and the plurality of second features to be processed into the pooling layer of a plurality of second processing layers, performs pooling processing on the input data through the pooling layer to obtain the output result of the first second processing layer.

[0121] In one embodiment, the convolutional kernel of the convolutional layer in the second processing layer can be 3*3, the stride can be 1, and the padding can be 1. The pooling layer in the second processing layer can be a max pooling layer, with a window size of 2*2 and a stride of 2.

[0122] In the above embodiments, by additionally inputting multiple second features to be processed into each second processing layer, the probability of model overfitting can be reduced, thereby improving the accuracy of the output results.

[0123] In one embodiment, determining the audio detection result corresponding to the audio to be detected based on the first detection result and the second detection result corresponding to each audio segment includes: determining the model weights corresponding to the first detection model and the second detection model respectively; determining a first ratio between normal audio segments and total audio segments in the audio to be detected based on the first detection result corresponding to each audio segment, and correcting the first ratio according to the model weights corresponding to the first detection model to obtain a first corrected value; determining a second ratio between normal audio segments and total audio segments in the audio to be detected based on the second detection result corresponding to each audio segment, and correcting the second ratio according to the model weights corresponding to the second detection model to obtain a second corrected value; and determining the audio detection result corresponding to the audio to be detected based on the first corrected value and the second corrected value.

[0124] Specifically, since the audio detection result of the audio to be detected is determined by combining the first detection result output by the first detection model and the second detection result output by the second detection model, the weights of each detection model can be determined based on the detection accuracy of the first and second detection models. The first and second detection results can then be corrected using these weights to improve the accuracy of the final audio detection result.

[0125] When the first and second detection results corresponding to each audio segment are obtained, the computer device can obtain the first model weight of the first detection model and the second model weight of the second detection model. Based on the first detection result, it calculates the first ratio between normal audio segments and the total number of audio segments in the audio to be detected, and based on the second detection result, it calculates the second ratio between normal audio segments and the total number of audio segments in the audio to be detected. Further, the computer device corrects the first ratio based on the first model weight to obtain a first corrected value, and corrects the second ratio based on the second model weight to obtain a second corrected value. Finally, it combines the first and second corrected values ​​to obtain the audio detection result corresponding to the audio to be detected. For example, the computer device multiplies the first model weight by the first ratio to obtain the first corrected value, multiplies the second model weight by the second ratio to obtain the second corrected value, and sums the first and second corrected values ​​to obtain a total corrected value. It then multiplies the total corrected value by the total number of audio segments included in the audio to be detected to obtain the number of normal audio segments included in the audio to be detected, and based on the number of normal audio segments, it obtains the audio detection result corresponding to the audio to be detected.

[0126] In this embodiment, the detection results are corrected by the model weights corresponding to each detection model, so that the audio detection results corresponding to the audio to be detected, determined based on the corrected detection results, can be more accurate.

[0127] In one embodiment, determining the audio detection result corresponding to the audio to be detected based on a first correction value and a second correction value includes: obtaining a first quantity value of normal audio segments and a second quantity value of abnormal audio segments included in the corrected audio to be detected based on the first correction value and the second correction value; determining the audio to be detected as normal audio when the first quantity value is greater than the second quantity value; and determining the audio to be detected as abnormal audio when the first quantity value is less than or equal to the second quantity value.

[0128] Specifically, the computer device can sum the first correction value and the second correction value to obtain a total correction value. This total correction value is then multiplied by the total number of audio segments included in the audio to be detected to obtain the first number of normal audio segments in the audio to be detected. The total number of audio segments is then subtracted from the first number of normal audio segments to obtain the second number of abnormal audio segments. When the first number is greater than the second number, the computer device determines the audio to be detected to be normal audio; when the first number is less than or equal to the second number, the computer device determines the audio to be detected to be abnormal audio. For example, if the first detection model determines the first ratio between normal audio segments and total audio segments in the audio to be detected to be 8 / 10, and the second detection model determines the first ratio to total audio segments to be detected to be 4 / 10, with the first model weight of the first detection model being 0.6 and the second model weight of the second detection model being 0.4, then the first number of normal audio segments included in the audio to be detected is (0.8 * 0.6 + 0.4 * 0.4) * the total number of audio segments included in the audio to be detected.

[0129] In this embodiment, since the audio detection result of the audio to be detected is determined by combining the first correction value and the second correction value, the accuracy of the audio detection result can be improved.

[0130] In one embodiment, the audio detection method is executed by an audio detection model, which includes a first detection model and a second detection model. The audio detection model is obtained through a model training step. (Refer to...) Figure 3 The model training steps include:

[0131] Step S302: Obtain the first sample speech and the first sample label corresponding to the first sample speech, and obtain the second sample speech and the second sample label corresponding to the second sample speech.

[0132] Step S304: Extract sub-audio features of each audio segment in the first sample speech under multiple different audio dimensions to obtain the first sample feature set.

[0133] Step S306: Extract sub-audio features of the second sample speech under multiple different audio dimensions to obtain the second sample feature set; the second sample feature set contains sub-audio features that are not present in the first sample feature set.

[0134] Specifically, before training the first and second detection models, the computer device can collect audio from multiple normally playing devices and audio from multiple abnormally playing devices to obtain first sample speech and second sample speech. Based on the normality and abnormality of the playing devices, the first and second sample speech are labeled to obtain first sample labels and second sample labels. The first sample speech and first sample labels are used to train the first detection model, and the second sample speech and second sample labels are used to train the second detection model. Further, the computer device can extract sub-audio features from each audio segment in the first sample speech under multiple different audio dimensions to obtain a first sample feature set, and extract sub-audio features from each audio segment in the second sample speech under multiple different audio dimensions to obtain a second sample feature set.

[0135] Step S308: The first detection model to be trained performs audio prediction processing on the feature set of the first sample to obtain the first prediction result corresponding to the speech of the first sample.

[0136] Step S310: The second detection model to be trained performs audio prediction processing on the feature set of the second sample to obtain the second prediction result corresponding to the speech of the second sample.

[0137] Specifically, the computer device inputs a first sample feature set into a first detection model, which performs audio prediction processing on the first sample feature set to obtain a first prediction result corresponding to the first sample speech. The computer device then inputs a second sample feature set into a second detection model, which performs audio prediction processing on the second sample feature set to obtain a second prediction result for the second sample speech.

[0138] Step S312: Using the first prediction result and the first sample label, perform the first training on the first detection model until the first training stopping condition is met, and obtain the first detection model that has been trained.

[0139] Step S314: Using the second prediction result and the second sample label, perform a second training on the second detection model until the second training stopping condition is met, and obtain the trained second detection model.

[0140] Step S316: Combine the first and second detection models that have been trained to obtain the trained audio detection model.

[0141] Specifically, the first detection model can sample the aforementioned feature processing method for the first set of features to be processed, and perform audio prediction processing on the first sample feature set; similarly, the second detection model can sample the aforementioned feature processing method for the second set of features to be processed, and perform audio prediction processing on the second sample feature set. Further, the computer device determines the difference between the first prediction result and the corresponding first sample label, and trains the first detection model based on this difference until a first training stopping condition is reached, thus obtaining a trained first detection model. The computer device determines the difference between the second prediction result and the corresponding second sample label, and trains the second detection model based on this difference until a second training stopping condition is reached, thus obtaining a trained second detection model. Both the first and second training stopping conditions can be freely set according to requirements. Further, the computer device integrates the trained first and second detection models to obtain a trained audio detection model.

[0142] In one embodiment, multiple normal and abnormal sound-producing devices can be prepared in advance, and multiple sampling frequencies and sampling bit depths can be set. The audio acquisition device can then acquire the audio played by the normal and abnormal sound-producing devices according to the set sampling frequencies and sampling bit depths to obtain a first sample speech and a second sample speech. For example, refer to... Figure 4 The sampling frequency can be set to 16kb, 32kb, and 48kb, and the sampling bit depth can be set to 16bit. When the sound-producing device is a dual-channel sound-producing device, the audio acquisition device can acquire audio played by the normal sound-producing device and the abnormal sound-producing device at 16kb and 16bit to obtain sample speech. It can also acquire audio played by the normal sound-producing device and the abnormal sound-producing device at 32kb and 16bit to obtain another batch of sample speech, and vice versa. Furthermore, the computer divides the acquired sample speech into a first sample speech set and a second sample speech set. Figure 4 A schematic diagram of sample speech in one embodiment is shown.

[0143] In one embodiment, the computer device may use the speech played by normal and abnormal speech devices in a first device set as first sample speech, and the speech played by normal and abnormal speech devices in a second device set as second sample speech. The computer device sets the first sample label corresponding to the speech played by the normal speech devices in the first device set to "normal," and sets the first sample label corresponding to the speech played by the abnormal speech devices in the first device set to "abnormal." Correspondingly, the computer device may also set a second sample label for the second sample speech based on the speech source.

[0144] In one embodiment, when the sound-producing device is a dual-channel sound-producing device, the computer device can also convert the acquired audio into single-channel audio to obtain a first sample speech or a second sample speech.

[0145] In one embodiment, reference Figure 5 The computer equipment can collect sounds played by normal and abnormal pronunciation devices to obtain first and second sample speech. It then segments the first and second sample speech using preset step sizes and segment durations to obtain audio segments. The computer equipment converts each audio segment from the time domain to frequency, obtaining multiple spectrograms. Features are extracted from these spectrograms to obtain a first sample feature set and a second sample feature set. The computer equipment then trains a first detection model and a second detection model using the first and second sample feature sets, respectively, to obtain trained first and second detection models. Figure 5 A schematic diagram of the model training process in one embodiment is shown.

[0146] In the above embodiments, by training the first detection model and the second detection model, audio detection can be performed based on the trained first detection model and the second detection model to determine the audio detection result of the audio to be detected.

[0147] In one embodiment, a trained audio detection model is obtained by combining a first detection model and a second detection model, including: acquiring test speech and a test label corresponding to the test speech; performing a first speech test on the test speech using the trained first detection model to obtain a first test result, and determining a first difference between the first test result and the test label; performing a second speech test on the test speech using the trained second detection model to obtain a second test result, and determining a second difference between the second test result and the test label; determining the model weights corresponding to the first detection model and the second detection model based on the first difference and the second difference; and obtaining the trained audio detection model based on the trained first detection model, the model weights corresponding to the first detection model, the trained second detection model, and the model weights corresponding to the second detection model.

[0148] Specifically, the computer device can acquire multiple test voice samples and their corresponding test labels. The computer device sequentially inputs the multiple test voice samples into a first detection model, which performs voice testing processing on each sample to obtain a first test result for each sample. Correspondingly, the computer device can also input multiple test voice samples into a second detection model, which performs voice testing processing on each sample to obtain a second test result for each sample. The computer device determines a first difference between each first test result and its corresponding test label, and based on this first difference, determines the accuracy rate of the test results output by the first detection model. Similarly, the computer device determines a second difference between each second test result and its corresponding test label, and based on this second difference, determines the accuracy rate of the test results output by the second detection model.

[0149] The computer device determines the model weights for the first and second detection models based on the ratio between the accuracy rates of the test results output by the first detection model and the second detection model. Then, based on the trained first detection model, its corresponding model weights, the trained second detection model, and its corresponding model weights, a trained audio detection model is obtained. For example, if the accuracy rate of the first detection model is 80% and the accuracy rate of the second detection model is 40%, the model weight for the first detection model can be 2 / 3, and the model weight for the second detection model can be 1 / 3.

[0150] In one embodiment, reference Figure 6 , Figure 6A schematic diagram of the overall framework for model training in one embodiment is shown. The computer device can collect multiple sample speech data and extract multiple sub-audio features for each sample speech data across multiple audio dimensions through audio feature engineering. The user can design the model structures of the first and second detection models to be trained, and train the designed first and second detection models using the sub-audio features. When the training of the first and second detection models is complete, the computer device can also evaluate the first and second detection models using a test set to determine the respective model weights for the first and second detection models.

[0151] In the above embodiments, since the model weights of the corresponding detection models are determined based on the difference between the test results and the test labels, high model weights can be assigned to detection models with higher accuracy and low model weights to detection models with lower accuracy, thereby making the first ratio and the second ratio corrected based on the model weights more accurate.

[0152] This application also provides an application scenario in which the above-described audio detection method is applied. Specifically, the audio detection method is applied in this scenario as follows:

[0153] Facial recognition payment refers to a payment method where users scan their faces using a camera in front of a facial recognition device to verify their identity. During the payment process, the device can announce the payment result via a speaker. The production process for facial recognition devices mainly involves DVT (Design Verification and Testing), EVT1 (Engineering Verification and Testing Phase 1), EVT2 (Engineering Verification and Testing Phase 2), and MP (Mass Production). Quality inspection of the speaker in the facial recognition device can be performed during the DVT stage to identify abnormal speakers early. Quality inspection can also be performed during the MP stage for post-production diagnosis and analysis of abnormal speakers. When speaker quality inspection is required, refer to... Figure 7 The computer equipment can acquire the audio to be tested played by the speaker to be inspected, segment the audio to be tested into multiple audio segments, and generate a spectrogram corresponding to each audio segment. Multiple sub-audio features are extracted from the spectrograms of the computer equipment, and these sub-audio features are concatenated with the spectrograms to obtain a first feature to be processed and a second feature to be processed. The computer equipment inputs the first feature to be processed into a first detection model and the second feature to be processed into a second detection model. The first and second detection models output the first and second detection results corresponding to each audio segment, respectively. Based on the first and second detection results, the computer equipment votes and determines the audio detection result corresponding to the audio to be tested, thus obtaining the quality inspection result of the speaker to be inspected. Figure 7 A schematic diagram of the quality inspection process for a sound-producing device is shown in one embodiment.

[0154] This application also provides another application scenario in which the above-described audio detection method is applied. Specifically, the audio detection method is applied in this scenario as follows:

[0155] When it is necessary to inspect the speakers in the terminal, the computer equipment can collect the audio to be tested emitted by the speakers and perform audio detection on the audio to be tested according to the above-mentioned audio detection method, so as to determine the quality inspection result of the speakers based on the audio detection results.

[0156] The above application scenarios are merely illustrative. It is understood that the application of the audio detection methods provided in the embodiments of this application is not limited to the above scenarios.

[0157] In a specific implementation, refer to Figure 8 An audio detection method is provided, which includes the following steps:

[0158] S802, acquire the audio to be tested played by the sound device to be inspected, and segment the audio to be tested into multiple audio segments by using a preset segment duration and segment step size; determine the first detection model and the second detection model, wherein the first detection model and the second detection model are models obtained through ensemble learning.

[0159] S804: Determine the correspondence between the number of sampling points in the Fast Fourier Transform and the sub-audio features under different audio dimensions; perform Fast Fourier Transform on the audio segment using the multiple sampling points included in the correspondence to obtain multiple spectrograms.

[0160] S806, according to the correspondence, extract the corresponding sub-audio features from each spectrogram; the audio dimension includes at least the timbre dimension, pitch dimension, and saturation dimension; the sub-audio features of the timbre dimension include Mel cepstral features and spectral centroid features; the sub-audio features of the pitch dimension include Mel spectral features; the sub-audio features of the saturation dimension include zero-crossing rate features and spectral flatness features; the first feature set includes Mel spectral features and Mel cepstral features; the second feature set includes Mel spectral features, Mel cepstral features, spectral centroid features, zero-crossing rate features, and spectral flatness features.

[0161] S808, the spectrogram corresponding to the audio segment is concatenated with each sub-audio feature in the first feature set to obtain multiple first features to be processed; the first detection model includes multiple first processing layers; through the first first processing layer among the multiple first processing layers, the multiple first features to be processed are processed to obtain the output result of the first first processing layer.

[0162] S810, for each first processing layer other than the first first processing layer, the output result of the preceding first processing layer is used as the input data of the corresponding first processing layer. Each first processing layer performs feature processing on its own input data to obtain the output result of each first processing layer. Based on the output result of the last first processing layer and multiple first features to be processed, the feature processing result corresponding to the first feature set is obtained.

[0163] S812, the spectrogram corresponding to the audio segment is concatenated with each sub-audio feature in the second feature set to obtain multiple second features to be processed; wherein, the second feature set contains sub-audio features that are not in the first feature set.

[0164] S814, the second detection model includes multiple second processing layers; through the first of the multiple second processing layers, feature processing is performed on multiple second features to be processed, and the result output by the first second processing layer is obtained.

[0165] S816: For each second processing layer except the first second processing layer, the output of the previous second processing layer and multiple second features to be processed are convolved by the convolutional layer in the current second processing layer to obtain the convolution result.

[0166] S818: The convolution result and multiple second features to be processed are pooled through the pooling layer in the current second processing layer to obtain the pooling result, and the pooling result is used as the output of the current second processing layer; based on the output of the last second processing layer, the feature processing result of the second feature set is obtained.

[0167] S820, determine the model weights corresponding to the first detection model and the second detection model respectively; based on the first detection results corresponding to each audio segment, determine the first ratio between the normal audio segments and the total audio segments in the audio to be detected, and correct the first ratio according to the model weights corresponding to the first detection model to obtain the first corrected value.

[0168] S822, based on the second detection results corresponding to each audio segment, determine the second ratio between the normal audio segments and the total audio segments in the audio to be detected, and correct the second ratio according to the model weights corresponding to the second detection model to obtain the second corrected value.

[0169] S824, based on the first correction value and the second correction value, obtain the first quantity value of normal audio segments and the second quantity value of abnormal audio segments included in the corrected audio to be detected; when the first quantity value is greater than the second quantity value, determine that the audio to be detected is normal audio; when the first quantity value is less than or equal to the second quantity value, determine that the audio to be detected is abnormal audio.

[0170] S826, based on the audio detection results of the audio to be tested, determine the quality inspection results of the sound-producing device.

[0171] The aforementioned audio detection method, by acquiring audio segments of the audio to be detected, can extract features from these segments to obtain multiple sub-audio features under different audio dimensions. These sub-audio features can be divided into a first feature set and a second feature set. Audio detection processing can then be performed on the first feature set based on a first detection model to obtain a first detection result. Similarly, audio detection processing can be performed on the second feature set based on a second detection model to obtain a second detection result. By determining the first and second detection results corresponding to each audio segment, the first and second detection results can be combined to automatically generate an audio detection result corresponding to the audio to be detected, thereby improving the accuracy of the audio detection results. Since the detection model detects the audio to be detected, compared to traditional methods using hardware detection devices, this application eliminates the need for a complex experimental environment, thus improving the efficiency of audio detection.

[0172] In a specific implementation, refer to Figure 9 An audio detection method is provided, which includes the following steps:

[0173] S902, obtain the first sample speech and the first sample label corresponding to the first sample speech, and obtain the second sample speech and the second sample label corresponding to the second sample speech.

[0174] S904, extract sub-audio features of each audio segment in the first sample speech under multiple different audio dimensions to obtain the first sample feature set; extract sub-audio features of the second sample speech under multiple different audio dimensions to obtain the second sample feature set; the second sample feature set contains sub-audio features that are not in the first sample feature set.

[0175] S906, the first sample feature set is processed by the first detection model to be trained to obtain the first prediction result of the first sample speech; the second sample feature set is processed by the second detection model to be trained to obtain the second prediction result of the second sample speech.

[0176] S908, using the first prediction result and the first sample label, perform first training on the first detection model until the first training stopping condition is met, and obtain the first detection model that has been trained; using the second prediction result and the second sample label, perform second training on the second detection model until the second training stopping condition is met, and obtain the second detection model that has been trained.

[0177] S910 acquires the test voice and the test tag corresponding to the test voice.

[0178] S912, the first detection model, after training, performs a first speech test on the test speech to obtain a first test result and determine the first difference between the first test result and the test label.

[0179] S914, the trained second detection model is used to perform second speech test processing on the test speech to obtain a second test result, and the second difference between the second test result and the test label is determined.

[0180] S916, Based on the first difference and the second difference, determine the model weights corresponding to the first detection model and the second detection model respectively; Based on the trained first detection model, the model weights corresponding to the first detection model, the trained second detection model, and the model weights corresponding to the second detection model, obtain the trained audio detection model.

[0181] S918, acquire multiple audio segments from the audio to be detected, determine the first detection model and the second detection model, and extract sub-audio features of each audio segment under multiple different audio dimensions; wherein, the first detection model and the second detection model are models obtained through ensemble learning.

[0182] S920, using the first detection model, audio detection processing is performed on the first feature set consisting of at least one sub-audio feature of the audio segment to obtain the first detection result corresponding to the audio segment.

[0183] S922, using the second detection model, audio detection processing is performed on the second feature set consisting of at least one sub-audio feature of the audio segment to obtain a second detection result corresponding to the audio segment; wherein, the second feature set contains sub-audio features that are not present in the first feature set.

[0184] S924, based on the first and second detection results corresponding to each audio segment, determine the audio detection result corresponding to the audio to be detected.

[0185] It should be understood that, although Figures 2-3 , Figures 8-9The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figures 2-3 , Figures 8-9 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0186] In one embodiment, such as Figure 10 As shown, an audio detection device 1000 is provided. This device can be a software module, a hardware module, or a combination of both as part of a computer device. Specifically, the device includes: a feature acquisition module 1002, a detection processing module 1004, and a result determination module 1006, wherein:

[0187] The feature acquisition module 1002 is used to acquire multiple audio segments in the audio to be detected, determine a first detection model and a second detection model, wherein the first detection model and the second detection model are models obtained through ensemble learning; and extract sub-audio features of each audio segment under multiple different audio dimensions.

[0188] The detection processing module 1004 is used to perform audio detection processing on a first feature set consisting of at least one sub-audio feature of an audio segment using a first detection model to obtain a first detection result corresponding to the audio segment; and to perform audio detection processing on a second feature set consisting of at least one sub-audio feature of an audio segment using a second detection model to obtain a second detection result corresponding to the audio segment; wherein the second feature set contains sub-audio features that are not present in the first feature set.

[0189] The result determination module 1006 is used to determine the audio detection result corresponding to the audio to be detected based on the first detection result and the second detection result corresponding to each audio segment.

[0190] In one embodiment, reference Figure 11 The audio detection device 1000 is also used to acquire the audio to be detected played by the sound-producing device to be inspected, and to segment the audio to be detected by using preset segment duration and segment step size to obtain multiple audio segments; and to determine the quality inspection result of the sound-producing device based on the audio detection result of the audio to be detected.

[0191] In one embodiment, the audio detection device 1000 is further configured to extract sub-audio features under different audio dimensions; the audio dimensions include at least timbre dimension, pitch dimension, and saturation dimension; the sub-audio features of the timbre dimension include Mel cepstral features and spectral centroid features; the sub-audio features of the pitch dimension include Mel spectral features; the sub-audio features of the saturation dimension include zero-crossing rate features and spectral flatness features; the first feature set includes Mel spectral features and Mel cepstral features; the second feature set includes Mel spectral features, Mel cepstral features, spectral centroid features, zero-crossing rate features, and spectral flatness features.

[0192] In one embodiment, the feature acquisition module 1002 is further configured to determine the correspondence between the number of sampling points of the fast Fourier transform and the sub-audio features under different audio dimensions; perform fast Fourier transform on the audio segment using the multiple sampling points included in the correspondence to obtain multiple spectrograms; and extract the corresponding sub-audio features from each spectrogram according to the correspondence.

[0193] In one embodiment, the detection processing module 1004 further includes a first detection module 1041, which is used to perform feature concatenation processing on the spectrogram corresponding to the audio segment and each sub-audio feature in the first feature set to obtain multiple first features to be processed; perform feature processing on the multiple first features to be processed by the first detection model to obtain feature processing results, and obtain the first detection result of the audio segment based on the feature processing results.

[0194] In one embodiment, the first detection model includes multiple first processing layers; the first detection module 1041 is further configured to perform feature processing on multiple first features to be processed through the first first processing layer among the multiple first processing layers to obtain the output result of the first first processing layer; for each first processing layer other than the first first processing layer, the output result of the preceding first processing layer before the corresponding first processing layer is used as the input data of the corresponding first processing layer, and each first processing layer performs feature processing on its own input data to obtain the output result of each first processing layer; based on the output result of the last first processing layer and the multiple first features to be processed, a feature processing result corresponding to the first feature set is obtained.

[0195] In one embodiment, the detection processing module 1004 further includes a second detection module 1042, which is used to perform feature concatenation processing on the spectrogram corresponding to the audio segment and each sub-audio feature in the second feature set to obtain multiple second features to be processed; the second detection model performs feature processing on the multiple second features to be processed to obtain feature processing results, and obtains the second detection result of the audio segment based on the feature processing results.

[0196] In one embodiment, the second detection model includes multiple second processing layers; the second detection module 1042 is further configured to perform feature processing on multiple second features to be processed through the first second processing layer among the multiple second processing layers to obtain the output result of the first second processing layer; for each second processing layer other than the first second processing layer, the multiple second features to be processed and the output result of the preceding second processing layer are used as the input data of the corresponding second processing layer, and each second processing layer performs feature processing on its own input data to obtain the output result of each second processing layer; based on the output result of the last second processing layer, the feature processing result corresponding to the second feature set is obtained.

[0197] In one embodiment, the second processing layer includes a pooling layer and a convolutional layer; the second detection module 1042 is further configured to, for each second processing layer other than the first second processing layer, perform convolution processing on the output of the preceding second processing layer and multiple second features to be processed through the convolutional layer in the current second processing layer to obtain a convolution result; perform pooling processing on the convolution result and multiple second features to be processed through the pooling layer in the current second processing layer to obtain a pooling result, and use the pooling result as the output of the current second processing layer.

[0198] In one embodiment, the result determination module 1006 further includes a correction module 1061, configured to determine the model weights corresponding to the first detection model and the second detection model respectively; determine a first ratio between normal audio segments and total audio segments in the audio to be detected based on the first detection results corresponding to each audio segment, and correct the first ratio according to the model weights corresponding to the first detection model to obtain a first correction value; determine a second ratio between normal audio segments and total audio segments in the audio to be detected based on the second detection results corresponding to each audio segment, and correct the second ratio according to the model weights corresponding to the second detection model to obtain a second correction value; and determine the audio detection result corresponding to the audio to be detected based on the first correction value and the second correction value.

[0199] In one embodiment, the correction module 1061 is further configured to obtain a first quantity value of normal audio segments and a second quantity value of abnormal audio segments included in the corrected audio to be detected based on a first correction value and a second correction value; when the first quantity value is greater than the second quantity value, the audio to be detected is determined to be normal audio; when the first quantity value is less than or equal to the second quantity value, the audio to be detected is determined to be abnormal audio.

[0200] In one embodiment, the audio detection device 1000 further includes a training module 1008, configured to: acquire a first sample speech and a first sample label corresponding to the first sample speech; acquire a second sample speech and a second sample label corresponding to the second sample speech; extract sub-audio features of each audio segment in the first sample speech under multiple different audio dimensions to obtain a first sample feature set; extract sub-audio features of the second sample speech under multiple different audio dimensions to obtain a second sample feature set; the second sample feature set contains sub-audio features not present in the first sample feature set; perform audio prediction processing on the first sample feature set using a first detection model to be trained to obtain a first prediction result corresponding to the first sample speech; perform audio prediction processing on the second sample feature set using a second detection model to be trained to obtain a second prediction result corresponding to the second sample speech; perform first training on the first detection model using the first prediction result and the first sample label until a first training stopping condition is met, to obtain a trained first detection model; perform second training on the second detection model using the second prediction result and the second sample label until a second training stopping condition is met, to obtain a trained second detection model; and combine the trained first detection model and the second detection model to obtain a trained audio detection model.

[0201] In one embodiment, the training module 1008 is further configured to acquire test speech and test labels corresponding to the test speech; perform a first speech test on the test speech using a trained first detection model to obtain a first test result, and determine a first difference between the first test result and the test label; perform a second speech test on the test speech using a trained second detection model to obtain a second test result, and determine a second difference between the second test result and the test label; determine the model weights corresponding to the first detection model and the second detection model based on the first difference and the second difference; and obtain a trained audio detection model based on the trained first detection model, the model weights corresponding to the first detection model, the trained second detection model, and the model weights corresponding to the second detection model.

[0202] For specific limitations regarding the audio detection device, please refer to the limitations of the audio detection method above, which will not be repeated here. Each module in the aforementioned audio detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in a computer device, or stored in software in the memory of a computer device, so that the processor can call and execute the corresponding operations of each module.

[0203] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 12 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores audio detection data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements an audio detection method.

[0204] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0205] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0206] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0207] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0208] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0209] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0210] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An audio detection method, characterized in that, The method includes: Multiple audio segments from the audio to be detected emitted by the sound-producing device are acquired, and a first detection model and a second detection model are determined, wherein the first detection model and the second detection model are complementary models obtained through ensemble learning; Extract sub-audio features of each audio segment under multiple different audio dimensions; Using the first detection model, audio detection processing is performed on a first feature set consisting of at least one sub-audio feature of the audio segment to obtain a first detection result corresponding to the audio segment; The second detection model performs audio detection processing on a second feature set consisting of at least one sub-audio feature of the audio segment to obtain a second detection result corresponding to the audio segment; wherein, the second feature set contains sub-audio features not present in the first feature set; the first feature set and the second feature set are obtained by dividing the extracted sub-audio features according to the audio dimension and the model characteristics of the first detection model and the second detection model; both the first detection result and the second detection result are used to characterize the probability value that the corresponding audio segment is a normal audio segment; Based on the first and second detection results corresponding to each audio segment, the audio detection result corresponding to the audio to be detected is determined, and the quality inspection result of the sound-producing device is determined based on the audio detection result.

2. The method according to claim 1, characterized in that, The acquisition of multiple audio segments from the audio to be detected emitted by the sound-producing device includes: The audio to be tested, played by the sound-producing device to be inspected, is acquired and segmented into multiple audio segments by using preset segment duration and segment step size.

3. The method according to claim 1, characterized in that, The audio dimensions include at least a timbre dimension, a pitch dimension, and a saturation dimension; the sub-audio features of the timbre dimension include Mel cepstrum features and spectral centroid features; the sub-audio features of the pitch dimension include Mel spectral features; the sub-audio features of the saturation dimension include zero-crossing rate features and spectral flatness features; the first feature set includes Mel spectral features and Mel cepstrum features; the second feature set includes Mel spectral features, Mel cepstrum features, spectral centroid features, zero-crossing rate features, and spectral flatness features.

4. The method according to claim 3, characterized in that, The extraction of sub-audio features for each audio segment across multiple different audio dimensions includes: Determine the correspondence between the number of sampling points in the Fast Fourier Transform and the sub-audio features under different audio dimensions; By using the number of sampling points included in the correspondence, a fast Fourier transform is performed on the audio segment to obtain multiple spectrograms; Based on the correspondence, the corresponding sub-audio features are extracted from each spectrogram.

5. The method according to claim 1, characterized in that, The step of performing audio detection processing on a first feature set consisting of at least one sub-audio feature of the audio segment using the first detection model to obtain a first detection result corresponding to the audio segment includes: The spectrogram corresponding to the audio segment is concatenated with each sub-audio feature in the first feature set to obtain multiple first features to be processed. The first detection model performs feature processing on the plurality of first features to be processed to obtain feature processing results, and based on the feature processing results, the first detection result of the audio segment is obtained.

6. The method according to claim 5, characterized in that, The first detection model includes multiple first processing layers; the step of performing feature processing on the multiple first features to be processed by the first detection model to obtain feature processing results includes: The first processing layer among multiple first processing layers performs feature processing on the multiple first features to be processed, and obtains the output of the first processing layer. For each first processing layer except the first first processing layer, the output of the preceding first processing layer is used as the input data of the corresponding first processing layer. Each first processing layer performs feature processing on its own input data to obtain the output of each first processing layer. Based on the output of the last layer of the first processing layer and the multiple first features to be processed, the feature processing result corresponding to the first feature set is obtained.

7. The method according to claim 1, characterized in that, The step of performing audio detection processing on a second feature set consisting of at least one sub-audio feature of the audio segment using the second detection model to obtain a second detection result corresponding to the audio segment includes: The spectrogram corresponding to the audio segment is concatenated with each sub-audio feature in the second feature set to obtain multiple second features to be processed. The second detection model performs feature processing on the plurality of second features to be processed to obtain feature processing results, and based on the feature processing results, the second detection result of the audio segment is obtained.

8. The method according to claim 7, characterized in that, The second detection model includes multiple second processing layers; the feature processing performed by the second detection model on the multiple second features to be processed to obtain the feature processing result includes: The first of a plurality of second processing layers performs feature processing on the plurality of second features to be processed, and obtains the output of the first second processing layer. For each second processing layer other than the first second processing layer, multiple second features to be processed and the output results of the preceding second processing layer are used as the input data of the corresponding second processing layer. Each second processing layer performs feature processing on its own input data to obtain the output results of each second processing layer. Based on the output of the last second processing layer, the feature processing result corresponding to the second feature set is obtained.

9. The method according to claim 8, characterized in that, The second processing layer includes pooling layers and convolutional layers; each second processing layer performs feature processing on its respective input data to obtain the output results of each second processing layer, including: For each second processing layer except the first second processing layer, the output of the previous second processing layer and multiple second features to be processed are convolved by the convolutional layer in the current second processing layer to obtain the convolution result. The convolution result and multiple second features to be processed are pooled by the pooling layer in the current second processing layer to obtain the pooling result, and the pooling result is used as the output of the current second processing layer.

10. The method according to claim 1, characterized in that, The step of determining the audio detection result corresponding to the audio to be detected based on the first detection result and the second detection result corresponding to each audio segment includes: Determine the model weights corresponding to the first detection model and the second detection model respectively; Based on the first detection results corresponding to each audio segment, a first ratio between the normal audio segments and the total audio segments in the audio to be detected is determined, and the first ratio is corrected according to the model weights corresponding to the first detection model to obtain a first corrected value. Based on the second detection results corresponding to each audio segment, a second ratio between the normal audio segments and the total audio segments in the audio to be detected is determined, and the second ratio is corrected according to the model weights corresponding to the second detection model to obtain a second corrected value. Based on the first correction value and the second correction value, the audio detection result corresponding to the audio to be detected is determined.

11. The method according to claim 10, characterized in that, The step of determining the audio detection result corresponding to the audio to be detected based on the first correction value and the second correction value includes: Based on the first correction value and the second correction value, the first number of normal audio segments and the second number of abnormal audio segments included in the corrected audio to be detected are obtained. When the first quantity value is greater than the second quantity value, the audio to be detected is determined to be normal audio; When the first quantity value is less than or equal to the second quantity value, the audio to be detected is determined to be abnormal audio.

12. The method according to any one of claims 1 to 11, characterized in that, The audio detection method is executed by an audio detection model, which includes a first detection model and a second detection model. The audio detection model is obtained through a model training step, which includes: Obtain a first sample speech and a first sample label corresponding to the first sample speech; obtain a second sample speech and a second sample label corresponding to the second sample speech. Extract sub-audio features from each audio segment in the first sample speech under multiple different audio dimensions to obtain the first sample feature set; Extract sub-audio features from the second sample speech under multiple different audio dimensions to obtain a second sample feature set; the second sample feature set contains sub-audio features that are not present in the first sample feature set; The first detection model to be trained performs audio prediction processing on the first sample feature set to obtain a first prediction result corresponding to the speech of the first sample. The second sample feature set is processed by the second detection model to be trained to perform audio prediction processing to obtain a second prediction result corresponding to the speech of the second sample. The first detection model is trained using the first prediction result and the first sample label until the first training stopping condition is met, thus obtaining the first detection model that has been trained. The second detection model is trained using the second prediction result and the second sample label until the second training stopping condition is met, thus obtaining the trained second detection model. The first and second detection models, which have been trained, are combined to obtain the trained audio detection model.

13. The method according to claim 12, characterized in that, The first and second detection models, after comprehensive training, yield a trained audio detection model, including: Obtain the test audio and the test tag corresponding to the test audio; The first detection model, after training, performs a first speech test on the test speech to obtain a first test result, and determines a first difference between the first test result and the test label. The trained second detection model is used to perform a second speech test on the test speech to obtain a second test result, and a second difference between the second test result and the test label is determined. Based on the first difference and the second difference, determine the model weights corresponding to the first detection model and the second detection model respectively; The trained audio detection model is obtained based on the first detection model, the model weights corresponding to the first detection model, the second detection model, and the model weights corresponding to the second detection model.

14. An audio detection device, characterized in that, The device includes: The feature acquisition module is used to acquire multiple audio segments from the audio to be detected emitted by the sound-producing device, determine a first detection model and a second detection model, wherein the first detection model and the second detection model are complementary models obtained through ensemble learning; and extract sub-audio features of each audio segment under multiple different audio dimensions. The detection processing module is used to perform audio detection processing on a first feature set consisting of at least one sub-audio feature of the audio segment using the first detection model, to obtain a first detection result corresponding to the audio segment; and to perform audio detection processing on a second feature set consisting of at least one sub-audio feature of the audio segment using the second detection model, to obtain a second detection result corresponding to the audio segment; wherein the second feature set contains sub-audio features not present in the first feature set; the first feature set and the second feature set are obtained by dividing the extracted sub-audio features according to the audio dimension and the model characteristics of the first detection model and the second detection model; both the first detection result and the second detection result are used to characterize the probability value that the corresponding audio segment is a normal audio segment; The result determination module is used to determine the audio detection result corresponding to the audio to be detected based on the first detection result and the second detection result corresponding to each audio segment, and to determine the quality inspection result of the sound-producing device based on the audio detection result.

15. The apparatus according to claim 14, characterized in that, The audio detection device is also used to acquire the audio to be detected played by the sound-producing device to be inspected, and to segment the audio to be detected by using a preset segment duration and segment step size to obtain multiple audio segments.

16. The apparatus according to claim 14, characterized in that, The audio dimensions include at least a timbre dimension, a pitch dimension, and a saturation dimension; the sub-audio features of the timbre dimension include Mel cepstrum features and spectral centroid features; the sub-audio features of the pitch dimension include Mel spectral features; the sub-audio features of the saturation dimension include zero-crossing rate features and spectral flatness features; the first feature set includes Mel spectral features and Mel cepstrum features; the second feature set includes Mel spectral features, Mel cepstrum features, spectral centroid features, zero-crossing rate features, and spectral flatness features.

17. The apparatus according to claim 16, characterized in that, The feature acquisition module is also used to determine the correspondence between the number of sampling points of the fast Fourier transform and the sub-audio features under different audio dimensions; to perform fast Fourier transform on the audio segment using the multiple sampling points included in the correspondence, and to obtain multiple spectrograms; and to extract the corresponding sub-audio features from each spectrogram according to the correspondence.

18. The apparatus according to claim 14, characterized in that, The detection and processing module is also used to perform feature concatenation processing on the spectrogram corresponding to the audio segment and each sub-audio feature in the first feature set to obtain multiple first features to be processed. The first detection model performs feature processing on the plurality of first features to be processed to obtain feature processing results, and based on the feature processing results, the first detection result of the audio segment is obtained.

19. The apparatus according to claim 18, characterized in that, The first detection model includes multiple first processing layers; the detection processing module is further configured to perform feature processing on the multiple first features to be processed through the first first processing layer among the multiple first processing layers to obtain the output result of the first first processing layer; for each first processing layer other than the first first processing layer, the output result of the preceding first processing layer is used as the input data of the corresponding first processing layer, and each first processing layer performs feature processing on its own input data to obtain the output result of each first processing layer; based on the output result of the last first processing layer and the multiple first features to be processed, a feature processing result corresponding to the first feature set is obtained.

20. The apparatus according to claim 14, characterized in that, The detection and processing module is also used to perform feature concatenation processing on the spectrogram corresponding to the audio segment and each sub-audio feature in the second feature set to obtain multiple second features to be processed. The second detection model performs feature processing on the plurality of second features to be processed to obtain feature processing results, and based on the feature processing results, the second detection result of the audio segment is obtained.

21. The apparatus according to claim 20, characterized in that, The second detection model includes multiple second processing layers; the detection processing module is further configured to perform feature processing on the multiple second features to be processed through the first second processing layer among the multiple second processing layers to obtain the output result of the first second processing layer; for each second processing layer other than the first second processing layer, the multiple second features to be processed and the output result of the preceding second processing layer are used as the input data of the corresponding second processing layer, and each second processing layer performs feature processing on its own input data to obtain the output result of each second processing layer; based on the output result of the last second processing layer, the feature processing result corresponding to the second feature set is obtained.

22. The apparatus according to claim 21, characterized in that, The second processing layer includes a pooling layer and a convolutional layer; the detection processing module is further configured to, for each second processing layer other than the first second processing layer, perform convolution processing on the output of the preceding second processing layer and multiple second features to be processed through the convolutional layer in the current second processing layer to obtain a convolution result; perform pooling processing on the convolution result and multiple second features to be processed through the pooling layer in the current second processing layer to obtain a pooling result, and use the pooling result as the output of the current second processing layer.

23. The apparatus according to claim 14, characterized in that, The result determination module is further configured to determine the model weights corresponding to the first detection model and the second detection model respectively; determine a first ratio between normal audio segments and total audio segments in the audio to be detected based on the first detection results corresponding to each audio segment, and correct the first ratio according to the model weights corresponding to the first detection model to obtain a first corrected value; determine a second ratio between normal audio segments and total audio segments in the audio to be detected based on the second detection results corresponding to each audio segment, and correct the second ratio according to the model weights corresponding to the second detection model to obtain a second corrected value; and determine the audio detection result corresponding to the audio to be detected based on the first corrected value and the second corrected value.

24. The apparatus according to claim 23, characterized in that, The result determination module is further configured to obtain a first quantity value of normal audio segments and a second quantity value of abnormal audio segments included in the corrected audio to be detected based on the first correction value and the second correction value; when the first quantity value is greater than the second quantity value, the audio to be detected is determined to be normal audio; when the first quantity value is less than or equal to the second quantity value, the audio to be detected is determined to be abnormal audio.

25. The apparatus according to claim 14, characterized in that, The audio detection device further includes a training module, used to acquire a first sample speech and a first sample label corresponding to the first sample speech, acquire a second sample speech and a second sample label corresponding to the second sample speech; extract sub-audio features of each audio segment in the first sample speech under multiple different audio dimensions to obtain a first sample feature set; extract sub-audio features of the second sample speech under multiple different audio dimensions to obtain a second sample feature set; the second sample feature set contains sub-audio features not present in the first sample feature set; perform audio prediction processing on the first sample feature set using a first detection model to be trained to obtain a first prediction result corresponding to the first sample speech; perform audio prediction processing on the second sample feature set using a second detection model to be trained to obtain a second prediction result corresponding to the second sample speech; perform first training on the first detection model using the first prediction result and the first sample label until a first training stopping condition is met, to obtain a trained first detection model; perform second training on the second detection model using the second prediction result and the second sample label until a second training stopping condition is met, to obtain a trained second detection model; and combine the trained first detection model and the second detection model to obtain a trained audio detection model.

26. The apparatus according to claim 25, characterized in that, The training module is also used to acquire test speech and test labels corresponding to the test speech; to perform a first speech test on the test speech through the trained first detection model to obtain a first test result, and to determine a first difference between the first test result and the test label; The trained second detection model is used to perform a second speech test on the test speech to obtain a second test result, and a second difference between the second test result and the test label is determined; based on the first difference and the second difference, the model weights corresponding to the first detection model and the second detection model are determined respectively. The trained audio detection model is obtained based on the first detection model, the model weights corresponding to the first detection model, the second detection model, and the model weights corresponding to the second detection model.

27. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 13.

28. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 13.

29. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 13.

Images