Audio detection method, device, electronic equipment and medium

Through the combination of audio separation and detection models, the problem of low accuracy of abnormal sound detection is solved, and the accurate identification and processing of abnormal audio data is achieved, which improves the user experience.

CN118711615BActive Publication Date: 2024-11-19BEIJING XIAOMI MOBILE SOFTWARE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411187015.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-11-19
Estimated Expiration
2044-08-28

AI Technical Summary

Technical Problem

The detection results of existing abnormal sound detection technology are low in accuracy, resulting in the inability to process abnormal audio data in time, affecting the user experience.

Method used

By collecting target audio data for audio separation, audio clips with similar or identical sound characteristics are obtained, and each audio clip is detected separately. Pre-trained audio separation model and detection model are used for clustering and identification, improving the accuracy of detection sex.

Benefits of technology

The preliminary separation of abnormal audio data and normal audio data is achieved, the accuracy of audio detection is improved, and the abnormal audio data is facilitated in a timely manner and the user experience is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118711615B_ABST
    Figure CN118711615B_ABST
Patent Text Reader

Abstract

The present disclosure relates to an audio detection method, device, electronic device and medium. The audio detection method includes collecting target audio data in the current scene, performing audio separation processing on the target audio data, obtaining multiple audio segments separated from the target audio data, the similarity between the sound features in the same audio segment is greater than a preset threshold, and detecting each audio segment respectively to obtain a detection result for each audio segment. The detection result is used to characterize the type of the audio segment, and the type includes a normal type or an abnormal type. The present disclosure can more accurately determine the detection result of each audio segment, thereby improving the accuracy of audio detection, facilitating timely processing of abnormal audio data, and improving the user experience.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the technical field of audio detection, and in particular to an audio detection method, device, electronic equipment and medium. Background Art

[0002] Abnormal sound detection is a technology that identifies whether the collected audio data is normal audio data or abnormal audio data. Abnormal sound detection can help users monitor the operation of electronic devices and determine how to deal with abnormal audio data in a timely manner when abnormal audio data is generated.

[0003] However, since abnormal audio data has a wide range of audio changes, the accuracy of the detection results of abnormal sound detection is low, so that the abnormal audio data cannot be processed in time, resulting in a poor user experience. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides an audio detection method, device, electronic device and medium.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an audio detection method, including:

[0006] Collect target audio data in the current scene;

[0007] Performing audio separation processing on the target audio data to obtain multiple audio segments separated from the target audio data, wherein the similarity between sound features in the same audio segment is greater than a preset threshold;

[0008] Each audio segment is detected respectively to obtain a detection result of each audio segment, wherein the detection result is used to characterize a type of the audio segment, and the type includes a normal type or an abnormal type.

[0009] In some embodiments, performing audio separation processing on the target audio data to obtain a plurality of audio segments separated from the target audio data includes:

[0010] Calling a target audio separation model to perform audio separation processing on the target audio data to obtain a plurality of audio segments separated from the target audio data; the target audio separation model is used to perform audio separation based on the input audio data, and the target audio separation model is trained based on abnormal audio data and normal audio data;

[0011] The detecting each audio segment respectively to obtain the detection result of each audio segment includes:

[0012] The target audio detection model is called to detect each audio segment respectively to obtain the detection result of each audio segment; the target audio detection model is used to perform audio detection based on the input audio data, and the target audio detection model is trained based on abnormal audio data and normal audio data.

[0013] In some embodiments, the training process of the target audio separation model and the target audio detection model includes:

[0014] Acquire a plurality of first training data, a plurality of second training data, and a plurality of third training data; the first training data include normal audio data, the second training data include abnormal audio data, and the third training data include mixed audio data, wherein the mixed audio data is obtained by mixing the normal audio data and the abnormal audio data;

[0015] Based on the multiple first training data, respectively train the audio separation model to be trained and the audio detection model to be trained to obtain a first audio separation model and a first audio detection model;

[0016] Based on the multiple second training data, respectively train the first audio separation model and the first audio detection model to obtain a second audio separation model and a second audio detection model;

[0017] Based on the multiple third training data, the second audio separation model and the second audio detection model are jointly trained to obtain the target audio separation model and the target audio detection model.

[0018] In some embodiments, the jointly training the second audio separation model and the second audio detection model based on the plurality of third training data to obtain the target audio separation model and the target audio detection model includes:

[0019] For each third training data, calling the second audio separation model, performing audio separation processing on the third training data, and obtaining a plurality of predicted audio segments separated from the third training data;

[0020] Calling the second audio detection model to detect each predicted audio segment respectively, and obtaining a prediction detection result of each predicted audio segment;

[0021] Based on the multiple predicted detection results and the sample detection results corresponding to the third training data, the model parameters of the second audio separation model and the model parameters of the second audio detection model are adjusted.

[0022] In some embodiments, the method further comprises:

[0023] If the detection result of the audio segment is an abnormal type, a prompt message is displayed; the prompt message is used to prompt the target object to input reference information, and the reference information is used to describe the situation when the target audio data is generated in the current scene;

[0024] Acquire reference information input by the target object, and send the reference information to the cloud server.

[0025] In some embodiments, the method further comprises:

[0026] If the detection result of the audio segment is an abnormal type, log information when the target audio data is generated is obtained, and the log information is sent to the cloud server.

[0027] In some embodiments, the method further comprises:

[0028] A target processing method sent by the cloud server is received, where the target processing method is used to indicate a method for stopping the generation of the audio segment.

[0029] According to a second aspect of an embodiment of the present disclosure, there is provided an audio detection method, including:

[0030] receiving an audio segment sent by an electronic device and a detection result of the audio segment; the audio segment is audio data of which the detection result is an abnormal type, and the detection result is used to characterize the type of the audio segment, which includes a normal type or an abnormal type;

[0031] Determine, based on the audio segment and the detection result of the audio segment, a target generation reason and a target processing method of the audio segment; the target generation reason is used to indicate a reason for generating the audio segment, and the target processing method is used to indicate a method for stopping generating the audio segment;

[0032] The target processing mode is sent to the electronic device.

[0033] In some embodiments, the determining, based on the audio segment and the detection result of the audio segment, a target generation cause and a target processing method of the audio segment includes:

[0034] Based on a preset large language model, the audio segment and the detection result of the audio segment are analyzed to obtain a target generation reason of the audio segment; the preset large language model is used to predict the cause of abnormal audio data;

[0035] Based on the cause of the target generation, determine the target processing method.

[0036] In some embodiments, determining the target processing method based on the target generation cause includes:

[0037] Based on the cause of the target generation and the preset association relationship, the target processing method is determined; the preset association relationship is used to characterize the correspondence between the cause of the generation and the processing method.

[0038] In some embodiments, determining the target processing method based on the target generation cause and the preset association relationship includes:

[0039] If the preset association relationship includes the processing method corresponding to the cause of the target generation, the processing method corresponding to the cause of the target generation is determined as the target processing method;

[0040] If the preset association relationship does not include the processing method corresponding to the cause of the target generation, a reference processing method is generated, and the reference processing method is determined as the target processing method.

[0041] In some embodiments, the method further comprises:

[0042] receiving reference information sent by the electronic device;

[0043] The determining, based on the audio segment and the detection result of the audio segment, a target generation cause and a target processing method of the audio segment includes:

[0044] Based on the audio segment, the detection result of the audio segment and the reference information, a target generation cause and a target processing method of the audio segment are determined.

[0045] In some embodiments, the method further comprises:

[0046] Receiving log information sent by the electronic device;

[0047] The determining, based on the audio segment and the detection result of the audio segment, a target generation cause and a target processing method of the audio segment includes:

[0048] Based on the audio segment, the detection result of the audio segment and the log information, a target generation cause and a target processing method of the audio segment are determined.

[0049] According to a third aspect of an embodiment of the present disclosure, there is provided an audio detection device, including:

[0050] A collection module, configured to collect target audio data in the current scene;

[0051] A first processing module is configured to perform audio separation processing on the target audio data to obtain a plurality of audio segments separated from the target audio data, wherein the similarity between the sound features in the same audio segment is greater than a preset threshold;

[0052] The second processing module is configured to detect each audio segment respectively to obtain a detection result of each audio segment, wherein the detection result is used to characterize a type of the audio segment, and the type includes a normal type or an abnormal type.

[0053] According to a fourth aspect of an embodiment of the present disclosure, there is provided an audio detection device, characterized in that it includes:

[0054] A receiving module, configured to receive an audio segment sent by an electronic device and a detection result of the audio segment; the audio segment is audio data of an abnormal type of detection result, and the detection result is used to characterize the type of the audio segment, and the type includes an abnormal type;

[0055] a determination module configured to determine a target generation reason and a target processing method of the audio segment based on the audio segment and the detection result of the audio segment; the target generation reason is used to indicate a reason for generating the audio segment, and the target processing method is used to indicate a method for stopping generating the audio segment;

[0056] The sending module is configured to send the target processing method to the electronic device.

[0057] According to a fifth aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0058] processor;

[0059] a memory for storing processor-executable instructions;

[0060] The processor is configured to execute the audio detection method as described in the first aspect of the present disclosure or the second aspect of the present disclosure.

[0061] According to the sixth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the audio detection method as described in the first aspect of the present disclosure or as described in the second aspect of the present disclosure.

[0062] The above method of the present disclosure has the following beneficial effects: the present disclosure can separate the target audio data and obtain audio segments with similar or identical sound features, so that most of the abnormal audio data can be clustered into the same audio segment, and most of the normal audio data can be clustered into the same audio segment, and the separation of abnormal audio data and normal audio data can be preliminarily achieved. In this way, when detecting multiple audio segments, the detection result of each audio segment can be determined more accurately, thereby improving the accuracy of audio detection, facilitating timely processing of abnormal audio data, and improving the user experience.

[0063] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0065] Figure 1 is a flow chart of an audio detection method according to an exemplary embodiment.

[0066] Figure 2 is a flow chart of an audio detection method according to an exemplary embodiment.

[0067] Figure 3 is a flow chart of an audio detection method according to an exemplary embodiment.

[0068] Figure 4 is a flow chart of an audio detection method according to an exemplary embodiment.

[0069] Figure 5 It is a schematic diagram showing a prompt message according to an exemplary embodiment.

[0070] Figure 6 is a flow chart of an audio detection method according to an exemplary embodiment.

[0071] Figure 7 It is a block diagram of an audio detection device according to an exemplary embodiment.

[0072] Figure 8 It is a block diagram of an audio detection device according to an exemplary embodiment.

[0073] Fig. 9 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0074] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0075] Abnormal sound detection is a technology that identifies whether the collected audio data is normal audio data or abnormal audio data. Abnormal sound detection has many application scenarios. For example, abnormal sound detection can detect abnormal audio data when a machine fails, thereby monitoring the condition of the machine; abnormal sound detection can also monitor abnormal sounds in the environment, thereby monitoring and alarming abnormal audio data; abnormal sound detection can determine the operating status of certain devices and applications by analyzing the operating sounds and feedback sounds of certain devices and applications. Abnormal sound detection can help users monitor the operation of electronic devices and determine the processing method in a timely manner when abnormal audio data is generated.

[0076] However, abnormal audio data has a wide range of audio changes. For example, the independence, sparsity and low rank of abnormal audio data are difficult to define. Therefore, the accuracy of the detection results of abnormal sound detection is low, and the abnormal audio data cannot be processed in time, resulting in a poor user experience.

[0077] In order to solve the above problems, the present disclosure provides an audio detection method, including: collecting target audio data in the current scene, performing audio separation processing on the target audio data, obtaining multiple audio segments separated from the target audio data, detecting each audio segment respectively, and obtaining the detection result of each audio segment, and the detection result is used to characterize the type of the audio segment, and the type includes a normal type or an abnormal type. The present disclosure can achieve the separation of target audio data, obtain audio segments with similar or identical sound features, thereby clustering most of the abnormal audio data into the same audio segment, and clustering most of the normal audio data into the same audio segment, and preliminarily realizing the separation of abnormal audio data and normal audio data. In this way, when detecting multiple audio segments, the detection result of each audio segment can be determined more accurately, thereby improving the accuracy of audio detection, facilitating timely processing of abnormal audio data, and improving the user experience.

[0078] The audio detection method provided in the embodiment of the present disclosure is executed by an electronic device, which may be a mobile phone, a tablet computer, a notebook, an intelligent robot, a car computer, an intelligent wearable device, etc. In addition, the electronic device is also provided with various hardware resources and an energy storage device for providing power for the operation of various hardware resources.

[0079] Figure 1 is a flowchart of an audio detection method according to an exemplary embodiment, which is executed by an electronic device, see Figure 1 , the method comprises the following steps.

[0080] Step S101, collecting target audio data in the current scene.

[0081] Among them, the current scene is the scene in which the electronic device is located, and the target audio data includes scene audio data in the scene in which the electronic device is located, audio data emitted by the electronic device, etc. For example, the target audio data may include ambient sound, the speaking sound of the target object in the current scene, and the machine sound generated by the electronic device itself.

[0082] In some embodiments, the target audio data in the current scene may be collected by a sound sensor, wherein the sound sensor may be a capacitive sound sensor, a piezoresistive sound sensor, or the like.

[0083] Step S102: performing audio separation processing on the target audio data to obtain a plurality of audio segments separated from the target audio data, wherein the similarity between the sound features in the same audio segment is greater than a preset threshold.

[0084] In some embodiments, the target audio data may be subjected to audio separation processing using a sound separation technology, wherein the sound separation technology is a technology that can separate mixed audio data into multiple audio data.

[0085] Optionally, the sound separation technology may be a mask-based sound separation technology. In some embodiments, the sound separation technology may perform clustering processing on the target audio data, thereby clustering audio data with similar or identical sound features to obtain multiple audio segments. It should be noted that the similarity of the sound features clustered into the same audio segment is greater than a preset threshold, and the size of the preset threshold may be set based on actual needs, for example, the preset threshold may be 95%, 96%, etc. For example, if the target audio data includes audio data of multiple target objects (possibly multiple target audio data conversations), the audio data of multiple target objects may be clustered into the same audio segment. For another example, if the target audio data includes operating audio data of multiple internal devices of electronic devices, multiple operating audio data may be clustered into the same audio segment.

[0086] Step S103: Detect each audio segment respectively to obtain a detection result of each audio segment, where the detection result is used to characterize the type of the audio segment.

[0087] The type of the audio segment includes a normal type or an abnormal type. For example, the detection result of the audio segment is abnormal, or the detection result of the audio segment is normal.

[0088] In some embodiments, in addition to determining that each audio segment is of a normal type or an abnormal type, the specific type of each audio segment may also be determined. Optionally, the normal type may include a user's voice, environmental noise, laughter, shouting, etc., and the abnormal type may include an explosion sound, noise, abnormal speaker sound, buzzing sound, etc. For example, the detection result of a certain audio segment may be of an abnormal type, and the abnormal type is a buzzing sound.

[0089] The disclosed embodiments can separate the target audio data and obtain audio segments with similar or identical sound features, so that most of the abnormal audio data can be clustered into the same audio segment, and most of the normal audio data can be clustered into the same audio segment, thereby preliminarily separating the abnormal audio data from the normal audio data. In this way, when detecting multiple audio segments, the detection result of each audio segment can be determined more accurately, thereby improving the accuracy of audio detection, facilitating timely processing of abnormal audio data, and improving the user experience.

[0090] In some embodiments, a pre-trained audio separation model may be used to process the target audio data to obtain multiple audio segments, and a pre-trained audio detection model may be used to detect the multiple separated audio segments to obtain a detection result for each audio segment. Figure 2 The illustrated embodiment illustrates the processing of audio data.

[0091] Figure 2 is a flowchart of an audio detection method according to an exemplary embodiment, which is executed by an electronic device, see Figure 2 , the method comprises the following steps.

[0092] Step S201, calling a target audio separation model to perform audio separation processing on target audio data to obtain a plurality of audio segments separated from the target audio data.

[0093] The target audio separation model is used to perform audio separation based on the input audio data, and the target audio separation model is trained based on abnormal audio data and normal audio data. In this embodiment, the model framework used by the target audio separation model is not limited. For example, the Conv-TasNet (Convolutional Time Domain Separation Network) framework can be used. Conv-TasNet is an end-to-end sound separation model that uses sound waveforms for input and output.

[0094] It should be noted that the training process of the target audio separation model can be an unsupervised training process. Since unsupervised training does not require labeled training data, unsupervised training can be used to determine the structure and relationship between training data. For example, unsupervised training can achieve clustering, dimensionality reduction or feature extraction between training data. In one example, the target audio separation model can cluster the input target audio data, thereby clustering audio clips with similar or identical sound features to obtain multiple audio clips.

[0095] Step S202: calling the target audio detection model to detect each audio segment respectively to obtain a detection result for each audio segment.

[0096] Among them, the target audio detection model is used to perform audio detection based on the input audio data, and the target audio detection model is trained based on abnormal audio data and normal audio data. In this embodiment, the framework used by the target audio detection model is not limited. For example, the target audio detection model can use a self-researched framework. In addition, in this embodiment, the model parameters of the target audio detection model are not limited. For example, the batch size of the target audio detection model can be 2, the target audio detection model can use the Adam (Adaptive Moment Estimation) optimizer, the learning rate of the target audio detection model can be 0.0001, and the loss function of the target audio detection model can use the mean square error loss function of the input and output quantities, and train a preset number of times.

[0097] Optionally, the sound features of the audio clips may be processed to determine the detection result of each audio clip. For example, the sound waveform of each audio clip may be converted into a 128-dimensional Mel spectrogram with a frame size of 1024 and a frame shift of 512, and then the target audio detection model may be called to detect the processed audio clips to obtain the detection result of each processed audio clip.

[0098] It should be noted that the training process of the target audio detection model is a supervised training process. Supervised training uses known inputs and outputs (labels) to learn the mapping relationship between input features and outputs. In one example, the target audio detection model can determine the detection result of the input audio segment, and the detection result is related to the label in the training process. For example, when the label in the training process is the type of audio data, the detection result is the type of the audio segment.

[0099] The training process of the target audio separation model and the target audio detection model includes the joint training of the audio separation model and the audio detection model. Therefore, in order to more clearly determine the training process of the audio separation model and the audio detection model, the training process of the target audio separation model and the target audio detection model is explained below.

[0100] In some embodiments, the training process of the target audio separation model and the target audio detection model includes:

[0101] Obtain multiple first training data, multiple second training data, and multiple third training data, and based on the multiple first training data, respectively train the audio separation model to be trained and the audio detection model to be trained to obtain the first audio separation model and the first audio detection model, and based on the multiple second training data, respectively train the first audio separation model and the first audio detection model to obtain the second audio separation model and the second audio detection model, and based on the multiple third training data, jointly train the second audio separation model and the second audio detection model to obtain the target audio separation model and the target audio detection model. Among them, the first training data includes normal audio data, the second training data includes abnormal audio data, and the third training data includes mixed audio data, and the mixed audio data is obtained by mixing normal audio data and abnormal audio data. In addition, it should be noted that when using the first training data and the second training data to train the audio detection model, it is necessary to set the label of the training data. For example, the first training data can be set with a normal type label, and the second training data can be set with an abnormal type label.

[0102] Optionally, the audio data in the first training data, the second training data, and the third training data may include noise. For example, the normal audio data in the first training data includes noise, and the abnormal audio data in the second training data includes noise.

[0103] Optionally, when performing audio separation processing on the first training data or the third training data, noise reduction processing may be performed on normal audio data in the first training data or the third training data, so as to obtain clean normal audio data. When performing audio separation processing on the second training data or the third training data, noise reduction processing may not be performed on abnormal audio data in the second training data or the third training data, so as to determine the original abnormal audio data, so as to further ensure the accuracy of the detection result of the audio segment.

[0104] In one example, normal audio data in the training data may include user voices, laughter, animal calls (such as bird calls), keyboard tapping, metal collision sounds, etc., and abnormal audio data may include machine sounds (such as abnormal machine operating noises, beeps), etc.

[0105] In some embodiments, based on multiple third training data, the second audio separation model and the second audio detection model are jointly trained to obtain a target audio separation model and a target audio detection model, including: for each third training data, calling the second audio separation model, performing audio separation processing on the third training data, and obtaining multiple predicted audio segments separated from the third training data; calling the second audio detection model, respectively detecting each predicted audio segment, and obtaining a predicted detection result for each predicted audio segment; based on the multiple predicted detection results and the sample detection results corresponding to the third training data, adjusting the model parameters of the second audio separation model and the model parameters of the second audio detection model.

[0106] In some embodiments, the sample detection results corresponding to the third training data and the multiple predicted detection results can be compared. If the similarity between the sample detection results corresponding to the third training data and the multiple predicted detection results is greater than a preset value, the model parameters characterizing the second audio separation model and the model parameters of the second audio detection model are better. At this time, the model parameters of the second audio separation model and the model parameters of the second audio detection model can be fine-tuned. Conversely, if the similarity between the sample detection results corresponding to the third training data and the multiple predicted detection results is less than the preset value, the model parameters characterizing the second audio separation model and the model parameters of the second audio detection model are poor. At this time, the model parameters of the second audio separation model and the model parameters of the second audio detection model can be significantly adjusted until the similarity between the sample detection results corresponding to the third training data and the multiple predicted detection results is greater than the preset value. Among them, the size of the preset value can be set based on actual needs.

[0107] In addition, it should be noted that in the process of training the audio separation model and the audio detection model, audio data of an electronic device with a hardware structure similar to that of the electronic device can be used for training. For example, the audio data of an electronic device of a different model from the same series as the electronic device can be used to train the audio separation model and the audio detection model of the electronic device. If the hardware structure of another electronic device is significantly different from that of the electronic device, the target audio separation model and the target audio detection model that characterize the electronic device may not be applicable to the electronic device, and the audio separation model and the audio detection model need to be retrained for the electronic device.

[0108] In the disclosed embodiment, the target audio separation model can be used to separate the target audio data, and obtain audio segments with similar or identical sound features, thereby preliminarily separating abnormal audio data from normal audio data. In this way, when the target audio detection model is used to detect the audio segments, a more accurate detection result for each audio segment can be determined, thereby improving the accuracy of abnormal audio detection.

[0109] In some embodiments, no processing may be performed on audio clips whose detection results are normal types, while for audio clips whose detection results are abnormal types, the electronic device may send the audio clip and the detection results of the audio clip to the cloud server, and the cloud server may receive the audio clip and the detection results of the audio clip, and determine a solution for the audio clip. Among them, the cloud server can provide efficient, stable, and secure cloud computing services and cloud computing resources to realize functions such as data storage and data analysis. It should be noted that when the same electronic device simultaneously generates multiple audio clips with abnormal detection results, the electronic device may send multiple audio clips and the abnormal type of each audio clip to the cloud server in sequence or simultaneously, and the cloud server may process the received data (multiple audio clips and the abnormal type of each audio clip) in sequence or simultaneously.

[0110] Below through Figure 3 The illustrated embodiment illustrates the process of determining a resolution of an audio segment.

[0111] Figure 3 is a flowchart of an audio detection method according to an exemplary embodiment, which is executed by a cloud server, see Figure 3 , the method comprises the following steps.

[0112] Step S301: receiving an audio clip sent by an electronic device and a detection result of the audio clip.

[0113] The audio segment is audio data of an abnormal type for which the detection result is used to characterize the type of the audio segment, and the type includes the abnormal type. In one example, the detection result of the audio segment also includes a specific type of the abnormal type, and the specific type of the abnormal type can refer to the type of the abnormal type shown in step 103 in the above embodiment.

[0114] Step S302: determining a target generation cause and a target processing method of the audio segment based on the audio segment and the detection result of the audio segment.

[0115] The target generation reason is used to indicate the reason for generating the audio segment, and the target processing method is used to indicate the method for stopping generating the audio segment.

[0116] In some embodiments, the cloud server can analyze the audio clip and the detection result of the audio clip, and determine the target generation cause and the target processing method for the audio clip. The specific process of the cloud server performing the analysis can refer to the content shown below.

[0117] Step S303: sending the target processing method to the electronic device.

[0118] In some embodiments, the target processing method can be sent to the electronic device through the communication connection between the electronic device and the cloud server. Afterwards, the electronic device can receive the target processing method and make corresponding processing based on the target processing method, thereby stopping the generation of audio clips corresponding to the target processing method with the detection result of abnormal type.

[0119] The disclosed embodiment can analyze the audio clips and the detection results of the audio clips through a cloud server, so as to timely locate the causes of the audio clips and the processing methods, repair the possible problems of the electronic device, and provide users with a better user experience.

[0120] In some embodiments, the following Figure 4 The illustrated embodiment illustrates the process of determining the cause of a target and the target processing method.

[0121] Figure 4 is a flowchart of an audio detection method according to an exemplary embodiment, which is executed by an electronic device and a cloud server, see Figure 4 The interaction process between the electronic device and the cloud server shown includes the following steps.

[0122] Step S401: the electronic device collects target audio data in the current scene.

[0123] Step S402: the electronic device performs audio separation processing on the target audio data to obtain a plurality of audio segments separated from the target audio data.

[0124] In step S403, the electronic device detects each audio segment respectively to obtain a detection result of each audio segment.

[0125] Among them, step S401 to step S403 are the same as step S101 to step S103 in the above embodiment, and will not be repeated here.

[0126] Step S404: If the detection result of the audio segment is an abnormal type, the electronic device sends the audio segment and the detection result of the audio segment to the cloud server.

[0127] In one example, the electronic device may send the audio segment and the detection result of the audio segment to the cloud server through a communication connection between the electronic device and the cloud server.

[0128] Step S405: the cloud server receives the audio clip sent by the electronic device and the detection result of the audio clip.

[0129] The audio segment is audio data whose detection result is an abnormal type, and the detection result is used to characterize the type of the audio segment, and the type includes the abnormal type.

[0130] Step S406: If the detection result of the audio clip is an abnormal type, the electronic device displays a prompt message.

[0131] The prompt information is used to prompt the target object to input reference information. The target object may be a user of the electronic device. The reference information is used to describe the situation when the target audio data is generated in the current scene.

[0132] In one example, the prompt information may be displayed on the display screen of the electronic device in the form of a pop-up window or a prompt box, and the prompt information includes prompt content and an input box for inputting reference information. Figure 5 A schematic diagram of a prompt message is shown, in which the prompt message is displayed in a pop-up window on a display screen of an electronic device, and the pop-up window includes a prompt message "Please input the situation when the current abnormal sound is generated" and an input box for entering reference information.

[0133] In some embodiments, this embodiment does not limit the specific type of reference information, and the reference information may be text information, voice information, or audio information. Accordingly, the input box for inputting reference information may be a text box or a voice box.

[0134] Optionally, the reference information may be the condition of the electronic device when the target audio data is generated, for example, the reference information may be that the electronic device is running an application program and the electronic device is placed flat on a desktop. Optionally, the reference information may also be descriptive information related to the target audio data, for example, the reference information may be an onomatopoeia of the target audio data, such as a buzzing sound or a whirring sound; for another example, the reference information may be the duration and volume of the target audio data.

[0135] Step S407: the electronic device obtains the reference information input by the target object and sends the reference information to the cloud server.

[0136] The internal program in the electronic device can read the information input by the target object and send reference information to the cloud server through the communication connection between the electronic device and the cloud server. The specific process of the electronic device reading the information input by the target object is not discussed here.

[0137] Step S408: the cloud server receives the reference information sent by the electronic device.

[0138] In one example, after receiving the reference information sent by the electronic device, the cloud server may store the reference information.

[0139] Step S409: If the detection result of the audio segment is an abnormal type, the electronic device obtains log information when the target audio data is generated, and sends the log information to the cloud server.

[0140] The content of the log information can be set based on actual needs. The following are some possible contents of the log information.

[0141] Optionally, the log information may include the generation time and / or duration of the target audio data, for example, the target audio data is generated at 2024.2.12 15:00:06 and lasts for 10 seconds.

[0142] Optionally, the log information may include application information running in the foreground and / or background of the electronic device when the target audio data is generated. The application information may include an application identifier, application version information, developer information, etc. of the application.

[0143] Optionally, the log information may include the external environment in which the electronic device is located when the target audio data is generated. In one example, the external environment may be determined by a sound sensor or an image sensor provided on the electronic device. For example, the external environment may be determined to be a library environment by an image sensor.

[0144] Optionally, the log information may include the operation of the target object or the operation being performed by the electronic device when the target audio data is generated. For example, the target object is performing a click operation; for another example, the electronic device is performing a query operation, a refresh operation, etc. In addition, the object information targeted by the operation of the target object can also be obtained. For example, if the current click operation is a click operation based on a certain button, then when obtaining the click operation, information such as the function of the button can be obtained.

[0145] Optionally, the log information may include device information of the electronic device when the target audio data is generated, etc. The device information of the electronic device may include version information of the electronic device, storage information of the electronic device, developer information of the electronic device, system information of the electronic device, etc.

[0146] In some embodiments, all the acquired log information can be packaged, and the packaged information can be determined as the log information sent to the cloud server. It should be noted that packaging the log information can reduce the occupation of memory and computing resources during data transmission, and packaging the log information can facilitate data storage and maintain data consistency. Optionally, all log information can be packaged in text form to facilitate user viewing.

[0147] In this step, obtaining log information can solve the problem of difficulty in capturing relevant data when abnormal audio data is generated when the user uses an electronic device. Obtaining log information can improve the developer's efficiency in reproducing the scenario where abnormal audio data is generated, as well as improve the developer's problem analysis and location effect, making it easier for developers to solve problems in a timely manner and improving the user experience.

[0148] Step S410: the cloud server receives log information sent by the electronic device.

[0149] In one example, after receiving the log information sent by the electronic device, the cloud server can store the log information to facilitate use in subsequent processes.

[0150] In step S411, the cloud server determines the target generation cause and target processing method of the audio segment based on the audio segment, the detection result of the audio segment, the reference information and the log information.

[0151] In some embodiments, a pre-trained preset large language model is set in the cloud server, and the preset large language model can determine the target generation cause of the audio segment based on the information received by the cloud server, such as the audio segment, the detection result of the audio segment, any one of the reference information and the log information, or a combination of multiple information, to determine the target generation cause of the audio segment.

[0152] In some embodiments, the information received by the cloud server can be input into a preset large language model, and the received information can be analyzed based on the preset large language model to obtain the target cause of the audio segment. In some embodiments, the preset large language model can be determined as a preset audio database and a preset text database, the preset audio database includes audio data and corresponding causes, and the preset text database includes text data and corresponding causes. When the preset large language model receives audio data such as an audio segment, the cause corresponding to the audio segment can be searched in the preset audio database, and when the preset large language model receives text data such as the detection result, reference information or log information of the audio segment, the corresponding cause can be determined in the preset text database.

[0153] It should be noted that the target cause determined by the preset large language model may be multiple target causes. In some embodiments, the target processing method corresponding to the target cause can be determined based on the target cause. Accordingly, for multiple target causes, the target processing method corresponding to each target cause can be determined. In some embodiments, the target processing method can be determined based on the target cause and a preset association relationship. Among them, the preset association relationship can be used to characterize the correspondence between the cause and the processing method. It should be noted that the processing method corresponding to the cause in the preset association relationship is a processing method for the cause that has been verified. That is, the processing method in the preset association relationship can effectively stop the generation of audio clips whose detection results are abnormal types. In addition, the preset association relationship can be determined as a preset database set in a cloud server, and the preset association relationship / preset database includes multiple causes and processing methods corresponding to each cause.

[0154] In some embodiments, it may be determined whether the preset association relationship includes the target generation cause and the processing method corresponding to the target generation cause to determine the target processing method. Optionally, if the preset association relationship includes the processing method corresponding to the target generation cause, the processing method corresponding to the target generation cause is determined as the target processing method.

[0155] Optionally, if the preset association relationship does not include the processing method corresponding to the target cause, a reference processing method is generated and the reference processing method is determined as the target processing method. In some embodiments, when the preset association relationship does not include the processing method corresponding to the target cause, a reference processing method corresponding to the target cause can be generated by a preset large language model. Among them, the open source data set corresponding to the training data of the preset large language model can also include multiple causes and the processing method corresponding to each cause.

[0156] In some embodiments, the reference processing method generated by the preset large language model needs to be verified by the developer, and after the developer successfully verifies, the reference processing method is determined as the target processing method. Optionally, the developer to be contacted can be determined by the target generation cause, and after the developer to be contacted is determined, the target generation cause and the reference processing method are sent to the developer. After the developer receives the target generation cause and the reference processing method, the developer will verify the received reference processing method for the target generation cause.

[0157] Optionally, when the target cause is related to an application, the target cause and a reference processing method may be sent to the developer of the application, that is, the developer of the application verifies the reference processing method; or, when the target cause is related to an electronic device, the target cause and a reference processing method may be sent to the developer of the electronic device, that is, the developer of the electronic device verifies the reference processing method. Optionally, the developer of the electronic device may send the target cause and the reference processing method in a unified manner, and the developer of the electronic device may contact a corresponding technician to verify the reference processing method, and the corresponding technician may be the developer of the electronic device or the developer of the application.

[0158] In some embodiments, the developer can verify the reference processing method by simulating the target generation cause. If the reference processing method can stop the generation of the audio clip corresponding to the target generation cause, then the reference processing method is determined to be successfully verified and the reference processing method is determined to be the target processing method. On the contrary, if the reference processing method cannot stop the generation of the audio clip corresponding to the target generation cause, then the reference processing method is determined to have failed verification. If the reference processing method fails verification, the developer can re-determine the target processing method corresponding to the target generation cause.

[0159] Optionally, if the preset association relationship includes a processing method corresponding to the cause of target generation, the processing method corresponding to the cause of target generation may be verified and then determined as the target processing method.

[0160] In this step, the received information is analyzed by the preset large language model to generate the cause of the target, and generate a reference processing method when the target processing method cannot be determined. In this way, the preset large language model can assist developers in determining the processing method, improve the work quality and work efficiency of developers, and send the target processing method to the electronic device in a timely manner, thereby improving the user experience.

[0161] In step S412, the cloud server sends the target processing method of the audio segment to the electronic device.

[0162] In some embodiments, the cloud server can send the target processing method of the audio clip to the electronic device through the communication connection between the cloud server and the electronic device in the role of a developer.

[0163] Step S413, the electronic device receives the target processing method sent by the cloud server.

[0164] The target processing method is used to indicate a method for stopping the generation of the audio segment.

[0165] In some embodiments, after receiving the target processing method, the electronic device may display the target processing method on a display screen of the electronic device to facilitate the user to view the target processing method.

[0166] Optionally, the target processing method can be automatically executed by the electronic device or manually executed by the target object. For example, the target object can refer to the target processing method to process the electronic device or application, so as to achieve the effect of stopping the generation of audio clips with abnormal detection results. Optionally, during the processing, the target object can also view the processing progress in real time.

[0167] Optionally, the order in which the electronic device sends information to the cloud server in steps S401-S413 in the above embodiment can be adjusted based on actual conditions, and the above steps S406-S408 and steps S409-S410 are optional steps. That is, the electronic device can choose not to execute steps S406-S408 and / or steps S409-S410. For example, when steps S406-S408 are not executed, the electronic device sends the audio clip, the detection results of the audio clip, and log information to the cloud server, and step S411 is for the cloud server to determine the target cause of the audio clip and the target processing method based on the audio clip, the detection results of the audio clip, and the log information; when steps S409-S410 are not executed, the electronic device sends the audio clip, the detection results of the audio clip, and reference information to the cloud server, and step S411 is for the cloud server to determine the target cause of the audio clip and the target processing method based on the audio clip, the detection results of the audio clip, and the reference information; when steps S406-S408 and steps S409-S410 are not executed, the electronic device sends the audio clip and the detection results of the audio clip to the cloud server, and step S411 is for the cloud server to determine the target cause of the audio clip and the target processing method based on the audio clip and the detection results of the audio clip.

[0168] It should be noted that the cloud server may receive information from multiple electronic devices. In some embodiments, in order to distinguish different electronic devices, the electronic device can simultaneously send the identification information and time information of the electronic device when sending information such as audio clips, detection results of audio clips, reference information and log information to the cloud server. Among them, the identification information of the electronic device can be the device ID (Identity document, ID card identification number) of the electronic device, user ID, etc. Correspondingly, after the cloud server determines the target processing method corresponding to the audio clip of a certain electronic device, the cloud server can send the target processing method corresponding to the audio clip to the electronic device corresponding to the identification information based on the identification information of the electronic device for user viewing.

[0169] For ease of understanding, a specific example is given below to illustrate the interaction process between the electronic device and the cloud server:

[0170] Electronic device 1 sends the identification information of electronic device 1, audio segment A1 in target audio data A, and the detection result of audio segment A1 to the cloud server; electronic device 2 sends the identification information of electronic device 2, audio segment B1 in target audio data B, the detection result of audio segment B1, and reference information B2 to the cloud server; electronic device 3 sends the identification information of electronic device 3, audio segment C1 in target audio data C, the detection result of audio segment C1, reference information C2, and log information C3 to the cloud server. The cloud server receives information from electronic devices 1, 2, and 3, and determines the target generation cause and target processing method of audio segments A1, B1, and C1 respectively based on the received information. Afterwards, the cloud server sends the target generation cause and target processing method of audio segment A1 to electronic device 1 through the identification information of electronic device 1, and electronic device 1 receives the target generation cause and target processing method of audio segment A1. The cloud server sends the target generation cause and target processing method of audio segment B1 to electronic device 2 through the identification information of electronic device 2, and electronic device 2 receives the target generation cause and target processing method of audio segment B1. The cloud server sends the target generation cause and target processing method of the audio segment C1 to the electronic device 3 via the identification information of the electronic device 3, and the electronic device 3 receives the target generation cause and target processing method of the audio segment C1.

[0171] The disclosed embodiment provides a specific interaction process between the electronic device and the cloud server. Through the interaction between the electronic device and the cloud server, the target processing method corresponding to the audio segment with the detection result of abnormal type can be determined, thereby realizing abnormal audio detection. In addition, by adding the target audio separation model, the target audio detection model and the preset large language model, the audio detection method shown in the disclosed embodiment is more intelligent and the efficiency of determining the target processing method is higher.

[0172] Figure 6It is a flowchart of an audio detection method according to an exemplary embodiment, and the execution subject of the flowchart is an electronic device and a cloud server, wherein the steps executed by the electronic device include: the electronic device collects target audio data, and inputs the target audio data into a target audio separation model to obtain multiple audio segments, and then inputs the multiple audio segments into a target audio detection model to obtain the detection result of each audio segment. If the detection result of the audio segment is an abnormal type, the electronic device sends the audio segment and the detection result of the audio segment to the cloud server. The steps executed by the cloud server include: the preset large language model of the cloud server determines the target processing method based on the audio segment and the detection result of the audio segment, and the cloud server sends the target processing method to the electronic device. In addition, the electronic device can obtain reference information and log information, and send the reference information and log information to the cloud server. The preset large language model in the cloud server can determine the target processing method based on the detection result of the audio segment and the audio segment in combination with the reference information and log information.

[0173] Figure 7 is a block diagram of an audio detection device according to an exemplary embodiment, which is configured in an electronic device, see Figure 7 , the device comprises:

[0174] The acquisition module 701 is configured to acquire target audio data in the current scene;

[0175] The first processing module 702 is configured to perform audio separation processing on the target audio data to obtain multiple audio segments separated from the target audio data, and the similarity between the sound features in the same audio segment is greater than a preset threshold;

[0176] The second processing module 703 is configured to detect each audio segment respectively to obtain a detection result of each audio segment, where the detection result is used to characterize the type of the audio segment, including a normal type or an abnormal type.

[0177] In some embodiments, the first processing module 702 is configured to:

[0178] Calling a target audio separation model to perform audio separation processing on the target audio data to obtain multiple audio segments separated from the target audio data; the target audio separation model is used to perform audio separation based on the input audio data, and the target audio separation model is trained based on the abnormal audio data and the normal audio data;

[0179] Each audio clip is tested separately to obtain the test result of each audio clip, including:

[0180] The target audio detection model is called to detect each audio segment respectively to obtain the detection result of each audio segment; the target audio detection model is used to perform audio detection based on the input audio data, and the target audio detection model is trained based on abnormal audio data and normal audio data.

[0181] In some embodiments, the first processing module 702 is configured to:

[0182] Acquire a plurality of first training data, a plurality of second training data, and a plurality of third training data; the first training data includes normal audio data, the second training data includes abnormal audio data, and the third training data includes mixed audio data, where the mixed audio data is obtained by mixing the normal audio data and the abnormal audio data;

[0183] Based on the plurality of first training data, respectively train the audio separation model to be trained and the audio detection model to be trained to obtain a first audio separation model and a first audio detection model;

[0184] Based on the plurality of second training data, respectively train the first audio separation model and the first audio detection model to obtain a second audio separation model and a second audio detection model;

[0185] Based on the multiple third training data, the second audio separation model and the second audio detection model are jointly trained to obtain the target audio separation model and the target audio detection model.

[0186] In some embodiments, the first processing module 702 is configured to:

[0187] For each third training data, calling the second audio separation model, performing audio separation processing on the third training data, and obtaining a plurality of predicted audio segments separated from the third training data;

[0188] Calling the second audio detection model to detect each predicted audio segment respectively, and obtaining a prediction detection result of each predicted audio segment;

[0189] Based on the multiple predicted detection results and the sample detection results corresponding to the third training data, the model parameters of the second audio separation model and the model parameters of the second audio detection model are adjusted.

[0190] In some embodiments, the second processing module 703 is configured to:

[0191] If the detection result of the audio clip is an abnormal type, the audio clip and the detection result of the audio clip are sent to the cloud server.

[0192] In some embodiments, the second processing module 703 is configured to:

[0193] If the detection result of the audio clip is an abnormal type, a prompt message is displayed; the prompt message is used to prompt the target object to input reference information, and the reference information is used to describe the situation when the target audio data is generated in the current scene;

[0194] Get the reference information input by the target object and send the reference information to the cloud server.

[0195] In some embodiments, the second processing module 703 is configured to:

[0196] If the detection result of the audio clip is an abnormal type, the log information when the target audio data is generated is obtained and sent to the cloud server.

[0197] In some embodiments, the second processing module 703 is configured to:

[0198] Receive a target processing method sent by a cloud server, where the target processing method is used to indicate a method for stopping the generation of audio segments.

[0199] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0200] Figure 8 is a block diagram of an audio detection device according to an exemplary embodiment, which is configured in a cloud server, see Figure 8 , the device comprises:

[0201] The receiving module 801 is configured to receive an audio segment and a detection result of the audio segment sent by an electronic device; the audio segment is audio data of which the detection result is an abnormal type, and the detection result is used to characterize the type of the audio segment, and the type includes the abnormal type;

[0202] The determination module 802 is configured to determine a target generation reason and a target processing method of the audio segment based on the audio segment and the detection result of the audio segment; the target generation reason is used to indicate a reason for generating the audio segment, and the target processing method is used to indicate a method for stopping generating the audio segment;

[0203] The sending module 803 is configured to send the target processing method to the electronic device.

[0204] In some embodiments, the determination module 802 is configured to:

[0205] Based on the preset large language model, the audio clips and the detection results of the audio clips are analyzed to obtain the target generation cause of the audio clips; the preset large language model is used to predict the cause of abnormal audio data;

[0206] Determine how to handle the target based on the cause of the target.

[0207] In some embodiments, the determination module 802 is configured to:

[0208] The target processing method is determined based on the target generation cause and the preset association relationship; the preset association relationship is used to characterize the correspondence between the generation cause and the processing method.

[0209] In some embodiments, the determination module 802 is configured to:

[0210] If the preset association relationship includes a processing method corresponding to the cause of the target generation, the processing method corresponding to the cause of the target generation is determined as the target processing method;

[0211] If the preset association relationship does not include the processing method corresponding to the cause of the target generation, a reference processing method is generated and determined as the target processing method.

[0212] In some embodiments, the determination module 802 is configured to:

[0213] receiving reference information sent by an electronic device;

[0214] Based on the audio segment, the detection result of the audio segment and the reference information, a target generation cause and a target processing method of the audio segment are determined.

[0215] In some embodiments, the determination module 802 is configured to:

[0216] Receiving log information sent by electronic devices;

[0217] Based on the audio segment, the detection result of the audio segment and the log information, determine the target generation cause and target processing method of the audio segment.

[0218] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0219] An embodiment of the present disclosure further provides an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the audio detection method in the above embodiment.

[0220] Fig. 9 is a block diagram of an electronic device 900 according to an exemplary embodiment.

[0221] Reference Fig. 9, the electronic device 900 may include one or more of the following components: a processing component 902 , a memory 904 , a power component 906 , a multimedia component 908 , an audio component 910 , an input / output (I / O) interface 912 , a sensor component 914 , and a communication component 916 .

[0222] The processing component 902 generally controls the overall operation of the electronic device 900, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 902 may include one or more processors 920 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 902 may include one or more modules to facilitate the interaction between the processing component 902 and other components. For example, the processing component 902 may include a multimedia module to facilitate the interaction between the multimedia component 908 and the processing component 902.

[0223] The memory 904 is configured to store various types of data to support operations on the electronic device 900. Examples of such data include instructions for any application or method operating on the electronic device 900, contact data, phone book data, messages, pictures, videos, etc. The memory 904 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0224] The power supply component 906 provides power to various components of the electronic device 900. The power supply component 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 900.

[0225] The multimedia component 908 includes a screen that provides an output interface between the electronic device 900 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 908 includes a front camera and / or a rear camera. When the electronic device 900 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0226] The audio component 910 is configured to output and / or input audio signals. For example, the audio component 910 includes a microphone (MIC), and when the electronic device 900 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 904 or sent via the communication component 916. In some embodiments, the audio component 910 also includes a speaker for outputting audio signals.

[0227] I / O interface 912 provides an interface between processing component 902 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0228] The sensor assembly 914 includes one or more sensors for providing various aspects of status assessment for the electronic device 900. For example, the sensor assembly 914 can detect the open / closed state of the electronic device 900, the relative positioning of components, such as the display and keypad of the electronic device 900, and the sensor assembly 914 can also detect the position change of the electronic device 900 or a component of the electronic device 900, the presence or absence of user contact with the electronic device 900, the orientation or acceleration / deceleration of the electronic device 900, and the temperature change of the electronic device 900. The sensor assembly 914 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 914 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 914 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0229] The communication component 916 is configured to facilitate wired or wireless communication between the electronic device 900 and other devices. The electronic device 900 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 916 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 916 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0230] In an exemplary embodiment, the electronic device 900 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0231] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 904 including instructions, and the instructions can be executed by a processor 920 of an electronic device 900 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0232] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute the audio detection method provided by an exemplary embodiment of the present disclosure.

[0233] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0234] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An audio detection method, It is characterized in that include: Collect target audio data in the current scene; Performing audio separation processing on the target audio data to obtain multiple audio segments separated from the target audio data, wherein the similarity between sound features in the same audio segment is greater than a preset threshold; Detecting each audio segment respectively to obtain a detection result of each audio segment, wherein the detection result is used to characterize a type of the audio segment, wherein the type includes a normal type or an abnormal type; The performing audio separation processing on the target audio data to obtain a plurality of audio segments separated from the target audio data includes: Calling a target audio separation model to perform audio separation processing on the target audio data to obtain a plurality of audio segments separated from the target audio data; the target audio separation model is used to perform audio separation based on the input audio data, and the target audio separation model is trained based on abnormal audio data and normal audio data; The detecting each audio segment respectively to obtain the detection result of each audio segment includes: Calling a target audio detection model to detect each of the audio segments respectively to obtain a detection result of each of the audio segments; the target audio detection model is used to perform audio detection based on the input audio data, and the target audio detection model is trained based on abnormal audio data and normal audio data; The training process of the target audio separation model and the target audio detection model includes: Acquire a plurality of first training data, a plurality of second training data, and a plurality of third training data; the first training data include normal audio data, the second training data include abnormal audio data, and the third training data include mixed audio data, wherein the mixed audio data is obtained by mixing the normal audio data and the abnormal audio data; Based on the multiple first training data, respectively train the audio separation model to be trained and the audio detection model to be trained to obtain a first audio separation model and a first audio detection model; Based on the multiple second training data, respectively train the first audio separation model and the first audio detection model to obtain a second audio separation model and a second audio detection model; Based on the multiple third training data, the second audio separation model and the second audio detection model are jointly trained to obtain the target audio separation model and the target audio detection model.

2. The audio detection method according to claim 1, It is characterized in that The step of jointly training the second audio separation model and the second audio detection model based on the plurality of third training data to obtain the target audio separation model and the target audio detection model includes: For each third training data, calling the second audio separation model, performing audio separation processing on the third training data, and obtaining a plurality of predicted audio segments separated from the third training data; Calling the second audio detection model to detect each predicted audio segment respectively, and obtaining a prediction detection result of each predicted audio segment; Based on the multiple predicted detection results and the sample detection results corresponding to the third training data, the model parameters of the second audio separation model and the model parameters of the second audio detection model are adjusted.

3. The audio detection method according to claim 1, It is characterized in that The method further comprises: If the detection result of the audio segment is an abnormal type, the audio segment and the detection result of the audio segment are sent to a cloud server.

4. The audio detection method according to claim 3, It is characterized in that The method further comprises: If the detection result of the audio segment is an abnormal type, a prompt message is displayed; the prompt message is used to prompt the target object to input reference information, and the reference information is used to describe the situation when the target audio data is generated in the current scene; Acquire reference information input by the target object, and send the reference information to the cloud server.

5. The audio detection method according to claim 3, It is characterized in that The method further comprises: If the detection result of the audio segment is an abnormal type, log information when the target audio data is generated is obtained, and the log information is sent to the cloud server.

6. The audio detection method according to claim 3, It is characterized in that The method further comprises: A target processing method sent by the cloud server is received, where the target processing method is used to indicate a method for stopping the generation of the audio segment.

7. An audio detection device, It is characterized in that include: A collection module, configured to collect target audio data in the current scene; A first processing module is configured to perform audio separation processing on the target audio data to obtain a plurality of audio segments separated from the target audio data, wherein the similarity between the sound features in the same audio segment is greater than a preset threshold; A second processing module is configured to detect each audio segment respectively to obtain a detection result of each audio segment, wherein the detection result is used to characterize a type of the audio segment, wherein the type includes a normal type or an abnormal type; The first processing module is configured to: Calling a target audio separation model to perform audio separation processing on the target audio data to obtain a plurality of audio segments separated from the target audio data; the target audio separation model is used to perform audio separation based on the input audio data, and the target audio separation model is trained based on abnormal audio data and normal audio data; The second processing module is configured to: Calling a target audio detection model to detect each of the audio segments respectively to obtain a detection result of each of the audio segments; the target audio detection model is used to perform audio detection based on the input audio data, and the target audio detection model is trained based on abnormal audio data and normal audio data; The first processing module is further configured to: Acquire a plurality of first training data, a plurality of second training data, and a plurality of third training data; the first training data include normal audio data, the second training data include abnormal audio data, and the third training data include mixed audio data, wherein the mixed audio data is obtained by mixing the normal audio data and the abnormal audio data; Based on the multiple first training data, respectively train the audio separation model to be trained and the audio detection model to be trained to obtain a first audio separation model and a first audio detection model; Based on the multiple second training data, respectively train the first audio separation model and the first audio detection model to obtain a second audio separation model and a second audio detection model; Based on the multiple third training data, the second audio separation model and the second audio detection model are jointly trained to obtain the target audio separation model and the target audio detection model.

8. An electronic device, It is characterized in that include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the audio detection method according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium, It is characterized in that When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the audio detection method as described in any one of claims 1 to 6.