Audio processing method and device, equipment, storage medium and program product

Abstract

The invention provides an audio processing method and device, equipment, a storage medium and a program product, and relates to the field of audio processing or the field of intelligent cabins. The method comprises the following steps: in an audio playing process, carrying out abnormity identification on played audio data; in response to the identified audio abnormal event, acquiring an abnormal audio clip and an abnormal log; analyzing the abnormal audio clip and the abnormal log, and determining an abnormal recovery strategy; and executing exception recovery according to the exception recovery strategy. According to the method, abnormity identification can be actively carried out on the played audio data in real time in the playing process. And the automatic abnormal audio clip and the abnormal log can be analyzed after the abnormity is confirmed, and the abnormity recovery strategy is triggered, so that the continuity and the stability of audio playing are guaranteed, and the reliability of the system and the user experience are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of audio processing or intelligent cabin, and particularly relates to an audio processing method and device, equipment, storage medium and program product. BACKGROUND

[0002] With the development of computer technology and audio technology, audio is increasingly widely used in various scenarios such as entertainment, communication, conference, vehicle and smart home. During the playing process, audio abnormalities may occur due to various reasons, which will affect the product usability and user satisfaction.

[0003] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0004] The purpose of the present disclosure is to provide an audio processing method, device, equipment, storage medium and program product.

[0005] According to a first aspect of an embodiment of the present disclosure, an audio processing method is provided, comprising: in an audio playing process, identifying an abnormality of playing audio data; in response to identifying an audio abnormality event, acquiring an abnormal audio segment and an abnormal log; analyzing the abnormal audio segment and the abnormal log to determine an abnormality recovery strategy; and executing the abnormality recovery according to the abnormality recovery strategy.

[0006] The technical solution provided by the embodiments of the present disclosure can include the following beneficial effects:

[0007] The present disclosure can actively identify the abnormality of the playing audio data in real time during the playing process. After confirming the abnormality, the present disclosure can analyze the automatic abnormal audio segment and the abnormal log to trigger the abnormality recovery strategy, thereby ensuring the continuity and stability of the audio playing and significantly improving the reliability and user experience of the system.

[0008] In some embodiments, in the audio playing process, the abnormality of the playing audio data is identified, comprising: in the audio playing process, segmenting and collecting audio segments; and comparing each audio segment with sample abnormal data to determine whether an audio abnormality event occurs according to the comparison result.

[0009] In the above embodiments, precise abnormal event detection can be achieved by segmenting and collecting and sample comparison, thereby reducing the missed detection rate and improving the identification reliability.

[0010] In some embodiments, in response to identifying the audio abnormal event, the abnormal audio segment and the abnormal log are obtained, including: taking the audio segment identified as the audio abnormal event in the comparison as the abnormal audio segment; and extracting the abnormal log corresponding to the abnormal audio segment from the running logs of the plurality of audio subsystems in the audio processing link.

[0011] In the above embodiments, the abnormal audio segment and the multi-subsystem log can be combined to provide comprehensive diagnostic data, enhancing the accuracy and completeness of the abnormal analysis.

[0012] In some embodiments, the abnormal log includes at least one of: a connection state of a hardware module related to audio output, a fault diagnosis state of a hardware module related to audio output, node original audio data of a plurality of nodes on an audio processing link, and a running log of a plurality of audio subsystems in the audio processing link.

[0013] In the above embodiments, key log information such as hardware state and node data can be covered to provide sufficient basis for multi-dimensional fault location and support fine-grained analysis.

[0014] In some embodiments, the abnormal audio segment and the abnormal log are analyzed to determine an abnormal recovery strategy, including: determining an abnormal type of the abnormal audio segment; determining a hardware fault according to the connection state and the fault diagnosis state of the hardware module related to the audio output in the abnormal log; and matching the abnormal recovery strategy from a recovery strategy library according to at least one of the abnormal type and the hardware fault.

[0015] In the above embodiments, the recovery strategy can be dynamically matched based on the abnormal type and the hardware fault to achieve accurate and efficient abnormal processing and improve the system self-healing capability.

[0016] In some embodiments, determining the abnormal type of the abnormal audio segment includes: extracting an audio feature of the abnormal audio segment, matching the audio feature with a preset abnormal feature of an abnormal type audio, and determining the abnormal type according to a matching result; and / or inputting the abnormal audio segment into an audio abnormal classification model to output the abnormal type by the audio abnormal classification model; wherein the audio abnormal classification model is trained based on historical abnormal audio data.

[0017] In the above embodiments, the abnormal type can be classified in multiple ways through feature matching and AI model, improving the accuracy of abnormal type identification and adapting to complex audio scenarios.

[0018] In some embodiments, the audio processing method further includes: sending the abnormal audio segment and the abnormal log to the cloud.

[0019] In the above-mentioned embodiments, remote analysis of abnormal data can be achieved through cloud uploading, providing data support for subsequent optimization.

[0020] In some embodiments, the audio processing method further comprises: receiving a cloud-side abnormal recovery strategy generated by the cloud side based on the abnormal audio segment and the abnormal log; and performing abnormal recovery according to the abnormal recovery strategy, including: performing abnormal recovery according to the abnormal recovery strategy and the cloud-side abnormal recovery strategy.

[0021] In the above-mentioned embodiments, joint decision can be made in combination with local and cloud-side strategies, improving the global optimization capability and adaptability of the recovery strategy.

[0022] In some embodiments, the audio processing method further comprises: receiving an updated audio abnormality classification model and / or an audio abnormality analysis strategy sent by the cloud side; updating the local audio abnormality classification model according to the updated audio abnormality classification model, and / or updating the local recovery strategy library according to the audio abnormality analysis strategy.

[0023] In the above-mentioned embodiments, the local model and strategy can be updated through the cloud side, realizing continuous evolution of the system and improving the timeliness and accuracy of abnormality processing.

[0024] In some embodiments, the method is applied to a vehicle-mounted audio system; wherein the abnormality identification of the played audio data comprises: deploying a loudness detection algorithm at the input end and the output end of a digital signal processor in the vehicle-mounted audio system to perform abnormality identification; wherein the identified abnormality type includes at least one of the following: silence, abnormal sound, stuttering, and popping sound abnormality.

[0025] In the above-mentioned embodiments, the detection algorithm can be customized for the vehicle-mounted scenario, effectively identifying typical audio faults and ensuring the stability and safety of the vehicle-mounted audio system.

[0026] In some embodiments, the abnormal audio segment and the abnormal log are sent to the cloud side by using a cockpit monitoring platform.

[0027] In the above-mentioned embodiments, the cockpit platform can be used to realize efficient uploading of vehicle-mounted data, strengthen the cloud-side collaborative analysis capability, and improve the vehicle system diagnosis efficiency.

[0028] According to a second aspect of the embodiments of the present disclosure, a method for audio exception processing is provided, including: building an initial audio exception analysis model according to an audio exception analysis strategy; obtaining historical audio exception training data, training the initial audio exception analysis model according to the historical audio exception training data to obtain a trained audio exception analysis model; in response to receiving an abnormal audio segment and an exception log sent by a terminal, inputting the abnormal audio segment and the exception log into the trained audio exception analysis model to obtain a target exception root cause, and updating the audio exception analysis strategy according to the target exception root cause.

[0029] The technical solutions provided by the embodiments of the present disclosure can include the following beneficial effects:

[0030] The present disclosure can construct a cloud intelligent analysis system with self-evolution ability, establish an initial model through historical training data and continuously optimize the model. When receiving abnormal data reported by a terminal, the model can accurately analyze the exception root cause and dynamically update the analysis strategy accordingly, forming a virtuous cycle of data-driven optimization and significantly improving the accuracy and intelligence level of audio exception diagnosis.

[0031] In some embodiments, the method for audio exception processing further includes: analyzing the abnormal audio segment and the exception log according to the updated audio exception analysis strategy to determine a cloud exception recovery strategy; and sending the cloud exception recovery strategy to the terminal.

[0032] In the above embodiments, a complete cloud processing link from exception analysis to recovery strategy generation is achieved. After determining the root cause through model analysis, a targeted recovery strategy is automatically generated and sent to the terminal, forming a cloud decision support system and enhancing the exception handling capability and recovery effect of the terminal device.

[0033] In some embodiments, the method for audio exception processing further includes: updating the audio exception analysis strategy again according to the cloud exception recovery strategy; and in response to satisfying an update condition, sending the audio exception analysis strategy updated again to the terminal.

[0034] In the above embodiments, a strategy continuous optimization mechanism is constructed, and the analysis strategy is updated using the cloud recovery strategy. Through condition-triggered updating, the terminal always obtains the latest analysis strategy, realizing the collaborative evolution of cloud and terminal capabilities and continuously improving the overall intelligence level of the system.

[0035] In some implementations, an initial audio anomaly analysis model is built based on an audio anomaly analysis strategy, including: selecting abnormal features, root causes, and recovery actions related to in-vehicle audio from the audio anomaly analysis strategy based on the in-vehicle audio anomaly scenario; and building an in-vehicle audio anomaly analysis model based on the selected in-vehicle audio-related data.

[0036] In the above implementation, a customized analysis model can be built for in-vehicle scenarios, and the model's professionalism can be ensured by screening relevant features and root cause data. This method improves the accuracy of in-vehicle audio anomaly recognition and enhances the model's practicality and reliability in specific application scenarios.

[0037] In some implementations, the target anomaly root cause includes an anomaly severity level, and the method for audio anomaly processing further includes: triggering an alarm notification of the corresponding level based on the anomaly severity level.

[0038] In the above implementation, graded alarms can be triggered according to the severity level to achieve differentiated fault notification, optimize the operation and maintenance response process, and ensure that critical faults are handled with priority.

[0039] According to a third aspect of the present disclosure, an audio processing apparatus is provided, comprising: an identification unit for identifying anomalies in audio data during audio playback; an acquisition unit for acquiring an abnormal audio segment and an abnormal log in response to the identification of an audio anomaly event; an analysis unit for analyzing the abnormal audio segment and the abnormal log to determine an anomaly recovery strategy; and a recovery unit for performing anomaly recovery according to the anomaly recovery strategy.

[0040] According to a fourth aspect of the present disclosure, an apparatus for audio anomaly processing is provided, comprising: a building unit for building an initial audio anomaly analysis model according to an audio anomaly analysis strategy; a training unit for acquiring historical audio anomaly training data, training the initial audio anomaly analysis model according to the historical audio anomaly training data, and obtaining a trained audio anomaly analysis model; and a processing unit for responding to receiving an abnormal audio segment and anomaly log sent by a terminal, inputting the abnormal audio segment and anomaly log into the trained audio anomaly analysis model to obtain a target anomaly root cause, and updating the audio anomaly analysis strategy according to the target anomaly root cause.

[0041] According to a fifth aspect of the present disclosure, a vehicle is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described audio processing method or a method for audio anomaly handling.

[0042] According to a sixth aspect of the present disclosure, an electronic device is provided, characterized in that it includes: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described audio processing method or a method for audio anomaly processing.

[0043] According to a seventh aspect of the present disclosure, a computer-readable storage medium is provided, wherein when instructions in the storage medium are executed by a processor of a mobile terminal, the mobile terminal is enabled to perform the above-described audio processing method or the method for handling audio anomalies.

[0044] According to an eighth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described audio processing method or the method for handling audio anomalies.

[0045] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0046] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0047] Figure 1 This is a flowchart illustrating an audio processing method according to some embodiments of the present disclosure.

[0048] Figure 2 This is a flowchart illustrating an anomaly identification in an audio processing method according to some embodiments of the present disclosure.

[0049] Figure 3 This is a flowchart illustrating yet another audio processing method according to some embodiments of the present disclosure.

[0050] Figure 4 This is a flowchart illustrating an anomaly recovery process in an audio processing method according to some embodiments of the present disclosure.

[0051] Figure 5 This is a flowchart illustrating an abnormal update in an audio processing method according to some embodiments of the present disclosure.

[0052] Figure 6 This is a schematic diagram illustrating fault analysis using the cloud in an audio processing method according to some embodiments of the present disclosure.

[0053] Figure 7 This is a schematic diagram of a recovery strategy library in an audio processing method according to some embodiments of the present disclosure.

[0054] Figure 8 This is a flowchart illustrating a method for audio anomaly handling according to some embodiments of the present disclosure.

[0055] Figure 9 This is a block diagram illustrating an audio processing apparatus according to some embodiments of the present disclosure.

[0056] Figure 10 This is a block diagram illustrating an apparatus for audio anomaly processing according to some embodiments of the present disclosure.

[0057] Figure 11 This is a block diagram illustrating an apparatus for audio processing according to some embodiments of the present disclosure. Detailed Implementation

[0058] Exemplary embodiments of this disclosure will be described in detail herein, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. Various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but can be changed as will become apparent upon understanding this disclosure, except for operations that must be performed in a particular order. Furthermore, for clarity and brevity, descriptions of features known in the art may be omitted.

[0059] The flowchart shown in the attached diagram is merely an illustrative example and does not necessarily include all content and steps, nor does it necessarily have to be executed in the described order or in the order of the step numbers. For example, some steps can be broken down, while others can be combined or partially combined, and multiple steps can have their order interchanged or be executed simultaneously. Therefore, the actual execution order may change depending on the actual situation.

[0060] The embodiments described below, which are examples of some of the embodiments of this disclosure, do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0061] In some embodiments of this disclosure, the acquisition of data and information, as well as the collection, updating, analysis, processing, use, transmission, and storage of related user personal information, may comply with the laws and regulations of the country where the location is situated.

[0062] In some embodiments of this disclosure, data, information, etc., may be obtained after obtaining the user's consent.

[0063] The specific implementation methods of the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0064] Figure 1 This is a flowchart illustrating an audio processing method according to some embodiments of the present disclosure, such as... Figure 1 As shown, the audio processing method can be applied to electronic devices, including but not limited to in-vehicle terminals, smartphones, smart tablets, wearable devices, desktop computers, laptops, smart speakers, and other terminal devices. It can also include server-side components such as local servers and cloud servers, which can be deployed in a computer cluster or a combination of multiple computers. The audio processing method may include the following steps.

[0065] In step S110, during audio playback, anomaly detection is performed on the played audio data.

[0066] In this embodiment of the disclosure, during the normal audio playback process, continuous detection of the playing audio data can be performed to determine if any abnormalities exist. This is the foundation and starting step of the entire audio processing method, enabling timely detection of problems that occur during audio playback.

[0067] Anomaly detection in the played audio data can involve identifying anomalies in the audio stream data. This can be triggered automatically when an audio playback service is initiated. Audio playback services may include, for example, playing music, navigation, alarm sounds, and online meetings.

[0068] The audio stream data can be a digital representation of a continuously playing audio signal, or it can include multiple frames of continuous audio data packets of a preset duration, containing information such as volume and frequency. The audio stream data can be, for example, raw audio data for output, such as PCM (Pulse Code Modulation) data, or the signal strength of the raw audio data, such as loudness. Audio stream data of a preset duration can be continuously collected for anomaly detection.

[0069] In an exemplary embodiment, this method can be applied to an in-vehicle cockpit system, and can automatically trigger the collection of audio stream data in response to the activation of audio playback services in the in-vehicle cockpit system.

[0070] In an exemplary embodiment, the duration of the audio data streams collected for anomaly identification under different audio playback services can be different. For example, the preset duration for playing alarm sounds can be shorter (e.g., 0.5 seconds, 1 second, etc.) so that it can quickly identify whether the alarm sound is played normally, thus ensuring the normal playback of the alarm content; the preset duration for playing music can be slightly longer (e.g., 2 seconds, 3 seconds, etc.) to ensure the user's music experience; the preset duration for online meetings can be longer (e.g., 5 seconds, 6 seconds, etc.) to adapt to the normal situation of short periods of no one speaking in online meetings.

[0071] In step S120, in response to the detection of an audio anomaly event, abnormal audio segments and anomaly logs are obtained.

[0072] In this embodiment of the disclosure, when an audio anomaly is detected, immediate action can be taken to collect the abnormal audio data segment and obtain log information related to the anomaly. This data is crucial for subsequent problem analysis and solution development.

[0073] Audio anomalies can be situations that occur during audio playback that deviate from the normal playback state, such as sound stuttering, distortion, interruption, or noise. These anomalies can affect the user's normal reception and experience of audio content.

[0074] An abnormal audio segment can be the portion of audio data that becomes abnormal during audio playback. An exception log can be a file or record that documents relevant system information, operation logs, error codes, and other data related to when an exception occurs during audio playback. By analyzing the exception log, one can understand the system environment and operational steps at the time of the exception, which helps to accurately determine the cause of the exception.

[0075] The anomaly log can include the connection status of hardware modules, the fault diagnosis status of hardware modules, and the operation logs of multiple audio subsystems in the audio processing chain. The connection status of hardware modules can be real-time information on whether the physical or logical connections of audio output hardware (such as speakers, handsets, Bluetooth headsets, and USB sound cards) are functioning correctly, such as "Bluetooth device A2DP connection lost" or "Wired headset unplugged." Anomalies in the connection status often directly lead to audio problems, and the connection status of hardware modules can serve as direct clues to hardware faults for analysis.

[0076] The fault diagnosis status of a hardware module can be a health status and error code provided by the hardware driver or the hardware itself, such as "power amplifier chip overheating" or "audio DSP reports communication timeout". The fault diagnosis status can indicate whether the hardware is about to fail or what specific internal error has occurred.

[0077] The raw audio data from multiple nodes in the audio processing chain can be the raw audio data extracted from intermediate nodes (such as encoding nodes, effects processing nodes, mixing nodes, decoding nodes, and output nodes) before the final output. By comparing the data from different nodes, it is possible to pinpoint which processing stage the problem occurs in, thus enabling fault localization. For example, if the audio output from the "encoding node" is normal, but the audio output from the subsequent "effects processing node" contains noise, then it can be inferred that the problem lies in the effects processor.

[0078] The runtime logs of multiple audio subsystems in the audio processing chain can record detailed text information generated by each audio-related software component (subsystem) during runtime. This information may include system startup and shutdown records, error messages, parameter settings, and user commands. The runtime logs provide the software-level execution context and can be used to identify software problems that may cause audio anomalies.

[0079] In this embodiment of the disclosure, identifying audio anomaly events can be achieved by using existing sample anomaly data as a reference to analyze and judge the collected audio stream data to determine whether any audio anomalies exist. For example, machine learning algorithms or signal processing algorithms can be used to analyze and compare the sample anomaly data and the audio stream data.

[0080] Among them, audio abnormalities can refer to unexpected and undesirable states that occur during audio playback, such as silence, popping sounds (such as harsh or distorted sounds), noise (such as background noise that is not part of the content itself), audio stuttering (such as intermittent or discontinuous sound), and abnormal volume (such as the sound suddenly becoming too loud or too soft).

[0081] In this embodiment of the disclosure, the sample abnormal data may be a pre-built audio abnormality database or model, which may contain audio data or features of various audio abnormalities.

[0082] In an exemplary embodiment, the audio anomaly database may include various abnormal audio segments, such as silent segments (e.g., a completely silent or very quiet audio segment used to detect audio interruptions) and stuttering sample segments (e.g., discontinuous or intermittent audio caused by lost audio packets). The real-time acquired "audio stream data" can be compared with these abnormal audio segments to determine the type of anomaly the current audio belongs to, or whether it is normal.

[0083] In step S130, the abnormal audio segments and abnormal logs are analyzed to determine an abnormal recovery strategy.

[0084] In this embodiment of the disclosure, in-depth analysis can be performed based on the collected abnormal audio segments and abnormal logs to find the cause of the audio abnormality, and then determine a targeted recovery strategy that can effectively solve the abnormality problem.

[0085] Anomaly recovery strategies can be a series of measures and methods developed to resolve abnormal situations that occur during audio playback and restore normal audio playback. Anomaly recovery strategies can vary depending on factors such as the type and cause of the anomaly.

[0086] In step S140, anomaly recovery is performed according to the anomaly recovery strategy.

[0087] In this embodiment of the disclosure, specific measures can be taken to repair audio playback according to the determined anomaly recovery strategy, so that audio playback can return to normal. This is the ultimate goal of the entire audio processing method, ensuring that users can obtain a normal and smooth audio experience.

[0088] In an exemplary embodiment, upon detecting an audio anomaly, a pre-defined anomaly recovery strategy can be triggered and executed to resolve the audio anomaly, attempt to automatically fix the problem, restore normal audio playback, and avoid or reduce the impact on user experience.

[0089] Among them, the anomaly recovery strategy can be a pre-designed automated response plan, which is executed when a specific anomaly is detected in order to attempt to restore normal audio playback.

[0090] As can be seen from the above steps, the audio processing method provided in this disclosure can proactively identify anomalies in the audio data during playback in real time. Furthermore, after confirming an anomaly, it can analyze automatically generated abnormal audio segments and anomaly logs, triggering anomaly recovery strategies, thus ensuring the continuity and stability of audio playback and significantly improving system reliability and user experience.

[0091] In an exemplary embodiment, the abnormal audio stream data can also be processed to obtain new sample abnormal audio, thereby enriching and updating the audio anomaly database.

[0092] Figure 2 This is a flowchart illustrating an anomaly detection method in an audio processing method according to some embodiments of the present disclosure, such as... Figure 2 As shown, in some embodiments of this disclosure, anomaly identification of the played audio data during audio playback may include the following steps.

[0093] Step S210: During audio playback, audio segments are captured in segments.

[0094] In this embodiment of the disclosure, a continuous audio stream can be divided into multiple relatively independent segments (e.g., one segment every 3 seconds) according to certain rules for acquisition. This breaks down the overall audio data into smaller, more easily processed parts, facilitating detailed analysis of each segment and enabling more accurate location of any abnormal parts in the audio, avoiding the overlooking of subtle local anomalies due to overall analysis.

[0095] Step S220: Compare each audio segment with the sample abnormal data, and determine whether an audio abnormal event has occurred based on the comparison results.

[0096] In this embodiment, after an audio segment is acquired, it can be compared one by one with pre-defined sample anomaly data. The sample anomaly data can be collected, organized, and classified, covering characteristic data of various known audio anomalies. Through comparison, if a high similarity is found between the current audio segment and the sample anomaly data, it can be determined that the comparison result meets the preset anomaly judgment conditions, thus confirming that an audio anomaly event has occurred; conversely, if the comparison result does not meet the anomaly conditions, the audio segment is considered normal. This comparison method provides an objective and quantifiable basis for judging whether audio is abnormal.

[0097] Through the embodiments of this disclosure, accurate abnormal event detection can be achieved through segmented collection and sample comparison, reducing the false negative rate and improving the reliability of identification.

[0098] In some embodiments of this disclosure, in response to the identification of an audio anomaly event, acquiring an abnormal audio segment and an anomaly log includes: identifying the audio segment identified as having caused an audio anomaly event in the comparison as the abnormal audio segment; and extracting the anomaly log corresponding to the abnormal audio segment from the operation logs of multiple audio subsystems in the audio processing link.

[0099] In this embodiment of the disclosure, after comparing each audio segment with the sample abnormal data and determining that an audio abnormality event has occurred, the specific audio segment exhibiting the abnormality can be accurately located, extracted separately, and marked as an abnormal audio segment. This step is the foundation for subsequent anomaly analysis. By clarifying the specific location and content of the anomaly, key clues can be provided for further troubleshooting, which helps to deeply analyze the cause of the anomaly.

[0100] In this embodiment of the disclosure, the audio processing link may include multiple audio subsystems, such as an audio acquisition subsystem, an encoding / decoding subsystem, a transmission subsystem, and a playback subsystem, and each subsystem can generate corresponding operation logs during operation.

[0101] Once the abnormal audio segment is identified, log information related to that segment can be filtered from the operational logs of these numerous subsystems; this is known as the exception log. These exception logs record the operational status, parameter settings, and error messages of each subsystem at the time of the exception. They are crucial for a comprehensive understanding of the background and cause of the exception and provide rich data support for developing subsequent exception recovery strategies.

[0102] Through the embodiments of this disclosure, comprehensive diagnostic data can be provided by combining abnormal audio clips and multi-subsystem logs, thereby enhancing the accuracy and completeness of anomaly analysis.

[0103] In some implementations, the anomaly log includes at least one of the following: the connection status of the hardware module related to audio output, the fault diagnosis status of the hardware module related to audio output, the node raw audio data of multiple nodes in the audio processing link, and the operation logs of multiple audio subsystems in the audio processing link.

[0104] In this embodiment of the disclosure, when an audio abnormality event is detected and preparation is made to obtain the abnormality log, log information can be collected from multiple dimensions provided in this embodiment of the disclosure.

[0105] The embodiments disclosed herein can cover key log information such as hardware status and node data, providing sufficient basis for multi-dimensional fault location and supporting refined analysis.

[0106] In some implementations, analyzing the abnormal audio segments and abnormal logs to determine an abnormality recovery strategy includes: determining the abnormal type of the abnormal audio segments; determining a hardware fault based on the connection status and fault diagnosis status of the hardware modules related to audio output in the abnormal logs; and matching the abnormality recovery strategy from a recovery strategy library based on at least one of the abnormal type and the hardware fault.

[0107] In this embodiment, detailed feature extraction and analysis of abnormal audio segments can be performed, such as checking parameters like frequency, amplitude, and waveform, and comparing them with the characteristics of normal audio. This allows for the determination of the specific type of abnormality, such as noise interference, sound distortion, or audio interruption. Identifying the type of abnormality helps in more accurately locating the root cause of the problem and developing a recovery strategy.

[0108] The error log can contain key information about the hardware modules related to audio output. By comprehensively analyzing this information, it is possible to accurately determine whether a hardware fault exists and its specific location and nature, such as whether it is a speaker failure, an audio interface malfunction, or a problem with the audio processing chip.

[0109] Once the anomaly type and / or hardware failure is determined, a pre-built recovery strategy library can be used for matching. This library stores solutions and operational steps for various anomaly types and hardware failures. By comparing the currently identified anomaly type and hardware failure with entries in the library, the best-matching recovery strategy can be found. This matching method ensures the targeted nature and effectiveness of the recovery strategy, enabling quick and accurate resolution of audio anomaly problems.

[0110] Through the embodiments of this disclosure, recovery strategies can be dynamically matched based on the anomaly type and hardware fault, thereby achieving accurate and efficient anomaly handling and improving the system's self-healing capability.

[0111] The anomaly recovery strategy includes at least one of the following: restarting the audio subsystem, adjusting audio playback parameters, and adjusting the cache capacity used for audio playback.

[0112] In this embodiment of the disclosure, restarting the audio subsystem can be a process of re-initializing audio-related software or hardware settings, which can resolve audio problems caused by abnormal system operation.

[0113] Adjusting audio playback parameters, such as volume, pitch, and sampling rate, can optimize audio playback and adapt to different hardware conditions or abnormal situations. For example, reducing the data processing load by lowering the quality (such as sampling rate and bit rate) can resolve stuttering; enabling software filters or adjusting the gain (Volume) can suppress noise at specific frequencies, thus resolving noise issues; enabling "fade in / fade out" mode can resolve popping sounds; and increasing the output volume can resolve silent audio issues.

[0114] Adjusting the buffer size used for audio playback—either increasing or decreasing the size of the software buffer used to store the audio data to be played—can improve audio data buffering and transmission, avoiding audio stuttering and latency issues caused by insufficient or excessive buffering. For example, when there are network fluctuations or high system load, increasing the buffer size can store more data, counteracting the instability of data arrival and sacrificing latency for smoother playback, thus resolving stuttering or jitter issues.

[0115] In an exemplary embodiment, the type of the audio data stream in which the audio anomaly occurs can be determined, and an anomaly recovery strategy associated with the type of anomaly can be executed.

[0116] In this embodiment of the disclosure, after detecting an audio anomaly, the anomaly type can be further analyzed and identified. Then, an anomaly recovery strategy for that type of anomaly can be executed to attempt to eliminate the cause of the anomaly. The audio data stream anomaly type can include one or more types.

[0117] The anomaly type can be the result of classifying audio anomalies. Different anomaly types can have different characteristics and manifestations, such as silence, noise, stuttering, sudden volume changes, and popping sounds. One or more corresponding recovery strategies can be pre-defined for each anomaly type. Once the anomaly type is determined, the associated strategy can be directly called and executed to specifically resolve the current audio anomaly problem.

[0118] In an exemplary embodiment, if silence is detected, the anomaly type can be determined to be a silent anomaly. The corresponding anomaly recovery strategy can be configured as follows: increase the audio playback volume, automatically restart the audio playback service, or switch to a backup audio output device.

[0119] If stuttering is detected and the anomaly type is determined to be stuttering, the corresponding anomaly recovery strategy can be configured as follows: restart the audio subsystem, or automatically clear the current audio buffer and try to start playing from the latest data packet, sacrificing some latency for smoothness.

[0120] If noise or jarring is detected, and the anomaly type can be determined to be a jarring anomaly, the corresponding anomaly recovery strategy can be configured as follows: increase the size of the buffer, or start a software filter to try to suppress noise at a specific frequency, or fine-tune the output gain.

[0121] In an exemplary embodiment, the hardware module state related to the output of the audio stream data can be obtained, and an anomaly recovery strategy associated with the hardware module state can be executed; the hardware module state includes at least one of connection state and fault diagnosis state.

[0122] In this embodiment, audio playback relies on the collaborative operation of multiple hardware modules. The status of these hardware modules related to audio output (such as whether they are properly connected or faulty) can be checked to attempt to identify the cause of the audio anomaly at the hardware level. Recovery strategies associated with the abnormal hardware module status can be executed, and different hardware states can correspond to different processing methods to attempt to restore normal audio playback. The abnormal hardware module statuses may include one or more, or none at all.

[0123] The hardware module status can refer to the current working status information of physical or logical hardware components related to audio playback, which may include the connection status of the hardware module and / or the fault diagnosis status of the hardware module.

[0124] Connection status can refer to whether the physical or logical connection of hardware devices is normal. Connection status can include, for example, the plugging and unplugging status of wired headphones, the connection status of Bluetooth headphones, the connection status of HDMI cables, and the recognition status of USB audio devices.

[0125] Fault diagnostic status can refer to error codes or health status reported internally by a hardware module, and can be provided by the driver or the hardware itself. Fault diagnostic status can include, for example: no fault, overflow error reported by the audio encoder (DSP), clock out-of-synchronization reported by the digital-to-analog converter (DAC), over-temperature protection reported by the power amplifier, power supply abnormality reported by the power amplifier, etc.

[0126] In an exemplary embodiment, the anomaly type of the audio data stream and the state of the hardware module associated with outputting the audio stream data can be considered simultaneously to determine the anomaly recovery strategy to be executed. The execution priorities of the two can be pre-configured; for example, the anomaly recovery strategy associated with the anomaly type can be executed first, and if the anomaly is not resolved, the anomaly recovery strategy associated with the hardware module state can be executed. Alternatively, different execution priorities can be configured for different anomaly types, or different execution priorities can be configured for different states corresponding to the hardware module states (including no fault and different fault types).

[0127] For example, if the hardware module status report shows no faults, the exception recovery strategy associated with the exception type can be executed directly; if the hardware module status report includes a hardware driver freeze, the strategy of restarting the audio subsystem can be executed.

[0128] Through the embodiments disclosed herein, recovery strategies can be executed in a targeted manner according to the type of anomaly or the state of the hardware module, thereby achieving accurate fault recovery, avoiding the limitations of a single strategy, improving recovery efficiency, and reducing interference with normal playback.

[0129] In some embodiments of this disclosure, the sample abnormal data includes: abnormal features of sample abnormal audio corresponding to the business scenario of the audio playback; the sample abnormal audio includes at least one of the following: silent audio, noise audio, stuttering audio, and plosive audio.

[0130] In this embodiment of the disclosure, the sample abnormal data can correspond to the business scenario of audio playback. The sample abnormal data may be different in different business scenarios. Sample abnormal audio can be collected and organized for different business scenarios to improve the accuracy and pertinence of abnormal identification.

[0131] Among them, silent audio can be used to detect audio interruptions or mute faults. Noise audio can include various white noises, electrical noises, environmental noises, etc., and can be used to detect sound quality pollution. Stuttering audio can simulate audio interruptions caused by network latency or insufficient processing power. Popping audio can include sudden high-energy noises such as "crackling" and "poof" sounds, and can be used to detect poor interface contact or signal abrupt changes.

[0132] For example, in online meeting scenarios, we can focus on anomalies such as stuttering and interruptions, while tolerating slight noise and short periods of silence. Therefore, the sample abnormal audio in online meeting scenarios can include stuttering audio, interrupted audio, and long periods of silence, but not short periods of silence. Alternatively, it can exclude noisy audio, or exclude noisy audio with less noise and only include noisy audio with more noise.

[0133] For example, in a music playback scenario, we can focus on anomalies such as noise, pops, and stuttering. Therefore, the sample abnormal audio in a music playback scenario can include noise audio, pops audio, and stuttering audio.

[0134] For example, in alarm sound playback scenarios, we can focus on silent anomalies and anomalies with low volume. Therefore, the sample abnormal audio corresponding to alarm sound playback scenarios can include silent audio, low volume audio, etc.

[0135] In an exemplary embodiment, the anomalous features of the sample anomalous audio can be obtained by processing the PCM data of the sample anomalous audio or by processing the signal strength of the sample anomalous audio. For example, the anomalous features can be obtained by performing a time-domain to frequency-domain transformation on the PCM data of the sample anomalous audio.

[0136] In some implementations, determining the anomaly type of the abnormal audio segment includes: extracting audio features of the abnormal audio segment, matching the audio features with the anomaly features of a preset anomaly type audio, and determining the anomaly type based on the matching result; and / or, inputting the abnormal audio segment into an audio anomaly classification model, and having the audio anomaly classification model output the anomaly type; wherein the audio anomaly classification model is trained based on historical abnormal audio data.

[0137] In this embodiment of the disclosure, audio features of the audio stream data can be determined, and abnormal features of sample abnormal audio corresponding to the business scenario of the audio playback can be obtained. Then, the similarity between the audio features and the abnormal features can be determined. If the similarity exceeds a preset threshold, an audio anomaly is determined to have occurred.

[0138] In this embodiment of the disclosure, audio features can be parameters that reflect the essential attributes of audio, such as frequency, amplitude, timbre, etc. By analyzing these features, the state of the audio can be understood.

[0139] In this embodiment of the disclosure, the audio features of the extracted audio stream data can be compared with the abnormal features of a pre-determined sample abnormal audio, and the similarity between them can be calculated. The similarity can measure the degree of similarity between the two, and it can be used to determine whether the audio stream data and the sample abnormal audio have similar features.

[0140] In an exemplary embodiment, specific algorithms can be used for calculation, such as cosine similarity, Euclidean distance, or reasoning based on machine learning models. The similarity value can range from 0 to 1, with a value closer to 1 indicating greater similarity between the two features.

[0141] In this embodiment of the disclosure, the preset threshold can be a standard for determining whether an abnormality has occurred. The calculated similarity can be compared with the preset threshold, such as 80% or 85%. If the similarity exceeds the threshold, it means that the current audio is close to some known abnormality, and thus it can be determined that an audio abnormality has occurred; otherwise, the audio playback can be considered normal.

[0142] In an exemplary embodiment, different preset thresholds can be set for different business scenarios based on their importance or characteristics to balance false positives and false negatives. For example, a high threshold results in strict judgment, making false positives less likely but false negatives more likely; a low threshold results in sensitive judgment, making false positives less likely but false positives more likely.

[0143] Through the embodiments of this disclosure, audio features can be extracted and compared with pre-stored abnormal features to achieve accurate identification of multiple types of audio anomalies, reduce the false judgment rate, and provide a reliable basis for subsequent recovery strategies.

[0144] In this embodiment, the abnormal audio segment to be analyzed can also be used as input and provided to a pre-trained audio anomaly classification model. The model can receive audio data and process and analyze it. The audio anomaly classification model can use its internally learned knowledge and patterns to classify the input abnormal audio segment and output the corresponding anomaly type. This model can be trained based on historical abnormal audio data, and through learning from a large number of samples, it can automatically identify and classify different types of audio anomalies.

[0145] Through the embodiments disclosed herein, various methods can be used for classification through feature matching and AI models, thereby improving the accuracy of anomaly type identification and adapting to complex audio scenarios.

[0146] In some embodiments of this disclosure, the audio stream data includes at least one of the following: raw audio data and signal strength; the audio features include at least one of the following: spectral features obtained by time-domain to frequency-domain transformation based on the raw audio data, and time-domain energy features calculated based on the signal strength.

[0147] In this embodiment of the disclosure, the raw audio data can be raw waveform samples acquired by a microphone or internal loop, unprocessed or only subject to basic preprocessing. It can record the amplitude of sound at a given time point, and the raw audio data can be PCM data. The signal strength can be a preliminary calculated summary index from the raw audio data, used to intuitively and quickly describe the overall volume or energy of an audio segment. For example, it can be RMS (Root Mean Square) data or dBFS (Decibels Relative to Full Scale) data; where RMS data measures the average signal power, and dBFS data is a unit of volume representation in digital audio.

[0148] Among these methods, mathematical tools such as Fourier Transform (FFT) can be used to transform the raw audio data from the time domain (amplitude changing with time) to the frequency domain (energy distributed with frequency) to obtain spectral characteristics, allowing us to focus on the intrinsic components of the audio stream data. Alternatively, we can directly analyze and calculate the signal intensity (amplitude) in the time domain to obtain time-domain energy characteristics, allowing us to focus on the extrinsic intensity of the audio stream data.

[0149] In an exemplary embodiment, temporal energy features can quickly process obvious, macroscopic anomalies such as silence, excessively low volume, and excessively high volume (e.g., pops). Spectral features can be used to obtain deeper components of sound, thereby processing anomalies such as background noise and specific noise interference.

[0150] This disclosure illustrates the content of audio stream data and audio features, which can extract features from multiple aspects, helping to more comprehensively and accurately identify audio anomalies and improve the accuracy and reliability of anomaly detection.

[0151] In some embodiments of this disclosure, the abnormal feature is associated with an abnormal type; the audio processing method further includes: determining a target abnormal feature corresponding to a similarity exceeding a preset threshold; and determining the abnormal type of the audio data stream in which the audio abnormality occurs based on the target type of the target abnormal feature.

[0152] In this embodiment, a correspondence between abnormal features and abnormal types can be established in advance, laying the foundation for quickly locating the abnormal type based on the identified abnormal features. This association ensures that each abnormal feature has a clear abnormal type affiliation, facilitating the system's classification and processing of audio abnormalities.

[0153] In anomaly identification, if the similarity between real-time audio features and one or more sample anomaly features exceeds a preset threshold, the sample anomaly features exceeding the preset threshold can be identified as target anomaly features. Then, based on the target type of the identified target anomaly features (i.e., the anomaly type associated with the target anomaly feature), the specific anomaly type of the audio data stream where the audio anomaly occurred can be determined, thereby quickly realizing the transformation from specific anomaly features to abstract anomaly types, providing a basis for subsequent targeted anomaly recovery strategies.

[0154] Through the embodiments of this disclosure, abnormal features can be associated with abnormal types. While identifying abnormalities, the audio abnormality type can be determined based on the abnormality type of the matched target abnormal features. This can accurately locate audio abnormality problems, provide a type basis for subsequent targeted recovery strategies, and improve processing efficiency.

[0155] Figure 3 This is a flowchart illustrating yet another audio processing method according to some embodiments of the present disclosure.

[0156] In this embodiment of the disclosure, Figure 3 The audio processing method shown includes any one or a combination of any of steps S310, S320, S330, S340, and S350, wherein steps S310, S320, S340, and S350 are respectively related to... Figure 1 The steps S110, S120, S130, and S140 in the audio processing method shown correspond to each other and will not be repeated here.

[0157] In this embodiment of the disclosure, Figure 1 Based on the audio processing method shown, Figure 3 The audio processing method shown may also include the following steps.

[0158] Step S330: Send the abnormal audio clip and abnormal log to the cloud.

[0159] In this context, the cloud can be a remote server cluster built on cloud computing technology, providing various computing resources, storage resources, and software services over a network. The cloud possesses powerful computing and storage capabilities, enabling it to handle large-scale data and complex computational tasks. After receiving data from the device, the cloud server can utilize its powerful computing resources for fault analysis and processing.

[0160] The cloud server can use pre-configured algorithms and / or models to analyze multi-dimensional field data (i.e., abnormal audio clips and abnormal logs). These algorithms and / or models may include, for example, machine learning algorithms (such as decision trees, neural networks, etc.), statistical analysis methods (such as regression analysis, time series analysis, etc.), and rule engines based on expert experience. Through in-depth analysis of multi-dimensional field data, the cloud server can uncover the causes, locations, and severity of audio anomalies, and generate detailed fault analysis results to return to the device.

[0161] Through the embodiments disclosed herein, remote analysis of abnormal data can be achieved by uploading data to the cloud, providing data support for subsequent optimization.

[0162] Figure 4 This is a flowchart illustrating an anomaly recovery process in an audio processing method according to some embodiments of the present disclosure, such as... Figure 4 As shown, in some embodiments of this disclosure, abnormal recovery may include the following steps.

[0163] Step S410: Receive the cloud-based anomaly recovery strategy generated by the cloud based on the abnormal audio clip and the abnormal log.

[0164] In this embodiment, after the local device sends the abnormal audio clips and abnormal logs to the cloud, the cloud can utilize its powerful computing capabilities and abundant resources to analyze this data. After the analysis is complete, the cloud generates a targeted cloud-based anomaly recovery strategy and sends it back to the local device.

[0165] Step S420: Perform anomaly recovery according to the anomaly recovery strategy and the cloud-based anomaly recovery strategy.

[0166] In this embodiment, the local device possesses both its own generated anomaly recovery strategy and a cloud-based anomaly recovery strategy received from the cloud. During anomaly recovery, both strategies can be comprehensively analyzed and integrated.

[0167] For example, considering the different handling methods for hardware failures and software problems in the two strategies, and taking into account the actual situation and resources of the local equipment, determine the final recovery plan to be implemented.

[0168] Through the embodiments disclosed herein, joint decision-making can be carried out by combining local and cloud strategies, thereby improving the global optimization capability and adaptability of recovery strategies.

[0169] Figure 5 This is a flowchart illustrating an abnormal update in an audio processing method according to some embodiments of the present disclosure, such as... Figure 5 As shown, in some embodiments of this disclosure, abnormal recovery may include the following steps.

[0170] Step S510: Receive the updated audio anomaly classification model and / or audio anomaly analysis strategy sent from the cloud.

[0171] In this embodiment, a new model for classifying audio anomalies can be obtained in the cloud after retraining, optimization, or improvement. Compared with the previous model, this model can improve in terms of classification accuracy, ability to identify novel anomalies, and ability to process complex audio data.

[0172] Audio anomaly analysis strategies can be cloud-based strategies derived from the analysis of large amounts of data for analyzing and handling audio anomalies. These strategies can cover the entire process from anomaly detection, classification, root cause localization to recovery.

[0173] Step S520: Update the local audio anomaly classification model according to the updated audio anomaly classification model, and / or update the local recovery strategy library according to the audio anomaly analysis strategy.

[0174] In this embodiment of the disclosure, when an updated audio anomaly classification model is received, it can be replaced or merged with the existing local audio anomaly classification model.

[0175] If an audio anomaly analysis strategy is received, the local recovery strategy library can be updated accordingly. The audio anomaly analysis strategy can include more targeted handling methods for different anomaly types, optimized processing flows, and special strategies for specific scenarios. By updating the recovery strategy library, the local device can adopt more effective and comprehensive recovery measures when facing audio anomalies, improving the success rate and efficiency of anomaly recovery.

[0176] Through the embodiments of this disclosure, local models and strategies can be updated via the cloud, enabling continuous system evolution and improving the timeliness and accuracy of anomaly handling.

[0177] In some embodiments, the method is applied to an in-vehicle audio system; wherein, anomaly identification of the played audio data includes: deploying a loudness detection algorithm at the input and output terminals of the digital signal processor in the in-vehicle audio system for anomaly identification; wherein the identified anomaly type includes at least one of the following: no sound, abnormal noise, stuttering, or popping sound anomaly.

[0178] In this embodiment of the disclosure, the in-vehicle audio system can be a system installed inside a vehicle for playing audio content (such as music, radio, navigation voice, etc.). It may include audio source devices (such as CD players, USB interfaces, Bluetooth modules, etc.), digital signal processors, power amplifiers, speakers, and other components.

[0179] The digital signal processor (DSP) is the core processing component of the in-vehicle audio system, responsible for the digital processing, encoding and decoding, and sound effect adjustment of audio signals. Deploying a loudness detection algorithm at its input end allows for real-time monitoring of the loudness of the raw audio signal entering the DSP, promptly detecting any potential anomalies. Deploying it at the output end allows for the detection of the loudness of the audio signal output after DSP processing, determining whether any anomalies were introduced during processing or if there are any problems with the output signal itself.

[0180] Anomaly detection algorithms can be used to identify loudness. Loudness is a physical quantity that reflects the intensity of sound. Loudness detection algorithms can analyze the loudness characteristics of audio signals, such as the range and rate of change of loudness. When the loudness of an audio signal deviates from the normal range or pattern of change, it may be determined that an anomaly exists.

[0181] Through the embodiments disclosed herein, a customized detection algorithm can be developed for in-vehicle scenarios to effectively identify typical audio faults and ensure the stability and security of in-vehicle audio systems.

[0182] In some implementations, the abnormal audio clips and abnormal logs are sent to the cloud using a cockpit monitoring platform.

[0183] In this embodiment, the cockpit monitoring platform can be an integrated software and hardware system installed in the vehicle cockpit for real-time monitoring, data collection, and management of various systems within the vehicle cockpit (including audio systems, cockpit environment systems, vehicle control systems, etc.). The cockpit monitoring platform can act as a data transmission bridge between local (e.g., vehicle-side, in-vehicle audio system) and the cloud, securely and accurately sending abnormal audio clips and logs collected locally to the cloud.

[0184] In the above implementation, the cockpit platform can be used to achieve efficient uploading of vehicle data, enhance cloud-based collaborative analysis capabilities, and improve the diagnostic efficiency of the entire vehicle system.

[0185] Figure 6 This is a schematic diagram illustrating fault analysis using the cloud in an audio processing method according to some embodiments of the present disclosure.

[0186] like Figure 6 As shown, the architecture of a cockpit cloud monitoring platform 600 is illustrated. This platform can collect and process data from audio playback services in the cockpit system through a big data monitoring system 601. The big data monitoring system may include: a subsystem log acquisition module 602, a PCM (Pulse Code Modulation) data acquisition module 603, and a system recovery module 604.

[0187] The PCM data acquisition module 603 can acquire audio stream data in real time for real-time anomaly identification. After an audio anomaly is identified, the system recovery module 604 can be used to execute anomaly recovery strategies.

[0188] After identifying an audio anomaly, the subsystem log acquisition module 602 and PCM data acquisition module 603 can be used to obtain multi-dimensional on-site data (i.e., abnormal audio segments and abnormal logs) for fault analysis and obtain the fault analysis results of the audio anomaly.

[0189] The subsystem log acquisition module 602 can collect log information from subsystems such as the Android subsystem and the DSP (audio encoder) subsystem (these two subsystems are used as examples in this embodiment, but this disclosure is not limited to them). The PCM data acquisition module can capture the original audio data of multiple nodes in the audio processing link.

[0190] The data collected by the subsystem log acquisition module 602 and the PCM data acquisition module 603 can be uploaded to the cockpit cloud monitoring platform for centralized management and analysis, so as to realize the monitoring and management of audio playback services in the cockpit system.

[0191] In some embodiments of this disclosure, the fault analysis results include recovery suggestions for the audio anomaly; the audio processing method further includes updating the anomaly recovery strategy based on the recovery suggestions.

[0192] In this embodiment of the disclosure, during the process of obtaining audio anomaly fault analysis results based on multi-dimensional on-site data analysis, information such as the cause, location, and severity of the fault can be obtained. Targeted recovery suggestions can then be generated based on this information. For example, if the analysis indicates that the audio anomaly is caused by overheating of the audio decoding chip, recovery suggestions could include reducing the chip's operating frequency and improving heat dissipation.

[0193] In an exemplary embodiment, the generation of recovery recommendations can be based on the specific conclusions of the fault analysis, such as the type of hardware fault, the type of software error, etc.; it can also refer to past experience in handling similar faults, the technical specifications and performance parameters of the device, and general handling methods. For example, for a similar audio distortion fault in a specific model of audio equipment, if past experience indicates that the problem can be solved by replacing a specific capacitor, then in the results of this fault analysis, if it is determined to be a similar situation, a recovery recommendation to replace that capacitor can be given.

[0194] In this embodiment, a recovery strategy record can be added for the identified abnormal type of audio stream data, or a recovery strategy record can be added for the hardware module status corresponding to the current abnormality. This can override the original recovery strategy, or multiple recovery strategies can coexist.

[0195] Through the embodiments disclosed herein, the anomaly recovery strategy can be dynamically updated based on recovery recommendations, thereby enabling the system to self-optimize and continuously improve the anomaly handling effect.

[0196] Figure 7 This is a schematic diagram of a recovery strategy library in an audio processing method according to some embodiments of the present disclosure.

[0197] like Figure 7 As shown, the recovery strategy library can be categorized according to the business scenario of audio playback, such as music playback, horn playback, and navigation playback (this embodiment uses these three as examples, but is not limited to them). In music playback, anomaly types can include noise, stuttering, and silence (this embodiment uses these three as examples, but is not limited to them). These anomaly types can correspond to different spectral characteristics, and different anomaly types can correspond to different anomaly recovery strategies. For example, noise problems can be solved by increasing the size of the buffer; stuttering problems can be solved by restarting the audio subsystem; and silence problems can be solved by increasing the audio playback volume or checking whether the data source is silent audio.

[0198] Figure 8 This is a flowchart illustrating a method for audio anomaly handling according to some embodiments of this disclosure, such as... Figure 8 As shown, the method for audio anomaly handling can be applied to a server, such as a local server or a cloud server, which can be deployed in a computer cluster or a group of multiple computers. The method for audio anomaly handling can include the following steps.

[0199] In step S810, an initial audio anomaly analysis model is built based on the audio anomaly analysis strategy.

[0200] In this embodiment of the disclosure, the audio anomaly analysis strategy may include correlated abnormal audio data, root causes of the anomalies, and corresponding recovery actions. The audio anomaly analysis strategy can be a set of rules that defines the correlation between abnormal audio data, root causes of the anomalies, and corresponding recovery actions, providing a guiding framework for building an initial audio anomaly analysis model.

[0201] In this step, an audio anomaly analysis strategy can be used as a basis. By exploring the relationship between abnormal audio data, root causes of anomalies, and corresponding recovery actions within this strategy, a logical framework and knowledge base can be provided for model construction.

[0202] In step S820, historical audio anomaly training data is obtained, and the initial audio anomaly analysis model is trained based on the historical audio anomaly training data to obtain the trained audio anomaly analysis model.

[0203] In this embodiment of the disclosure, historical audio anomaly training data can be collected. This data can cover historical abnormal audio segments, historical abnormal logs, and corresponding root causes of the anomalies. Historical abnormal audio segments can directly reflect the sound characteristics when audio anomalies occur; historical abnormal logs record various relevant information when the anomaly occurs, such as time and system status; and the corresponding root causes of the anomalies indicate the fundamental reasons for the audio anomalies.

[0204] These historical data can be used to train the initial audio anomaly analysis model. During training, the model can learn patterns and regularities in the historical data, such as the correspondence between different abnormal audio segments and the root causes of the anomalies, and which information in the anomaly logs is associated with specific root causes. By adjusting the model parameters, it can more accurately predict the correct and reasonable root causes of anomalies based on the input audio segments and log information, ultimately resulting in a fully trained audio anomaly analysis model.

[0205] In step S830, in response to receiving abnormal audio segments and abnormal logs sent by the terminal, the abnormal audio segments and abnormal logs are input into the trained audio anomaly analysis model to obtain the target anomaly root cause, and the audio anomaly analysis strategy is updated according to the target anomaly root cause.

[0206] In this embodiment of the disclosure, the received abnormal audio segments and abnormal logs can be input into the trained audio anomaly analysis model. The model uses the knowledge and patterns learned during previous training to analyze and process these input data, thereby deriving the root cause of the target anomaly, that is, the most likely root cause of the current audio anomaly.

[0207] The audio anomaly analysis strategy can be updated based on the root causes of the target anomalies obtained from model analysis. As actual audio anomalies continue to emerge and be analyzed, new situations not covered by previous strategies can be discovered, or a deeper understanding of existing relationships can be gained.

[0208] As can be seen from the above steps, the audio processing method provided in this disclosure can be used to build a cloud-based intelligent analysis system with self-evolution capabilities. An initial model is established using historical training data and continuously optimized. When abnormal data is received from the terminal, the model can accurately analyze the root cause of the anomaly and dynamically update the analysis strategy accordingly, forming a virtuous cycle of data-driven optimization and significantly improving the accuracy and intelligence of audio anomaly diagnosis.

[0209] In an exemplary embodiment, hardware fault analysis results can be generated by comparing the connection status in the anomaly log and / or the fault diagnosis status with the normal status information.

[0210] In this embodiment of the disclosure, the connection status and / or fault diagnosis status of the hardware module can be compared with normal status information. Through this comparison, it is possible to quickly determine whether the hardware has faults such as loose connections or damage, thereby generating hardware fault analysis results.

[0211] For example, if the connection status shows that the speaker is not connected correctly, while the normal status should be that it is connected, then it can be determined that there is a hardware connection failure.

[0212] Hardware fault analysis results can be used to determine whether a hardware fault exists and its specific location and type, providing a basis for hardware repair or replacement.

[0213] In an exemplary embodiment, the multi-dimensional on-site data (i.e., abnormal audio segments and abnormal logs) can be matched with preset fault combination conditions to generate software fault analysis results, wherein the fault combination conditions include: the original audio data of the node belongs to a preset abnormal type, and the connection status and / or the fault diagnosis status is normal.

[0214] In this embodiment, multi-dimensional field data can be matched with preset fault combination conditions. The fault combination conditions define certain characteristics of a software fault, such as the node's original audio data belonging to a preset anomaly type, while the hardware connection status and / or fault diagnosis status are normal. When these conditions are met, it indicates that the audio anomaly may be caused by a software problem, thereby generating software fault analysis results.

[0215] For example, if the raw audio data shows stuttering characteristics, but the hardware connection and diagnostic status are normal, then it can be inferred that there is a problem in the software processing. Furthermore, by determining whether the raw audio data of each audio processing node is abnormal or what type of abnormality it belongs to, the node in the software chain that caused the abnormality can be inferred, and the algorithm, process or configuration that caused the abnormality on that node can be further analyzed.

[0216] Software fault analysis results can indicate whether audio anomalies are caused by software problems, as well as the software modules and causes of the faults that may be involved, which helps software developers debug and optimize their code.

[0217] In an exemplary embodiment, fault development trend prediction can be performed based on historical multi-dimensional field data and the multi-dimensional field data to generate fault trend analysis results.

[0218] In this embodiment, historical multi-dimensional field data and current multi-dimensional field data can be used to predict fault development trends. By analyzing the changing trends of historical and current data, potential fault risks can be identified in advance, the possible development direction and severity of faults can be predicted, and fault trend analysis results can be generated.

[0219] For example, if the number of communication timeouts in a subsystem increases from "1 time / day" to "5 times / day", it can be predicted that "continuous timeouts may occur in 3 days". As another example, if the current power amplifier chip temperature is trending upwards, and the rate of temperature rise is very similar to the pattern before the three previous audio interruptions due to overheating, it can be predicted that overheat protection may be triggered after a period of time, and a suggestion to "reduce the output volume to prevent overheating" can be obtained.

[0220] Fault trend analysis results can provide early warnings of potential fault risks, thus providing a reference for system maintenance personnel to formulate preventive maintenance strategies and avoid further deterioration or expansion of faults.

[0221] In exemplary embodiments, fault trend analysis can be performed using statistical model-based methods (such as regression analysis) or machine learning-based methods (such as supervised learning, reinforcement learning, etc.), and this disclosure does not limit the scope of the analysis.

[0222] In some embodiments, the method for handling audio anomalies further includes: analyzing the abnormal audio segments and the abnormal logs according to an updated audio anomaly analysis strategy to determine a cloud-based anomaly recovery strategy; and sending the cloud-based anomaly recovery strategy to the terminal.

[0223] In this embodiment, an updated strategy can be used to perform in-depth analysis of received abnormal audio segments and abnormal logs. The updated strategy integrates the target anomaly root cause information obtained from previous model analysis, optimizes and adjusts the correlation between abnormal audio data, anomaly root causes, and corresponding recovery actions, and can more accurately reflect the actual situation.

[0224] By defining the relationships within the strategy, the root cause of the current audio anomaly can be precisely located, allowing for the determination of the most suitable cloud-based anomaly recovery strategy. For example, if the analysis reveals that the root cause is a software algorithm failure, the cloud-based anomaly recovery strategy could be to push a new algorithm patch to the terminal.

[0225] This disclosure provides a complete cloud-based processing chain, from anomaly analysis to recovery strategy generation. After identifying the root cause through model analysis, a targeted recovery strategy is automatically generated and distributed to the terminal, forming a cloud-based decision support system that enhances the anomaly handling capabilities and recovery effectiveness of terminal devices.

[0226] In some embodiments, the method for handling audio anomalies further includes: updating the audio anomaly analysis strategy again according to the cloud anomaly recovery strategy; and sending the updated audio anomaly analysis strategy to the terminal in response to meeting the update conditions.

[0227] In this embodiment, the cloud-based anomaly recovery strategy is formulated based on the analysis and processing of the current audio anomaly situation, reflecting effective measures for actually resolving audio anomaly problems. Updating the audio anomaly analysis strategy based on this strategy allows for the updating of the audio anomaly analysis strategy to include new anomaly patterns, root causes of anomalies, and their correlation with recovery actions that may be discovered during the actual recovery process, or to update existing correlations.

[0228] In this way, the audio anomaly analysis strategy can continuously absorb the lessons learned from actual processing, continuously optimize and improve itself, enhance the ability to predict and handle future audio anomalies, and make the entire audio anomaly processing system more intelligent and efficient.

[0229] In this embodiment of the disclosure, the update conditions can be a set of preset rules or standards used to determine whether to send the updated audio anomaly analysis strategy to the terminal. The update conditions can be time-related, such as updating the strategy at regular intervals; they can also be quantity-related, such as updating after processing a certain number of audio anomaly cases; or they can be triggered by a specific event, such as the terminal sending a relevant request.

[0230] When the update conditions are met, the updated audio anomaly analysis strategy can be sent from the cloud to the terminal device. Upon receiving the updated strategy, the terminal can store it and apply it to subsequent audio anomaly handling. This allows the terminal to perform more accurate analysis and judgment based on the latest strategy when handling audio anomalies, and also ensures consistency between the terminal and the cloud in audio anomaly handling strategies.

[0231] This disclosure establishes a continuous strategy optimization mechanism, utilizing cloud-based recovery strategies to improve analysis strategies. Condition-triggered updates ensure the terminal always receives the latest analysis strategies, enabling collaborative evolution of cloud and terminal capabilities and continuously enhancing the overall system intelligence.

[0232] In some implementations, an initial audio anomaly analysis model is built based on an audio anomaly analysis strategy, including: selecting abnormal features, root causes, and recovery actions related to in-vehicle audio from the audio anomaly analysis strategy based on the in-vehicle audio anomaly scenario; and building an in-vehicle audio anomaly analysis model based on the selected in-vehicle audio-related data.

[0233] In this embodiment of the disclosure, the in-vehicle audio system may have a unique operating environment and abnormal behavior, such as being affected by vibrations during vehicle operation, electromagnetic interference, and the impact of different driving modes (such as high speed and low speed) on the audio equipment. An overall audio anomaly analysis strategy can identify anomaly characteristics closely related to the in-vehicle audio scenario (such as specific manifestations like audio noise, interruptions, and abnormal volume fluctuations), the root causes of the anomalies (which may be hardware-related, such as speaker damage or poor wiring connections, or software-related, such as audio driver failure or system conflicts), and recovery actions (such as hardware repair and replacement, or software updates and resets).

[0234] The selected audio anomaly analysis strategies can be used to construct models using appropriate modeling methods (such as decision trees and neural network algorithms in machine learning). A well-constructed in-vehicle audio anomaly analysis model can quickly and accurately analyze anomalies occurring in the in-vehicle audio system. When a new audio anomaly occurs, the model can predict possible root causes based on the input anomaly characteristics and provide corresponding recovery action suggestions, thus providing strong support for maintenance personnel or automated system handling, and improving the stability and reliability of the in-vehicle audio system.

[0235] This disclosure provides an embodiment that allows for the customized construction of analytical models for in-vehicle scenarios. By filtering relevant features and root cause data, the professionalism of the model can be ensured. This method improves the accuracy of in-vehicle audio anomaly identification and enhances the practicality and reliability of the model in specific application scenarios.

[0236] In some embodiments of this disclosure, the target anomaly root cause includes an anomaly severity level, and the method for audio processing further includes: triggering an alarm notification of the corresponding level based on the anomaly severity level.

[0237] In this embodiment of the disclosure, during the analysis and processing of audio anomalies, the severity of the obtained root causes of the anomalies can be assessed and classified, and a corresponding severity level can be assigned to them. For example, the severity level of the anomaly can be divided into different levels (such as minor, general, severe, urgent, or low, medium, high) based on factors such as the degree of impact of the audio anomaly event on the audio system function and the degree of interference with the user experience.

[0238] In an exemplary embodiment, the anomaly can be categorized into different severity levels by considering factors such as the audio playback business scenario, the type of the anomaly, the number of times the anomaly occurred within a certain period, and the fault causes identified from the target anomaly root cause. The fault causes can encompass hardware status (e.g., audio chip temperature, voltage, speaker impedance), audio signal characteristics (e.g., audio distortion, signal-to-noise ratio, frequency response), and software operating conditions (e.g., audio processing algorithm execution time, error logs).

[0239] For example, in a music playback scenario, if occasional stuttering occurs, and analysis confirms that the hardware module's connection status and fault diagnosis status are normal, and the suspected cause is insufficient buffer capacity, then the severity level of the anomaly can be determined as "low". In the same scenario, if occasional stuttering occurs, and analysis confirms that the hardware module's connection status has timed out several times with an increasing trend, then the severity level can be determined as "medium". In an alarm tone playback scenario, if a silent sound occurs, and analysis confirms that the subsystem status code shows "DAC (Digital-to-Analog Converter) overloaded, audio output has stopped", and multiple attempts to implement recovery strategies fail to eliminate the silent sound, then the severity level can be determined as "high".

[0240] In an exemplary embodiment, the severity level of the anomaly can also be assessed based on preset factors. Preset factors may include, for example, the degree of impact of the anomaly on audio quality (slight distortion, severe distortion, complete silence, etc.), the threat to system stability (whether it may cause system crash), and the damage to user experience (whether it affects normal listening, whether it generates harsh noise, whether it affects user safety, etc.).

[0241] In this embodiment of the disclosure, alarm notification methods may include, for example, cloud platform message push, email, SMS, pop-up alarm, audio-visual alert, emergency pop-up phone notification, and continuous ringing. Different levels of anomaly severity can trigger corresponding levels of alarm notification. For example, the alarm notification method corresponding to a "low" anomaly severity level may be cloud platform message push or email notification, which is non-mandatory and can be used to remind relevant personnel to pay attention. The alarm notification method corresponding to a "medium" anomaly severity level may be cloud platform pop-up alarm, SMS notification requiring confirmation of read, or local audio-visual alert at the edge (such as a flashing red light on the gateway). The alarm notification method corresponding to a "high" anomaly severity level may be an emergency pop-up on the cloud platform, phone notification (allowing priority contact with the responsible person), or local alarm sound at the edge (continuous ringing) to ensure timely attention.

[0242] Through the embodiments disclosed herein, graded alarms can be triggered according to the severity level, enabling differentiated fault notifications, optimizing the operation and maintenance response process, and ensuring that critical faults are handled with priority.

[0243] It should be noted that the above figures are merely illustrative representations of the processes included in methods according to some embodiments of this disclosure, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0244] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0245] Figure 8 This is a block diagram illustrating an audio processing apparatus according to some embodiments of the present disclosure. (Refer to...) Figure 8 The device includes: an identification unit 901, an acquisition unit 902, an analysis unit 903, a recovery unit 904, a first transmission unit 905, and a first reception unit 906.

[0246] The identification unit 901 is used to identify anomalies in the audio data during audio playback; the acquisition unit 902 is used to acquire abnormal audio segments and abnormal logs in response to the identification of an audio abnormal event; the analysis unit 903 is used to analyze the abnormal audio segments and abnormal logs to determine an anomaly recovery strategy; and the recovery unit 904 is used to perform anomaly recovery according to the anomaly recovery strategy.

[0247] In some embodiments of this disclosure, during audio playback, the identification unit 901 performs anomaly identification on the played audio data, including: collecting audio segments in segments during audio playback; comparing each audio segment with sample abnormal data; and determining whether an audio abnormal event has occurred based on the comparison result.

[0248] In some embodiments of this disclosure, the acquisition unit 902, in response to recognizing an audio abnormal event, acquires abnormal audio segments and abnormal logs, including: identifying audio segments that are identified as having caused audio abnormal events in the comparison as abnormal audio segments; and extracting abnormal logs corresponding to the abnormal audio segments from the operation logs of multiple audio subsystems in the audio processing link.

[0249] In some embodiments of this disclosure, the anomaly log includes at least one of the following: the connection status of the hardware module related to audio output, the fault diagnosis status of the hardware module related to audio output, the node raw audio data of multiple nodes in the audio processing link, and the operation log of multiple audio subsystems in the audio processing link.

[0250] In some embodiments of this disclosure, the analysis unit 903 analyzes the abnormal audio segment and the abnormal log to determine an abnormal recovery strategy, including: determining the abnormal type of the abnormal audio segment; determining a hardware fault based on the connection status and fault diagnosis status of the hardware module related to audio output in the abnormal log; and matching the abnormal recovery strategy from the recovery strategy library based on at least one of the abnormal type and the hardware fault.

[0251] In some embodiments of this disclosure, the analysis unit 903 determines the abnormal type of the abnormal audio segment by: extracting audio features of the abnormal audio segment, matching the audio features with abnormal features of a preset abnormal type of audio, and determining the abnormal type based on the matching result; and / or, inputting the abnormal audio segment into an audio abnormality classification model, and outputting the abnormal type by the audio abnormality classification model; wherein the audio abnormality classification model is trained based on historical abnormal audio data.

[0252] In some embodiments of this disclosure, the first sending unit 905 is configured to: send the abnormal audio clip and abnormal log to the cloud.

[0253] In some embodiments of this disclosure, the first receiving unit 906 is configured to: receive a cloud-based anomaly recovery strategy generated by the cloud based on the abnormal audio segment and the abnormal log; and the recovery unit 904 performs anomaly recovery according to the anomaly recovery strategy, including: performing anomaly recovery according to the anomaly recovery strategy and the cloud-based anomaly recovery strategy.

[0254] In some embodiments of this disclosure, the first receiving unit 906 is further configured to: receive an updated audio anomaly classification model and / or audio anomaly analysis strategy sent from the cloud; the analysis unit 903 is further configured to: update the local audio anomaly classification model according to the updated audio anomaly classification model, and / or update the local recovery strategy library according to the audio anomaly analysis strategy.

[0255] In some embodiments of this disclosure, the method is applied to an in-vehicle audio system; wherein, the identification unit 901 performs anomaly identification on the played audio data, including: deploying a loudness detection algorithm at the input and output terminals of the digital signal processor in the in-vehicle audio system to perform anomaly identification; wherein, the identified anomaly type includes at least one of the following: no sound, abnormal noise, stuttering, and popping sound anomaly.

[0256] In some embodiments of this disclosure, the first sending unit 905 uses a cockpit monitoring platform to upload the abnormal audio clips and abnormal logs to the cloud.

[0257] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0258] Figure 10 This is a block diagram illustrating an apparatus for audio anomaly processing according to some embodiments of the present disclosure. (Refer to...) Figure 10 The device includes: a setup unit 1001, a training unit 1002, a processing unit 1003, a second sending unit 1004, and a notification unit 1005.

[0259] The setup unit 1001 is used to build an initial audio anomaly analysis model according to the audio anomaly analysis strategy; the training unit 1002 is used to acquire historical audio anomaly training data, train the initial audio anomaly analysis model according to the historical audio anomaly training data, and obtain a trained audio anomaly analysis model; the processing unit 1003 is used to respond to receiving abnormal audio segments and abnormal logs sent by the terminal, input the abnormal audio segments and abnormal logs into the trained audio anomaly analysis model, obtain the target anomaly root cause, and update the audio anomaly analysis strategy according to the target anomaly root cause.

[0260] In some embodiments of this disclosure, the apparatus for audio anomaly processing further includes a second sending unit 1004; the processing unit 1003 is further configured to analyze the abnormal audio segment and the abnormal log according to the updated audio anomaly analysis strategy, and determine a cloud anomaly recovery strategy; the second sending unit 1004 is configured to send the cloud anomaly recovery strategy to the terminal.

[0261] In some embodiments of this disclosure, the processing unit 1003 is further configured to update the audio anomaly analysis strategy again according to the cloud anomaly recovery strategy; the second sending unit 1004 is further configured to send the updated audio anomaly analysis strategy to the terminal in response to the fulfillment of the update conditions.

[0262] In some embodiments of this disclosure, the building unit 1001 builds an initial audio anomaly analysis model according to the audio anomaly analysis strategy, including: selecting abnormal features, root causes and recovery actions related to vehicle audio from the audio anomaly analysis strategy according to the vehicle audio anomaly scenario; and building the vehicle audio anomaly analysis model based on the selected vehicle audio-related data.

[0263] In some embodiments of this disclosure, the target anomaly root cause includes an anomaly severity level, and the device for audio anomaly processing further includes a notification unit 1005, configured to: trigger an alarm notification of the corresponding level based on the anomaly severity level.

[0264] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0265] Figure 11 This is a block diagram illustrating an audio processing apparatus 1100 according to some embodiments of the present disclosure. For example, apparatus 1100 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0266] Reference Figure 11 The device 1100 may include one or more of the following components: a processing component 1102, a memory 1104, a power component 1106, a multimedia component 1108, an audio component 1110, an input / output (I / O) interface 1112, a sensor component 1114, and a communication component 1116.

[0267] Processing component 1102 typically controls the overall operation of device 1100, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 1102 may include one or more processors 1120 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 1102 may include one or more modules to facilitate interaction between processing component 1102 and other components. For example, processing component 1102 may include a multimedia module to facilitate interaction between multimedia component 1108 and processing component 1102.

[0268] Memory 1104 is configured to store various types of data to support the operation of device 1100. Examples of such data include instructions for any application or method operating on device 1100, contact data, phonebook data, messages, pictures, videos, etc. Memory 1104 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0269] The power supply component 1106 provides power to the various components of the device 1100. The power supply component 1106 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 1100.

[0270] Multimedia component 1108 includes a screen that provides an output interface between the device 1100 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 1108 includes a front-facing camera and / or a rear-facing camera. When the device 1100 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0271] Audio component 1110 is configured to output and / or input audio signals. For example, audio component 1110 includes a microphone (MIC) configured to receive external audio signals when device 1100 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1104 or transmitted via communication component 1116. In some embodiments, audio component 1110 also includes a speaker for outputting audio signals.

[0272] I / O interface 1112 provides an interface between processing component 1102 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0273] Sensor assembly 1114 includes one or more sensors for providing status assessments of various aspects of device 1100. For example, sensor assembly 1114 may detect the on / off state of device 1100, the relative positioning of components such as the display and keypad of device 1100, changes in the position of device 1100 or a component of device 1100, the presence or absence of user contact with device 1100, the orientation or acceleration / deceleration of device 1100, and temperature changes of device 1100. Sensor assembly 1114 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1114 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 1114 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0274] Communication component 1116 is configured to facilitate wired or wireless communication between device 1100 and other devices. Device 1100 can access wireless networks based on communication standards, such as WiFi, 3G, 4G, 5G, other communication standards, or combinations thereof. In some embodiments of this disclosure, communication component 1116 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In some embodiments of this disclosure, communication component 1116 further includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0275] In some embodiments of this disclosure, the apparatus 1100 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0276] In some embodiments of this disclosure, a computer-readable storage medium including instructions is also provided, such as a memory 904 including instructions, which can be executed by a processor 920 of device 900 to perform the above-described method. For example, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0277] A vehicle may be a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other type of vehicle. The vehicle may be a driver-assisted vehicle, a semi-driver-assisted vehicle, or a driver-free vehicle.

[0278] Vehicles can include various subsystems, such as infotainment systems, perception systems, decision control systems, drive systems, and computing platforms. A vehicle can also include more or fewer subsystems, and each subsystem can include multiple components. Furthermore, each subsystem and each component of the vehicle can be interconnected via wired or wireless means.

[0279] In some embodiments, an infotainment system may include a communication system, an entertainment system, and a navigation system, etc.

[0280] The perception system may include several types of sensors used to sense information about the environment surrounding the vehicle. For example, the perception system may include a global positioning system (which may be GPS, BeiDou, or other positioning systems), an inertial measurement unit (IMU), lidar, millimeter-wave radar, ultrasonic radar, and camera devices.

[0281] The decision control system may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.

[0282] A drive system may include components that provide powered motion to a vehicle. In one embodiment, a drive system may include an engine, an energy source, a transmission system, and wheels. The engine may be one or a combination of internal combustion engines, electric motors, and compressed air engines. The engine is capable of converting energy provided by the energy source into mechanical energy.

[0283] Some or all of the vehicle's functions are controlled by a computing platform. The computing platform may include at least one processor and memory, the processor being able to execute instructions stored in the memory.

[0284] The processor can be any conventional processor, such as a commercially available CPU. The processor can also include graphics processing units (GPUs), field-programmable gate arrays (FPGAs), systems-on-chips (SoCs), application-specific integrated circuits (ASICs), or combinations thereof.

[0285] Memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0286] In addition to instructions, memory can also store data, such as road maps, route information, and vehicle position, direction, and speed. The data stored in memory can be used by the computing platform.

[0287] In this embodiment of the disclosure, the processor may execute instructions to complete all or part of the steps of the above method.

[0288] A computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to perform all or part of the steps of the above method.

[0289] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0290] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An audio processing method, characterized in that, include: During audio playback, anomaly detection is performed on the played audio data; In response to the detection of an audio anomaly event, retrieve the abnormal audio segment and the anomaly log; The abnormal audio segments and abnormal logs are analyzed to determine the abnormal recovery strategy; Perform anomaly recovery according to the aforementioned anomaly recovery strategy.

2. The method according to claim 1, characterized in that, During audio playback, anomaly detection is performed on the played audio data, including: During audio playback, audio segments are captured in stages; Each audio segment is compared with the sample abnormal data, and the comparison results are used to determine whether an audio abnormality event has occurred.

3. The method according to claim 2, characterized in that, In response to the detection of an audio anomaly event, the abnormal audio segment and anomaly log are retrieved, including: Audio segments identified as having audio anomalies during the comparison are classified as anomalous audio segments. Extract the abnormal logs corresponding to the abnormal audio segments from the operation logs of multiple audio subsystems in the audio processing chain.

4. The method according to claim 3, characterized in that, The anomaly log includes at least one of the following: the connection status of the hardware module related to audio output, the fault diagnosis status of the hardware module related to audio output, the original audio data of multiple nodes in the audio processing link, and the operation logs of multiple audio subsystems in the audio processing link.

5. The method according to claim 1, characterized in that, The abnormal audio segments and abnormal logs are analyzed to determine anomaly recovery strategies, including: Determine the anomaly type of the abnormal audio segment; The hardware fault is determined based on the connection status and fault diagnosis status of the hardware modules related to audio output in the anomaly log. Based on at least one of the anomaly type and the hardware failure, the anomaly recovery strategy is matched from the recovery strategy library.

6. The method according to claim 5, characterized in that, Determining the abnormal type of the abnormal audio segment includes: Extract the audio features of the abnormal audio segment, match the audio features with the abnormal features of audio of a preset abnormal type, and determine the abnormal type based on the matching result; and / or, The abnormal audio segment is input into the audio anomaly classification model, and the audio anomaly classification model outputs the anomaly type; wherein, the audio anomaly classification model is trained based on historical abnormal audio data.

7. The method according to claim 1, characterized in that, The method further includes: The abnormal audio clips and abnormal logs are sent to the cloud.

8. The method according to claim 7, characterized in that, The method further includes: Receive the cloud-based anomaly recovery strategy generated by the cloud based on the abnormal audio fragments and abnormal logs; In addition, performing anomaly recovery according to the anomaly recovery strategy includes: performing anomaly recovery according to the anomaly recovery strategy and the cloud-based anomaly recovery strategy.

9. The method according to claim 7, characterized in that, The method further includes: Receive updated audio anomaly classification models and / or audio anomaly analysis strategies sent from the cloud; Update the local audio anomaly classification model according to the updated audio anomaly classification model, and / or update the local recovery strategy library according to the audio anomaly analysis strategy.

10. The method according to claim 1, characterized in that, The method is applied to an in-vehicle audio system; wherein, anomaly identification of the played audio data includes: A loudness detection algorithm is deployed at the input and output of the digital signal processor in the vehicle audio system to identify anomalies; wherein the identified anomaly types include at least one of the following: silence, abnormal noise, stuttering, and popping sound anomalies.

11. The method according to claim 10, characterized in that, The abnormal audio clips and abnormal logs are sent to the cloud using the cockpit monitoring platform.

12. A method for audio anomaly processing, characterized in that, include: An initial audio anomaly analysis model was built based on the audio anomaly analysis strategy; Acquire historical audio anomaly training data, and train the initial audio anomaly analysis model based on the historical audio anomaly training data to obtain the trained audio anomaly analysis model. In response to receiving abnormal audio segments and abnormal logs from the terminal, the abnormal audio segments and abnormal logs are input into the trained audio anomaly analysis model to obtain the target anomaly root cause, and the audio anomaly analysis strategy is updated according to the target anomaly root cause.

13. The method according to claim 12, characterized in that, The method further includes: Based on the updated audio anomaly analysis strategy, the abnormal audio segments and the abnormal logs are analyzed to determine the cloud-based anomaly recovery strategy. The cloud-based anomaly recovery strategy is sent to the terminal.

14. The method according to claim 13, characterized in that, The method further includes: The audio anomaly analysis strategy is updated again based on the cloud-based anomaly recovery strategy. In response to the fulfillment of the update conditions, the updated audio anomaly analysis strategy is sent to the terminal.

15. The method according to claim 12, characterized in that, An initial audio anomaly analysis model is built based on the audio anomaly analysis strategy, including: Based on the abnormal scenarios of in-vehicle audio, the abnormal features, root causes of abnormalities, and recovery actions related to in-vehicle audio are selected from the audio abnormality analysis strategy. Based on the selected in-vehicle audio-related data, an in-vehicle audio anomaly analysis model was built.

16. The method according to claim 12, characterized in that, The target anomaly root cause includes an anomaly severity level, and the method further includes: triggering an alarm notification of the corresponding level based on the anomaly severity level.

17. An audio processing apparatus, characterized in that, include: The identification unit is used to identify anomalies in the audio data during audio playback. The acquisition unit is used to acquire abnormal audio segments and abnormal logs in response to the detection of an audio abnormality event; The analysis unit is used to analyze the abnormal audio segments and abnormal logs to determine the abnormal recovery strategy; The recovery unit is used to perform abnormal recovery according to the abnormal recovery strategy.

18. The apparatus according to claim 17, characterized in that, During audio playback, the recognition unit performs anomaly detection on the played audio data, including: During audio playback, audio segments are captured in stages; Each audio segment is compared with the sample abnormal data, and the comparison results are used to determine whether an audio abnormality event has occurred.

19. The apparatus according to claim 17, characterized in that, The device further includes a first transmitting unit for: The abnormal audio clips and abnormal logs are sent to the cloud.

20. The apparatus according to claim 19, characterized in that, The device further includes a first receiving unit for: Receive the cloud-based anomaly recovery strategy generated by the cloud based on the abnormal audio fragments and abnormal logs; In addition, the recovery unit performs abnormal recovery according to the abnormal recovery strategy, including performing abnormal recovery according to the abnormal recovery strategy and the cloud-based abnormal recovery strategy.

21. An apparatus for audio anomaly processing, characterized in that, include: The building unit is used to build an initial audio anomaly analysis model based on the audio anomaly analysis strategy; The training unit is used to acquire historical audio anomaly training data and train the initial audio anomaly analysis model based on the historical audio anomaly training data to obtain the trained audio anomaly analysis model. The processing unit is configured to respond to receiving abnormal audio segments and abnormal logs sent by the terminal, input the abnormal audio segments and abnormal logs into the trained audio anomaly analysis model, obtain the target anomaly root cause, and update the audio anomaly analysis strategy according to the target anomaly root cause.

22. The apparatus according to claim 21, characterized in that, The device further includes a second sending unit, and the processing unit is further configured to analyze the abnormal audio segments and the abnormal logs according to the updated audio anomaly analysis strategy, and determine the cloud anomaly recovery strategy. The second sending unit is also used to send the cloud-based anomaly recovery strategy to the terminal.

23. The apparatus according to claim 22, characterized in that, The processing unit is also used to update the audio anomaly analysis strategy again according to the cloud anomaly recovery strategy; The second sending unit is also configured to send the updated audio anomaly analysis strategy to the terminal in response to the fulfillment of the update conditions.

24. A vehicle, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the steps of the method according to any one of claims 1-16.

25. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the steps of the method according to any one of claims 1-16.

26. A computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of a mobile terminal, enable the mobile terminal to perform the steps of the method according to any one of claims 1-16.

27. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-16.

Images