Audio processing method, device, equipment, medium and program product

Abstract

The embodiment of the invention discloses an audio processing method and device, equipment, a medium and a program product. The method comprises the following steps: receiving an audio operation request sent by target terminal equipment; acquiring equipment information of the target terminal equipment in response to the audio operation request; determining configuration parameters matched with the target terminal equipment from the audio control system according to the equipment information of the target terminal equipment; and executing the audio operation requested by the audio operation request according to the configuration parameters matched with the target terminal equipment. According to the embodiment of the invention, cross-platform audio acquisition and playing can be realized, and repeated development of audio acquisition and playing services is avoided.

Description

Technical Field

[0001] The present application relates to the field of computer technology, in particular to the field of audio and video, and specifically to an audio processing method, an audio processing device, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the rapid development of multimedia technology, audio acquisition and playback have become basic service requirements for many application functions (such as live broadcast, conferencing or voice calls, etc.) or platforms (such as Android or Apple, etc.).

[0003] Currently, personalized development of audio collection and playback for different application functions or different platforms is supported; for example, audio collection and playback adapted to the corresponding platform is developed for different application platforms to achieve compatibility and adaptation between the platform and audio collection and playback. However, this exclusive development method for audio collection and playback for different applications or platforms will face repeated development of audio collection and playback in different functions or platforms, resulting in high development costs for audio collection and playback. Summary of the invention

[0004] The embodiments of the present application provide an audio processing method, apparatus, device, medium and program product, which can realize cross-platform audio collection and playback and avoid repeated development of audio collection and playback services.

[0005] On the one hand, an embodiment of the present application provides an audio processing method, which is applied to an audio control system, and the audio control system is used to provide audio acquisition and playback services for terminal devices; the audio control system pre-stores configuration parameters corresponding to a plurality of terminal devices, and the configuration parameters corresponding to each of the plurality of terminal devices are pre-configured to the audio control system to ensure the compatibility between the corresponding terminal device and the audio control system; the method includes:

[0006] receiving an audio operation request sent by a target terminal device; the audio operation request is used to request the audio control system to perform an audio operation on an audio acquisition and playback service; the target terminal device is any one of a plurality of terminal devices;

[0007] Acquire device information of the target terminal device in response to the audio operation request; the device information includes hardware information and system information, the hardware information is used to indicate information of the hardware device deployed in the target terminal device, and the system information is used to indicate system information of the operating system deployed in the target terminal device;

[0008] According to the device information of the target terminal device, determining the configuration parameters adapted to the target terminal device from the audio control system; the configuration parameters are pre-set based on the hardware information and software information included in the device information;

[0009] The audio operation requested by the audio operation request is performed according to the configuration parameters adapted to the target terminal device.

[0010] On the other hand, an embodiment of the present application provides an audio processing device, which is equipped with an audio control system, and the audio control system is used to provide audio collection and playback services for terminal devices; the audio control system pre-stores configuration parameters corresponding to a variety of terminal devices, and the configuration parameters corresponding to each of the multiple terminal devices are pre-configured to the audio control system to ensure the compatibility between the corresponding terminal device and the audio control system; the device includes:

[0011] A receiving unit, configured to receive an audio operation request sent by a target terminal device; the audio operation request is used to request the audio control system to perform an audio operation on an audio acquisition and playback service; the target terminal device is any one of a plurality of terminal devices;

[0012] A processing unit, configured to obtain device information of a target terminal device in response to an audio operation request; the device information includes hardware information and system information, the hardware information is used to indicate information of a hardware device deployed in the target terminal device, and the system information is used to indicate system information of an operating system deployed in the target terminal device;

[0013] The processing unit is further used to determine configuration parameters adapted to the target terminal device from the audio control system according to the device information of the target terminal device; the configuration parameters are pre-set based on the hardware information and software information included in the device information;

[0014] The processing unit is further used to execute the audio operation requested by the audio operation request according to the configuration parameters adapted to the target terminal device.

[0015] In one implementation, the audio control system includes a platform interface layer, which encapsulates platform interfaces corresponding to multiple platform types; the system information includes the platform type of the operating system carried by the target terminal device; the configuration parameters adapted to the target terminal device include the target platform interface;

[0016] The processing unit is used to determine the configuration parameters adapted to the target terminal device from the audio control system according to the device information of the target terminal device, specifically to:

[0017] According to the platform type of the operating system installed on the target terminal device, a target platform interface having the same platform type as the platform type of the operating system installed on the target terminal device is selected in the platform interface layer;

[0018] The target platform interface is used as a system interface for communication and transmission between the target terminal device and the audio control system.

[0019] In one implementation, the system information further includes a system version of an operating system installed in the target terminal device; the hardware device deployed in the target terminal device includes an audio device, and the hardware information includes sub-device information of the audio device; the configuration parameters adapted to the target terminal device include a target audio format compatible with the system version; and the processing unit is further used to:

[0020] According to the system version of the operating system installed on the target terminal device, a target audio format compatible with the system version is obtained from the audio control system;

[0021] Among them, the target audio format is pre-configured based on the coordination between the modules within the audio control system and the compatibility between the audio control system and the system version; the target audio format includes at least one of the following: audio sampling rate, audio mode, number of audio channels and transmission interval time of the callback function corresponding to the audio data.

[0022] In one implementation, the configuration parameters adapted to the target terminal device include a target platform interface; and the processing unit is configured to, when performing the audio operation requested by the audio operation request according to the configuration parameters adapted to the target terminal device, specifically:

[0023] Acquire the implementation class of the target platform interface included in the configuration parameters adapted to the target terminal device; the implementation class is used to instruct the audio control system to provide the service logic of the audio acquisition and playback service through the target platform interface;

[0024] Instantiate the implementation class of the target platform interface and register the callback function to start the audio acquisition and playback thread in the target terminal device;

[0025] Execute the audio operation requested by the audio operation request in the audio acquisition and playback thread.

[0026] In one implementation, the audio acquisition and playback service includes audio acquisition and audio playback; the audio operation request is used to request the audio control system to perform an audio operation on audio playback, and the audio acquisition and playback thread is the audio playback thread; the processing unit is used to perform the audio operation requested by the audio operation request in the audio acquisition and playback thread, specifically for:

[0027] In the audio playback thread, the first audio data to be played is obtained from the remote end through the callback function;

[0028] Sending the first audio data to an audio device in a target terminal device for playing; and,

[0029] The first audio data is cached in a cache space in the audio control system, and the first audio data in the cache space is used as an echo cancellation reference signal in the audio acquisition process.

[0030] In one implementation, the system information includes a system version of an operating system installed on the target terminal device; the processing unit is further configured to:

[0031] Check whether the system version of the operating system installed on the target terminal device reaches the version threshold;

[0032] If the system version of the operating system installed in the target terminal device reaches the version threshold, the step of sending the first audio data to the audio device in the target terminal device for playback is triggered; wherein, before playing the first audio data, the target terminal device uses the sampling conversion module deployed in the target terminal device to convert the first audio data into an audio format; or,

[0033] If the system version of the operating system installed in the target terminal device does not reach the version threshold, the sampling conversion module deployed in the audio control system is used to convert the first audio data into an audio format, and trigger the execution of the step of sending the first audio data to the audio device in the target terminal device for playback;

[0034] The audio format of the first audio data after the audio format conversion is the same as the audio format required by the target terminal device.

[0035] In one implementation, the audio acquisition and playback service includes audio acquisition and audio playback; the audio operation request is used to request the audio control system to perform an audio operation on audio acquisition, and the audio acquisition and playback thread is the audio acquisition thread; the processing unit is used to perform the audio operation requested by the audio operation request in the audio acquisition and playback thread, specifically for:

[0036] In the audio collection thread, the second audio data collected by the audio device in the target terminal device is obtained through the callback function;

[0037] Performing audio processing on the second audio data to obtain processed second audio data;

[0038] The processed second audio data is transmitted to the remote end through the callback function.

[0039] In one implementation, the system information includes a system version of an operating system installed on the target terminal device; the processing unit is further configured to:

[0040] Check whether the system version of the operating system installed on the target terminal device reaches the version threshold;

[0041] If the system version of the operating system installed in the target terminal device reaches the version threshold, the step of performing audio processing on the second audio data to obtain the processed second audio data is triggered; wherein, before the second audio data is called back to the audio control system, the sampling conversion module in the target terminal device performs audio format conversion on the second audio data; or,

[0042] If the system version of the operating system installed in the target terminal device does not reach the version threshold, the sampling conversion module deployed in the audio control system is used to convert the second audio data into an audio format, and the step of performing audio processing on the second audio data to obtain the processed second audio data is triggered;

[0043] The audio format of the second audio data after the audio format conversion is the same as the audio format required by the audio control system.

[0044] In one implementation, the target terminal device includes one or more audio devices, and the audio devices are used for collecting and playing audio data; the processing unit is further used for:

[0045] Starting a target audio device in a target terminal device; the target audio device is an audio device in a selected state among one or more audio devices; the target audio device is a default one, or is selected by a user in a selection window provided by an application or a setting interface provided by the target terminal device;

[0046] Get the startup status of the target audio device, and when the startup status indicates that the target audio device is successfully started, detect the working status of the callback function;

[0047] If the working status of the callback function indicates that the callback function is in a normal working state, a startup success log is printed.

[0048] In one implementation, the processing unit is further configured to:

[0049] If the startup status indicates that the target audio device fails to start, or the working status of the callback function indicates that the callback function is in an abnormal working state, the target audio device is restarted and the number of restarts is recorded. When the number of restarts exceeds the threshold, the audio acquisition and playback service is terminated and a startup failure log is printed; or,

[0050] If the startup status indicates that the target audio device failed to start, or the working status of the callback function indicates that the callback function is in an abnormal working state, the target terminal device is controlled to output a prompt message, and the prompt message is used to prompt the object to reselect a new audio device as the audio acquisition or audio playback device; and receive a reselection operation for the prompt message, and start the audio device indicated by the reselection operation.

[0051] In one implementation, the target terminal device includes one or more audio devices, and the audio devices are used for collecting and playing audio data; the processing unit is further used for:

[0052] Receiving a device switching event sent by a target terminal device, where the device switching event is used to indicate that an audio device in the target terminal device has changed;

[0053] Perform audio device switching processing according to device switching events;

[0054] The switching process includes any of the following:

[0055] If the device switching event indicates that the target audio device in the selected state in the target terminal device has changed, the audio device is switched;

[0056] If the device switching event indicates that there is an audio device in the target terminal device that is disconnected from the target terminal device, then query the device list of the target terminal device, which is used to store the device identifiers of the audio devices deployed in the target terminal device; if the device list is empty, stop the audio acquisition and playback service and print the service failure log; if the device list is not empty, check whether the disconnected audio device is the target audio device in the selected state; if so, switch to a new audio device;

[0057] If the device switching event indicates that a new audio device is added to the target terminal device, the device list is queried; if the device list only includes the new audio device, the new audio device is switched to.

[0058] In one implementation, the processing unit, when being used to obtain device information of a target terminal device in response to an audio operation request, is specifically used to:

[0059] In response to the audio operation request, obtaining device information of the target terminal device from the main thread of the target terminal device;

[0060] The audio control system is deployed on the target terminal device, the audio operation request is generated when the object performs a startup operation on the audio function provided by the target terminal device, and the main thread refers to the thread that executes the audio function of the target terminal device; or

[0061] The audio control system is deployed in the application running in the target terminal device. The audio operation request is generated when the object performs a startup operation on the target function provided by the application. The main thread refers to the thread that executes the target function in the application; the target function includes at least one of the following: live broadcast function, conference function, voice call function and music playback function.

[0062] On the other hand, an embodiment of the present application provides a terminal device, the device comprising:

[0063] a processor adapted to execute a computer program;

[0064] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned audio processing method is implemented.

[0065] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is suitable for being loaded by a processor and executing the audio processing method as described above.

[0066] On the other hand, an embodiment of the present application provides a computer program product, the computer program product comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. When the computer instructions are executed by a processor, the above-mentioned audio processing method is implemented.

[0067] In an embodiment of the present application, an audio control system is provided, in which configuration parameters corresponding to a plurality of terminal devices are pre-stored, and the configuration parameters corresponding to each terminal device are pre-configured to ensure the compatibility between the corresponding terminal device and the audio control system. In this way, the audio control system runs on different terminal devices (specifically, terminal devices equipped with different operations / platforms), and different terminal devices can realize audio acquisition and playback with the audio control system through the pre-stored corresponding configuration parameters. Taking any terminal device as a target terminal device as an example, when the audio control system receives an audio operation request sent by the target terminal device, the device information of the target terminal device can be obtained from the target terminal device in response to the audio operation request, and then the configuration parameters adapted to the terminal device can be determined from the audio control system based on the device information of the target terminal device, so that the audio operation requested by the audio operation request (such as the operation of audio acquisition or the operation of audio playback) can be executed based on the configuration parameters. It can be seen that the audio control system provided by the embodiment of the present application is a control system that supports cross-platform audio acquisition and playback services; when different terminal devices use the audio control system for audio acquisition and playback, it is possible to determine the appropriate configuration parameters for the corresponding terminal device from the configuration parameters pre-stored in the audio control system for audio acquisition and playback, so that the audio control system has the advantage of being compatible with different platforms and operating systems. In addition, compared to developing audio acquisition and playback services for different platforms, the one-time development of the embodiment of the present application to integrate configuration parameters that are compatible with different platforms in the audio control system can avoid repeated and multiple development of the basic service "audio acquisition and playback service", improve development efficiency, and effectively reduce development costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0069] Figure 1 is a structural schematic diagram of an audio control system provided by an exemplary embodiment of the present application;

[0070] Figure 2 is a schematic diagram of an audio processing scenario provided by an exemplary embodiment of the present application;

[0071] Figure 3 is a flowchart of an audio processing method provided by an exemplary embodiment of the present application;

[0072] Figure 4 This is a schematic diagram of an interface for starting an audio collection and playback service provided by an exemplary embodiment of the present application;

[0073] Figure 5 It is a flowchart of an audio collection and playback process provided by an exemplary embodiment of the present application;

[0074] Figure 6 is a flowchart of another audio processing method provided by an exemplary embodiment of the present application;

[0075] Figure 7 It is a schematic diagram of compatibility between a system version and an audio format provided by an exemplary embodiment of the present application;

[0076] Figure 8 is a schematic diagram of starting abnormality detection provided by an exemplary embodiment of the present application;

[0077] Fig. 9 is a schematic diagram of an interface for a user to specify a target audio device provided by an exemplary embodiment of the present application;

[0078] Fig.10 is a schematic diagram of an interface for a user to reselect an audio device provided by an exemplary embodiment of the present application;

[0079] Fig.11 This is a schematic diagram of an interface for prompting a non-audio device provided by an exemplary embodiment of the present application;

[0080] Fig.12 is a schematic diagram of detecting switching of an audio device provided by an exemplary embodiment of the present application;

[0081] Fig.13 This is a schematic diagram of an interface for a user to select an audio device from a setting interface provided by an exemplary embodiment of the present application;

[0082] Fig.14 It is a schematic diagram of an interface for prompting a user to switch audio devices provided by an exemplary embodiment of the present application;

[0083] Fig.15 is a structural schematic diagram of an audio processing device provided by an exemplary embodiment of the present application;

[0084] Fig.16 It is a structural diagram of a terminal device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0085] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0086] In an embodiment of the present application, an audio processing solution is provided, which can provide audio acquisition and playback services for different platforms and target functions (or called application functions, which can be simply referred to as functions).

[0087] in:

[0088] ① Audio collection and playback services include collection services (or audio collection services, audio collection) and playback services (or audio playback services, audio playback) for audio. Among them, audio can refer to sound waves in a certain sound frequency range (such as 20 Hz to 20 kHz) that can be heard by the human ear; audio covers all signals that can be perceived by the human ear, including human voice, music, noise, whistle sounds or machine tool sounds, etc. For example, conversation audio is involved in the call scenario using a communication device, and live audio is involved when watching live broadcasts using a smart device, etc.; the specific type of audio is not limited in the embodiments of the present application. The audio collection service can be simply understood as a service that collects the sounds of the surrounding environment through a built-in or external audio collection device (such as a microphone or a pickup) of the terminal device; at this time, the collected audio data (or voice signal, audio, etc.) may include the voice of the user in the environment, environmental noise, or the sound being played by the terminal device. Audio playback service can be simply understood as a service that plays audio data obtained from a remote end (such as received from other devices through the network) through a built-in or external audio playback device (such as a speaker) of a terminal device; such as the user's voice obtained from a remote end, or the sound of a video played over the Internet.

[0089] ② The platform involved in the embodiment of the present application may refer to the operating system installed on the terminal device. The operating system may include: Windows operating system, Mac / iOS operating system, Android operating system and Linux operating system, etc. Among them, according to the difference in the hardware structure of the terminal device, the operating system installed on the terminal device is not the same; for example, the Windows operating system is installed on a PC (Personal Computer) device; for example, the iOS operating system and the Android operating system are installed on a smart phone; and so on. Among them, the embodiment of the present application does not limit the terminal device and the operating system installed on the terminal device, which is specially explained here; the terminal device may include but is not limited to: smart phones, tablet computers, portable personal computers, mobile Internet devices (Mobile Internet Devices, MID), smart TVs, vehicle-mounted devices, head-mounted devices and aircraft and other devices. For example, a smart phone can play music and record voice; for example, the vehicle-mounted device can be a map navigation device installed on a vehicle, so that the map navigation device can collect questions from people in the car and play corresponding answers to the questions (such as the broadcast of a certain path, etc.).

[0090] ③ The target function involved in the embodiment of the present application may refer to a function with audio collection and playback requirements. The target function may include but is not limited to: VoIP (Voice over Internet Protocol, voice transmission based on IP) function (IP is the Internet Protocol, the full name is Internet Protocol), live broadcast function, conference function, voice call function, intelligent question and answer function, music playback function, etc.; these target functions are provided by the application installed in the terminal device (such as the application is a music application, the music application can play music, and can also record the user's singing voice, etc.); the same application may have one or more target functions. Among them, the application installed in the terminal device may refer to a computer program for completing one or more specific tasks. By classifying the application according to different dimensions (such as the operation mode and function of the application), the types of the same application in different dimensions can be obtained. According to the classification of the operation mode of the application, the application may include but is not limited to: a client installed in the terminal, a small program that can be used without downloading and installing, a web application opened through a browser, etc. According to the functional type of the application, the application may include but not limited to: IM (Instant Messaging) application, content interaction application, etc.; among them, the instant messaging application refers to the application for instant communication of messages and social interaction based on the Internet, and the instant messaging application may include but not limited to social applications, map applications with audio collection and playback capabilities, game applications, music applications, browser applications, etc. Content interaction applications refer to applications that can realize content interaction, such as personal space, news and other applications.

[0091] In actual applications, audio acquisition and playback services, as basic services for different target functions or platforms, need to be customized and developed according to the function type and platform type (such as the type of operating system); this makes developers face the problem of repeated development for different target functions or different platforms. For example, if the same application has two functions (such as live broadcast function and conference function), then it is necessary to develop corresponding audio acquisition and playback services for the two functions according to the function types of the two functions, that is, it is necessary to develop two sets of audio acquisition and playback services; when any of the two functions is started in the application, the audio acquisition and playback service corresponding to the function is called to realize audio acquisition and audio playback. For another example, when the same target function is deployed to different platforms (specifically, the application with the target function is installed on different platforms), in order to enable the target function to adapt to the control logic of the platform, it is necessary to develop different audio acquisition and playback services for the target function according to the platform type, so that audio acquisition and audio playback can be realized when the target function is deployed to a certain platform.

[0092] In order to ensure the cross-platform compatibility of audio acquisition and playback services and avoid the cost pressure caused by repeated development, the audio processing scheme provided in the embodiment of the present application proposes a cross-platform real-time audio acquisition and playback integrated control system (or simply referred to as an audio control system). The audio control system is used to provide audio acquisition and playback services for terminal devices, specifically by providing a unified external input and output interface for different function calls, and can provide audio acquisition services and audio playback services for different applications and businesses on different platforms, with good cross-platform compatibility, thereby ensuring that the audio control system can be applied to different target functions and platforms. In addition, the audio control system is also equipped with some other services to provide reliable audio acquisition and playback services for target functions; for example, in the system external interface, the startup mode, the system interface (API) and the audio format of the call can be configured; for example, in the case of changes or disconnections of audio devices (such as microphones for audio acquisition, or speakers for audio playback, etc.), the logic of device switching can be entered to switch to the audio device that works normally; for example, when a startup exception occurs, the audio control system can also enter the exception handling logic to perform exception handling, reporting and log printing.

[0093] For example, the structural diagram of the audio control system involved in the embodiment of the present application can be found in Figure 1 .like Figure 1 As shown, the general structure of the audio control system may include: a platform interface layer and an internal logic control layer (i.e., WwkDevice). In detail, the internal logic control layer of the audio control system is connected to the functional layer (or application layer) of the terminal device, so that the audio control system can provide audio acquisition and playback services for different functions in the functional layer, achieving the purpose of cross-platform and cross-functionality, so that there is no need to customize and develop different audio acquisition and playback services for different functions. In addition, the internal logic control layer and the platform interface layer inside the audio control system are interconnected, and the platform interface in the platform interface layer can upload data in the terminal device to the internal logic control layer, and the internal logic control layer can download relevant data of the audio acquisition and playback service to the platform interface layer, so that the platform interface layer can send it to the terminal device through the platform interface that is compatible with the terminal device for corresponding processing. The following is an introduction to the various layers included in the audio control system, among which:

[0094] 1) Platform interface layer. The platform interface layer integrates platform interfaces corresponding to different platform types; when the audio control system is installed on different platforms, the audio control system can communicate and transmit through the platform interface in the platform interface layer that is adapted to the corresponding platform. Among them, the correspondence between the platform interface and the platform type (i.e., the type of operating system) in the platform interface layer may include but is not limited to: when the terminal device is a PC, specifically when the platform type installed on the terminal device is a windows operating system, a Core Audio interface and a Wave interface are provided in the platform interface layer; in this way, the windows operating system and the audio control system can communicate and transmit through the Core Audio interface and the Wave interface. When the platform type of the platform is Mac / iOS (or it is understood that the system type of the operating system deployed in the platform is Mac / iOS), an Audio unit interface is provided in the platform interface layer; in this way, the Mac / iOS operating system and the audio control system can communicate and transmit through the Audiounit interface. When the platform type of the platform is Android (or understood as the operating type of the operating system deployed in the platform is Android), the platform interface layer is provided with an AAudio interface and an OpenSL ES interface; thus, the Android operating system and the audio control system can communicate and transmit via the AAudio interface and the OpenSL ES interface. When the platform type of the platform is Linux (or understood as the operating type of the operating system deployed in the platform is Linux), the platform interface layer is provided with a PulseAudio interface; thus, the Linux operating system and the audio control system can communicate and transmit via the PulseAudio interface.

[0095] 2) Internal logic control layer. The internal logic control layer is the main layer for the audio control system to implement the audio processing solution provided by the embodiment of the present application; in other words, the internal logic control layer includes most of the processing or control logic for implementing the audio processing solution. Specifically, the embodiment of the present application supports encapsulating the control or processing logic of different tasks (such as audio acquisition, audio playback or switching of audio devices, etc.) in the internal logic control layer into modules; such as logic control modules (such as for implementing logic control of different platforms or operating systems), audio processing modules (such as for audio processing of collected audio data, so that the processed audio data meets the acquisition requirements), device detection modules (mainly used to implement the management of audio devices carried in terminal devices) and exception handling modules (such as processing logic when startup exceptions are detected), etc. Further, it supports the use of c++ to encapsulate a unified external interface for the internal logic control module, and the external interface can be uniformly exported as a functional interface of c language type; in this way, the audio control system can perform a task by calling the functional interface in the internal logic control layer, simplifying the internal structure of the audio control system to a certain extent, and ensuring that the audio control system can expand other tasks by encapsulating the interface, and improving the generalizability of the functions provided by the audio control system. The following is an exemplary introduction to each module or module interface in the internal logic control layer:

[0096] ① The parameter configuration interface can be implemented as different platforms or terminal devices sending out corresponding configuration parameters to realize the audio control system compatibility with different platforms or terminal devices; wherein, the audio control system pre-stores configuration parameters corresponding to a variety of terminal devices (specifically, a variety of platforms), and the configuration parameters corresponding to each of the multiple terminal devices are pre-configured to the audio control system to ensure the compatibility between the corresponding terminal device and the audio control system. ② The audio acquisition start / end interface can be used to realize audio acquisition. ③ The audio playback start / end interface can be used to realize audio playback. ④ The audio system replay start / end interface is used to share other sounds being played in the terminal device; for example, in the process of using application 1 in the terminal device for audio acquisition, if application 2 in the terminal device is playing audio, then the audio system replay start / end interface can be used to control whether application 1 needs to collect the audio played by application 2 when collecting audio. ⑤ The volume control interface can be used to realize volume control during audio playback. ⑥ The device list query interface can be used to query the audio devices carried in the terminal device, and the audio devices here can include devices for realizing audio acquisition and devices for realizing audio playback. ⑦ The device selection interface can be used to implement the selection or switching of the audio device installed in the terminal device. ⑧ The log interface can be used to record various events in the terminal device, such as the successful startup log of the audio device, or the failed startup log of the audio device, etc. ⑨ The callback interface can be used to implement the callback of audio data; such as obtaining the voice signal of the remote user from the remote end, etc. It should be understood that the modules and interfaces given above are exemplary modules and interfaces of the internal control logic layer; in actual applications, the modules or interfaces can be changed in the internal control logic layer according to the needs of the audio acquisition and playback service, and there is no limitation on this.

[0097] Based on the above Figure 1The structure of the audio control system is introduced below. Taking the case where the audio control system is deployed in an application program, and the application program is installed in a target terminal device (such as any of the various terminal devices equipped with different operating systems mentioned above), and the user initiates a VoIP call as an example, the general principle of the audio processing solution provided by the embodiment of the present application is briefly introduced. In a specific implementation, when the user is using the target terminal device, the user can initiate a VoIP call through the target terminal device, so that the audio control system can be started; after the audio control system is successfully started, the device information transmitted by the terminal device can be parsed in the internal logic control module in the audio control system, and according to the platform type, system version and configuration parameters of the terminal device that are compatible with the target terminal device, the platform interface layer that matches the platform type of the operating system equipped with the target terminal device is called, and the audio acquisition and playback thread (such as the audio acquisition thread or the audio playback thread) is started in the target terminal device, and the callback function is registered to realize the acquisition or playback of the audio.

[0098] Furthermore, when the audio device in the terminal device collects audio data, the audio data can be reported to the audio control system, so that the audio control system can perform audio processing (such as howling suppression, echo elimination and noise reduction) on the audio data through the audio processing module, and then send the processed audio data to the application layer (i.e., the application layer) through the callback function. Figure 1 Functional layer in the audio control system). At the same time, the audio control system can also receive remote audio signals from the application layer of the terminal device, and send them to the audio device (such as a speaker) installed in the terminal device for audio playback through the corresponding platform interface in the platform layer. Furthermore, if the audio device changes during the above-mentioned audio acquisition or playback process, the device detection module can handle the corresponding event to ensure that the internal logic control layer WwkDevice always uses the correct audio device for audio acquisition and playback. In addition, when an exception occurs during the startup of the audio device or the callback status of the callback function is abnormal, the exception handling module can enable the exception handling logic, delay or change the device to restart the audio acquisition and playback, report the exception through the callback, and record the status through the log system.

[0099] In summary, on the one hand, the audio control system provided by the embodiment of the present application provides services such as device detection, exception handling and audio processing, and specifically encapsulates functional modules corresponding to different services (such as logic control models or audio processing modules, etc.) within the audio control system; on the basis of the audio control system providing audio acquisition and audio playback services, functions such as audio device management, abnormal situation handling and audio optimization are realized, which effectively improves the ability of the audio control system to generalize functions. On the other hand, the audio control system supports cross-platform provision of audio acquisition and playback services, and can realize audio acquisition and playback when different terminal devices use the audio control system for audio acquisition and playback, thereby achieving cross-platform and cross-functional use of the same audio control system, greatly ensuring the compatibility of the audio control system, and effectively reducing the development cost of audio acquisition and playback services. Among them, the cross-platform of the audio control system is specifically embodied in: in order to prevent the functional modules encapsulated inside the audio control system (or the audio acquisition and playback services provided by the audio control system) from being repeatedly developed according to the platform type, the embodiment of the present application not only deploys configuration parameters corresponding to different platforms (such as audio formats, etc.) in each functional module, but also sets a platform interface layer for the audio control system, in which platform interfaces corresponding to different platforms are deployed (such as the aforementioned windows system is provided with Core Audio interface and Wave interface). In this way, when the application program deployed with the audio control system is installed or run on a terminal device of any platform, and when the audio control system is called to interact with the terminal device to realize audio acquisition and playback, communication and transmission can be carried out through the platform interface in the platform interface layer of the audio control system that is compatible with the platform of the terminal device, which makes it unnecessary to customize and develop audio acquisition and playback services for deploying applications to different platforms, such as customizing audio acquisition and playback services that are compatible with the iOS system for applications, and customizing audio acquisition and playback services that are compatible with the windows system for applications, thereby ensuring that audio acquisition and playback services can be provided when the application is deployed to different platforms, realizing cross-platform audio acquisition and playback integrated control, and avoiding the cost of repeated development for different platforms.

[0100] The audio processing solution provided in the embodiment of the present application can be applied to any application scenario that requires audio acquisition and playback services. For example, application scenarios that require VoIP functional requirements may include but are not limited to: conference scenarios that require calls based on IP (i.e., VoIP), live broadcast scenarios, and voice call scenarios, etc.; for another example, application scenarios that can perform voice acquisition / playback may include but are not limited to: intelligent question-and-answer scenarios, in-car navigation scenarios, or shopping mall robot intelligent question-and-answer scenarios, etc. The embodiment of the present application does not exhaustively list the application scenarios to which the audio processing solution is applicable. The following takes the application scenario of the live broadcast scenario as an example to briefly introduce the audio processing system involved in the live broadcast scenario.

[0101] For example, the schematic diagram of the audio processing scenario in the live broadcast scenario can be as follows: Figure 2 As shown, the audio processing system may include a terminal device 201 used by the anchor and a terminal device 202 used by the audience watching the live broadcast. The embodiment of the present application does not limit the number of terminal devices 202; of course, the live broadcast service is often provided by the application with live broadcast function deployed in the terminal device 201 and the terminal device 202, so the audio processing system also includes a server 203 corresponding to the application. Among them, the types of terminal devices 201 and terminal devices 201 in the above description, as well as the types of applications, can be found in the above-mentioned related descriptions, and are not repeated here. Server 203 can be a background server of an application with live broadcast function, which is used to interact with the terminal device running the application to provide computing and application service support for the application; specifically, server 203 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. Server 203 can include but is not limited to: data processing servers, Web servers, application servers, cloud servers (or simply referred to as cloud servers), and other devices with complex computing capabilities. Among them, the terminal device 201 and the terminal 202 can be directly or indirectly connected to the server 203 through wired or wireless methods. The embodiment of the present application does not limit the connection method between the terminal device and the server.

[0102] like Figure 2 In the live broadcast scenario shown, the audio control system provided by the embodiment of the present application is deployed in the application running in the terminal device 201 used by the anchor; in this way, when the anchor initiates a live broadcast demand through the application, the audio control system can be started to provide audio acquisition and playback services. Specifically, after the terminal device 201 collects the anchor's voice signal (or audio data) through the audio device, the voice signal can be transmitted to the audio control system for audio processing, and the processed audio data is transmitted to the application layer so that the remote audience can play the audio through the terminal device 202 used. Similarly, when the anchor and the audience are connected, the audience can initiate audio acquisition in the terminal device 202 used by them, and the terminal device 202 also performs similar operations to the terminal device 201, and transmits the audience's audio data to the anchor side for audio playback.

[0103] In the above process, considering that the audio control system provided by the embodiment of the present application has the advantage of being cross-platform, when the same application developed and deployed with the audio control system is deployed on different platforms, the audio control system can provide audio collection and playback services on different platforms. Figure 2The terminal devices held by the anchor and the audience shown in the figure do not need to consider whether the platforms of the terminal devices held by the anchor and the audience are the same, that is, it is not necessary to require Figure 2 The platform type of the terminal device 201 used by the anchor must be the same as the platform type of the terminal device 202 used by the audience, so as to achieve the purpose and effect of cross-platform audio collection and playback, and effectively expand the promotion of the application.

[0104] Furthermore, it can be seen from the above description that the audio control system provided in the embodiment of the present application also provides a device detection function, etc. The device detection function means that the audio control system can detect changes in the audio devices (such as the audio acquisition device "microphone" or the audio playback device "speaker") installed on the terminal device, and can simultaneously match the audio devices before and after the change. Figure 2 In the live broadcast scenario shown, if the microphone on the terminal device held by the anchor fails, the anchor can use a new microphone to access the terminal device; at this time, the device detection module encapsulated inside the audio control system can detect the device update event in time, and send down the configuration parameters that are compatible with the new microphone, so that the audio control system can call the new microphone to realize audio collection. It can be seen that by encapsulating various functions (such as device detection function) in the audio control system, and interacting with the outside world through a unified interface, the audio control system can realize device detection when deployed on different platforms, without having to develop corresponding device detection programs for different platform adaptability. This not only saves development costs, but also can provide reliable and stable audio collection and playback services for functions such as VoIP and live broadcast.

[0105] Furthermore, from the above description, it can be known that the audio control system provided in the embodiment of the present application can also provide audio collection and playback services for different functions, such as live broadcast function and conference function, which can call the audio control system to provide audio collection and playback services. Figure 2 In the live broadcast scenario shown, if the live broadcast application scenario used by the anchor also provides a conference function, and the anchor needs to enter the conference to speak during the live broadcast, then the anchor can maintain or exit the live broadcast screen, and after entering the conference interface, call the audio control system through the conference function to perform audio collection or audio playback. It can be seen that the audio control system supports providing audio collection and playback services for different functions of the same application, ensuring the reliability and stability of the operation of various functions in the application.

[0106] Based on the above brief introduction to the audio processing solution provided in the embodiment of the present application and the application scenarios to which the solution is adapted, the embodiment of the present application further needs to explain the following points:

[0107] ① The above Figure 2This is only an exemplary audio processing system involved in the embodiments of the present application and does not limit the embodiments of the present application. For example, in a vehicle navigation scenario, a vehicle occupant initiates an inquiry (such as route query) through a vehicle application and does not involve Figure 2 Among the multiple terminal devices in the vehicle, there will only be a single terminal device installed in the vehicle; that is, the audio collection and audio playback are realized by the single vehicle-mounted device installed in the vehicle. At this time, there is no logic in the audio processing system that audio data is transmitted from one terminal device to another terminal device.

[0108] ② The collection and processing of relevant data in the embodiments of this application should be strictly in accordance with the requirements of relevant laws and regulations. The acquisition of personal information requires the knowledge or consent of the individual subject (or the legal basis for obtaining the information), and the subsequent use and processing of data shall be carried out within the scope of authorization of laws and regulations and the subject of personal information. For example, when the embodiments of this application are applied to specific products or technologies, such as when collecting the user's voice signal, the user's permission or consent is required, and the collection, use and processing of relevant data (such as the collection and release of the barrage posted by the object, etc.) need to comply with the relevant laws, regulations and standards of the relevant regions.

[0109] Based on the audio processing scheme described above, the present application embodiment proposes a more detailed audio processing method. The audio processing method proposed in the present application embodiment will be described in detail below in conjunction with the accompanying drawings. Figure 3 , Figure 3 A flowchart of an audio processing method provided by an exemplary embodiment of the present application is shown; the audio processing method can be executed by the aforementioned target terminal device deployed with an audio control system, and the audio control system can be directly deployed on the target terminal device or indirectly deployed on the target terminal device through an application. The method may include steps S301-S304:

[0110] S301: receiving an audio operation request sent by a target terminal device.

[0111] The audio operation request, also known as an audio operation event, an audio operation event message, etc., is used to request the audio control system running in the target terminal device to perform an audio operation on the audio acquisition and playback service. As described above, the audio acquisition and playback service includes an audio acquisition service and an audio playback service; therefore, when a single audio operation request is used to request the audio control system to perform an audio operation on the audio acquisition service, the audio operation here is an audio acquisition operation, or, when it is used to request the audio control system to perform an audio operation on the audio playback service, the audio operation here is an audio playback service. The embodiment of the present application does not limit the specific operation content of the audio operation requested by the audio operation request in a single audio acquisition and playback service.

[0112] In a specific implementation, when a user has a need for audio collection or audio playback, the user can use the target terminal device to open and start the audio control system in the target terminal device; for example, the user opens an application in the target terminal device and performs a user operation for audio collection or playback (such as a long press operation, a click operation, or a double-click operation, etc.) in the service interface provided by the application. Then, the target terminal device detects the user operation performed by the user in the service interface, and determines that the user has a need for audio collection or audio playback, and then an audio operation request can be generated based on the user operation. In this way, the target terminal device can report the audio operation request to the audio control system, and at this time the audio control system determines that it has received the audio operation request sent by the target terminal device.

[0113] It should be understood that the application scenarios involved in starting the audio collection and playback service from different applications may be different, and the application scenarios involved in starting the audio collection and playback service from the same application may also be different; the present application embodiment does not limit the application scenarios when starting the audio collection service or audio playback service in the application in the above description process. Figure 4 As shown, assuming that a social application is installed in the target terminal device, the social application includes multiple social sessions corresponding to the user; if the user has a need to send voice in one of the multiple social sessions, the user can trigger the icon 401 of the social application in the main interface of the target terminal device to open the social application, and further open the session page 402 of the social session, and start the audio collection service by long pressing the voice button 403 (or button, option, control, etc.) in the session page 402. At this time, when the target terminal device detects the user's long press operation on the voice button 403, it generates an audio operation request based on the long press operation and sends it to the social application, so as to be sent to the audio control system deployed in the social application.

[0114] S302: Acquire device information of the target terminal device in response to the audio operation request.

[0115] S303: According to the device information of the target terminal device, determine the configuration parameters adapted to the target terminal device from the audio control system.

[0116] In steps S302-S303, it can be seen from the above description that the audio control system pre-stores configuration parameters corresponding to a variety of terminal devices, specifically configuration parameters corresponding to a variety of platform types or operating systems; therefore, after the audio control system receives the audio operation request sent by the target terminal device, it is necessary to obtain the device information of the target terminal device, so as to match the configuration parameters adapted to the target terminal device from the audio control system according to the device information of the target terminal device, so as to provide audio acquisition and playback services for the target terminal device. It is precisely because the embodiment of the present application pre-sets the configuration parameters adapted to different terminal devices into the audio control system that the audio control system provided by the embodiment of the present application can adapt to different terminal devices and provide audio acquisition and playback services for terminal devices of different platforms or operating systems, so as to ensure the cross-platform nature of the audio control system.

[0117] In a specific implementation, when a user has a need to use a target function (such as a VoIP function), the target terminal device will generate an audio operation request based on the need, and send the audio operation request to the audio control system. At this time, the audio control system can obtain the device information of the target terminal device from the main thread of the target terminal device in response to the audio operation request, so as to determine the platform type and system version corresponding to the target terminal device based on the device information of the target terminal device, and set and parse the configuration parameters of the target terminal device. It is worth noting that the audio operation request and the main thread mentioned above may be different depending on the deployment location of the audio control system. Optionally, if the audio control system is directly deployed on the target terminal device, such as the target terminal device comes with an audio control system, then the audio operation request can be generated by the user (or object) performing a startup operation on the audio function provided by the target terminal device (such as the audio playback function when setting the alarm clock), and the main thread can refer to the thread that executes the audio function of the target terminal device. Optionally, if the audio control system is deployed in an application running in the target terminal device, such as a social application deployed in the target terminal device, then the audio operation request is generated when the object performs a startup operation on the target function provided by the application, and the main thread may refer to the thread that executes the target function in the application (such as a live broadcast function, a conference function, or a voice call function, etc.).

[0118] Among them, the device information of the target terminal device can be roughly divided into two categories, namely hardware information and software information. Among them: ① Hardware information can be used to indicate the information of the hardware devices deployed in the target terminal device; for example, if the hardware devices in the target terminal device include audio devices (such as microphones for audio acquisition, or speakers for audio playback, etc.), the hardware information included in the device information of the target terminal device can include sub-device information of the audio device, such as the audio sampling rate, number of audio channels, total number and distribution location of the audio device "microphone". ② Software information can be used to indicate the system information of the operating system deployed in the target terminal device; in detail, the software information may include but is not limited to: the platform type of the operating system installed on the target terminal device (such as Windows operating system or Linux operating system, etc.) and the system version of the operating system installed on the target terminal device; the platform type of the operating system installed on the target terminal device and the system version of the operating system installed on the target terminal device refer to the version information of the operating system installed on the target terminal device, such as Windows 10 and Windows 11 are two different system versions of the same operating system. The higher the system version, the better the performance.

[0119] Furthermore, configuration parameters adapted to the target terminal device are determined from the audio control system based on the device information of the target terminal device; that is, the configuration parameters adapted to the target terminal device can be understood as being pre-set based on the hardware information and software information of the target terminal device and stored in the audio control system. Configuration parameters may include but are not limited to: a target platform interface adapted to the target terminal device, and a target audio format compatible with the system version, etc. Among them: ① For the target platform interface adapted to the target terminal device, the determination method can be summarized as follows: according to the platform type of the operating system equipped by the target terminal device, a target platform interface with the same platform type as the platform type of the operating system equipped by the target terminal device is selected in the platform interface layer of the audio control system; in this way, the target platform interface can be used as a system interface (or simply referred to as an interface) for communication and transmission between the target terminal device and audio control information. For example, if the platform type of the operating system equipped by the target terminal device is Android, then Figure 1The target platform interface determined from the platform interface layer of the audio control system is shown as an AAudio interface or an OpenSL ES interface. ② For the target audio format compatible with the system version, the determination method can be summarized as follows: according to the system version of the operating system carried by the target terminal device, the target audio format compatible with the system version is obtained from the audio control system; the target audio format is pre-configured according to the coordination between the modules within the audio control system, and the compatibility between the audio control system and the system version. In other words, in order to enable better coordination between the modules within the audio control system, and to ensure better compatibility between the audio control system and different system versions, to improve the compatibility between the audio control system and different system versions, and to ensure that the audio control system is cross-platform; the embodiment of the present application will pre-set one or more audio formats to be stored in the audio control system, so that the audio control system can select a target audio format that is more compatible with the audio control system and the system version according to the platform type and system version for audio acquisition and playback. Among them, the target audio format (or any audio format) may include at least one of the following: audio sampling rate (such as a sampling rate of 16 kHz), audio data bit depth (such as an integer of 16-bit depth, int16), number of audio channels (such as multiple audio channels can form a stereo model or surround sound mode, etc.), and the transmission interval of the callback function corresponding to the audio data (such as transmitting data once every 20ms), etc. It should be understood that the embodiment of the present application does not limit the device information of the target terminal device and the specific types of configuration parameters adapted to the target terminal device; the above is only an exemplary introduction to the device information and configuration parameters.

[0120] S304: Execute the audio operation requested by the audio operation request according to the configuration parameters adapted to the target terminal device.

[0121] After determining the configuration parameters that are compatible with the target terminal device from the audio control system based on step S303, the target terminal device can start the audio acquisition and playback thread based on the configuration parameters, so as to perform audio acquisition or audio playback operations in the audio acquisition and playback thread; specifically, the implementation class of the target platform interface corresponding to the target terminal device is instantiated and called based on the platform type and configuration parameters, so as to start the audio acquisition and playback thread and register the callback function in the target terminal device, thereby realizing audio acquisition and playback.

[0122] In the specific implementation, the audio control system obtains the implementation class of the target platform interface according to the target platform interface included in the configuration parameters; wherein, the implementation class is for the interface, and is mainly used to implement the logic of the content agreed in the interface. In the embodiment of the present application, the implementation class is used to indicate the service logic of the audio control system to provide audio acquisition and playback services through the target platform interface. Then, the implementation class of the target platform interface is instantiated and the callback function is registered (that is, the callback function is set for the object during the instantiation process, such as setting the callback function in the audio acquisition logic to obtain the collected second audio data from the audio device) to start the audio acquisition and playback thread. Among them, the implementation class (or simply referred to as the class) is an abstraction of things with common attributes and behaviors. In order to realize the class, the implementation class needs to be instantiated to convert the abstract implementation class into a real entity (that is, an object) that can be seen. The object is used to implement the function of the class, that is, the instantiation of the implementation class can be simply understood as the process of generating an object of the implementation class. A callback function is a function that can be passed as a parameter. In simple terms, a callback function allows a function to be passed as a parameter to another function, and the callback function is automatically called after the other function completes its task. The callback function can improve the reusability of the code, and in the embodiment of the present application, the development cost can be reduced and the flexibility of development can be increased. Finally, the audio operation requested by the audio operation request is executed in the started audio acquisition and playback thread, such as executing the audio operation of audio acquisition or audio playback in the audio acquisition and playback thread.

[0123] Furthermore, depending on the audio operation requested by the audio operation request, the specific implementation process of executing the audio operation requested by the audio operation request in the audio acquisition and playback thread is different. The following introduces the specific implementation process of the audio operation requested by the audio operation request for audio acquisition and audio playback respectively; wherein:

[0124] (1) The audio operation request is used to request the audio control system to perform an audio operation of audio playback; in other words, the audio operation is an operation of using the audio device "speaker" to perform audio playback; in this implementation, the audio acquisition and playback thread used to implement audio playback is represented as the audio playback thread. In the specific implementation, in the audio playback thread, the audio control system can obtain the first audio data to be played from the remote end (i.e., the end that sends the audio signal or voice signal) through the callback function registered when starting the audio playback thread; when the callback function obtains the first audio data, it specifically obtains it according to the transmission interval time of the callback function. If the transmission interval time is 20ms, the callback function obtains the first audio data from the remote end through the network (or the Internet) every 20ms. Then, the audio control system sends the first audio data to the audio device in the target terminal device (the audio device is a speaker at this time) for playback to implement the audio playback operation.

[0125] In addition, considering that the target terminal device is performing audio collection while playing the first audio data, the second audio data collected by the audio includes both the original sound to be collected and the echo (i.e., the echo of the first audio data); under such a vicious cycle, the echo in the collected second audio data will increase, a buzzing sound will appear, and the quality of the second audio data will be reduced. In order to eliminate the echo in the second audio data and improve the audio quality of the second audio data, the embodiment of the present application supports caching the first audio data to be played into the cache space in the audio control system when the callback function is used to obtain the first audio data to be played in the audio playback thread. The cache space can be understood as a buffer set in the audio control system for caching remote audio data. The first audio data in the cache space is used as an echo cancellation reference signal in the audio collection process.

[0126] (2) The audio operation request is used to request the audio control system to perform an audio operation of audio collection; in other words, the audio operation is an operation of using the audio device "microphone" to collect audio; in this implementation, the audio collection and playback thread used to implement audio collection is represented as the audio collection thread. In a specific implementation, in the audio collection thread, the audio control system can obtain the second audio data collected by the audio device of the target terminal device through the callback function registered when the audio collection thread is started; specifically, the second audio data is collected from the audio device of the target terminal device according to the transmission time interval (such as 20ms) through the callback function, and the second audio data collected here includes a near-end voice signal (such as a voice signal emitted by the user holding the target terminal device), a far-end voice signal (such as the first audio data being played by the target terminal device) and environmental noise (such as the noise in the physical environment where the user holding the target terminal device is located). Then, the audio control system uses its internal audio processing module to perform audio processing on the second audio data to obtain processed second audio data; wherein, an audio processing algorithm is deployed in the audio processing module to achieve howling suppression, noise reduction, echo cancellation and other operations on the second audio data through the audio processing algorithm to improve the audio quality of the second audio data; the embodiment of the present application does not limit the specific algorithm content of the audio processing algorithm, such as the audio processing algorithm is a self-developed algorithm. Finally, the processed second audio data is transmitted to the remote end through the callback function for audio playback; specifically, the processed second audio data is transmitted to the application layer of the target terminal device through the callback function, and the second audio data is transmitted to the remote user through the application layer.

[0127] It can be seen that the exemplary process of audio collection and playback shown in the above steps S302-S304 can be referred to Figure 5 .like Figure 5As shown, the object / user holding the target terminal device can use the target terminal device to start the target function with audio collection / playback capabilities, such as initiating a VoIP call in a social application; at this time, the audio control system obtains the device information of the target terminal device in the main thread of the target terminal device to determine the platform type and system version of the operating system carried by the target terminal device. Then, based on the device information, the configuration parameters adapted to the target terminal device are obtained from the audio control system; for example, if the target terminal device is deployed with a windows operating system, then it is determined to use the coreaudio interface as the interface for communication and transmission between the audio control system and the target terminal device, and set the audio sampling rate to 16k Hz, stereo mode, and the callback function to transmit 20ms of data each time. Then, the audio control system instantiates and calls the implementation class of the target platform interface that matches the target terminal device according to the platform type and configuration parameters, starts the audio collection and playback thread (specifically the audio collection thread or the audio playback thread) and registers the callback function to realize audio collection / audio playback in the audio collection and playback thread. Furthermore, within the audio playback thread, the audio control system can obtain the first audio data of the remote user through the network every 20 ms through the callback function, and send it to the speaker of the target terminal device for audio playback, while storing the first audio data in the buffer of the remote data as an echo cancellation reference signal during the audio acquisition process. Similarly, within the audio acquisition thread, the audio control system collects the second audio data from the microphone every 20 ms through the callback function. The collected second audio data includes the near-end voice, the echo of the far-end signal, and the environmental noise; and uses the self-developed algorithm in the audio processing module in the audio control system to perform howling suppression, noise reduction, and echo elimination operations on the second audio data to obtain the processed second audio data; and then transmits the processed second audio data to the application layer through the callback function, so that it can be transmitted to the remote user through the application layer.

[0128] To summarize, the embodiments of the present application support the pre-deployment of configuration parameters compatible with different terminal devices in the audio control system, so that the audio control system has the advantage of being compatible with different platforms and operating systems; compared with developing audio acquisition and playback services for different platforms, this method of integrating configuration parameters compatible with different platforms in the audio control system can avoid repeated and multiple development of audio acquisition and playback services for different platforms or functions, thereby improving development efficiency and effectively reducing development costs.

[0129] See also Figure 6 , Figure 6A flowchart of another audio processing method provided by an exemplary embodiment of the present application is shown; the audio processing method can be executed by the aforementioned target terminal device deployed with an audio control system, and the audio control system can be directly deployed on the target terminal device or indirectly deployed on the target terminal device through an application. The method may include steps S601-S605:

[0130] S601: Receive an audio operation request sent by a target terminal device.

[0131] S602: Acquire device information of the target terminal device from the target terminal device in response to the audio operation request.

[0132] S603: According to the device information of the target terminal device, determine the configuration parameters adapted to the target terminal device from the audio control system.

[0133] It should be noted that the specific implementation process shown in steps S601-S603 can be found in the aforementioned Figure 3 The relevant description of the specific implementation process shown in steps S301-S303 in the illustrated embodiment is not repeated here.

[0134] S604: Acquire the implementation class of the target platform interface included in the configuration parameters, instantiate the implementation class of the target platform interface, and register a callback function to start an audio acquisition and playback thread.

[0135] S605: Execute the audio operation requested by the audio operation request in the audio acquisition and playback thread.

[0136] It should be noted that the specific implementation process shown in steps S604-S605 can be found in the aforementioned Figure 3 The description of the relevant contents shown in step S304 in the illustrated embodiment is not repeated here.

[0137] It is worth noting that in order to ensure the stability and compatibility of the audio acquisition and playback service, it is also necessary to ensure that the system version of the target terminal device and the audio format of the audio data (such as the first audio data and the second audio data) are compatible. Based on this, the embodiment of the present application also supports audio format conversion during the audio acquisition and audio playback process mentioned in step S304 to ensure that the audio format supported by the target terminal device and the audio format supported by the audio control system are compatible and matchable, thereby ensuring the normal operation of the audio acquisition and playback service. The implementation process of audio format conversion during audio acquisition and audio playback is introduced below, wherein:

[0138] (1) Audio format conversion during audio playback. In a specific implementation, after obtaining the first audio data to be played through the callback function, it is detected whether the system version of the operating system installed on the target terminal device reaches the version threshold. If the system version of the operating system installed on the target terminal device reaches the version threshold, it indicates that the target terminal device has a higher system version, and the higher system version often has a better audio format conversion capability. Then, the audio control system can directly use the target audio format pre-set by the audio control system to start the audio acquisition thread, and use the sampling conversion module (or audio conversion sampling module, etc.) of the target terminal device to achieve audio format matching inside the operating system. Specifically, before the target terminal device plays the first audio data, the sampling conversion module deployed in the target terminal device converts the first audio data into an audio format; the audio format of the first audio data after the audio format conversion is the same as the audio format required by the target terminal device. In more detail, the audio control system sends the first audio data that conforms to the target audio format to the target terminal device; in this way, the target terminal device uses the sampling conversion module provided by the target terminal device to convert the first audio data from the target audio format to the default audio format used by the audio device in the target terminal device; then, the audio device in the target terminal device plays the first audio data after the audio format conversion. On the contrary, if the system version of the operating system installed in the target terminal device does not reach the version threshold, indicating that the target terminal device does not have a higher system version, the audio control system uses the sampling conversion module deployed internally to convert the first audio data into an audio format, and sends the first audio data after format conversion to the audio device in the target terminal device for playback.

[0139] (2) Audio format conversion during audio acquisition. In a specific implementation, after obtaining the collected second audio data from the audio device "microphone" in the target terminal device through the callback function, it is detected whether the system version of the operating system installed on the target terminal device reaches the version threshold. If the system version of the operating system installed on the target terminal device reaches the version threshold, it indicates that the target terminal device has a higher system version, and the higher system version often has a better audio format conversion capability, then the capture is triggered to perform audio processing on the second audio data to obtain the processed second audio data; wherein, before the second audio data is called back to the audio control system, the sampling conversion module in the target terminal device performs audio format conversion on the second audio data, and the audio format of the converted second audio data is the target audio format required by the audio control system. That is, in the case where the system version of the target terminal device is higher, the sampling conversion module in the target terminal device can automatically perform audio format conversion on the collected second audio data, and the audio format of the second audio data after the audio format conversion is the same as the audio format required by the audio control system; that is, the audio control system directly obtains the second audio data that conforms to the target audio format, and the audio control system needs to perform audio format conversion on the second audio data.

[0140] On the contrary, if the system version of the operating system installed in the target terminal device does not reach the version threshold, the sampling conversion module deployed in the audio control system is used to convert the audio format of the second audio data, and the audio processing of the second audio data is triggered to obtain the processed second audio data. In other words, if the system version of the target terminal device is low, indicating that the target terminal device does not have a good audio format conversion capability, the target terminal device directly sends the second audio data collected by the audio device that conforms to the default audio format of the audio device to the audio control system, and the audio control system uses the sampling conversion module deployed internally to convert the audio format of the second audio data into the target audio format.

[0141] In summary, it can be seen that the embodiment of the present application supports the judgment of the system version of the operating system deployed in the target terminal device. When it is determined that the system version is higher, the sampling conversion module that comes with the operating system can be used to convert the audio format, so that the audio format of the converted audio data (such as the first audio data and the second audio data) meets the use requirements of the audio device in the target terminal device; on the contrary, when it is determined that the system version is lower, the conversion module deployed inside the audio control system is used to convert the audio format, so that the audio format of the converted audio data meets the use requirements of the audio device in the target terminal device. It can be seen that when it is determined that the system version of the target terminal device is higher, the sampling conversion module that comes with the target terminal device is used to convert the audio format, which can effectively reduce the workload of the audio control system, so that the audio control system has more resources for other processing of audio data.

[0142] To facilitate understanding of the above-described system version and audio format compatibility, the following takes the target terminal device as a Windows platform as an example. Figure 7 The principle of compatibility between the audio acquisition and playback system version and the audio format provided in the embodiment of the present application is introduced. Figure 7 As shown, the object / user holding the target terminal device can use the target terminal device to start the target function with audio collection / playback capabilities, such as initiating a VoIP call in a social application; at this time, the audio control system can receive the device information transmitted by the target terminal device, and parse the device information to determine the configuration parameters adapted to the target terminal device from the audio control system, and the configuration parameters include the target audio format. Further, according to the different system versions of the target terminal device, different platform interfaces can be deployed in the platform interface layer of the audio control system; therefore, after the audio control system determines the system version number of the target terminal device (i.e., referred to as the system version) based on the device information of the target terminal device, it can instantiate and call the implementation class of the target platform interface compatible with the system version according to the system version of the target terminal device to execute the audio format conversion logic of the implementation class. Specifically, if the system version of the target terminal device is greater than the Windows 10 version (i.e., the version threshold), the audio control system directly uses the target audio format to start audio collection and playback; specifically, with the help of the sampling conversion module that comes with the target terminal device, the audio format matching is realized within the operating system.

[0143] On the contrary, if the system version of the target terminal device is less than or equal to the Windows 10 version, the audio control system queries the default audio format of the audio device in the target terminal device, and uses the default audio format of the audio device for audio acquisition and playback; specifically, in the audio acquisition thread, the audio device in the target terminal device is used to collect the second audio data, and at this time, the audio format of the second audio data is the audio format of the audio device, and then, the sampling conversion module is used inside the audio control system to perform audio format conversion (or called sampling conversion) on the second audio data, and the audio format of the converted second audio data is the target audio format, and the audio processing module inside the audio control system is used to perform audio processing and other operations on the second audio data that conforms to the target audio format. Similarly, in the audio playback thread, the audio control system converts the first audio data in the target audio format to the default audio format of the audio device of the target terminal device, and then plays the first audio data.

[0144] The audio control system provided in the embodiment of the present application also supports the detection of startup abnormalities, specifically detecting the startup status of the target audio device in the target terminal device and the working status of the callback function. Only when both are operating normally, will the audio control system be successfully started to provide audio acquisition and playback services. By deploying the function of startup abnormality detection in the audio control system, it is beneficial to promptly feedback the working status of the audio control system (such as startup success or failure) to the user, so that the user can make timely feedback on startup failures.

[0145] Combine the following Figure 8 An exemplary process for starting anomaly detection is given; Figure 8 As shown, after starting the audio control system, first, the target audio device in the target terminal device is started, and the target audio device is an audio device in a selected state among one or more audio devices included in the target terminal device. Among them, the target audio device is in a selected state, indicating that the target terminal device will use the target audio device to collect or play audio. In addition, the target audio device is the default, or is selected by the user in a selection window provided by the application or a setting window provided by the terminal device. In detail, the method for determining the target audio device may include: judging whether there is a user-specified audio device in one or more audio devices included in the target terminal device; if so, starting the user-specified audio device for audio collection or audio playback, and the user-specified audio device is the target audio device; if not, starting the default audio device of the target terminal device as the target audio device to realize audio collection / audio playback. An exemplary interface diagram for a user to specify a target audio device or start a default target audio device can be seen in Fig. 9 .like Fig. 9In the first figure shown, when the target terminal device has a built-in microphone for audio collection and a speaker for user audio playback, the user can select the system built-in / default audio device "microphone" for audio collection, or the system built-in / default audio device "speaker" for audio playback; if the user does not select an audio device, the target terminal device also uses the system built-in / default audio device for audio collection / playback. Fig. 9 As shown in the second figure, the user can also specify a microphone from multiple microphones in the target terminal device to collect audio. Similarly, the user can also specify a speaker from multiple speakers in the target terminal device to play audio.

[0146] Then, the startup status of the target audio device is obtained; when the startup status of the target audio device is successful startup, it means that the target audio device can perform audio acquisition or audio playback normally. When the startup status indicates that the target audio device is successfully started, continue to detect the working status of the callback function; if the working status of the callback function indicates that the callback function is in a normal working state, that is, the callback function can correctly call back the relevant data, and the surface audio control system can be started normally, then print the startup success log. On the contrary, if the startup status of the target audio device indicates that the target audio device fails to start, or the working status of the callback function indicates that the callback function is in an abnormal working state, then it is determined that the audio control system cannot start normally, and the target audio device can be restarted, and the number of restarts can be recorded; if the restart is successful, continue to judge the working status of the callback function, and if the working status of the callback function indicates that the callback function can work normally, print the startup success log; if the number of restarts of the target audio device exceeds the number threshold, then the audio acquisition and playback service can be ended, and the startup failure log can be printed. Alternatively, if the startup status of the target audio device indicates that the target audio device failed to start, or the working status of the callback function indicates that the callback function is in an abnormal working state, in addition to restarting the target audio device as described above, the embodiment of the present application also supports outputting a startup failure prompt to the user so that the user can promptly grasp the startup status; specifically, the target terminal device is controlled to output prompt information, which is used to prompt the object to reselect a new audio device as the device for audio collection or audio playback. At this time, the object can receive a reselection operation for the prompt information and start the audio device indicated by the reselection operation.

[0147] An exemplary interface diagram for prompting an object to reselect an audio device can be found in Fig.10 ;like Fig.10As shown, when it is determined that the target audio device cannot be restarted successfully, the audio control system can send a startup failure message to the target terminal device, so that the target terminal device can output a prompt message 1001 to the user on the terminal screen. In this way, after the user observes the prompt message 1001, he can reselect a new audio device to perform audio collection or audio playback, thereby achieving the purpose of restarting the audio collection and playback service. Of course, if the user starts the target function (such as a VoIP call) without installing an audio device (such as a microphone) in the target terminal device, the target terminal device directly displays a prompt that there is no microphone to select, so that the user can understand the existence of the microphone in the target terminal device; Fig.11 In the first figure shown in FIG. 1 , a prompt "No device currently" is displayed in the selection window 1101, and the mute icon 1102 in the call interface is displayed as muted, to prompt the user that there is currently no microphone available. Further, if the user inserts a microphone into the target terminal device, and the user does not select a microphone, and does not start the default microphone selection function, then Fig.11 In the second figure shown in FIG. 1 , the name of the newly inserted microphone can be displayed in the selection window, but the mute icon 1102 is still displayed as being muted. Further, if the user selects a microphone from the selection window, then Fig.11 As shown in the third figure, the mute icon 1102 is switched from a muted state to an unmuted state.

[0148] The audio control system provided by the embodiment of the present application also has the ability to detect the switching and updating of audio devices in real time, which enables the audio control system to provide audio acquisition and playback services based on the latest selected audio device in a timely manner. In a specific implementation, if the audio device in the target terminal device changes (such as a new audio device is inserted, or the original audio device is unplugged, or the speaker used for audio playback is reselected, etc.), the target terminal device can generate a device switching event based on the change of the audio device. The device switching event is used to indicate that the audio device in the target terminal device has changed. The target terminal device sends the device switching event to the audio control system; in this way, the device detection thread and the audio acquisition and playback thread in the audio control system can receive the device switching event, and after filtering out the same device switching events, the audio device can be switched according to the device switching event, that is, the switching event processing flow is entered. The following is combined with Fig.12 The following briefly introduces some of the switching processing situations that may be involved in the embodiments of the present application; among them:

[0149] ① A device switching event is an event in which the target audio device used for audio capture / playback changes. In other words, a device switching event indicates that a target audio device selected in the target terminal device changes; the change may be caused by the user reselecting the audio device, or it may be caused by the automatic switching of the audio device due to reasons such as the failure of the target audio device to start or connect. In this case, the switching process performed by the audio control system is to switch the audio device, specifically, to switch from the target audio device to the newly selected audio device; in more detail, the audio control system disconnects the target audio device and instead starts a new audio device to facilitate communication with the new audio device to achieve audio capture / playback. Among them, the schematic diagram of the user selecting the target audio device can be found in the aforementioned Fig.10 or Fig.11 , Fig.10 and Fig.11 The schematic diagram shows a user selecting a target audio device from a call interface provided by an application; that is, the audio control system provided by the embodiment of the present application can respond to the user's reselection of the target audio device from the service interface provided by the application (such as the call interface). It is worth noting that the audio control system provided by the embodiment of the present application also supports responding to the user's reselection of the target audio device from the setting interface provided by the target terminal device (such as Fig.13 In short, no matter the user selects an audio device from the service interface provided by the application or the setting interface of the target terminal device, the audio control system provided by the embodiment of the present application can respond to the selection and provide audio acquisition and playback services.

[0150] ② The device change event is an event in which an audio device is unplugged in the target terminal device. That is, the device switching event indicates that there is a disconnection between the audio device in the target terminal device and the target terminal device, such as the external microphone of the target terminal device is unplugged. In this case, the audio control system will query the device list of the target terminal device, which is used to store the device identification of the audio device deployed in the target terminal device. Further, if the device list is empty, that is, after the device change event occurs, there is no audio device in the target terminal device, then the audio acquisition and playback service is stopped (or the operation of the audio control system is stopped) and the service failure log is printed. On the contrary, if the device list is not empty, that is, after the device change event occurs, there are still audio devices in the target terminal device, then it is detected whether the disconnected audio device is the target audio device in the selected state; if not, the target audio device is continued to be used for audio acquisition / or playback; if so, it indicates that the target audio device can no longer be used for audio acquisition / playback, then switch to a new audio device for audio acquisition / playback, the new audio device can be the default, or customized by the user after the prompt information is output, and there is no limitation on this. In addition, in order to facilitate the user to intuitively perceive that the audio device used for audio collection / playback has been switched to a new audio device, the embodiment of the present application supports outputting a prompt box (such as Fig.14 The prompt box 1401 shown is used to intuitively and timely remind the user of the audio device currently being used, so that the user can understand the device switching status.

[0151] ③ The device change event is an event of audio device insertion. In other words, the device switching event indicates that a new audio device is inserted or added to the target terminal device, such as inserting a new microphone into the target terminal device. In this case, the audio control system will query the device list of the target terminal device; if the device list only includes new audio devices, that is, the device list is empty before the device change event, it will directly switch to the new audio device for audio acquisition / playback; if the device list includes not only new audio devices but also one or more original audio devices in the target terminal device, it will continue to determine whether the selected target audio device has changed. If it has changed, it will switch to the newly selected audio device. Otherwise, the device will not be switched, that is, the original target audio device will be kept for audio acquisition / playback.

[0152] In summary, the embodiment of the present application provides an audio control system for cross-platform real-time audio collection and playback, supports pre-deployment of configuration parameters adapted to different terminal devices in the audio control system, so that the audio control system has the advantage of being compatible with different platforms and operating systems; compared with developing audio collection and playback services for different platforms, this way of integrating configuration parameters adapted to different platforms in the audio control system can avoid repeated and multiple development of audio collection and playback services for different platforms or functions, improve development efficiency, and effectively reduce development costs. In addition, the audio control system provided by the embodiment of the present application also has a detection function for abnormal startup, which can prompt the user about the startup of the audio control system in time, which is conducive to the user to give feedback on the startup in time. In addition, the audio control system provided by the embodiment of the present application can also match the compatible target audio format according to the system version of the target terminal device, so that the target terminal device and the audio control system can provide normal audio collection and playback services based on the compatible target audio format, and improve the stability and compatibility of the audio collection and playback service. In addition, the audio control system provided by the embodiment of the present application also supports detection of audio devices, ensuring that the audio control system can maintain communication with the audio device newly selected by the user or working normally, thereby improving stable audio collection and playback services, and improving the consistency and service quality of audio collection and playback services.

[0153] The method of the embodiment of the present application is described in detail above. In order to facilitate the above-mentioned scheme of the embodiment of the present application to be better implemented, the device of the embodiment of the present application is provided below accordingly. In the embodiment of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an overall module or unit that includes the module or unit function.

[0154] Fig.15 A schematic diagram of the structure of an audio processing device provided by an exemplary embodiment of the present application is shown; the audio processing device can be used to perform Figure 3 or Figure 6 Some or all of the steps in the method embodiment shown. Fig.15The audio processing device is equipped with an audio control system, which is used to provide audio collection and playback services for terminal devices; the audio control system pre-stores configuration parameters corresponding to a variety of terminal devices, and the configuration parameters corresponding to each of the multiple terminal devices are pre-configured to the audio control system to ensure the compatibility between the corresponding terminal device and the audio control system; the device includes the following units:

[0155] The receiving unit 1501 is used to receive an audio operation request sent by a target terminal device; the audio operation request is used to request the audio control system to perform an audio operation on an audio acquisition and playback service; the target terminal device is any one of a plurality of terminal devices;

[0156] The processing unit 1502 is used to obtain device information of the target terminal device in response to the audio operation request; the device information includes hardware information and system information, the hardware information is used to indicate information of the hardware device deployed in the target terminal device, and the system information is used to indicate system information of the operating system deployed in the target terminal device;

[0157] The processing unit 1502 is further used to determine, from the audio control system, configuration parameters adapted to the target terminal device according to the device information of the target terminal device; the configuration parameters are pre-set based on the hardware information and software information included in the device information;

[0158] The processing unit 1502 is further configured to execute the audio operation requested by the audio operation request according to the configuration parameters adapted to the target terminal device.

[0159] In one implementation, the audio control system includes a platform interface layer, which encapsulates platform interfaces corresponding to multiple platform types; the system information includes the platform type of the operating system carried by the target terminal device; the configuration parameters adapted to the target terminal device include the target platform interface;

[0160] The processing unit 1502 is used to determine the configuration parameters adapted to the target terminal device from the audio control system according to the device information of the target terminal device, specifically to:

[0161] According to the platform type of the operating system installed on the target terminal device, a target platform interface having the same platform type as the platform type of the operating system installed on the target terminal device is selected in the platform interface layer;

[0162] The target platform interface is used as a system interface for communication and transmission between the target terminal device and the audio control system.

[0163] In one implementation, the system information further includes a system version of an operating system installed in the target terminal device; the hardware device deployed in the target terminal device includes an audio device, and the hardware information includes sub-device information of the audio device; the configuration parameters adapted to the target terminal device include a target audio format compatible with the system version; the processing unit 1502 is further used to:

[0164] According to the system version of the operating system installed on the target terminal device, a target audio format compatible with the system version is obtained from the audio control system;

[0165] Among them, the target audio format is pre-configured based on the coordination between the modules within the audio control system and the compatibility between the audio control system and the system version; the target audio format includes at least one of the following: audio sampling rate, audio mode, number of audio channels and transmission interval time of the callback function corresponding to the audio data.

[0166] In one implementation, the configuration parameters adapted to the target terminal device include a target platform interface; the processing unit 1502 is used to perform the audio operation requested by the audio operation request according to the configuration parameters adapted to the target terminal device, specifically to:

[0167] Acquire the implementation class of the target platform interface included in the configuration parameters adapted to the target terminal device; the implementation class is used to instruct the audio control system to provide the service logic of the audio acquisition and playback service through the target platform interface;

[0168] Instantiate the implementation class of the target platform interface and register the callback function to start the audio acquisition and playback thread in the target terminal device;

[0169] Execute the audio operation requested by the audio operation request in the audio acquisition and playback thread.

[0170] In one implementation, the audio acquisition and playback service includes audio acquisition and audio playback; the audio operation request is used to request the audio control system to perform an audio operation on audio playback, and the audio acquisition and playback thread is the audio playback thread; the processing unit 1502 is used to perform the audio operation requested by the audio operation request in the audio acquisition and playback thread, specifically for:

[0171] In the audio playback thread, the first audio data to be played is obtained from the remote end through the callback function;

[0172] Sending the first audio data to an audio device in a target terminal device for playing; and,

[0173] The first audio data is cached in a cache space in the audio control system, and the first audio data in the cache space is used as an echo cancellation reference signal in the audio acquisition process.

[0174] In one implementation, the system information includes a system version of an operating system installed on the target terminal device; the processing unit 1502 is further configured to:

[0175] Check whether the system version of the operating system installed on the target terminal device reaches the version threshold;

[0176] If the system version of the operating system installed in the target terminal device reaches the version threshold, the step of sending the first audio data to the audio device in the target terminal device for playback is triggered; wherein, before playing the first audio data, the target terminal device uses the sampling conversion module deployed in the target terminal device to convert the first audio data into an audio format; or,

[0177] If the system version of the operating system installed in the target terminal device does not reach the version threshold, the sampling conversion module deployed in the audio control system is used to convert the first audio data into an audio format, and trigger the execution of the step of sending the first audio data to the audio device in the target terminal device for playback;

[0178] The audio format of the first audio data after the audio format conversion is the same as the audio format required by the target terminal device.

[0179] In one implementation, the audio acquisition and playback service includes audio acquisition and audio playback; the audio operation request is used to request the audio control system to perform an audio operation on audio acquisition, and the audio acquisition and playback thread is the audio acquisition thread; the processing unit 1502 is used to perform the audio operation requested by the audio operation request in the audio acquisition and playback thread, specifically for:

[0180] In the audio collection thread, the second audio data collected by the audio device in the target terminal device is obtained through the callback function;

[0181] Performing audio processing on the second audio data to obtain processed second audio data;

[0182] The processed second audio data is transmitted to the remote end through the callback function.

[0183] In one implementation, the system information includes a system version of an operating system installed on the target terminal device; the processing unit 1502 is further configured to:

[0184] Check whether the system version of the operating system installed on the target terminal device reaches the version threshold;

[0185] If the system version of the operating system installed in the target terminal device reaches the version threshold, the step of performing audio processing on the second audio data to obtain processed second audio data is triggered; wherein, before the second audio data is called back to the audio control system, the sampling conversion module in the target terminal device performs audio format conversion on the second audio data; or,

[0186] If the system version of the operating system installed in the target terminal device does not reach the version threshold, the sampling conversion module deployed in the audio control system is used to convert the second audio data into an audio format, and the step of performing audio processing on the second audio data to obtain the processed second audio data is triggered;

[0187] The audio format of the second audio data after the audio format conversion is the same as the audio format required by the audio control system.

[0188] In one implementation, the target terminal device includes one or more audio devices, and the audio devices are used for collecting and playing audio data; the processing unit 1502 is further used to:

[0189] Starting a target audio device in a target terminal device; the target audio device is an audio device in a selected state among one or more audio devices; the target audio device is a default one, or is selected by a user in a selection window provided by an application or a setting interface provided by the target terminal device;

[0190] Get the startup status of the target audio device, and when the startup status indicates that the target audio device is successfully started, detect the working status of the callback function;

[0191] If the working status of the callback function indicates that the callback function is in a normal working state, a startup success log is printed.

[0192] In one implementation, the processing unit 1502 is further configured to:

[0193] If the startup status indicates that the target audio device fails to start, or the working status of the callback function indicates that the callback function is in an abnormal working state, the target audio device is restarted and the number of restarts is recorded. When the number of restarts exceeds the threshold, the audio acquisition and playback service is terminated and the startup failure log is printed; or,

[0194] If the startup status indicates that the target audio device failed to start, or the working status of the callback function indicates that the callback function is in an abnormal working state, the target terminal device is controlled to output a prompt message, and the prompt message is used to prompt the object to reselect a new audio device as the audio acquisition or audio playback device; and receive a reselection operation for the prompt message, and start the audio device indicated by the reselection operation.

[0195] In one implementation, the target terminal device includes one or more audio devices, and the audio devices are used for collecting and playing audio data; the processing unit 1502 is further used to:

[0196] Receiving a device switching event sent by a target terminal device, where the device switching event is used to indicate that an audio device in the target terminal device has changed;

[0197] Perform audio device switching processing according to device switching events;

[0198] The switching process includes any of the following:

[0199] If the device switching event indicates that the target audio device in the selected state in the target terminal device has changed, the audio device is switched;

[0200] If the device switching event indicates that there is an audio device in the target terminal device that is disconnected from the target terminal device, then query the device list of the target terminal device, which is used to store the device identifiers of the audio devices deployed in the target terminal device; if the device list is empty, stop the audio acquisition and playback service and print the service failure log; if the device list is not empty, check whether the disconnected audio device is the target audio device in the selected state; if so, switch to a new audio device;

[0201] If the device switching event indicates that a new audio device is added to the target terminal device, the device list is queried; if the device list only includes the new audio device, the new audio device is switched to.

[0202] In one implementation, the processing unit 1502 is configured to obtain device information of a target terminal device in response to an audio operation request, specifically to:

[0203] In response to the audio operation request, obtaining device information of the target terminal device from the main thread of the target terminal device;

[0204] The audio control system is deployed on the target terminal device, the audio operation request is generated when the object performs a startup operation on the audio function provided by the target terminal device, and the main thread refers to the thread that executes the audio function of the target terminal device; or

[0205] The audio control system is deployed in the application running in the target terminal device. The audio operation request is generated when the object performs a startup operation on the target function provided by the application. The main thread refers to the thread that executes the target function in the application; the target function includes at least one of the following: live broadcast function, conference function, voice call function and music playback function.

[0206] According to one embodiment of the present application, Fig.15The various units in the audio processing device shown can be individually or completely combined into one or several other units to form, or one (some) of the units can be further divided into multiple functionally smaller units to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In practical applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the audio processing device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units. According to another embodiment of the present application, it can be executed by running on a general computing device such as a computer that includes processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM). Figure 3 and Figure 6 A computer program (including program code) for each step involved in the corresponding method shown in FIG. Fig.15 The audio processing device shown in the embodiment of the present application is implemented by the audio processing method. The computer program can be recorded on a computer-readable recording medium, for example, and loaded into the above-mentioned computing device through the computer-readable recording medium and run therein.

[0207] Based on the same inventive concept, the principles and beneficial effects of solving the problems by the audio processing device provided in the embodiment of the present application are similar to the principles and beneficial effects of solving the problems by the audio processing method in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0208] Fig.16 FIG. 1 shows a schematic diagram of the structure of a terminal device provided by an exemplary embodiment of the present application. Fig.16 , the terminal device includes a processor 1601, a communication interface 1602 and a computer-readable storage medium 1603. The processor 1601, the communication interface 1602 and the computer-readable storage medium 1603 may be connected via a bus or other means. The communication interface 1602 is used to receive and send data. The computer-readable storage medium 1603 may be stored in a memory of a computer device, the computer-readable storage medium 1603 is used to store a computer program, and the processor 1601 is used to execute the computer program stored in the computer-readable storage medium 1603. The processor 1601 (or CPU (Central Processing Unit)) is the computing core and control core of the terminal device, which is suitable for implementing one or more computer programs, and is specifically suitable for loading and executing one or more computer programs to implement corresponding method flows or corresponding functions.

[0209] The embodiment of the present application also provides a computer-readable storage medium (Memory), which is a memory device in a terminal device for storing programs and data. It is understandable that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and the extended storage medium supported by the terminal device. The computer-readable storage medium provides a storage space that stores the processing system of the terminal device. In addition, one or more computer programs suitable for being loaded and executed by the processor 1601 are also stored in the storage space. It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage; optionally, it can also be at least one computer-readable storage medium located away from the aforementioned processor.

[0210] In one embodiment, one or more computer programs are stored in the computer-readable storage medium; the processor 1601 loads and executes the one or more computer programs stored in the computer-readable storage medium to implement the corresponding steps in the above audio processing method embodiment; in a specific implementation, an audio control system is deployed in the computer device, and the audio control system is used to provide audio acquisition and playback services for the terminal device; the audio control system pre-stores configuration parameters corresponding to a variety of terminal devices, and the configuration parameters corresponding to each of the multiple terminal devices are pre-configured to the audio control system to ensure the compatibility between the corresponding terminal device and the audio control system; the one or more computer programs in the computer-readable storage medium are loaded by the processor 1601 and execute the following steps:

[0211] receiving an audio operation request sent by a target terminal device; the audio operation request is used to request the audio control system to perform an audio operation on an audio acquisition and playback service; the target terminal device is any one of a plurality of terminal devices;

[0212] Acquire device information of the target terminal device in response to the audio operation request; the device information includes hardware information and system information, the hardware information is used to indicate information of the hardware device deployed in the target terminal device, and the system information is used to indicate system information of the operating system deployed in the target terminal device;

[0213] According to the device information of the target terminal device, determining the configuration parameters adapted to the target terminal device from the audio control system; the configuration parameters are pre-set based on the hardware information and software information included in the device information;

[0214] The audio operation requested by the audio operation request is performed according to the configuration parameters adapted to the target terminal device.

[0215] In one implementation, the audio control system includes a platform interface layer, which encapsulates platform interfaces corresponding to multiple platform types; the system information includes the platform type of the operating system carried by the target terminal device; the configuration parameters adapted to the target terminal device include the target platform interface;

[0216] The one or more computer programs in the computer-readable storage medium are loaded by the processor 1601 and when executing, according to the device information of the target terminal device, determine the configuration parameters adapted to the target terminal device from the audio control system, specifically perform the following steps:

[0217] According to the platform type of the operating system installed on the target terminal device, a target platform interface having the same platform type as the platform type of the operating system installed on the target terminal device is selected in the platform interface layer;

[0218] The target platform interface is used as a system interface for communication and transmission between the target terminal device and the audio control system.

[0219] In one implementation, the system information further includes a system version of an operating system installed in the target terminal device; the hardware device deployed in the target terminal device includes an audio device, and the hardware information includes sub-device information of the audio device; the configuration parameters adapted to the target terminal device include a target audio format compatible with the system version; one or more computer programs in a computer-readable storage medium are loaded by the processor 1601 and further perform the following steps:

[0220] According to the system version of the operating system installed on the target terminal device, a target audio format compatible with the system version is obtained from the audio control system;

[0221] Among them, the target audio format is pre-configured based on the coordination between the modules within the audio control system and the compatibility between the audio control system and the system version; the target audio format includes at least one of the following: audio sampling rate, audio mode, number of audio channels and transmission interval time of the callback function corresponding to the audio data.

[0222] In one implementation, the configuration parameters adapted to the target terminal device include a target platform interface; one or more computer programs in a computer-readable storage medium are loaded by the processor 1601 and when executing the audio operation requested by the audio operation request according to the configuration parameters adapted to the target terminal device, the following steps are specifically performed:

[0223] Acquire the implementation class of the target platform interface included in the configuration parameters adapted to the target terminal device; the implementation class is used to instruct the audio control system to provide the service logic of the audio acquisition and playback service through the target platform interface;

[0224] Instantiate the implementation class of the target platform interface and register the callback function to start the audio acquisition and playback thread in the target terminal device;

[0225] Execute the audio operation requested by the audio operation request in the audio acquisition and playback thread.

[0226] In one implementation, the audio acquisition and playback service includes audio acquisition and audio playback; the audio operation request is used to request the audio control system to perform an audio operation on audio playback, and the audio acquisition and playback thread is the audio playback thread; one or more computer programs in a computer-readable storage medium are loaded by the processor 1601 and when executing the audio operation requested by the audio operation request in the audio acquisition and playback thread, the following steps are specifically performed:

[0227] In the audio playback thread, the first audio data to be played is obtained from the remote end through the callback function;

[0228] Sending the first audio data to an audio device in a target terminal device for playing; and,

[0229] The first audio data is cached in a cache space in the audio control system, and the first audio data in the cache space is used as an echo cancellation reference signal in the audio acquisition process.

[0230] In one implementation, the system information includes a system version of an operating system installed on the target terminal device; one or more computer programs in a computer-readable storage medium are loaded by the processor 1601 and further perform the following steps:

[0231] Check whether the system version of the operating system installed on the target terminal device reaches the version threshold;

[0232] If the system version of the operating system installed in the target terminal device reaches the version threshold, the step of sending the first audio data to the audio device in the target terminal device for playback is triggered; wherein, before playing the first audio data, the target terminal device uses the sampling conversion module deployed in the target terminal device to convert the first audio data into an audio format; or,

[0233] If the system version of the operating system installed in the target terminal device does not reach the version threshold, the sampling conversion module deployed in the audio control system is used to convert the first audio data into an audio format, and trigger the execution of the step of sending the first audio data to the audio device in the target terminal device for playback;

[0234] The audio format of the first audio data after the audio format conversion is the same as the audio format required by the target terminal device.

[0235] In one implementation, the audio acquisition and playback service includes audio acquisition and audio playback; the audio operation request is used to request the audio control system to perform an audio operation on audio acquisition, and the audio acquisition and playback thread is the audio acquisition thread; one or more computer programs in a computer-readable storage medium are loaded by the processor 1601 and when executing the audio operation requested by the audio operation request in the audio acquisition and playback thread, the following steps are specifically performed:

[0236] In the audio collection thread, the second audio data collected by the audio device in the target terminal device is obtained through the callback function;

[0237] Performing audio processing on the second audio data to obtain processed second audio data;

[0238] The processed second audio data is transmitted to the remote end through the callback function.

[0239] In one implementation, the system information includes a system version of an operating system installed on the target terminal device; one or more computer programs in a computer-readable storage medium are loaded by the processor 1601 and further perform the following steps:

[0240] Check whether the system version of the operating system installed on the target terminal device reaches the version threshold;

[0241] If the system version of the operating system installed in the target terminal device reaches the version threshold, the step of performing audio processing on the second audio data to obtain the processed second audio data is triggered; wherein, before the second audio data is called back to the audio control system, the sampling conversion module in the target terminal device performs audio format conversion on the second audio data; or,

[0242] If the system version of the operating system installed in the target terminal device does not reach the version threshold, the sampling conversion module deployed in the audio control system is used to convert the second audio data into an audio format, and the step of performing audio processing on the second audio data to obtain the processed second audio data is triggered;

[0243] The audio format of the second audio data after the audio format conversion is the same as the audio format required by the audio control system.

[0244] In one implementation, the target terminal device includes one or more audio devices, and the audio devices are used for collecting and playing audio data; one or more computer programs in a computer-readable storage medium are loaded by the processor 1601 and further perform the following steps:

[0245] Starting a target audio device in a target terminal device; the target audio device is an audio device in a selected state among one or more audio devices; the target audio device is a default one, or is selected by a user in a selection window provided by an application or a setting interface provided by the target terminal device;

[0246] Get the startup status of the target audio device, and when the startup status indicates that the target audio device is successfully started, detect the working status of the callback function;

[0247] If the working status of the callback function indicates that the callback function is in a normal working state, a startup success log is printed.

[0248] In one implementation, one or more computer programs in the computer-readable storage medium are loaded by the processor 1601 and further perform the following steps:

[0249] If the startup status indicates that the target audio device fails to start, or the working status of the callback function indicates that the callback function is in an abnormal working state, the target audio device is restarted and the number of restarts is recorded. When the number of restarts exceeds the threshold, the audio acquisition and playback service is terminated and the startup failure log is printed; or,

[0250] If the startup status indicates that the target audio device failed to start, or the working status of the callback function indicates that the callback function is in an abnormal working state, the target terminal device is controlled to output a prompt message, and the prompt message is used to prompt the object to reselect a new audio device as the audio acquisition or audio playback device; and receive a reselection operation for the prompt message, and start the audio device indicated by the reselection operation.

[0251] In one implementation, the target terminal device includes one or more audio devices, and the audio devices are used for collecting and playing audio data; one or more computer programs in a computer-readable storage medium are loaded by the processor 1601 and further perform the following steps:

[0252] Receiving a device switching event sent by a target terminal device, where the device switching event is used to indicate that an audio device in the target terminal device has changed;

[0253] Perform audio device switching processing according to device switching events;

[0254] The switching process includes any of the following:

[0255] If the device switching event indicates that the target audio device in the selected state in the target terminal device has changed, the audio device is switched;

[0256] If the device switching event indicates that there is an audio device in the target terminal device that is disconnected from the target terminal device, then query the device list of the target terminal device, which is used to store the device identifiers of the audio devices deployed in the target terminal device; if the device list is empty, stop the audio acquisition and playback service and print the service failure log; if the device list is not empty, check whether the disconnected audio device is the target audio device in the selected state; if so, switch to a new audio device;

[0257] If the device switching event indicates that a new audio device is added to the target terminal device, the device list is queried; if the device list only includes the new audio device, the new audio device is switched to.

[0258] In one implementation, one or more computer programs in a computer-readable storage medium are loaded by the processor 1601 and, when executing to obtain device information of a target terminal device in response to an audio operation request, specifically perform the following steps:

[0259] In response to the audio operation request, obtaining device information of the target terminal device from the main thread of the target terminal device;

[0260] The audio control system is deployed on the target terminal device, the audio operation request is generated when the object performs a startup operation on the audio function provided by the target terminal device, and the main thread refers to the thread that executes the audio function of the target terminal device; or

[0261] The audio control system is deployed in the application running in the target terminal device. The audio operation request is generated when the object performs a startup operation on the target function provided by the application. The main thread refers to the thread that executes the target function in the application; the target function includes at least one of the following: live broadcast function, conference function, voice call function and music playback function.

[0262] Based on the same inventive concept, the principles and beneficial effects of solving the problems provided by the terminal device in the embodiment of the present application are similar to the principles and beneficial effects of solving the problems by the data processing method in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0263] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the above-mentioned audio processing method is implemented.

[0264] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0265] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes a computer program (one or more). When a computer program is loaded and executed on a computer device, the computer program executes the above-mentioned process or function of the embodiment of the present application. The computer device can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer program can be stored in a computer-readable storage medium or transmitted by a computer-readable storage medium. The computer program can be transmitted from a website site, a computer device, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer device, server or data center. The computer-readable storage medium can be any available medium that a computer device can access or a data storage device such as a server or a data center that contains one or more available media integrations. Available media can be magnetic media (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state drive (Solid State Disk, SSD)) and the like.

[0266] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An audio processing method, characterized in that: The method is applied to an audio control system, which is used to provide audio acquisition and playback services for terminal devices; the audio control system pre-stores configuration parameters corresponding to a plurality of terminal devices, and the configuration parameters corresponding to each of the plurality of terminal devices are pre-configured to the audio control system to ensure compatibility between the corresponding terminal device and the audio control system; the method comprises: Receiving an audio operation request sent by a target terminal device; the audio operation request is used to request the audio control system to perform an audio operation on the audio acquisition and playback service; the target terminal device is any one of the multiple terminal devices; Acquire device information of the target terminal device in response to the audio operation request; the device information includes hardware information and system information, the hardware information is used to indicate information of the hardware device deployed in the target terminal device, and the system information is used to indicate system information of the operating system deployed in the target terminal device; According to the device information of the target terminal device, determining configuration parameters adapted to the target terminal device from the audio control system; the configuration parameters are pre-set based on the hardware information and software information included in the device information; The audio operation requested by the audio operation request is performed according to the configuration parameters adapted to the target terminal device.

2. The method according to claim 1, characterized in that The audio control system includes a platform interface layer, and the platform interface layer encapsulates platform interfaces corresponding to multiple platform types; the system information includes the platform type of the operating system installed on the target terminal device; Configuration parameters adapted to the target terminal device include a target platform interface; The step of determining, from the audio control system, configuration parameters adapted to the target terminal device according to the device information of the target terminal device comprises: According to the platform type of the operating system installed on the target terminal device, selecting in the platform interface layer a target platform interface having the same platform type as the platform type of the operating system installed on the target terminal device; The target platform interface serves as a system interface for communication and transmission between the target terminal device and the audio control system.

3. The method according to claim 2, characterized in that The system information also includes the system version of the operating system installed in the target terminal device; the hardware device deployed in the target terminal device includes an audio device, and the hardware information includes sub-device information of the audio device; The configuration parameters adapted to the target terminal device include a target audio format compatible with the system version; the method further includes: According to the system version of the operating system installed on the target terminal device, acquiring a target audio format compatible with the system version from the audio control system; Among them, the target audio format is pre-configured based on the coordination between the modules inside the audio control system and the compatibility between the audio control system and the system version; the target audio format includes at least one of the following: audio sampling rate, audio mode, number of audio channels and transmission interval time of the callback function corresponding to the audio data.

4. The method according to claim 1, characterized in that The configuration parameters adapted to the target terminal device include a target platform interface; and the step of executing the audio operation requested by the audio operation request according to the configuration parameters adapted to the target terminal device includes: Acquire an implementation class of a target platform interface included in the configuration parameters adapted to the target terminal device; the implementation class is used to indicate the service logic of the audio control system to provide an audio acquisition and playback service through the target platform interface; Instantiating the implementation class of the target platform interface and registering a callback function to start an audio acquisition and playback thread in the target terminal device; The audio operation requested by the audio operation request is performed in the audio acquisition and playback thread.

5. The method according to claim 4, characterized in that The audio collection and playback service includes audio collection and audio playback; the audio operation request is used to request the audio control system to perform an audio operation related to audio playback, and the audio collection and playback thread is an audio playback thread; The performing of the audio operation requested by the audio operation request in the audio acquisition and playback thread includes: In the audio playback thread, obtaining first audio data to be played from a remote end through the callback function; sending the first audio data to the audio device in the target terminal device for playing; and, The first audio data is cached in a cache space in the audio control system, and the first audio data in the cache space is used as an echo cancellation reference signal in an audio acquisition process.

6. The method according to claim 5, characterized in that The system information includes the system version of the operating system installed on the target terminal device; the method further includes: Detecting whether the system version of the operating system installed on the target terminal device reaches a version threshold; If the system version of the operating system installed in the target terminal device reaches the version threshold, the step of sending the first audio data to the audio device in the target terminal device for playback is triggered; wherein, before playing the first audio data, the target terminal device uses the sampling conversion module deployed in the target terminal device to convert the first audio data into an audio format; or, If the system version of the operating system installed in the target terminal device does not reach the version threshold, the sampling conversion module deployed in the audio control system is used to convert the first audio data into an audio format, and the step of sending the first audio data to the audio device in the target terminal device for playback is triggered; The audio format of the first audio data after the audio format conversion is the same as the audio format required by the target terminal device.

7. The method according to claim 4, characterized in that The audio collection and playback service includes audio collection and audio playback; the audio operation request is used to request the audio control system to perform an audio operation related to audio collection, and the audio collection and playback thread is an audio collection thread; The performing of the audio operation requested by the audio operation request in the audio acquisition and playback thread includes: In the audio collection thread, obtaining the second audio data collected by the audio device in the target terminal device through the callback function; Performing audio processing on the second audio data to obtain processed second audio data; The processed second audio data is transmitted to a remote end through the callback function.

8. The method according to claim 7, characterized in that The system information includes the system version of the operating system installed on the target terminal device; the method further includes: Detecting whether the system version of the operating system installed on the target terminal device reaches a version threshold; If the system version of the operating system installed in the target terminal device reaches the version threshold, the step of performing audio processing on the second audio data to obtain processed second audio data is triggered; wherein, before the second audio data is called back to the audio control system, the sampling conversion module in the target terminal device performs audio format conversion on the second audio data; or, If the system version of the operating system installed in the target terminal device does not reach the version threshold, the sampling conversion module deployed in the audio control system is used to convert the audio format of the second audio data, and the step of performing audio processing on the second audio data to obtain the processed second audio data is triggered; The audio format of the second audio data after the audio format conversion is the same as the audio format required by the audio control system.

9. The method according to any one of claims 1, 2 or 4, characterized in that: The target terminal device includes one or more audio devices, and the audio devices are used for collecting and playing audio data; the method also includes: Starting a target audio device in the target terminal device; the target audio device is an audio device in a selected state among the one or more audio devices; the target audio device is a default one, or is selected by a user in a selection window provided by the application or a setting interface provided by the target terminal device; Acquire the startup status of the target audio device, and when the startup status indicates that the target audio device is successfully started, detect the working status of the callback function; If the working status of the callback function indicates that the callback function is in a normal working state, a startup success log is printed.

10. The method according to claim 9, characterized in that The method further comprises: If the startup status indicates that the target audio device fails to start, or the working status of the callback function indicates that the callback function is in an abnormal working state, the target audio device is restarted, and the number of restarts is recorded. When the number of restarts exceeds a threshold, the audio acquisition and playback service is terminated, and a startup failure log is printed; or, If the startup status indicates that the target audio device has failed to start, or the working status of the callback function indicates that the callback function is in an abnormal working state, the target terminal device is controlled to output a prompt message, wherein the prompt message is used to prompt the object to reselect a new audio device as the device for audio collection or audio playback; and a reselection operation for the prompt message is received, and the audio device indicated by the reselection operation is started.

11. The method according to claim 1, characterized in that The target terminal device includes one or more audio devices, and the audio devices are used for collecting and playing audio data; the method also includes: Receiving a device switching event sent by the target terminal device, wherein the device switching event is used to indicate that an audio device in the target terminal device has changed; Performing a switching process of the audio device according to the device switching event; The switching process includes any one of the following: If the device switching event indicates that a target audio device in a selected state in the target terminal device changes, switching the audio device; If the device switching event indicates that an audio device in the target terminal device is disconnected from the target terminal device, query the device list of the target terminal device, where the device list is used to store device identifiers of audio devices deployed in the target terminal device; if the device list is empty, stop the audio acquisition and playback service and print a service failure log; if the device list is not empty, detect whether the disconnected audio device is a target audio device in a selected state; if so, switch to a new audio device; If the device switching event indicates that a new audio device is added to the target terminal device, a device list is queried; if the device list only includes the new audio device, the new audio device is switched to.

12. The method according to claim 1, characterized in that The acquiring device information of the target terminal device in response to the audio operation request includes: In response to the audio operation request, acquiring device information of the target terminal device from the main thread of the target terminal device; Wherein, the audio control system is deployed on the target terminal device, the audio operation request is generated when the object performs a startup operation on the audio function provided by the target terminal device, and the main thread refers to a thread that executes the audio function of the target terminal device; or, The audio control system is deployed in an application running in the target terminal device. The audio operation request is generated when the object performs a startup operation on the target function provided by the application. The main thread refers to the thread that executes the target function in the application. The target function includes at least one of the following: live broadcast function, conference function, voice call function and music playback function.

13. An audio processing device, characterized in that: The audio processing device is equipped with an audio control system, which is used to provide audio collection and playback services for terminal devices; the audio control system pre-stores configuration parameters corresponding to a variety of terminal devices, and the configuration parameters corresponding to each of the multiple terminal devices are pre-configured to the audio control system to ensure the compatibility between the corresponding terminal device and the audio control system; the device includes: A receiving unit, configured to receive an audio operation request sent by a target terminal device; the audio operation request is used to request the use of the audio control system to perform an audio operation on the audio acquisition and playback service; the target terminal device is any one of the multiple terminal devices; a processing unit, configured to obtain device information of the target terminal device in response to the audio operation request; the device information includes hardware information and system information, the hardware information is used to indicate information of a hardware device deployed in the target terminal device, and the system information is used to indicate system information of an operating system deployed in the target terminal device; The processing unit is further used to determine, from the audio control system, configuration parameters adapted to the target terminal device according to the device information of the target terminal device; the configuration parameters are pre-set based on the hardware information and software information included in the device information; The audio operation requested by the audio operation request is performed according to the configuration parameters adapted to the target terminal device.

14. A terminal device, characterized in that: include: a processor adapted to execute a computer program; A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the audio processing method according to any one of claims 1 to 12 is implemented.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer application, and when the computer application is executed, the audio processing method according to any one of claims 1 to 12 is implemented.

16. A computer program product, characterized in that The computer program product comprises computer instructions, and when the computer instructions are executed by a processor, the audio processing method according to any one of claims 1 to 12 is implemented.

Images