An audio signal acquisition method and device

Abstract

An embodiment of the present application provides an audio signal acquisition method and device, which relate to the terminal field and can reduce the power consumption of an electronic device and increase the battery life of the electronic device. The method includes: in response to a first operation, displaying a first interface of a first application, the first interface including a first microphone icon; after receiving the first operation, setting the current microphone device to a first microphone device; in the case of setting the current microphone device to the first microphone device, the electronic device acquires an audio signal through a microphone component; in response to a second operation, displaying a second interface of the first application, the second interface including a second microphone icon, the second microphone icon being different from the first microphone icon; after receiving the second operation, setting the current microphone device to a second microphone device; in the case of setting the current microphone device to the second microphone device, the electronic device does not acquire an audio signal through the microphone component.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of terminals, and in particular, to an audio signal acquisition method and apparatus. Background Art

[0002] An electronic device may include a microphone device, which can be used to acquire an audio signal (audio stream).

[0003] Currently, in some scenarios (e.g., a meeting scenario), even if the user triggers a mute operation, the microphone device still does not stop acquiring the audio stream. During the process of acquiring the audio signal, the electronic device generates power consumption, resulting in an increase in the overall power consumption of the electronic device and a reduction in the battery life of the electronic device. Summary of the Invention

[0004] Embodiments of the present application provide an audio signal acquisition method and apparatus, which can reduce the power consumption of the electronic device and increase the battery life of the electronic device.

[0005] To achieve the above object, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, there is provided an audio signal acquisition method, which is applied to an electronic device. A first application is installed on the electronic device, and the electronic device includes a microphone device. The method includes: in response to a first operation, displaying a first interface of the first application, the first interface including a first microphone icon; after receiving the first operation, setting the current microphone device to a first microphone device; when the current microphone device is set to the first microphone device, the electronic device acquires an audio signal through the microphone device; in response to a second operation, displaying a second interface of the first application, the second interface including a second microphone icon, and the second microphone icon is different from the first microphone icon; after receiving the second operation, setting the current microphone device to a second microphone device; when the current microphone device is set to the second microphone device, the electronic device does not acquire an audio signal through the microphone device.

[0007] Based on the audio signal acquisition method provided by the embodiments of the present application, after receiving the second operation (e.g., a mute operation), the currently used microphone device (microphone driver) can be set to a second microphone device (virtual microphone device), and the virtual microphone device does not trigger the microphone device to acquire the audio stream and perform audio stream processing, thereby reducing the power consumption of the electronic device and increasing the battery life of the electronic device.

[0008] In a possible implementation, after receiving the second operation, setting the current microphone device to the second microphone device includes: after receiving the second operation, based on the first application being muted, setting the current microphone device to the second microphone device. In some embodiments, it is possible to determine whether the first application is muted based on the first microphone icon, solving the problem that it is difficult to obtain the microphone state (muted / unmuted state) of a closed-source third-party application (the first application).

[0009] In a possible implementation, a second application is also installed on the electronic device, and the method further includes: in response to a third operation, displaying a third interface of the second application, where the third interface includes a third microphone icon; after receiving the third operation, setting the current microphone device to the first microphone device. After receiving the third operation, the current microphone device can be set to the first microphone device, avoiding the problem of reduced user experience caused by the second application being unable to collect the audio stream.

[0010] In a possible implementation, after receiving the third operation, setting the current microphone device to the first microphone device includes: after receiving the third operation, based on the second application not being muted, setting the current microphone device to the first microphone device. It is possible to determine whether the second application is muted based on the third microphone icon. In the case where the second application is not muted, the current microphone device can be set to the first microphone device, avoiding the problem of reduced user experience caused by the second application being unable to collect the audio stream.

[0011] In a possible implementation, the method further includes: in response to a fourth operation, closing the audio session of the second application; after receiving the fourth operation, setting the current microphone device to the second microphone device. After the audio session of the second application is closed, there is no need to collect the audio stream. Therefore, the current microphone device can be set to the second microphone device, and the second microphone device will not trigger the microphone device to collect the audio stream and perform audio stream processing, thereby reducing the power consumption of the electronic device.

[0012] In a possible implementation, the electronic device further includes a third application. After receiving the first operation, the method further includes: the third application determines whether the process name of the first application matches the whitelist, where the whitelist includes the process names of one or more preset processes; in the case where the process name of the first application matches the whitelist, the third application takes a screenshot of the first interface to obtain a first screenshot including the first microphone icon, and determines whether the first application is muted according to the first screenshot; in the case where the first application is not muted, install the second microphone device and disable the second microphone device. In this way, it is possible to determine whether the first application is muted based on the first screenshot (including the first microphone icon), solving the problem that it is difficult to obtain the microphone state (muted / unmuted state) of a closed-source third-party application (the first application).

[0013] In a possible implementation, after receiving the second operation, based on the first application being muted, setting the current microphone device to the second microphone device includes: after receiving the second operation, a third application takes a screenshot of the second interface to obtain a second screenshot including the second microphone icon, and determines whether the first application is muted according to the second screenshot; in the case where the first application is muted, the third application enables the second microphone device and sets the current microphone device to the second microphone device. Compared with continuously performing screenshot detection, performing screenshot detection after responding to the second operation (i.e., detecting whether the first application is muted according to the first screenshot) can effectively improve the detection efficiency and save power consumption.

[0014] In a possible implementation, the electronic device further includes a fourth application. Determining whether the first application is muted according to the first screenshot includes: the third application sends the first screenshot to the fourth application; the fourth application determines whether the first application is muted according to the first screenshot and returns the determination result to the third application.

[0015] In a possible implementation, determining whether the first application is muted according to the first screenshot includes: inputting the first screenshot into a detection model to obtain an output result of the detection model, and the output result is used to indicate whether the first application is muted. Among them, the detection model can be trained by multiple pictures including different types of mute icons and non - mute icons.

[0016] In a possible implementation, the second operation includes a mouse operation, a keyboard operation, or a touch - screen operation. In the Windows system, the mute state of an application program can be set based on a mouse operation, a keyboard operation, or a touch - screen operation. Therefore, after detecting a mouse operation, a keyboard operation, or a touch - screen operation, screenshot detection can be performed (i.e., detecting whether the first application is muted according to the first screenshot), which can effectively improve the detection efficiency and save power consumption.

[0017] In a possible implementation, the electronic device includes at least one of a laptop computer, a tablet computer, a portable computer, a desktop computer, and a personal digital assistant (PDA).

[0018] In a second aspect, the present application provides a chip system, which includes one or more interface circuits and one or more processors. The interface circuit and the processor are interconnected by a line. The above - mentioned chip system can be applied to an electronic device including a communication module and a memory. The interface circuit is used to receive a signal from the memory of the electronic device and send the received signal to the processor, and the signal includes computer instructions stored in the memory. When the processor executes the computer instructions, the electronic device can execute the method described in the first aspect and any of its possible design manners.

[0019] In a third aspect, the present application provides a computer-readable storage medium, which includes computer instructions. When the computer instructions run on an electronic device (such as a mobile phone), the electronic device is caused to execute the method described in the first aspect and any possible design thereof.

[0020] In a fourth aspect, the present application provides a computer program product. When the computer program product runs on a computer, the computer is caused to execute the method described in the first aspect and any possible design thereof.

[0021] In a fifth aspect, an embodiment of the present application provides a process management device for an application program, including a processor. The processor is coupled to a memory, and the memory stores program instructions. When the program instructions stored in the memory are executed by the processor, the device is caused to implement the method described in the first aspect and any possible design thereof. The device may be an electronic device or a server device; or may be a component of an electronic device or a server device, such as a chip.

[0022] In a sixth aspect, an embodiment of the present application provides a process management device for an application program. The device may be divided into different logical units or modules according to functions, and each unit or module performs different functions, so that the device executes the method described in the first aspect and any possible design thereof.

[0023] It can be understood that for the beneficial effects that can be achieved by the chip system described in the second aspect, the computer-readable storage medium described in the third aspect, the computer program product described in the fourth aspect, and the devices described in the fifth and sixth aspects, reference may be made to the beneficial effects in the first aspect and any possible design thereof, which will not be elaborated herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application;

[0025] Figure 2 It is a schematic diagram of the software architecture of an electronic device provided by an embodiment of the present application;

[0026] Figure 3 It is a schematic diagram of a process provided by an embodiment of the present application;

[0027] Figure 4 It is a schematic diagram of signal interaction between modules provided by an embodiment of the present application Figure 1 ;

[0028] Figure 5 It is a schematic diagram of a display provided by an embodiment of the present application Figure 1 ;

[0029] Figure 6 A display schematic provided by an embodiment of the present application Figure 2 ;

[0030] Figure 7 A display schematic provided by an embodiment of the present application Figure 3 ;

[0031] Figure 8 A display schematic provided by an embodiment of the present application Figure 4 ;

[0032] Figure 9 A display schematic provided by an embodiment of the present application Figure 5 ;

[0033] Figure 10 A display schematic provided by an embodiment of the present application Figure 6 ;

[0034] Figure 11 A display schematic provided by an embodiment of the present application Figure 7 ;

[0035] Figure 12 A signal interaction schematic between modules provided by an embodiment of the present application Figure 2 ;

[0036] Figure 13 A structural schematic diagram of a chip system provided by an embodiment of the present application. Detailed implementation manners

[0037] For the sake of clear and concise description of the following embodiments, a brief introduction to relevant concepts or technologies is given first:

[0038] Currently, in some scenarios (for example, a meeting scenario), even if the user triggers a mute operation, the microphone device still does not stop collecting the audio stream. During the process of collecting the audio signal by the electronic device, power consumption is generated, resulting in an increase in the overall power consumption of the electronic device and reducing the battery life of the electronic device.

[0039] In a possible design, when the microphone is muted, the electronic device can control the microphone device to stop collecting the audio stream through the microphone driver. However, controlling the microphone device through the microphone driver requires cooperation and negotiation among the vendor of the operating system of the electronic device (e.g., a PC) (e.g., Microsoft®), the vendor of the microphone driver (e.g., Intel®), the vendor of the microphone device, and the device manufacturer (e.g., Honor), and the process is complex. Moreover, since the operating system of the PC (e.g., the Windows® operating system) and the microphone driver (e.g., the Intel® driver or other third-party drivers) are closed-source, it is difficult for the device manufacturer (e.g., Honor) to control the microphone device through the microphone driver (e.g., control the microphone device to stop collecting the audio stream). Additionally, the source code of third-party applications (e.g., conference applications) is usually also closed-source, and it is difficult to obtain the microphone status (muted / unmuted status) of third-party applications.

[0040] An embodiment of the present application provides an audio signal acquisition method, which can identify the microphone status (muted / unmuted status) of a third-party application. When it is identified that the microphone status is the muted state (i.e., the microphone is muted), the currently used microphone device (microphone driver) can be replaced with a virtual microphone device, and the virtual microphone device will not trigger the microphone device to collect the audio stream (audio signal) and perform audio stream processing, thereby reducing the power consumption of the electronic device and increasing the battery life of the electronic device.

[0041] The process management method of the application program provided by the embodiment of the present application can be applied to an electronic device. The electronic device can be, for example, a tablet computer, a notebook computer (laptop), a desktop computer (desktop), a handheld computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a mobile phone, etc. The embodiment of the present application does not impose special restrictions on the specific form of the electronic device.

[0042] Figure 1 It is a schematic structural diagram of an electronic device 100 provided by an embodiment of the present application. As Figure 1As shown in the figure, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone device 170C, a headphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0043] Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0044] It can be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0045] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0046] The electronic device 100 can implement the shooting function through the ISP, camera 193, video codec, GPU, display screen 194, application processor, etc. The ISP is used to process the data fed back by the camera 193. The camera 193 is used to capture static images or videos. The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. The video codec is used to compress or decompress digital videos. The electronic device 100 can support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple coding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0047] The camera 193 can include 1 to N cameras. Each camera includes a photosensitive element (CCD / CMOS), and can sense light through the photosensitive element (CCD / CMOS), collect photons and convert them into charges.

[0048] The methods in the following embodiments can all be implemented in the electronic device 100 with the above hardware structure.

[0049] The software system of the above electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservices architecture, or cloud architecture. In the embodiments of the present invention, the Windows® operating system (OS) with a layered architecture is taken as an example to exemplarily illustrate the software structure of the electronic device 100.

[0050] In some embodiments, as Figure 2 shown, the Windows® operating system can include a user mode and a kernel mode. It should be noted that the embodiments of the present application take the Windows® operating system as an example. In other operating systems, as long as the functions implemented by each functional module are similar to those of the embodiments of the present application, the solutions of the present application can also be implemented.

[0051] Among them, the user mode is the mode in which applications run in the Windows operating system. In the user mode, applications can access some system resources, such as the file system, network, processes, and threads, etc., but cannot directly access the kernel mode or underlying hardware devices. The user mode can also be used to implement components such as the user interface of the Windows operating system (including the desktop, task bar, windows, menus, and dialog boxes). These components provide a way for users to interact and operate with the operating system. The user mode can include user programs. Among them, the user program can be a 32-bit application or a 64-bit application.

[0052] In the embodiments of the present application, the user mode may include Windows API (Application Program Interface), a first application (e.g., a conferencing application), a second application (e.g., a calling application), a third application (e.g., a housekeeper application), and a fourth application (e.g., a smart middle platform application).

[0053] Among them, Windows API can be used to manage the microphone device. The housekeeper application can switch the microphone device through Windows API.

[0054] The conferencing application can be used for multiple people to hold an online meeting (video conference or audio conference).

[0055] The calling application can be used for users to make voice calls / video calls to each other.

[0056] The housekeeper application can be used to manage the application programs installed on the PC, and can perform virus killing, junk cleaning and other processes.

[0057] In some embodiments, the housekeeper application can take screenshots of the interfaces of application programs (such as conferencing applications, calling applications), and can send the screenshots to the smart middle platform application.

[0058] The smart middle platform application can be used to determine the status (mute status or non-mute status) of the application program according to the screenshots of the application programs (such as conferencing applications, calling applications). The smart middle platform application may include a detection model, and the detection model can be trained through multiple pictures containing different types of mute icons and non-mute icons.

[0059] In some other embodiments, the housekeeper application can take screenshots of the interfaces of application programs (such as conferencing applications, calling applications), and determine the status (mute status or non-mute status) of the application program according to the screenshots.

[0060] The kernel mode may include components such as Windows Executive, Windows Kernel, device drivers, and hardware abstraction layer. Among them, Windows Executive may include basic system services, such as memory manager, process and thread manager, security management, I / O management, network, and inter-process communication. Windows Kernel may include underlying system functions, such as thread scheduling, interrupt, exception dispatching, and multi-core synchronization functions. Windows Kernel also provides some routines and basic objects for implementing high-level structures. Device drivers may include hardware device drivers, software drivers (such as file and network drivers). The hardware abstraction layer is a layer of code independent of the kernel, which separates the differences between device drivers and platforms.

[0061] In the embodiments of the present application, the device drivers in kernel mode may include an audio driver and a virtual audio driver.

[0062] Among them, the audio driver may include a microphone driver. The microphone driver can control the microphone device to collect audio signals.

[0063] The virtual audio driver can "pretend" to be an audio driver (such as a microphone driver). It does not have a corresponding physical device (microphone device) and cannot drive the microphone device to collect audio signals. The virtual audio driver corresponds to a virtual microphone device. The virtual microphone device cannot collect audio signals.

[0064] Next, the technical solutions in the embodiments of the present application will be described in conjunction with the accompanying drawings in the embodiments of the present application. Among them, in the description of the present application, unless otherwise specified, "at least one" means one or more, and "a plurality" means two or more than two. In addition, in order to facilitate a clear description of the technical solutions in the embodiments of the present application, in the embodiments of the present application, terms such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the terms "first", "second", etc. do not limit the quantity and execution order, and the terms "first", "second", etc. do not necessarily limit being different.

[0065] For the sake of easy understanding, the following will specifically introduce the audio signal acquisition method provided in the embodiments of the present application in conjunction with the accompanying drawings.

[0066] As Figure 3 shown, the embodiments of the present application provide an audio signal acquisition method, which is applied to an electronic device. The electronic device is installed with a first application, and the electronic device includes a microphone device. The method includes:

[0067] 301. In response to a first operation, display a first interface of the first application. The first interface includes a first microphone icon.

[0068] The first application may be an application program with an audio session function. Among them, the audio session function may include functions such as local recording, audio call / video call, etc.

[0069] Exemplarily, the first application may be a conference application. The conference application can be used for multiple people to hold an online conference (video conference or audio conference). The first application (such as a conference application) may be a three-party application, and its source code is usually closed source.

[0070] The first operation may be an operation of joining a meeting (e.g., a video conference). For example, a user (e.g., User A) may create / initiate a meeting and join it, or the user may join a meeting created by another user (e.g., User B), which is not limited in this application. The first interface may be a meeting interface, and the meeting interface may include a first microphone icon. The first microphone icon may be used to indicate that the current microphone is in an unmuted state (i.e., the first application is not muted).

[0071] 302. After receiving the first operation, set the current microphone device to the first microphone device.

[0072] Specifically, after receiving the first operation, based on the fact that the first application is not muted, the electronic device may set the current microphone device to the first microphone device.

[0073] Among them, the fact that the first application is not muted may be determined according to the first microphone icon on the first interface. The first microphone icon may be used to indicate that the current microphone is in an unmuted state (i.e., the first application is not muted).

[0074] It should be noted that when the current microphone device is set to the first microphone device, the electronic device may collect an audio signal through the microphone device. Among them, the first microphone device is the driver corresponding to the microphone device (e.g., an Intel® driver or other third-party drivers), which can drive the microphone device to collect an audio signal. The source code of the first microphone device is closed source.

[0075] 303. In response to the second operation, display a second interface of the first application. The second interface includes a second microphone icon, and the second microphone icon is different from the first microphone icon.

[0076] Among them, the second operation may be a muting operation. For example, the second operation may be a click operation by the user on the first microphone icon. The second microphone icon may be used to indicate that the current microphone is in a muted state (i.e., the first application is muted).

[0077] 304. After receiving the second operation, set the current microphone device to the second microphone device.

[0078] After receiving the second operation, based on the fact that the first application is muted, the electronic device may set the current microphone device to the second microphone device. The current microphone device refers to the current system default microphone device.

[0079] When the current microphone device is set to the second microphone device, the electronic device does not collect audio signals through the microphone component. Here, the second microphone device is a virtual microphone driver (virtual microphone device) without a corresponding physical component (microphone component) and cannot drive the microphone component to collect audio signals. Therefore, when the current microphone device is set to the second microphone device, the electronic device does not collect audio signals through the microphone component.

[0080] Based on the method provided in the embodiments of the present application, the microphone status (mute / non-mute status) of a third-party application (such as the first application) can be identified based on microphone icons (the first microphone icon and the second microphone icon), solving the problem that it is difficult to obtain the microphone status (mute / non-mute status) of a closed-source third-party application (the first application). Further, when it is recognized that the microphone status is the mute status (i.e., the microphone is muted), the currently used microphone device (microphone driver) can be replaced with a virtual microphone device, which will not trigger the microphone component to collect an audio stream (audio signal) and perform audio stream processing, thereby reducing the power consumption of the electronic device and increasing the battery life of the electronic device. In this way, there is no need to control the microphone component through the original closed-source microphone driver (the first microphone device), and there is no need for cooperation and negotiation among the supplier of the microphone driver (the first microphone device) of the electronic device (such as a PC) (such as Intel®), the supplier of the microphone component, the device (such as a PC) manufacturer, and the third-party application (such as a conferencing application, a calling application, etc.) manufacturer, simplifying the control process and saving human, material, and time costs.

[0081] In some embodiments, a second application is also installed on the electronic device, and the method further includes:

[0082] 305. In response to a third operation, display a third interface of the second application, where the third interface includes a third microphone icon.

[0083] Exemplarily, the second application can be a calling application, which can be used for users to make voice calls / video calls. The third operation can be an operation to initiate a voice call (video call) or answer a voice call (video call). The third interface can be an audio / video call interface, and the audio / video call interface can include a third microphone icon, and the third microphone icon can indicate that the current microphone is non-muted.

[0084] 306. After receiving the third operation, set the current microphone device to the first microphone device.

[0085] Specifically, after receiving the third operation, based on the fact that the second application is not muted, the current microphone device can be set to the first microphone device.

[0086] Among them, that the second application is not muted can be determined according to the third microphone icon on the third interface. The third microphone icon can indicate that the current microphone is in a non-muted state (i.e., the second application is not muted).

[0087] When the current microphone device is set to the first microphone device, the electronic device can collect an audio signal through the microphone component. The second application can send the audio signal to the call peer device.

[0088] 307. In response to a fourth operation, close the audio session of the second application.

[0089] Among them, the audio session can include recording, audio call / video call, etc. Exemplarily, the fourth operation can be an operation to close / exit recording, audio call / video call.

[0090] 308. After receiving the fourth operation, set the current microphone device to the second microphone device.

[0091] Specifically, after receiving the fourth operation, based on the first application being muted, set the current microphone device to the second microphone device.

[0092] Based on the audio signal collection method provided by the embodiments of the present application, when it is recognized that an application is muted, the currently used microphone device (microphone driver) can be set to the second microphone device (virtual microphone device), and this virtual microphone device will not trigger the microphone component to collect the audio stream and perform audio stream processing, thereby reducing the power consumption of the electronic device and increasing the battery life of the electronic device.

[0093] As Figure 4 shown, the embodiments of the present application provide an audio signal collection method, taking the electronic device as a PC as an example for illustration. Among them, the PC can include a first application (e.g., a conference application), a second application (e.g., a call application), a third application (e.g., a housekeeper application), a fourth application (e.g., a smart middle platform application), a first microphone device, a second microphone device, and a microphone component. The method includes:

[0094] 401a. Start the housekeeper application.

[0095] In some embodiments, in response to a user's click operation on the icon of the housekeeper application, the electronic device can start the housekeeper application. Alternatively, the user can set the housekeeper application to start automatically when the electronic device is powered on, so that the housekeeper application is automatically launched (started) when the electronic device is powered on.

[0096] 401b. The housekeeper application calls the first interface to monitor the state of the microphone component.

[0097] After the steward application is launched, the first interface of the Windows API can be called. The first interface can be used to monitor whether an application enters an audio session, and can report the process name of the application that enters the audio session to the steward application. After the application enters the audio session, the microphone device can be started through the Windows API.

[0098] Exemplarily, the first interface can be OnDefaultDeviceChanged().

[0099] 402a. In response to a first operation (such as an operation to join a meeting), the meeting application displays a first interface, and the first interface includes a first microphone icon.

[0100] Exemplarily, in response to an operation by the user on the icon of the meeting application (for example, an operation of clicking the icon of the meeting application with a mouse), the PC can start the meeting application (that is, the meeting application is launched). As Figure 5 shown, after the meeting application is launched, the PC can display window 601, and window 601 can include a join meeting control 602. Window 601 can also include a join session control 603. In response to the user's operation on the join meeting control 602 (for example, an operation of clicking the join meeting control 602 with a mouse) and the operations of inputting a meeting ID and a meeting password, or, in response to the user's operation on the join session control 603 (for example, an operation of clicking the join session control 603 with a mouse), the corresponding meeting (for example, XX meeting) can be joined.

[0101] Exemplarily, as Figure 6 shown, in response to the user's operation on the join meeting control 602 and the operations of inputting a meeting ID and a meeting password (an example of the first operation), or, in response to the user's operation on the join session control 603 (another example of the first operation), the PC can display a meeting interface 610 (the first interface), and the meeting interface 610 can include a microphone icon 611 (an example of the first microphone icon). The microphone icon 611 is used to indicate that the current microphone status is non-muted.

[0102] 402b. The meeting application sets the current microphone device to the first microphone device.

[0103] After receiving the first operation, the meeting application can enter the audio session, set the current microphone device to the first microphone device. The first microphone device can start the microphone device to collect an audio stream.

[0104] 403a. The first interface of the Windows API monitors that the meeting application enters the audio session.

[0105] After the first interface monitors that the conferencing application enters an audio session (i.e., the conferencing application activates the microphone device), step 403b can be executed, that is, the process name (or application name, in this embodiment of the present application, the process name is used as an example for illustration) of the conferencing application is sent to the steward application.

[0106] 403b. The first interface of the Windows API sends the process name of the conferencing application to the steward application.

[0107] 404. The steward application determines whether the process name of the conferencing application matches the whitelist.

[0108] The steward application can determine whether the process name of the conferencing application matches the process name in the whitelist. That is, it determines whether the process name of the conferencing application exists in the whitelist. Among them, the whitelist may include the process names of one or more preset processes. The preset processes may include processes related to the audio session. The processes related to the audio session may refer to processes that can call the microphone driver (the first microphone device) to control the microphone device to collect audio data.

[0109] Exemplarily, the processes related to the audio session may include the process of the conferencing application, the process of the call application (such as the phone application, chat application, etc.), the process of the recording application, etc. That is, the whitelist may include the process name of the conferencing application, the process name of the call application, the process name of the recording application, etc.

[0110] In a possible design, the whitelist may not include the process name of the browser. The scenario of conducting an audio session (such as joining a meeting or making an audio call) through the browser (web page) belongs to an abnormal scenario and is not within the scope of discussion of this application.

[0111] When the steward application determines that the process name of the conferencing application matches the process name in the whitelist, step 405 can be executed. Otherwise, the steward application can wait for the notification message sent by the first interface next time and determine whether the process name carried in the notification message sent next time matches the process name in the whitelist.

[0112] In some embodiments, when the steward application determines that the process name of the conferencing application matches the process name in the whitelist, it can also determine whether the current is an audio session scenario according to the command line in the process details. For example, if the command line includes "type=iron", it indicates that the current is an audio session scenario, that is, the user has joined a meeting.

[0113] 405. The steward application takes a screenshot of the interface (such as the first interface) of the conferencing application to obtain a first screenshot.

[0114] The steward application can take a screenshot of the interface of the meeting application (e.g., the first interface) based on a preset rule to obtain a first screenshot, and the first screenshot includes a first microphone icon. The preset rule can be to take a screenshot of a preset area in the interface (or window) of the meeting application at a preset ratio. That is, the area of the screenshot is the preset area in the interface of the meeting application, and the ratio of the size of the screenshot to the size of the interface of the meeting application can be fixed. The preset area can be, for example, the lower left area, the lower center area, the lower right area, etc., and the present application does not make specific limitations.

[0115] In some embodiments, different preset applications in the whitelist can be bound to different preset areas, that is, the preset areas corresponding to different preset applications can be different. The preset area corresponding to each preset application can be designed according to the characteristics of the preset application (the position where the microphone icon is displayed in the preset application). In this way, different preset areas can be intercepted for different applications more pertinently, making the screenshot result more accurate.

[0116] Exemplarily, as Figure 7 shown in (a) of, when the window of the meeting application is maximized, the area of the screenshot can be the bottom area 631 of the interface 630; as Figure 7 shown in (b) of, when the window of the meeting application is restored, the area of the screenshot can be the bottom area 633 of the interface 632; as Figure 7 shown in (c) of, when the window of the meeting application is minimized, the area of the screenshot can be the bottom area 635 of the interface 634.

[0117] 406. The steward application sends the first screenshot to the intelligent middle platform application.

[0118] The steward application can send the first screenshot to the intelligent middle platform application based on the interprocess communication (IPC) mechanism.

[0119] 407. The intelligent middle platform application determines whether the meeting application is muted according to the first screenshot and returns the determination result to the steward application.

[0120] In a possible design, the intelligent middle platform application can input the first screenshot into a detection model to determine whether the meeting application is muted through the detection model. Among them, the detection model can be trained by multiple pictures including different types of mute icons and non - mute icons.

[0121] Among them, the detection result includes two cases. The first case is that the meeting application is muted, and the second case is that the meeting application is not muted. When the detection result is that the meeting application is not muted, step 408 can be executed. When the detection result is that the meeting application is muted, step 415 can be executed.

[0122] 408. Based on the fact that the conferencing application is not muted, the steward application installs the second microphone driver and sets the second microphone device to the disabled state (i.e., disables the second microphone device).

[0123] Exemplarily, the steward application can disable the second microphone device through SetEndpointVisibility(). After the second microphone device is set to the disabled state, the second microphone device is invisible to the user, avoiding accidental user operations.

[0124] Wherein, the current microphone device refers to the microphone driver currently used by the electronic device. Exemplarily, as shown in (a) of Figure 8 interface 640 includes a control 641. In response to an operation of clicking on the control 641 by the mouse pointer, as shown in (b) of Figure 8 a list box 642 can be displayed. The list box 642 includes all the microphone drivers installed in the electronic device, and the current microphone driver is the selected microphone driver 643 (consistent with the system (microphone (6 - USB audio driver (AudioDevice), an example of the first microphone device))). Optionally, the list box 642 can also include speaker drivers, etc., which are not limited in this application.

[0125] When the current microphone device is the first microphone device (e.g., 6 - USB Audio Device), the electronic device can collect audio signals through the microphone device. Among them, the first microphone device is the driver corresponding to the microphone device and can drive the microphone device to collect audio signals.

[0126] 409. The steward application calls the second interface of the Windows API to monitor the target operation.

[0127] Wherein, the target operation can include mouse operations, keyboard operations, and touch screen operations.

[0128] Exemplarily, the second interface can be SetWindowsHookEx. The second interface can be used to monitor mouse operations, keyboard operations, and touch screen operations.

[0129] In some embodiments, the second interface can monitor mouse operations or touch screen operations in a preset area (e.g., the area below the window of the conferencing application), and keyboard operations of a preset key (e.g., the screenshot shortcut key).

[0130] After the second interface monitors mouse operations, keyboard operations, and touch screen operations, it can notify the steward application so that the steward application can timely detect whether the application program (e.g., the conferencing application) is muted.

[0131] In the embodiments of the present application, step 409 and subsequent steps (such as steps 410 - 414) can be executed in parallel with steps 404 - 408.

[0132] 410. In response to a second operation (for example, an operation to mute by using a mouse, a keyboard, or a touch screen), the conferencing application displays a second interface, and the second interface includes a second microphone icon, which is different from the first microphone icon.

[0133] Exemplarily, the second operation can be an operation to mute by using a mouse, a keyboard, or a touch screen. For example, as shown in (a) of Figure 9 , the interface 650 (an example of the first interface) can include a microphone icon 651 (an example of the first microphone icon). In response to the second operation, as shown in (b) of Figure 9 , the PC can display an interface 652 (an example of the second interface), and the interface 652 includes a microphone icon 653 (an example of the second microphone icon), and the microphone icon 653 is an icon indicating that the microphone is muted.

[0134] 411a. The second interface of the Windows API monitors the second operation (an operation to mute by using a mouse, a keyboard, or a touch screen).

[0135] 411b. The second interface of the Windows API notifies the steward application that the conferencing application has currently received the second operation (an operation to mute by using a mouse, a keyboard, or a touch screen).

[0136] 412. The steward application takes a screenshot of the second interface of the conferencing application to obtain a second screenshot.

[0137] Wherein, the second screenshot includes the second microphone icon.

[0138] 413. The steward application sends the second screenshot to the intelligent middle - platform application.

[0139] 414. The intelligent middle - platform application determines whether the conferencing application is muted according to the second screenshot, and returns the determination result to the steward application.

[0140] For the relevant descriptions of steps 412 - 414, reference can be made to steps 405 - 407, and details are not described herein again.

[0141] It should be noted that, compared with continuously performing screenshot detection, performing screenshot detection after the second operation (that is, detecting whether the conferencing application is muted according to the screenshot of the preset area of the conferencing application) can effectively improve the detection efficiency and save power consumption.

[0142] In the case where it is determined according to the second screenshot that the conferencing application is muted, step 415 can be executed.

[0143] 415. Based on the conferencing application being muted, the steward application sets the second microphone device to the enabled state and sets the current microphone device to the second microphone device.

[0144] Exemplarily, the steward application can enable the second microphone device through SetEndpointVisibility(), that is, set the second microphone device to the enabled state. The steward application can set the current microphone device to the second microphone device through SetDefaultEndpoint().

[0145] Among them, the second microphone device is a virtual microphone driver without a corresponding physical device (microphone device) and cannot drive the microphone device to collect audio signals. Therefore, when the current microphone device is set to the second microphone device, the electronic device does not collect audio signals through the microphone device.

[0146] After the second microphone device is set to the startup state, the second microphone device is visible to the user. Exemplarily, after setting the current microphone device to the second microphone device, as shown in (a) of Figure 10 in response to an operation of clicking on control 641 by the mouse pointer, as shown in (b) of Figure 10 a list box 642 can be displayed. The list box 642 includes all the microphone drivers installed on the electronic device, and the current microphone driver (system default microphone driver) is the selected microphone driver 644 (Honor Audio Driver, an example of the second microphone device). Optionally, the list box 642 can also include speaker drivers, etc., which are not limited in this application.

[0147] In this way, when the conferencing application is muted, the electronic device does not collect audio signals through the microphone device, which can reduce the power consumption of the electronic device and increase the battery life of the electronic device.

[0148] Furthermore, the embodiments of the present application may further include the following steps:

[0149] 416a. In response to an operation of closing the audio session of the conferencing application, the conferencing application closes the relevant interface of the audio session (such as, the first interface or the second interface).

[0150] Exemplarily, as shown in (a) of Figure 11 interface 650 (an example of the first interface) may include control 661. In response to an operation of the user on control 661 (for example, an operation of clicking on control 661 by the mouse), as shown in Figure 11As shown in (b) of , a pop-up window 662 can be displayed. The pop-up window 662 can include multiple options, such as options to leave the meeting, end the meeting, and cancel, etc. In response to the user's operation on the option to leave the meeting or end the meeting (for example, the operation of clicking the option to leave the meeting or end the meeting by mouse), the meeting application can exit the audio session and close the interface 650.

[0151] 416b. The first interface of the Windows API monitors that the meeting application exits the audio session.

[0152] The first interface can also be used to monitor whether an application exits the audio session, and can report the process name corresponding to the application that exits the audio session to the steward application.

[0153] 416c. The first interface of the Windows API notifies the steward application that the meeting application exits the audio session.

[0154] 416d. The steward application disables the second microphone device and sets the current microphone device to the first microphone device.

[0155] The steward application disables the second microphone device, that is, the steward application sets the second microphone device to the disabled state. And, the steward application can set the current microphone device to the first microphone device (for example, the steward application can set the current microphone device to the first microphone device through SetDefaultEndpoint()). In this way, it can be avoided that the next audio session cannot use the microphone device to collect the audio stream.

[0156] In some embodiments, when the housekeeper application determines that the current scenario is an audio session scenario, it may store the identifier corresponding to the current audio session and the sound state (mute state or non-mute state) corresponding to the current audio session in the audio session list. That is, the audio session list may store the identifiers of the currently active audio sessions and the sound states corresponding to these audio sessions. Among them, the identifier corresponding to the audio session may be the application name, process name, or session identifier corresponding to the audio session, which is not limited herein. Subsequently, when a new audio session is started or an already started audio session is closed, the housekeeper application may update the audio session list. For example, when a new audio session is added, the housekeeper application may store the identifier corresponding to the new audio session and its corresponding sound state in the audio session list. When an audio session is closed, the identifier and sound state corresponding to the closed audio session may be deleted from the audio session list. Further, the housekeeper application may set the current microphone device according to the updated audio session list. When all the remaining audio sessions in the updated audio session list are in the mute state, the housekeeper application may set the current microphone device to the second microphone device. In this way, the electronic device does not collect audio signals through the microphone device, which can save power consumption. When there is a non-mute audio session among all the remaining audio sessions in the updated audio session list, the current microphone device may be set to the first microphone device. In this way, the problem that a non-mute audio session cannot use the microphone device to collect the audio stream can be avoided.

[0157] Taking the audio session list including the audio session corresponding to the first application (the identifier corresponding to this audio session may be 1, and the sound state corresponding to this audio session may be the mute state) as an example, the situation of the PC starting a new audio session and closing an existing (already started) audio session will be exemplified below.

[0158] As Figure 12 shown, after step 415 and before step 416a, the method provided by the embodiments of the present application may further include:

[0159] 420. In response to a third operation, display a third interface of a second application (for example, a call application), where the third interface includes a third microphone icon.

[0160] Exemplarily, the third operation may be an operation to start an audio session (such as initiating an audio call or answering an audio call). The third interface may be an audio / video call interface, and the audio / video call interface may include a third microphone icon, and the third microphone icon may indicate that the current microphone is non-muted.

[0161] An audio session based on the second application is started. The steward application can store the identifier corresponding to the audio session of the second application (e.g., 2) and the sound state corresponding to this audio session (such as, non-muted state) in the audio session list. In this case, the audio session list can be as shown in Table 1:

[0162] Table 1

[0163] Identifier corresponding to the audio session Sound state corresponding to the audio session 1 (corresponding to the first application) Mute state 2 (corresponding to the second application) Non - mute state

[0164] 421a. The first interface of the Windows API monitors that the call application enters the audio session.

[0165] After the first interface monitors that the call application starts the audio session, it can execute step 421b, that is, send the process name of the conference application to the steward application.

[0166] 421b. The first interface of the Windows API sends the process name of the call application to the steward application.

[0167] 422. The steward application determines whether the process name of the call application matches the whitelist.

[0168] 423. The steward application takes a screenshot of the window of the call application to obtain a third screenshot.

[0169] Among them, the third screenshot includes a third microphone icon.

[0170] 424. The steward application sends the third screenshot to the intelligent middle platform application.

[0171] 425. The intelligent middle platform application determines whether the call application is muted according to the third screenshot and returns the judgment result to the steward application.

[0172] Steps 422 - 425 can refer to the relevant descriptions of the above steps 404 - 407 and will not be elaborated here.

[0173] 426. Based on the call application not being muted, the steward application sets the current microphone device to the first microphone device.

[0174] That is, after receiving the third operation, based on the second application not being muted, the steward application can set the current microphone device to the first microphone device (that is, switch from the second microphone device to the first microphone device). In the case of setting the current microphone device to the first microphone device, the audio signal can be collected through the microphone device so that the second application can perform an audio call normally.

[0175] After step 426, the following steps can also be included:

[0176] 427a. In response to a fourth operation to close the audio session of the call application, the call application closes the relevant interface of the audio session.

[0177] 427b. The first interface of the Windows API monitors that the call application exits the audio session.

[0178] The first interface can also be used to monitor whether an application exits the audio session, and can report the process name corresponding to the application that exits the audio session to the steward application.

[0179] 427c. The first interface of the Windows API notifies the steward application that the call application exits the audio session.

[0180] 428. The steward application sets the current microphone device to the second microphone device.

[0181] Exemplarily, the fourth operation can be an operation to close the audio session of the second application. Based on the closing of the audio session of the second application, the steward application can delete the audio session of the second application from the audio session list, and query the remaining audio sessions in the audio session list after deletion and their mute states. In this case, the audio session list can be as shown in Table 2:

[0182] Table 2

[0183] Identifier corresponding to the audio session Sound state corresponding to the audio session 1 Mute state

[0184] When all the remaining audio sessions in the audio session list (such as Table 2) are in the mute state, the steward application can set the current microphone device to the second microphone device (that is, switch from the first microphone device to the second microphone device). In this way, the electronic device does not collect audio signals through the microphone device, which can save power consumption.

[0185] In addition, if there are non - mute audio sessions among all the remaining audio sessions in the audio session list, the steward application can not switch the microphone device, that is, keep the current microphone device as the first microphone device. In this way, the problem that non - mute audio sessions cannot use the microphone device to collect audio streams can be avoided.

[0186] It should be noted that the above execution order is only an exemplary illustration, and this embodiment does not specifically limit the execution sequence between each step.

[0187] Based on the audio signal acquisition method provided in the embodiments of the present application, when it is recognized that the application is muted, the currently used microphone device (microphone driver) can be set to a second microphone device (virtual microphone device), and this virtual microphone device will not trigger the microphone device to collect the audio stream and perform audio stream processing, thereby reducing the power consumption of the electronic device and increasing the battery life of the electronic device.

[0188] Some embodiments of the present application provide an electronic device, which may include: a touch screen, a memory, and one or more processors. The touch screen, the memory, and the processor are coupled. The memory is used to store computer program code, and the computer program code includes computer instructions. When the processor runs the computer instructions, the electronic device can execute each function or step performed by the electronic device in the above method embodiments. The structure of the electronic device may refer to Figure 1 the structure of the electronic device 100 shown.

[0189] Embodiments of the present application further provide a chip system (for example, a system on a chip (SoC)), as Figure 13 shown, the chip system includes at least one processor 1301 and at least one interface circuit 1302. The processor 1301 and the interface circuit 1302 can be interconnected by a line. For example, the interface circuit 1302 can be used to receive signals from other devices (such as the memory of the electronic device). For another example, the interface circuit 1302 can be used to send signals to other devices (such as the processor 1301 or the touch screen of the electronic device). Exemplarily, the interface circuit 1302 can read the instructions stored in the memory and send the instructions to the processor 1301. When the instructions are executed by the processor 1301, the electronic device can execute each step in the above embodiments. Of course, the chip system may also include other discrete devices, and the embodiments of the present application do not make specific limitations on this.

[0190] Embodiments of the present application further provide a computer-readable storage medium, which includes computer instructions. When the computer instructions run on the above electronic device, the electronic device is enabled to execute each function or step performed by the electronic device (such as a mobile phone) in the above method embodiments.

[0191] Embodiments of the present application further provide a computer program product. When the computer program product runs on the electronic device, the electronic device is enabled to execute each function or step performed by the electronic device (such as a mobile phone) in the above method embodiments.

[0192] From the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be allocated to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0193] In several embodiments provided in this application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be electrical, mechanical or other forms.

[0194] The units described as separate components may or may not be physically separated. The components displayed as units can be one physical unit or multiple physical units, that is, they can be located in one place or distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0195] In addition, in each embodiment of this application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0196] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read only memory (ROM), random access memory (RAM), magnetic disks or optical discs and other various media that can store program codes.

[0197] The above content is only a specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be covered within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims

1. A method for collecting audio signals, characterized in that: Applied to an electronic device, the electronic device has a first application installed thereon, and the electronic device includes a microphone device, the method comprising: In response to a first operation, displaying a first interface of the first application, wherein the first interface includes a first microphone icon; After receiving the first operation, setting the current microphone device as the first microphone device; in the case where the current microphone device is set as the first microphone device, the electronic device collects audio signals through the microphone device; In response to a second operation, displaying a second interface of the first application, the second interface including a second microphone icon, the second microphone icon being different from the first microphone icon; After receiving the second operation, the current microphone device is set as the second microphone device; when the current microphone device is set as the second microphone device, the electronic device does not collect audio signals through the microphone device.

2. The method according to claim 1, characterized in that After receiving the second operation, setting the current microphone device as the second microphone device comprises: After receiving the second operation, based on the first application being muted, setting the current microphone device to the second microphone device.

3. The method according to claim 1 or 2, characterized in that: The electronic device also has a second application installed thereon, and the method further includes: In response to a third operation, displaying a third interface of the second application, the third interface including a third microphone icon; the third microphone icon is used to indicate that the second application is not muted; After receiving the third operation, the current microphone device is set as the first microphone device.

4. The method according to claim 3, characterized in that After receiving the third operation, setting the current microphone device to the first microphone device includes: After receiving the third operation, based on the second application not being muted, setting a current microphone device to the first microphone device.

5. The method according to claim 4, characterized in that The method further comprises: In response to a fourth operation, closing the audio session of the second application; After receiving the fourth operation, the current microphone device is set as the second microphone device.

6. The method according to claim 1 or 2, characterized in that: The electronic device further includes a third application. After receiving the first operation, the method further includes: The third application determines whether the process name of the first application matches a whitelist, where the whitelist includes process names of one or more preset processes; When the process name of the first application matches the whitelist, the third application takes a screenshot of the first interface to obtain a first screenshot including the first microphone icon, and determines whether the first application is muted according to the first screenshot; In a case where the first application is not muted, the second microphone device is installed and the second microphone device is disabled.

7. The method according to claim 6, characterized in that The step of, after receiving the second operation, setting the current microphone device to the second microphone device based on that the first application is muted, comprises: After receiving the second operation, the third application takes a screenshot of the second interface to obtain a second screenshot including the second microphone icon, and determines whether the first application is muted according to the second screenshot; In a case where the first application is muted, the third application enables the second microphone device and sets the current microphone device as the second microphone device.

8. The method according to claim 6, characterized in that The electronic device further includes a fourth application, and the determining whether the first application is muted according to the first screenshot includes: The third application sends the first screenshot to the fourth application; The fourth application determines whether the first application is muted according to the first screenshot, and returns the determination result to the third application.

9. The method according to claim 8, characterized in that The determining, according to the first screenshot, whether the first application is muted includes: The first screenshot is input into a detection model to obtain an output result of the detection model, where the output result is used to indicate whether the first application is muted.

10. The method according to claim 1 or 2, characterized in that: The second operation includes mouse operation, keyboard operation or touch screen operation.

11. The method according to claim 1 or 2, characterized in that: The electronic device includes at least one of a notebook computer, a tablet computer, a portable computer, a desktop computer, and a PDA.

12. An electronic device, characterized in that: The electronic device comprises a processor, and the processor is used to run a computer program stored in a memory, so that the electronic device implements the method according to any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on a processor, the method according to any one of claims 1 to 11 is implemented.

Images