Multi-device multi-sound card audio synchronous acquisition method and system thereof
By establishing a second thread in the non-primary audio device, synchronous acquisition of data from multiple sound cards is achieved, solving the problem that the Android system can only acquire data from one audio device sound card. This supports intelligent stereo acquisition and playback in multi-sound source scenarios, improving the convenience of video conferencing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 上海赛连信息科技有限公司
- Filing Date
- 2023-01-10
- Publication Date
- 2026-06-05
AI Technical Summary
In existing technologies, the Android system can only collect sound card data from one audio device at a time, and cannot support the synchronous collection of multi-sound card data from multiple audio devices. In particular, it cannot achieve intelligent stereo collection and playback in multi-sound source scenarios.
A second thread is created in the non-primary audio device to call multiple audio input sound cards for data acquisition. The data acquisition of multiple sound cards is synchronized through caching and merging, and users can select multiple audio inputs to acquire data at the same time.
It enables simultaneous acquisition of data from multiple audio devices and multiple sound cards, improving the convenience of video conferencing and expanding application scenarios. In particular, it provides a better user experience and stereo effect in scenarios where multiple sound sources are distributed across different audio devices.
Smart Images
Figure CN116185329B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of video communication technology, and more specifically, to a method and system for synchronous audio acquisition across multiple devices and multiple sound cards. Background Technology
[0002] Internet-based video communication technology is widely used in video conferencing scenarios for both work and personal life. However, traditional Android systems only support simultaneous capture of sound from multiple sound cards on a single audio device, such as... Figure 1a and Figure 1b As shown, in the Android system's user interface, i.e., the app or hardware terminal device on the phone, the audio acquisition chain consists of one process, one acquisition thread, one acquisition thread corresponding to one audio device, and one audio device can correspond to multiple sound cards, but only one audio device's sound card can be acquired at a time. Figure 1a The acquisition thread shown can only acquire data from sound cards 0 to 2 that primarily drive the audio device. Figure 1b The acquisition thread shown can only acquire data from sound card 3, which is not the primary audio device; that is, it can only acquire data from device A or device B, not both simultaneously. Users cannot select multiple or single audio inputs. Generally, the Android system will automatically switch the acquisition thread that acquires data from different audio devices based on the order in which user devices are connected, such as automatically switching to the USB audio device corresponding to the most recently connected USB audio input.
[0003] The aforementioned existing technology can only collect sound card data from one audio device at a time, which is extremely unfriendly to scenarios where the sound source comes from different audio devices. It can only forcibly collect sound card data from the audio device with the highest priority, and it cannot achieve intelligent stereo sound collection and playback from multiple audio devices at different distances based on the user's own location. Summary of the Invention
[0004] This invention provides a method and system for synchronous audio acquisition across multiple devices and sound cards, such as... Figure 2As shown, modifications were made to the audio device selection process. If an audio input corresponding to a non-primary audio device is selected, the request is routed to that non-primary audio device. Within the non-primary audio device, in addition to operating the capture thread of its own sound card, if the user selects an audio input not belonging to that non-primary audio device, the non-primary audio device will create a separate thread, the second thread. This second thread opens other sound cards not belonging to this driver and captures the data, caching it. As shown in the diagram, the second thread of the non-primary audio device calls sound cards 0-2 of the primary audio device to capture and cache data. The capture thread not only reads data captured by the sound cards of the non-primary audio devices but also reads the cached data captured by the sound cards of the primary audio devices.
[0005] Therefore, in this invention, users can select one or more audio inputs, and even if the selected audio inputs come from different audio devices, their sound card data can be collected simultaneously.
[0006] In a first aspect, the present invention provides a method for simultaneous audio acquisition across multiple devices and multiple sound cards, characterized in that the method comprises:
[0007] When the user selects the first audio input belonging to a non-master audio device, a capture thread is established with the non-master audio device.
[0008] The first sound card corresponding to the first audio input is sampled to obtain the first sound card data;
[0009] When the user selects a second audio input that does not belong to the non-master audio device, the non-master audio device establishes a second thread;
[0010] The second thread calls the second sound card corresponding to the second audio input and collects data from the second sound card.
[0011] Cache the data from the second sound card;
[0012] The data from the first sound card and the data from the second sound card are read, merged, and output through the acquisition thread.
[0013] Secondly, the present invention also provides a multi-device, multi-sound-card audio synchronous acquisition system, characterized in that the system comprises: an establishment device, an acquisition device, a non-master-driven audio device, a buffer device, and a reading device; wherein...
[0014] The establishment device is used to establish a acquisition thread with the non-primary audio device when the user selects the first audio input belonging to the non-primary driving audio device.
[0015] The acquisition device is used to acquire data from the first sound card corresponding to the first audio input, thereby obtaining the first sound card data;
[0016] When the user also selects a second audio input that does not belong to the non-master audio device, the non-master audio device is used to establish a second thread;
[0017] The second thread is used to call the second sound card corresponding to the second audio input, and the acquisition device is also used to acquire data from the second sound card.
[0018] The caching device is used to cache the data of the second sound card;
[0019] The reading device is used to read the first sound card data and the second sound card data, merge the two data, and output them through the acquisition thread.
[0020] The multi-device, multi-sound-card audio synchronous acquisition method and system provided by this invention, by establishing a second thread in a non-master audio device, can arbitrarily acquire data from multiple sound cards of multiple audio devices, regardless of the audio device itself. It supports simultaneous acquisition of data from multiple audio devices and multiple sound cards, which is particularly beneficial for video conferencing scenarios with multiple sound sources distributed across different audio devices, greatly improving the ease of use of video conferencing terminals and expanding into many new application scenarios. Attached Figure Description
[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0022] Figure 1a and Figure 1b This is a schematic diagram of the audio acquisition chain in the existing Android system;
[0023] Figure 2 This is a schematic diagram of a multi-device, multi-sound-card audio synchronous acquisition link provided in an embodiment of the present invention;
[0024] Figure 3 This is a flowchart of the multi-device, multi-sound-card audio synchronous acquisition method provided in this embodiment of the invention;
[0025] Figure 4 This is the audio input selection interface provided in the embodiments of the present invention;
[0026] Figure 5This is a schematic diagram of a multi-device, multi-sound-card audio synchronous acquisition system provided in an embodiment of the present invention. Detailed Implementation
[0027] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Invention Overview
[0029] As mentioned above, the present invention provides a method and system for simultaneous audio acquisition across multiple devices and multiple sound cards, which supports simultaneous acquisition of data from multiple audio devices and multiple sound cards. It is mainly used in video conferencing scenarios where there are multiple sound sources distributed across different audio devices.
[0030] Exemplary methods
[0031] Figure 3 This is a flowchart of a multi-device, multi-sound-card audio synchronous acquisition method provided in an embodiment of the present invention. This embodiment includes the following steps:
[0032] S301: When the user selects the first audio input belonging to a non-master audio device, an acquisition thread is established with the non-master audio device.
[0033] Audio devices connected to user equipment include main driver audio devices and non-main driver audio devices. Main driver audio devices are generally built into the user equipment, while non-main driver audio devices are generally external to the user equipment. For example, a USB audio input that can be plugged into and unplugged from the user equipment is the first audio input belonging to the category of non-main driver audio devices.
[0034] Since there can only be one acquisition thread per process in the audio acquisition link, and one acquisition thread can only correspond to one audio device, in order to ensure that the sound card data corresponding to the audio input selected by the user can be acquired, whenever it is detected that the user selects the first audio input belonging to a non-main driving audio device, an acquisition thread is only established with the non-main driving audio device.
[0035] For example, such as Figure 2 As shown, when the user selects both the first audio input belonging to the non-primary audio device and the second audio input belonging to the primary audio device, in this embodiment, only the acquisition thread is established with the non-primary audio device.
[0036] S302: Collect data from the first sound card corresponding to the first audio input to obtain the first sound card data.
[0037] After the user selects the first audio input, the system will operate the thread of the sound card device corresponding to the first audio input, based on the required sound card device. Figure 2As shown, sound card 3 is the first sound card. Data is collected from sound card 3.
[0038] The first sound card data is the raw stereo data without mixing. The acquisition step retains the raw stereo sound card data to prepare for the mixing of all subsequent sound card data.
[0039] S303: When the user selects a second audio input that does not belong to the non-master audio device, the non-master audio device establishes a second thread.
[0040] The second audio input includes a Phoenix head input, a 3.5mm line input, an HDMI input, an RCA input, or a Bluetooth input.
[0041] Furthermore, in order to simultaneously acquire sound card data from the first audio input and sound card data from the second audio input, a second thread is created in the non-main audio device that can call the sound card from the second audio input.
[0042] It should be noted that the user can only select the first audio input and / or the second audio input after a connection is established with the user device.
[0043] For example, such as Figure 4 The audio input selection interface shown displays five audio inputs that the user can select, indicating that all five audio inputs are connected to the user's device. Audio inputs not displayed, i.e., those unselectable by the user, are those not connected to the user's device. In the diagram, the user has selected the second audio input (USB) and the first audio input (HDMI). This will establish a capture thread with the USB audio device and create a second thread within the USB audio device to access the HDMI audio input. Simultaneously, data will be captured from both the first sound card corresponding to the USB audio input and the second sound card corresponding to the HDMI audio input.
[0044] Generally, user devices will have a few commonly used audio inputs connected. When the user turns on the device, they are prompted to select the audio input to connect. Users can set default selections according to their needs and readjust them by clicking the audio input options when necessary. Because this invention introduces a second thread into the non-primary audio device, it can simultaneously collect sound card data from different audio devices without considering whether the sound cards being collected belong to the same audio device. Therefore, any audio input connected to the user device—that is, the audio inputs displayed in the audio input selection interface—can be arbitrarily selected by the user, and its corresponding sound card data can be collected simultaneously.
[0045] When the first audio input and / or the second audio input establish a connection with the user equipment during operation, the audio input that establishes the connection is automatically selected.
[0046] For example, if user A is using their video terminal device for a video conference, and then inserts a USB audio input into the terminal device, the system will automatically select the USB audio input and establish a capture thread with the USB audio device without user A needing to enable the audio input option or check the USB audio input option.
[0047] Considering that users adding new audio input connections during meetings generally indicates that they need to enable that audio input, there's no need for users to manually modify the audio input options; it can be automatically selected. This method saves users time and effort, making it more efficient and practical.
[0048] S304: The second thread calls the second sound card corresponding to the second audio input and collects data from the second sound card.
[0049] The second sound card data is also the raw stereo without any mixing processing.
[0050] After the user selects the second audio input, based on the sound card device to be acquired, in addition to the thread operating the sound card device that is not the primary audio device, a second thread will be created to open the second sound card that does not belong to the non-primary audio device and perform acquisition, such as... Figure 2 As shown, sound cards 0-2 are the second sound cards. The second thread calls sound cards 0-2 and collects data from them.
[0051] S305: Cache the data from the second sound card.
[0052] Figure 2 The second thread shown not only collects data from sound cards 0 to 2, but also caches the collected data from sound cards 0 to 2 to facilitate the subsequent merging and transmission of all sound card data.
[0053] S306: Read the data from the first sound card and the data from the second sound card, merge the two data, and output them through the acquisition thread.
[0054] like Figure 2 As shown, the acquisition thread reads the data acquired by the first sound card and simultaneously reads the data acquired by the second sound card cached in the non-main driver audio device. It then merges all the sound card data and outputs it to the user interface.
[0055] Furthermore, the user interface receives the merged sound card data and performs mixing processing on the original sound card data as needed.
[0056] like Figure 2The collected sound card data shown are all raw two-channel audio. Users can select the channels and their combinations according to their needs. For example, sound card 0 has channels a and b; sound card 1 has channels c and d; sound card 2 has channels e and f; and sound card 3 has channels g and h. Users can choose to mix channels a, c, and h as one channel and channels d and f as another channel in the mixing interface, based on their personal preferences. In another scenario, the receiving end may want adaptive mixing. For example, one channel might always hear the sound from sound card 3, while the other channel can choose to mix in the sound from sound cards 0, 1, and 2. In this case, the optimal sound from sound cards 0, 1, and 2 will be adaptively selected to mix with sound card 3.
[0057] Additionally, the system can detect the orientation, distance, and facial orientation of each audio input relative to the user. Based on this information, the data from each audio input is stereo-mixed, allowing the user to discern the location and distance of each input. When the user's facial orientation changes, the volume of the audio input directly in front of the user's face is amplified. The orientation of each audio input can be manually set during the initial setup phase. For example, by capturing a video of the meeting room, the administrator can click on the location of each audio input in the frame using a mouse. Since the audio inputs typically remain unchanged during the meeting, this setting only needs to be done once. For microphones worn by the speaker, their location can be determined using other wireless positioning methods, such as UWB wireless positioning similar to that used in Apple's AirTag.
[0058] For example, in a large conference room giving a foreign language presentation, there is one speaker (A), one simultaneous interpreter (B), and 100 attendees. The conference room is equipped with cameras to capture the position, distance, and facial orientation of each audio input relative to the attendees. Each attendee wears headphones. Assuming speaker A's microphone is 10 meters to the right front of attendee A, and simultaneous interpreter B's audio input is 15 meters to the left rear of attendee A, the volume ratio of attendee A's left and right channels can be adjusted so that attendee A can distinguish speaker A's voice as coming from the right front and simultaneous interpreter B's voice as coming from the left rear, with simultaneous interpreter B's voice being slightly quieter due to the greater distance. When attendee A turns their face towards simultaneous interpreter B, the volume of simultaneous interpreter B's voice in the headphones will be increased. When the cameras cannot detect attendee A, the default mixing mode is activated. The cameras can detect the orientation of each attendee through facial recognition and perform different mixing processes on the left and right channels of each attendee based on the detected orientation, resulting in a higher level of immersion for the audience. For remote attendees, the system can detect their orientation using their camera and mix the audio signals provided to them based on the orientation of the camera relative to each audio input source, giving them an immersive experience.
[0059] In addition to simultaneously capturing the location, distance, and facial orientation of all listeners through cameras, data can also be collected through headphones or VR glasses with sensors. Compared to headphones with sensors for each person, cameras are obviously less expensive.
[0060] The method dynamically mixes stereo audio from various audio inputs based on the user's location, distance, and facial orientation. This is more intelligent and faster than manual parameter settings and is more suitable for scenarios involving multiple sound sources. The method can also be used in multiplayer VR battle games to achieve a strong sense of presence.
[0061] The following describes the steps included in this embodiment when the user deselects audio input.
[0062] First, when the user deselects the first audio input and the second audio input, the acquisition thread is closed; the first sound card, the second thread, and the second sound card are closed; and the data cache of the second sound card is cleared.
[0063] That is, if the user cancels all previously selected audio inputs, all previously enabled threads and sound cards will be closed, and the cache will be cleared.
[0064] Second, when the user deselects the first audio input but does not deselect the second audio input, the acquisition thread is maintained; the first sound card is turned off; the second thread is maintained and data acquisition from the second sound card continues; and the data from the second sound card is cached.
[0065] That is, if the data of the first sound card, which is not the main audio device, does not need to be collected, but the data of the second sound card, which is not a main audio device, needs to be collected, then only the first sound card is turned off, the data collection of the first sound card is stopped, the collection thread and the second thread are kept open, and the data collection and buffering of the second sound card continue.
[0066] For example, Figure 2 When only the audio input corresponding to sound card 3 is deselected, sound card 3 is turned off and the acquisition of data from sound card 3 is stopped. The acquisition thread and the second thread continue to be retained. The second thread still calls sound cards 0 to 2 and acquires and caches their data.
[0067] Third, when the user keeps selecting the first audio input and deselects the second audio input, the acquisition thread is maintained; the second sound card and the second thread are turned off; and the data cache of the second sound card is cleared.
[0068] For example, Figure 2 When the audio input corresponding to sound cards 0-2 is deselected, the second thread is closed, sound cards 0-2 are closed, data acquisition of sound cards 0-2 is stopped, the data cache of sound cards 0-2 is cleared, the acquisition thread is kept open, and data acquisition of sound card 3 continues.
[0069] In summary, even if an audio input is deselected midway, it will not affect the sound card data acquisition of other audio inputs that have not been deselected. The audio inputs that have not been deselected will still be acquired according to the original acquisition thread. This method effectively avoids the interruption of sound from other audio inputs due to the deselection of an audio input.
[0070] Exemplary System
[0071] Accordingly, this invention also provides a multi-device, multi-sound-card audio synchronous acquisition system. Figure 5 This is a schematic diagram of a multi-device, multi-sound-card audio synchronous acquisition system 100 provided in an embodiment of the present invention, as shown below. Figure 5 As shown, the system 100 provided in this embodiment includes: an establishment device 110, an acquisition device 120, a non-master-driven audio device 130, a buffer device 140, and a reading device 150; wherein
[0072] The establishment device 110 is used to establish a acquisition thread with the non-main driving audio device 130 when the user selects the first audio input belonging to the non-main driving audio device 130.
[0073] The acquisition device 120 is used to acquire data from the first sound card corresponding to the first audio input, thereby obtaining the first sound card data;
[0074] When the user selects a second audio input that does not belong to the non-master audio device 130, the non-master audio device 130 is used to establish a second thread;
[0075] The second thread is used to call the second sound card corresponding to the second audio input, and the acquisition device 120 is also used to acquire data from the second sound card.
[0076] The caching device 140 is used to cache the second sound card data;
[0077] The reading device 150 is used to read the first sound card data and the second sound card data, merge the two data, and output them through the acquisition thread.
[0078] The system 100 also includes a mixing device 160, which is used to receive the merged sound card data and mix it as needed.
[0079] The system 100 also includes a shut-off device 170;
[0080] When the user deselects the first audio input and the second audio input, the shutting device 170 is used to shut down the acquisition thread, shut down the first sound card, the second thread and the second sound card, and clear the data cache of the second sound card.
[0081] When the user deselects the first audio input but does not deselect the second audio input...
[0082] The acquisition device 120 is also used to maintain the acquisition thread;
[0083] The shut-off device 170 is used to shut down the first sound card;
[0084] The acquisition device 120 is also used to maintain the second thread and continue to acquire data from the second sound card;
[0085] The caching device 140 is also used to cache the data of the second sound card.
[0086] When the user keeps selecting the first audio input and deselects the second audio input,
[0087] The acquisition device 120 is also used to maintain the acquisition thread;
[0088] The shutdown device 170 is used to shut down the second sound card and the second thread, and clear the data cache of the second sound card.
[0089] The first audio input includes a USB audio input;
[0090] The second audio input includes a Phoenix head input, a 3.5mm line input, an HDMI input, an RCA input, or a Bluetooth input.
[0091] Both the first sound card data and the second sound card data are raw stereo audio without any mixing processing.
[0092] The system 100 also includes a selection device 180;
[0093] When the first audio input and / or the second audio input establish a connection with the user equipment, the selection device 180 is used by the user to select the established audio input.
[0094] When the first audio input and / or the second audio input establish a connection with the user equipment during operation, the selection device 180 is used to automatically select the audio input that has established the connection.
[0095] The mixing device 160 includes a detection unit 161 and a stereo mixing unit 162; wherein
[0096] The detection unit 161 is used to detect the position and distance of each audio input relative to the user, as well as the user's facial orientation.
[0097] The stereo mixing unit 162 is used to perform stereo mixing on the data from each audio input according to the detected location, distance and facial orientation of each audio input relative to the user, so that the user can hear the location and distance of the audio input.
[0098] The mixing device 160 also includes a volume adjustment unit 163, which amplifies the volume of the audio input facing the user's face when the user's face orientation changes.
[0099] It should be noted that although the operation of the multi-device, multi-sound-card audio synchronous acquisition method of the present invention is described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.
[0100] Furthermore, although several devices, units, or modules of a multi-device, multi-sound-card audio synchronous acquisition system have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules described above can be embodied in a single module. Conversely, the features and functions of a single module described above can be further divided and embodied by multiple modules.
[0101] While the spirit and principles of the invention have been described with reference to several specific embodiments, it should be understood that the invention is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for ease of description. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
[0102] 1. A method for synchronous audio acquisition across multiple devices and sound cards, characterized in that the method comprises:
[0103] When the user selects the first audio input belonging to a non-master audio device, a capture thread is established with the non-master audio device.
[0104] The first sound card corresponding to the first audio input is sampled to obtain the first sound card data;
[0105] When the user selects a second audio input that does not belong to the non-master audio device, the non-master audio device establishes a second thread;
[0106] The second thread calls the second sound card corresponding to the second audio input and collects data from the second sound card.
[0107] Cache the data from the second sound card;
[0108] The data from the first sound card and the data from the second sound card are read, merged, and output through the acquisition thread.
[0109] 2. The audio acquisition method according to item 1, characterized in that the method further includes: receiving the merged sound card data and performing audio mixing processing on it as needed.
[0110] 3. The audio acquisition method according to claim 1 or 2, characterized in that the method further includes:
[0111] The acquisition thread is closed when the user deselects both the first and second audio inputs.
[0112] Disable the first sound card, the second thread, and the second sound card;
[0113] Clear the data cache of the second sound card.
[0114] 4. The audio acquisition method according to item 1 or 2, characterized in that the method further includes:
[0115] The acquisition thread is maintained when the user deselects the first audio input but does not deselect the second audio input.
[0116] Turn off the first sound card;
[0117] Maintain the second thread and continue acquiring data from the second sound card;
[0118] The data from the second sound card is cached.
[0119] 5. The audio acquisition method according to item 1 or 2, characterized in that the method further includes:
[0120] The acquisition thread is maintained while the user keeps the first audio input selected and deselects the second audio input;
[0121] Turn off the second sound card and the second thread;
[0122] Clear the data cache of the second sound card.
[0123] 6. The audio acquisition method according to any one of claims 1-5, wherein the first audio input includes a USB audio input;
[0124] The second audio input includes a Phoenix head input, a 3.5mm line input, an HDMI input, an RCA input, or a Bluetooth input.
[0125] 7. The audio acquisition method according to any one of claims 1-6, characterized in that the first sound card data and the second sound card data are both raw stereo data without mixing processing.
[0126] 8. The audio acquisition method according to any one of claims 1-7, characterized in that the user can select the first audio input and / or the second audio input only after a connection is established with the user equipment.
[0127] 9. The audio acquisition method according to any one of claims 1-8, characterized in that, when the first audio input and / or the second audio input establish a connection with the user equipment during the operation of the user equipment, the audio input that establishes the connection is automatically selected.
[0128] 10. The audio acquisition method according to item 2, characterized in that the step of receiving the merged sound card data and performing mixing processing on it according to requirements further includes:
[0129] Detect the position and distance of each audio input relative to the user, as well as the user's facial orientation;
[0130] Based on the detected location, distance, and facial orientation of each audio input relative to the user, the data from each audio input are stereo-mixed so that the user can hear the location and distance of the audio input.
[0131] 11. The audio acquisition method according to item 10, characterized in that the step of receiving the merged sound card data and performing mixing processing on it according to requirements further includes:
[0132] When the user's face changes direction, the volume of the audio input facing the user's face is amplified.
[0133] 12. A multi-device, multi-sound-card audio synchronous acquisition system, characterized in that the system comprises: an establishment device, an acquisition device, a non-master-driven audio device, a buffer device, and a reading device; wherein...
[0134] The establishment device is used to establish a acquisition thread with the non-primary audio device when the user selects the first audio input belonging to the non-primary driving audio device.
[0135] The acquisition device is used to acquire data from the first sound card corresponding to the first audio input, thereby obtaining the first sound card data;
[0136] When the user also selects a second audio input that does not belong to the non-master audio device, the non-master audio device is used to establish a second thread;
[0137] The second thread is used to call the second sound card corresponding to the second audio input, and the acquisition device is also used to acquire data from the second sound card.
[0138] The caching device is used to cache the data of the second sound card;
[0139] The reading device is used to read the first sound card data and the second sound card data, merge the two data, and output them through the acquisition thread.
[0140] 13. The audio acquisition system according to item 12, characterized in that the system further includes a mixing device for receiving the merged sound card data and mixing it as needed.
[0141] 14. The audio acquisition system according to claim 12 or 13, characterized in that the system further includes a shut-off device;
[0142] When the user deselects the first audio input and the second audio input, the shutting device is used to shut down the acquisition thread, shut down the first sound card, the second thread and the second sound card, and clear the data cache of the second sound card.
[0143] 15. The audio acquisition system according to claim 12 or 13, characterized in that the system further includes a shut-off device;
[0144] When the user deselects the first audio input but does not deselect the second audio input...
[0145] The acquisition device is also used to maintain the acquisition thread;
[0146] The shut-off device is used to shut down the first sound card;
[0147] The acquisition device is also used to maintain the second thread and continue to acquire data from the second sound card;
[0148] The caching device is also used to cache the data of the second sound card.
[0149] 16. The audio acquisition system according to claim 12 or 13, characterized in that the system further includes a shut-off device;
[0150] When the user keeps selecting the first audio input and deselects the second audio input,
[0151] The acquisition device is also used to maintain the acquisition thread;
[0152] The shutdown device is used to shut down the second sound card and the second thread, and clear the data cache of the second sound card.
[0153] 17. The audio acquisition system according to any one of claims 12-16, wherein the first audio input includes a USB audio input;
[0154] The second audio input includes a Phoenix head input, a 3.5mm line input, an HDMI input, an RCA input, or a Bluetooth input.
[0155] 18. The audio acquisition system according to any one of claims 12-17, wherein the first sound card data and the second sound card data are both raw stereo data without mixing processing.
[0156] 19. The audio acquisition system according to any one of claims 12-18, characterized in that the system further includes a selection device;
[0157] When the first audio input and / or the second audio input establish a connection with the user equipment, the selection device is used by the user to select the audio input that has been established.
[0158] 20. The audio acquisition system according to any one of claims 12-19, characterized in that the system further includes a selection device;
[0159] When the first audio input and / or the second audio input establish a connection with the user equipment during operation, the selection device is used to automatically select the audio input that has established the connection.
[0160] 21. The audio acquisition system according to claim 13, characterized in that the mixing device comprises a detection unit and a stereo mixing unit; wherein...
[0161] The detection unit is used to detect the position and distance of each audio input relative to the user, as well as the user's facial orientation.
[0162] The stereo mixing unit is used to perform stereo mixing on the data from each audio input according to the detected location, distance, and facial orientation of each audio input relative to the user, so that the user can hear the location and distance of the audio input.
[0163] 22. The audio acquisition system according to claim 22, wherein the mixing device further includes a volume adjustment unit for increasing the volume of the audio input facing the user's face when the user's facial orientation changes.
Claims
1. A method for synchronous audio acquisition across multiple devices and sound cards, characterized in that, The method operates on the Android operating system and includes: When the user selects the first audio input belonging to a non-primary audio device, a first acquisition thread is established with the non-primary audio device to drive the sound card of the non-primary audio device to acquire audio. The first acquisition thread acquires data from the first sound card corresponding to the first audio input, thereby obtaining the first sound card data; When the user further selects a second audio input that does not belong to the non-master audio device, a second thread is established inside the non-master audio device. The second thread calls the second sound card corresponding to the second audio input and collects data from the second sound card. Cache the data from the second sound card; Read the data from the first sound card and the data from the second sound card, merge the two data, and output them through the first acquisition thread; When a user selects both the first audio input belonging to a non-primary audio device and the second audio input belonging to the primary audio device, the first acquisition thread is established only with the non-primary audio device.
2. The method for synchronous audio acquisition across multiple devices and sound cards according to claim 1, characterized in that, The method further includes: receiving the merged sound card data and mixing it according to requirements.
3. The method for synchronous audio acquisition across multiple devices and sound cards according to claim 1 or 2, characterized in that, The method further includes: The first acquisition thread is closed when the user deselects both the first audio input and the second audio input. Disable the first sound card, the second thread, and the second sound card; Clear the data cache of the second sound card.
4. The method for synchronous audio acquisition across multiple devices and sound cards according to claim 1 or 2, characterized in that, The method further includes: The first acquisition thread is maintained when the user deselects the first audio input but does not deselect the second audio input. Turn off the first sound card; Maintain the second thread and continue acquiring data from the second sound card; The data from the second sound card is cached.
5. The method for synchronous audio acquisition across multiple devices and sound cards according to claim 1 or 2, characterized in that, The method further includes: The first acquisition thread is maintained when the user keeps the first audio input selected and deselects the second audio input; Turn off the second sound card and the second thread; Clear the data cache of the second sound card.
6. The method for synchronous audio acquisition across multiple devices and sound cards according to claim 1 or 2, characterized in that, The first audio input includes a USB audio input; The second audio input is the master audio input.
7. The method for synchronous audio acquisition across multiple devices and sound cards according to claim 1 or 2, characterized in that, Both the first sound card data and the second sound card data are raw stereo audio without any mixing processing.
8. The method for synchronous audio acquisition across multiple devices and sound cards according to claim 1 or 2, characterized in that, The user can only select an audio input once the first audio input and / or the second audio input are connected to the user device.
9. The method for synchronous audio acquisition across multiple devices and sound cards according to claim 1 or 2, characterized in that, When the first audio input and / or the second audio input establish a connection with the user equipment during operation, the audio input that establishes the connection is automatically selected.
10. The method for synchronous audio acquisition across multiple devices and sound cards according to claim 2, characterized in that, The step of receiving and merging the sound card data and mixing it according to requirements also includes: Detect the position and distance of each audio input relative to the user, as well as the user's facial orientation; Based on the detected location, distance, and facial orientation of each audio input relative to the user, the data from each audio input are stereo-mixed so that the user can hear the location and distance of the audio input.
11. The method for synchronous audio acquisition across multiple devices and sound cards according to claim 10, characterized in that, The step of receiving and merging the sound card data and mixing it according to requirements also includes: When the user's face changes direction, the volume of the audio input facing the user's face is amplified.
12. A multi-device, multi-sound-card audio synchronous acquisition system, characterized in that, The system, running on the Android operating system, includes: a setup device, a capture device, a non-primary audio device, a cache device, and a read device; wherein... The establishment device is used to establish a first acquisition thread with the non-primary audio device when the user selects the first audio input belonging to the non-primary audio device, so as to drive the sound card of the non-primary audio device to perform audio acquisition. The acquisition device is used to acquire data from the first sound card corresponding to the first audio input through the first acquisition thread, thereby obtaining the first sound card data. When the user further selects a second audio input that does not belong to the non-master audio device, the non-master audio device establishes a second thread within it; The second thread is used to call the second sound card corresponding to the second audio input, and the acquisition device is also used to acquire data from the second sound card. The caching device is used to cache the data of the second sound card; The reading device is used to read the first sound card data and the second sound card data, merge the two data and output them through the first acquisition thread; When a user selects both the first audio input belonging to a non-primary audio device and the second audio input belonging to the primary audio device, the first acquisition thread is established only with the non-primary audio device.
13. The multi-device, multi-sound-card audio synchronous acquisition system according to claim 12, characterized in that, The system also includes a mixing unit, which receives the merged sound card data and mixes it as needed.
14. The multi-device, multi-sound-card audio synchronous acquisition system according to claim 12 or 13, characterized in that, The system also includes a shut-off device; When the user deselects the first audio input and the second audio input, the shutting device is used to shut down the first acquisition thread, shut down the first sound card, the second thread and the second sound card, and clear the data cache of the second sound card.
15. The multi-device, multi-sound-card audio synchronous acquisition system according to claim 12 or 13, characterized in that, The system also includes a shut-off device; When the user deselects the first audio input but does not deselect the second audio input... The acquisition device is also used to maintain the first acquisition thread; The shut-off device is used to shut down the first sound card; The acquisition device is also used to maintain the second thread and continue to acquire data from the second sound card; The caching device is also used to cache the data of the second sound card.
16. The multi-device, multi-sound-card audio synchronous acquisition system according to claim 12 or 13, characterized in that, The system also includes a shut-off device; When the user keeps selecting the first audio input and deselects the second audio input, The acquisition device is also used to maintain the first acquisition thread; The shutdown device is used to shut down the second sound card and the second thread, and clear the data cache of the second sound card.
17. The multi-device, multi-sound-card audio synchronous acquisition system according to claim 12 or 13, characterized in that, The first audio input includes a USB audio input; The second audio input includes a Phoenix head input, a 3.5mm line input, an HDMI input, an RCA input, or a Bluetooth input.
18. The multi-device, multi-sound-card audio synchronous acquisition system according to claim 12 or 13, characterized in that, Both the first sound card data and the second sound card data are raw stereo audio without any mixing processing.
19. The multi-device, multi-sound-card audio synchronous acquisition system according to claim 12 or 13, characterized in that, The system also includes a selection device; When the first audio input and / or the second audio input establish a connection with the user equipment, the selection device is used by the user to select the audio input that has been established.
20. The multi-device, multi-sound-card audio synchronous acquisition system according to claim 12 or 13, characterized in that, The system also includes a selection device; When the first audio input and / or the second audio input establish a connection with the user equipment during operation, the selection device is used to automatically select the audio input that has established the connection.
21. The multi-device, multi-sound-card audio synchronous acquisition system according to claim 13, characterized in that, The mixing device includes a detection unit and a stereo mixing unit; wherein The detection unit is used to detect the position and distance of each audio input relative to the user, as well as the user's facial orientation. The stereo mixing unit is used to perform stereo mixing on the data from each audio input according to the detected location, distance, and facial orientation of each audio input relative to the user, so that the user can hear the location and distance of the audio input.
22. The multi-device, multi-sound-card audio synchronous acquisition system according to claim 21, characterized in that, The mixing device also includes a volume adjustment unit for amplifying the volume of the audio input facing the user's face when the user's facial orientation changes.