Audio acquisition method and device, display screen, electronic device and storage medium

Abstract

The application provides an audio acquisition method and device, a display screen, electronic equipment and a storage medium, wherein the method comprises: obtaining a first recording request; under the triggering of the first recording request, controlling a hardware abstraction layer to issue a second recording request carrying an audio recording parameter to a real-time operating system; under the triggering of the second recording request, controlling the real-time operating system to acquire audio played by a sound card corresponding to the second recording request based on the audio recording parameter to obtain source audio; and obtaining the source audio recorded by the real-time operating system. The method, device, electronic equipment and storage medium provided by the application realize the acquisition and recording of the source audio played by the sound card, guarantee the availability and convenience of the source audio acquisition, and provide conditions for subsequent direct use of the acquired source audio for voice processing such as voice transcription and voice translation.

Description

Technical Field

[0001] This invention relates to the field of audio acquisition technology, and in particular to an audio acquisition method, device, display screen, electronic device, and storage medium. Background Technology

[0002] Currently, Android smart screens support input sources such as HDMI (High Definition Multimedia Interface), VGA (Video Graphics Array), and OPS (Open Pluggable Specification). The smart screen is used simply as a display.

[0003] In existing technologies, in Android mode, smart screens use AudioRecord to record audio for speech transcription. However, AudioRecord can only collect audio data from microphones or sounds played by the Android system and then upload the audio data to the transcription server for transcription. It cannot collect audio output from sources connected to the Android smart screen.

[0004] Therefore, Android smart screens cannot capture source audio in source mode, nor can they perform speech transcription or translation. Summary of the Invention

[0005] This invention provides an audio acquisition method, device, display screen, electronic device, and storage medium to solve the defect in the prior art that Android smart screens cannot acquire source audio for speech transcription.

[0006] This invention provides an audio acquisition method, comprising:

[0007] Get the first recording request;

[0008] Upon triggering the first recording request, the control hardware abstraction layer sends a second recording request carrying audio recording parameters to the real-time operating system;

[0009] Upon triggering the second recording request, the real-time operating system is controlled to collect the audio played by the sound card corresponding to the second recording request based on the audio recording parameters, thereby obtaining the source audio.

[0010] Obtain the source audio recorded by the real-time operating system.

[0011] According to an audio acquisition method provided by the present invention, the control hardware abstraction layer sends a second recording request carrying audio recording parameters to the real-time operating system, including:

[0012] The hardware abstraction layer is controlled to search for the sound card, obtain the target sound card parameters, and then the target sound card parameters are set into the audio recording parameters;

[0013] The hardware abstraction layer is controlled to generate a second recording request carrying the audio recording parameters and send it to the real-time operating system.

[0014] According to an audio acquisition method provided by the present invention, before the hardware abstraction layer generates a second recording request carrying the audio recording parameters, the method further includes:

[0015] The application parameters are placed into the audio recording parameters, which include at least one of the number of channels, sampling rate, and bit depth; the application parameters are transmitted from the framework layer to the hardware abstraction layer.

[0016] According to an audio acquisition method provided by the present invention, the step of controlling the real-time operating system to acquire the audio played by the sound card corresponding to the second recording request based on the audio recording parameters includes:

[0017] The real-time operating system is controlled to record the audio played by the sound card corresponding to the second recording request based on the pulse code modulation device, so as to obtain the source audio with parameters consistent with the audio recording parameters, and store it in the storage space specified by the second recording request;

[0018] The step of acquiring the source audio recorded by the real-time operating system includes:

[0019] The source audio is retrieved from the storage space specified in the second recording request.

[0020] According to an audio acquisition method provided by the present invention, after acquiring the source audio recorded by the real-time operating system, the method further includes:

[0021] Based on the source audio, speech transcription and / or speech translation are performed to obtain the text corresponding to the source audio and display it.

[0022] According to an audio acquisition method provided by the present invention, the step of performing speech transcription and / or speech translation based on the source audio includes:

[0023] Acquire system audio and / or ambient audio acquired via microphone;

[0024] The system audio and / or the ambient audio are merged with the source audio to obtain merged audio;

[0025] The merged audio is sent to a server to request the server to perform speech transcription and / or speech translation on the merged audio.

[0026] The present invention also provides an audio acquisition device, comprising:

[0027] The acquisition unit is used to acquire the first recording request;

[0028] The sending unit is used to control the hardware abstraction layer to send a second recording request carrying audio recording parameters to the real-time operating system when triggered by the first recording request;

[0029] The acquisition unit is used to control the real-time operating system to acquire the audio played by the sound card corresponding to the second recording request based on the audio recording parameters when the second recording request is triggered, so as to obtain the source audio.

[0030] The source audio acquisition unit is used to acquire source audio recorded by the real-time operating system.

[0031] The present invention also provides a display screen, the display screen being equipped with a processor, the processor loading a display system, the display system including a framework layer, a hardware abstraction layer, and a real-time operating system, characterized in that the framework layer is used to acquire a first recording request and send the first recording request to the hardware abstraction layer; the hardware abstraction layer is used to, upon triggering the first recording request, send a second recording request carrying audio recording parameters to the real-time operating system; the real-time operating system is used to, upon triggering the second recording request, collect audio played by the sound card corresponding to the second recording request based on the audio recording parameters, to obtain source audio; the framework layer is also used to acquire the source audio and acquire the text corresponding to the source audio for display on the display screen.

[0032] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the audio acquisition method described above.

[0033] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the audio acquisition method as described above.

[0034] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements any of the audio acquisition methods described above.

[0035] The audio acquisition method, device, display screen, electronic device, and storage medium provided by this invention, under the trigger of a second recording request, control the real-time operating system to acquire the audio played by the sound card corresponding to the second recording request based on audio recording parameters, thereby obtaining the source audio and then acquiring the source audio recorded by the real-time operating system. Thus, the acquisition and recording of source audio played by the sound card is realized, ensuring the availability and convenience of source audio acquisition, and providing conditions for subsequent direct use of the acquired source audio for speech transcription, speech translation, and other speech processing. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0037] Figure 1 This is one of the flowcharts illustrating the audio acquisition method provided by the present invention;

[0038] Figure 2 This is a flowchart illustrating step 120 in the audio acquisition method provided by the present invention;

[0039] Figure 3 This is a schematic diagram of the speech transcription and / or speech translation process provided by the present invention;

[0040] Figure 4 This is the second flowchart illustrating the audio acquisition method provided by the present invention;

[0041] Figure 5 This is a schematic diagram of the audio acquisition device provided by the present invention;

[0042] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0044] The terms "first," "second," etc., used in the specification and claims of this invention are used to distinguish similar objects and are not used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and that the objects distinguished by "first," "second," etc., are generally of the same class.

[0045] This invention provides an audio acquisition method. Figure 1 This is one of the flowcharts illustrating the audio acquisition method provided by the present invention, such as... Figure 1 As shown, this method can be applied to processors of various display screens, such as those of smart screens, to achieve the acquisition of source audio. The following method uses an Android smart screen as an example and includes:

[0046] Step 110: Obtain the first recording request.

[0047] Specifically, current display screen processors typically run an operating system. Taking Android as an example, it can be divided into a framework layer, a hardware abstraction layer, and a real-time operating system. In step 110, the framework layer can obtain a first recording request. Here, the framework layer can be the Framework layer, and the first recording request can be a recording request for source audio. The first recording request can be issued by the user through any of various human-computer interaction methods, such as voice interaction or remote control. The source can be HDMI, VGA, or OPS; this embodiment of the invention does not specifically limit this.

[0048] Step 120: Upon triggering the first recording request, control the hardware abstraction layer to send a second recording request carrying audio recording parameters to the real-time operating system.

[0049] Specifically, after receiving the first recording request, the hardware abstraction layer can be controlled to send a second recording request carrying audio recording parameters to the real-time operating system, triggered by the first recording request. Here, the hardware abstraction layer can be AudioHAL (hardware abstraction layer). It can be understood that all hardware-related operations required by the upper layers of the Android system need to call HAL-related APIs (Application Programming Interfaces), such as GPS (Global Positioning System), BT (Bit-Torrent), input devices, graphocs, cameras, and audio.

[0050] The audio recording parameters here refer to the audio recording settings, which may include the number of channels, the sampling rate, and the bit depth. The number of channels is an integer; mono means one channel, and stereo means two channels. The sampling rate refers to the number of times the source audio is sampled per second. The bit depth refers to the quantization precision; the higher the quantization precision, the higher the resolution, and the more realistic the recorded and played-out sound. The bit depth can be 16 bits, 32 bits, or 8 bits; this embodiment of the invention does not specifically limit this.

[0051] Here, the audio recording parameters can be passed to the hardware abstraction layer by the framework layer calling JNI (Java Native Interface), or they can be pre-stored in the hardware abstraction layer, or they can be found in the hardware abstraction layer. This embodiment of the invention does not specifically limit this.

[0052] The second recording request here could be an ioctl (input / output control) call. ioctl is a system call specifically for device input / output operations. This call takes a device-specific request code as input, and the system call's functionality depends entirely on the request code. The Real-Time Operating System (RTOS) here could be a kernel.

[0053] Step 130: When the second recording request is triggered, the real-time operating system is controlled to collect the audio played by the sound card corresponding to the second recording request based on the audio recording parameters, so as to obtain the source audio.

[0054] Specifically, upon triggering a second recording request, the real-time operating system can be controlled to acquire the audio played by the sound card corresponding to the second recording request based on the audio recording parameters carried in the second recording request, thus obtaining the source audio. For example, the real-time operating system can acquire the audio played by the sound card corresponding to the second recording request through a Pulse Code Modulation (PCM) device based on the audio recording parameters, thus obtaining the source audio. Here, source audio refers to audio recorded in source mode.

[0055] Step 140: Obtain the source audio recorded by the real-time operating system.

[0056] Specifically, after the real-time operating system (RTOS) completes the acquisition of the source audio, the framework layer can obtain the source audio recorded by the ROS. For example, the ROS can return the recorded source audio to the hardware abstraction layer (HAL), and then the HAL can return it to the framework layer. Thus, the Android smart screen can complete the acquisition and recording of source audio played through the sound card.

[0057] The method provided in this embodiment of the invention, when triggered by a second recording request, controls the real-time operating system to collect the audio played by the sound card corresponding to the second recording request based on audio recording parameters, thereby obtaining the source audio and then acquiring the source audio recorded by the real-time operating system. Thus, the acquisition and recording of the source audio played by the sound card is realized, ensuring the availability and convenience of the source audio acquisition, and providing conditions for the subsequent direct use of the acquired source audio for speech transcription, speech translation and other speech processing.

[0058] Based on the above embodiments, Figure 2 This is a flowchart illustrating step 120 of the audio acquisition method provided by the present invention, as follows: Figure 2 As shown, step 120 includes:

[0059] Step 121: Control the hardware abstraction layer to search for the sound card, obtain the target sound card parameters, and put the target sound card parameters into the audio recording parameters.

[0060] Specifically, upon triggering the first recording request, the hardware abstraction layer can be controlled to perform a sound card lookup, obtaining target sound card parameters including the target sound card identifier and the target device identifier on the target sound card, and then setting the target sound card parameters into the audio recording parameters. Here, the target sound card identifier and the target device identifier on the target sound card can be IDs (Identifications). For example, the device ID on the target sound card indicates which device it is on the sound card; assuming the driver remains unchanged, the device ID is usually fixed.

[0061] The hardware abstraction layer can be controlled to locate the sound card through the SAN (Storage Area Network) and obtain the target sound card parameters. SAN here refers to a storage technology that relies on Fibre Channel to provide higher throughput for connections between servers and storage devices, supporting longer distances and more reliable connectivity.

[0062] After obtaining the target sound card parameters, these parameters can be incorporated into the audio recording parameters. For example, the target sound card identifier and the device identifier on the target sound card can be incorporated into the audio recording parameters. The resulting audio recording parameters may include the target sound card identifier, the device identifier on the target sound card, the number of channels, the sampling rate, and the bit depth. This embodiment of the invention does not impose specific limitations on these parameters.

[0063] Step 122: Control the hardware abstraction layer to generate a second recording request carrying the audio recording parameters, and send it to the real-time operating system.

[0064] Specifically, after obtaining the audio recording parameters, the hardware abstraction layer can be controlled to generate a second recording request carrying the audio recording parameters and send it to the real-time operating system.

[0065] For example, the framework layer can control the hardware abstraction layer to generate a second recording request carrying audio recording parameters using the open-source framework ALSA (Advanced Linux SoundArchitecture) and send it to the kernel.

[0066] ALSA here stands for Advanced Linux Audio Architecture. At the application layer, ALSA provides alsa-lib, and applications can control the underlying audio hardware simply by calling the API (libasound.so) provided by alsa-lib.

[0067] The method provided in this embodiment of the invention controls the hardware abstraction layer to generate a second recording request carrying audio recording parameters and sends it to the real-time operating system. This facilitates the subsequent control of the real-time operating system to collect the audio played by the sound card corresponding to the second recording request based on the audio recording parameters, thereby obtaining the source audio.

[0068] Based on the above embodiments, before step 122, the method further includes:

[0069] The application parameters are placed into the audio recording parameters, which include at least one of the number of channels, sampling rate, and bit depth; the application parameters are transmitted from the framework layer to the hardware abstraction layer.

[0070] Specifically, before executing step 122, application parameters can be placed into audio recording parameters. These application parameters are transmitted from the framework layer to the hardware abstraction layer. The application parameters may include the number of channels, sampling rate, and bit depth, or the number of channels and sampling rate, or the number of channels. This embodiment of the invention does not specifically limit this.

[0071] As shown in Table 1, the number of channels here is an integer. Mono means one channel, and stereo means two channels. The sampling rate refers to the number of times the source audio is sampled per second. The bit depth refers to the quantization precision; the higher the quantization precision, the higher the resolution, and the more realistic the recorded and played-out sound. The bit depth can be 16 bits, 32 bits, or 8 bits; this embodiment of the invention does not impose a specific limitation on this.

[0072] Table 1. Application Parameters

[0073] Channels Rate Bits Number of channels Sampling rate Number of digits Int Int Int

[0074] The method provided in this embodiment of the invention incorporates application parameters into audio recording parameters. The application parameters include at least one of the number of channels, sampling rate, and bit depth, providing a parameter basis for subsequent audio recording of the source.

[0075] Based on the above embodiments, step 130 includes:

[0076] The real-time operating system is controlled to record the audio played by the sound card corresponding to the second recording request based on the pulse code modulation device, so as to obtain the source audio with parameters consistent with the audio recording parameters, and store it in the storage space specified by the second recording request.

[0077] Specifically, when the second recording request is triggered, the real-time operating system can be controlled to record the audio played by the sound card corresponding to the second recording request based on the pulse code modulation device, so as to obtain the source audio with parameters consistent with the audio recording parameters.

[0078] After obtaining the source audio with parameters consistent with the audio recording parameters, the source audio can be stored in the storage space specified in the second recording request. This storage space can be a buffer, but this embodiment of the invention does not specifically limit its use.

[0079] Accordingly, step 140 includes:

[0080] The source audio is retrieved from the storage space specified in the second recording request.

[0081] Specifically, after storing the source audio in the storage space specified in the second recording request, the source audio can be retrieved from the storage space specified in the second recording request.

[0082] For example, the source audio can be stored in the buffer specified by the second recording request, and the source audio can be retrieved from the buffer specified by the second recording request. The buffer specified by the second recording request usually exists in the real-time operating system, and the source audio can be retrieved from the real-time operating system. The source audio then goes through the hardware abstraction layer and returns to the framework layer.

[0083] Based on the above embodiments, after step 140, the method further includes:

[0084] Based on the source audio, speech transcription and / or speech translation are performed to obtain the text corresponding to the source audio and display it.

[0085] Specifically, after acquiring the source audio recorded by the real-time operating system, speech-to-text and / or speech-to-text translation can be performed based on the source audio to obtain and display the corresponding text.

[0086] This invention can perform speech transcription based on the source audio, speech translation based on the source audio, or both speech transcription and speech translation based on the source audio. The embodiments of the present invention do not specifically limit this.

[0087] Here, based on the source audio, speech transcription and / or speech translation can be performed using a cascaded multilayer convolutional neural network (CNN), a deep neural network (DNN), or a combination of CNN and DNN, etc. The embodiments of the present invention do not specifically limit this.

[0088] For example, in source mode, set-top boxes, computers, etc., can connect to the smart screen as source devices, and the smart screen plays the audio and video provided by the source. Assuming the user controls the smart screen to enter subtitle mode, the smart screen can obtain the source audio based on steps 110-140 above, and perform speech-to-text transcription to obtain the corresponding text. This text is then displayed as subtitles while the video provided by the source is played. For instance, when watching a foreign language film provided by a set-top box, the smart screen can obtain the audio of the foreign language film through audio acquisition methods, and then obtain and display the corresponding subtitles through speech-to-text transcription and translation. Similarly, when watching a conference video connected to a computer, the smart screen can obtain the real-time conference audio through audio acquisition methods, and then perform speech-to-text transcription to create conference subtitles and meeting minutes.

[0089] The method provided in this invention performs speech transcription and / or speech translation based on source audio, obtains and displays the text corresponding to the source audio, thereby improving the convenience of speech transcription and / or speech translation.

[0090] Considering that in practical applications, speech processing such as speech translation and speech transcription requires uploading to a server for processing, and the audio that needs to be processed may include not only the source audio but also audio collected from other channels, in order to improve processing efficiency, this embodiment of the invention provides the following method:

[0091] Figure 3 This is a flowchart illustrating the speech transcription and / or speech translation process provided by the present invention, such as... Figure 3 As shown, the process of speech transcription and / or speech translation based on the source audio includes:

[0092] Step 310: Acquire system audio and / or ambient audio acquired via microphone.

[0093] Specifically, system audio and / or ambient audio captured by the microphone can be acquired. Here, system audio refers to the audio played by the Android system, and ambient audio refers to the speaker's audio captured by the microphone.

[0094] Here, system audio can be acquired, ambient audio collected by the microphone can be acquired, or both system audio and ambient audio collected by the microphone can be acquired. This embodiment of the invention does not specifically limit this.

[0095] Step 320: Merge the system audio and / or the ambient audio with the source audio to obtain merged audio.

[0096] Specifically, after acquiring system audio and / or ambient audio obtained from a microphone, the system audio and / or ambient audio can be merged with the source audio to obtain merged audio.

[0097] For example, system audio, ambient audio, and source audio can be processed to ensure that the sampling rate and quantization precision parameters of multiple audio signals are consistent. After processing the system audio, ambient audio, and source audio, a mixing algorithm can be used to merge the system audio, ambient audio, and source audio to obtain a combined audio. The mixing algorithm here refers to the process of performing calculations on multiple audio input signals according to certain rules (adding multiple audio signals and then performing amplitude limiting) to obtain a single mixed audio signal, which is then used as the output.

[0098] Step 330: Send the merged audio to the server to request the server to perform speech transcription and / or speech translation on the merged audio.

[0099] Specifically, after obtaining the merged audio, the merged audio can be sent to a server to request the server to perform speech transcription and / or speech translation on the merged audio. Here, the server can use a cascaded multilayer convolutional neural network, a deep neural network, or a combination of multilayer convolutional neural networks and deep neural networks to perform speech transcription and / or speech translation on the merged audio, etc., and the embodiments of the present invention do not specifically limit this.

[0100] The method provided in this invention merges system audio and / or ambient audio with source audio to obtain merged audio. The merged audio covers richer audio information. The merged audio is sent to a server to request the server to perform speech transcription and / or speech translation, thereby improving the comprehensiveness of speech transcription and / or speech translation.

[0101] Based on the above embodiments, Figure 4 This is the second flowchart illustrating the audio acquisition method provided by the present invention, as shown below. Figure 4As shown, an audio acquisition method includes the following steps:

[0102] The first step is that the APP installed on the smart screen obtains the first recording request by interacting with the user, and sends the first recording request to the framework layer, which then sends the first recording request to the hardware abstraction layer.

[0103] The second step involves the Hardware Abstraction Layer (HAL) performing a sound card lookup upon triggering the first recording request. The HAL obtains the target sound card parameters and sets them into the audio recording parameters. Furthermore, the HAL also sets the application parameters issued by the framework layer into the audio recording parameters. These application parameters include at least one of the following: number of channels, sampling rate, and bit depth. Finally, the HAL generates a recording request carrying the audio recording parameters and sends it to the real-time operating system.

[0104] Third, triggered by the second recording request, the real-time operating system records the audio played by the sound card corresponding to the second recording request using a pulse code modulation device, obtains source audio with parameters consistent with the audio recording parameters, and stores it in the storage space specified by the second recording request.

[0105] Fourth, the framework layer retrieves the source audio recorded by the real-time operating system from the storage space specified in the second recording request.

[0106] Fifth, the framework layer can send the source audio to the app, where the app performs speech-to-text transcription and / or speech translation to obtain the corresponding text and display it. Additionally, the app can also acquire system audio and / or ambient audio captured by the microphone, then merge the system audio and / or ambient audio with the source audio to obtain a merged audio; this merged audio is then sent to the server to request speech-to-text transcription and / or speech translation.

[0107] The audio acquisition device provided by the present invention is described below. The audio acquisition device described below and the audio acquisition method described above can be referred to in correspondence.

[0108] Based on any of the above embodiments, the present invention provides an audio acquisition device. Figure 5 This is a schematic diagram of the audio acquisition device provided by the present invention, as shown below. Figure 5 As shown, the device includes:

[0109] Acquisition unit 510 is used to acquire the first recording request;

[0110] The sending unit 520 is used to control the hardware abstraction layer to send a second recording request carrying audio recording parameters to the real-time operating system when the first recording request is triggered.

[0111] The acquisition unit 530 is used to control the real-time operating system to acquire the audio played by the sound card corresponding to the second recording request based on the audio recording parameters when the second recording request is triggered, so as to obtain the source audio.

[0112] The source audio acquisition unit is used to acquire source audio recorded by the real-time operating system.

[0113] The device provided in this embodiment of the invention, when triggered by a second recording request, controls the real-time operating system to collect the audio played by the sound card corresponding to the second recording request based on audio recording parameters, thereby obtaining the source audio and then acquiring the source audio recorded by the real-time operating system. Thus, the acquisition and recording of the source audio played by the sound card is realized, ensuring the availability and convenience of the source audio acquisition, and providing conditions for the subsequent direct use of the acquired source audio for speech transcription, speech translation and other speech processing.

[0114] Based on any of the above embodiments, the sending unit is specifically used for:

[0115] The sound card lookup unit is used to control the hardware abstraction layer to perform a sound card lookup, obtain the target sound card parameters, and put the target sound card parameters into the audio recording parameters;

[0116] The sending subunit is used to control the hardware abstraction layer to generate a second recording request carrying the audio recording parameters and send it to the real-time operating system.

[0117] Based on any of the above embodiments, before issuing the subunit, the method further includes:

[0118] The application parameters are placed into the audio recording parameters, which include at least one of the number of channels, sampling rate, and bit depth; the application parameters are transmitted from the framework layer to the hardware abstraction layer.

[0119] Based on any of the above embodiments, the acquisition unit is specifically used for:

[0120] The real-time operating system is controlled to record the audio played by the sound card corresponding to the second recording request based on the pulse code modulation device, so as to obtain the source audio with parameters consistent with the audio recording parameters, and store it in the storage space specified by the second recording request;

[0121] The step of acquiring the source audio recorded by the real-time operating system includes:

[0122] The source audio is retrieved from the storage space specified in the second recording request.

[0123] Based on any of the above embodiments, after acquiring the source audio recorded by the real-time operating system, the method further includes:

[0124] Based on the source audio, speech transcription and / or speech translation are performed to obtain the text corresponding to the source audio and display it.

[0125] Based on any of the above embodiments, the step of performing speech transcription and / or speech translation based on the source audio includes:

[0126] Acquire system audio and / or ambient audio acquired via microphone;

[0127] The system audio and / or the ambient audio are merged with the source audio to obtain merged audio;

[0128] The merged audio is sent to a server to request the server to perform speech transcription and / or speech translation on the merged audio.

[0129] Based on any of the above embodiments, the present invention provides a display screen, the display screen being equipped with a processor, the processor loading a display system, the display system including a framework layer, a hardware abstraction layer, and a real-time operating system, characterized in that the framework layer is used to acquire a first recording request and send the first recording request to the hardware abstraction layer; the hardware abstraction layer is used to, upon triggering the first recording request, send a second recording request carrying audio recording parameters to the real-time operating system; the real-time operating system is used to, upon triggering the second recording request, collect audio played by the sound card corresponding to the second recording request based on the audio recording parameters, to obtain source audio; the framework layer is also used to acquire the source audio and acquire the text corresponding to the source audio for display on the display screen.

[0130] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, communications interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute an audio acquisition method, which includes: acquiring a first recording request; under the trigger of the first recording request, controlling the hardware abstraction layer to send a second recording request carrying audio recording parameters to the real-time operating system; under the trigger of the second recording request, controlling the real-time operating system to acquire the audio played by the sound card corresponding to the second recording request based on the audio recording parameters, thereby obtaining the source audio; and acquiring the source audio recorded by the real-time operating system.

[0131] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0132] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the audio acquisition method provided by the above methods. The method includes: obtaining a first recording request; under the trigger of the first recording request, controlling the hardware abstraction layer to send a second recording request carrying audio recording parameters to the real-time operating system; under the trigger of the second recording request, controlling the real-time operating system to acquire the audio played by the sound card corresponding to the second recording request based on the audio recording parameters, thereby obtaining the source audio; and acquiring the source audio recorded by the real-time operating system.

[0133] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the audio acquisition method provided by the above methods. The method includes: obtaining a first recording request; under the trigger of the first recording request, controlling the hardware abstraction layer to send a second recording request carrying audio recording parameters to the real-time operating system; under the trigger of the second recording request, controlling the real-time operating system to acquire the audio played by the sound card corresponding to the second recording request based on the audio recording parameters, thereby obtaining source audio; and acquiring the source audio recorded by the real-time operating system.

[0134] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0135] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An audio acquisition method, characterized in that, include: Get the first recording request; Upon triggering the first recording request, the control hardware abstraction layer sends a second recording request carrying audio recording parameters to the real-time operating system; The audio recording parameters include the number of channels, sampling rate, and bit depth; Upon triggering the second recording request, the real-time operating system is controlled to collect the audio played by the sound card corresponding to the second recording request based on the audio recording parameters, thereby obtaining the source audio. The source audio is input audio via HDMI, VGA, or OPS connection. Obtain the source audio recorded by the real-time operating system; The control of the real-time operating system to collect the audio played by the sound card corresponding to the second recording request based on the audio recording parameters includes: The real-time operating system is controlled to record the audio played by the sound card corresponding to the second recording request based on the pulse code modulation device, so as to obtain the source audio with parameters consistent with the audio recording parameters, and store it in the storage space specified by the second recording request; The step of acquiring the source audio recorded by the real-time operating system includes: The source audio is retrieved from the storage space specified in the second recording request.

2. The audio acquisition method according to claim 1, characterized in that, The control hardware abstraction layer sends a second recording request carrying audio recording parameters to the real-time operating system, including: The hardware abstraction layer is controlled to search for the sound card, obtain the target sound card parameters, and then the target sound card parameters are set into the audio recording parameters; The hardware abstraction layer is controlled to generate a second recording request carrying the audio recording parameters and send it to the real-time operating system.

3. The audio acquisition method according to claim 2, characterized in that, Before the hardware abstraction layer generates a second recording request carrying the audio recording parameters, the method further includes: The application parameters are placed into the audio recording parameters, which include at least one of the number of channels, sampling rate, and bit depth; the application parameters are transmitted from the framework layer to the hardware abstraction layer.

4. The audio acquisition method according to any one of claims 1 to 3, characterized in that, After acquiring the source audio recorded by the real-time operating system, the method further includes: Based on the source audio, speech transcription and / or speech translation are performed to obtain the text corresponding to the source audio and display it.

5. The audio acquisition method according to claim 4, characterized in that, The process of speech transcription and / or speech translation based on the source audio includes: Acquire system audio and / or ambient audio acquired via microphone; The system audio and / or the ambient audio are merged with the source audio to obtain merged audio; The merged audio is sent to a server to request the server to perform speech transcription and / or speech translation on the merged audio.

6. An audio acquisition device, characterized in that, include: The acquisition unit is used to acquire the first recording request; The sending unit is used to control the hardware abstraction layer to send a second recording request carrying audio recording parameters to the real-time operating system when triggered by the first recording request; The audio recording parameters include the number of channels, sampling rate, and bit depth; The acquisition unit is used to control the real-time operating system to acquire the audio played by the sound card corresponding to the second recording request based on the audio recording parameters when the second recording request is triggered, so as to obtain the source audio. The source audio is input audio via HDMI, VGA, or OPS connection. The source audio acquisition unit is used to acquire source audio recorded by the real-time operating system; The acquisition unit is specifically used for: The real-time operating system is controlled to record the audio played by the sound card corresponding to the second recording request based on the pulse code modulation device, so as to obtain the source audio with parameters consistent with the audio recording parameters, and store it in the storage space specified by the second recording request; The source audio acquisition unit is specifically used for: The source audio is retrieved from the storage space specified in the second recording request.

7. A display screen, wherein the display screen is provided with a processor, the processor is loaded with a display system, the display system comprising a framework layer, a hardware abstraction layer, and a real-time operating system, characterized in that, The framework layer is used to obtain a first recording request and send the first recording request to the hardware abstraction layer; The hardware abstraction layer is used to send a second recording request carrying audio recording parameters to the real-time operating system when the first recording request is triggered. The real-time operating system is used to control the real-time operating system to record the audio played by the sound card corresponding to the second recording request based on the pulse code modulation device when the second recording request is triggered, so as to obtain the source audio with parameters consistent with the audio recording parameters, and store it in the storage space specified by the second recording request; The framework layer is also used to call back the source audio from the storage space specified by the second recording request, and obtain the text corresponding to the source audio for display on the display screen; The source audio is the audio input source connected via HDMI, VGA, or OPS.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the audio acquisition method as described in any one of claims 1 to 5.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the audio acquisition method as described in any one of claims 1 to 5.

Images