Subtitle generation method, device, and storage medium

Abstract

This application discloses a subtitle generation method, device, and storage medium, relating to the field of speech recognition technology. The method includes: obtaining a cross-process file descriptor through a first channel between a server and a client, wherein the cross-process file descriptor is used to characterize the reading end of a second channel; responding to a subtitle startup command, sending an audio capture command through the first channel, and reading an audio data stream from the reading end of the second channel according to the cross-process file descriptor, wherein the audio data stream is captured by the server and written to the writing end of the second channel; and parsing and recognizing the audio data stream to obtain subtitle text data. This application constructs an architecture that decouples hierarchical permissions from cross-process communication, with the server possessing system-level permissions responsible for audio capture. This eliminates the need for cumbersome pop-up confirmation operations when the client application starts subtitles, thus eliminating interruptions to the user's viewing continuity and achieving seamless interaction from startup.

Description

Technical Field

[0001] This application relates to the field of speech recognition technology, and in particular to a method, device and storage medium for generating subtitles. Background Technology

[0002] Starting with Android 10, to enhance user privacy protection, the Android framework has strictly limited internal audio recording permissions. Third-party client applications wishing to capture the system's internal audio output must call the official MediaProjection API (Application Programming Interface) and request screen recording permission. This API is designed to require the system to dynamically display an explicit user confirmation dialog box when an application initiates a recording request. Only after the user manually clicks the confirmation or agreement button can the third-party client application obtain audio and video data.

[0003] However, in the specific usage scenario of television, the aforementioned permission interaction mechanism has significant applicability issues. Television users typically rely on remote controls for far-field operations, which involves long operation paths and low input efficiency. If users have to face pop-up confirmation dialog boxes and repeatedly press buttons on the remote control to move focus and confirm every time they need to use a third-party client application to enable subtitles or record audio and video, it will severely disrupt the continuity of viewing, making the operation process cumbersome and affecting the expected interaction in the television scenario. Summary of the Invention

[0004] The main purpose of this application is to provide a subtitle generation method, device and storage medium, which aims to solve the technical problem of the cumbersome interactive process of subtitle generation using third-party client applications.

[0005] To achieve the above objectives, this application proposes a subtitle generation method for use on a client-side application. The subtitle generation method includes: The cross-process file descriptor transmitted by the server is obtained through the first channel between the server and the client, wherein the cross-process file descriptor is used to characterize the read end of the second channel between the server and the client, and the second channel is a pipe established by the server. In response to the subtitle start command, an audio capture command is sent to the server through the first channel, and an audio data stream is read from the read end of the second channel according to the cross-process file descriptor, wherein the audio data stream is captured by the server based on the audio capture command and written to the write end of the second channel; The audio data stream is parsed and identified to obtain subtitle text data.

[0006] Furthermore, to achieve the above objectives, this application also proposes a subtitle generation method applied to a server, the subtitle generation method comprising: Establish a pipe between the client and the server to obtain a second channel, and obtain the cross-process file descriptor corresponding to the read end of the second channel; The cross-process file descriptor is transmitted to the client through the first channel between the client and the server. When the client responds to the subtitle start command and sends an audio capture command through the first channel, the audio data stream is captured based on the audio capture command, and the audio data stream is written to the write end of the second channel, so that the client reads the audio data stream from the read end of the second channel according to the cross-process file descriptor, and parses and identifies the audio data stream to obtain subtitle text data.

[0007] In addition, to achieve the above objectives, this application also proposes an electronic device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the subtitle generation method as described above.

[0008] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the subtitle generation method described above. Attached Figure Description

[0009] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating an embodiment of the subtitle generation method of this application. Figure 2 This is a flowchart illustrating Embodiment 2 of the subtitle generation method of this application; Figure 3 This is a flowchart illustrating Embodiment 3 of the subtitle generation method of this application; Figure 4 This is a schematic diagram of the system architecture of the subtitle generation method provided in Embodiment 3 of this application; Figure 5 This is a schematic diagram of the audio data flow of the subtitle generation method provided in Embodiment 3 of this application; Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the subtitle generation method in this application embodiment.

[0012] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0013] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0014] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0015] The main solution of this application embodiment is as follows: A cross-process file descriptor transmitted by the server is obtained through a first channel between the server and the client, wherein the cross-process file descriptor is used to characterize the read end of a second channel between the server and the client, and the second channel is a pipe established by the server; in response to a subtitle start command, an audio capture command is sent to the server through the first channel, and an audio data stream is read from the read end of the second channel according to the cross-process file descriptor, wherein the audio data stream is captured by the server based on the audio capture command and written to the write end of the second channel; the audio data stream is parsed and identified to obtain subtitle text data.

[0016] The aforementioned permission interaction mechanism suffers from significant applicability issues in the specific usage scenario of television. Television users typically rely on remote controls for far-field operations, resulting in long operation paths and low input efficiency. If users must face pop-up confirmation dialog boxes and repeatedly press buttons on the remote to move focus and confirm every time they need to enable subtitles or record audio / video using a third-party client application, it severely disrupts the continuity of viewing, leading to cumbersome operation processes and impacting the expected interaction in a television setting.

[0017] This application provides a solution by constructing an architecture that decouples hierarchical permissions from cross-process communication. A server with preset system permissions is responsible for audio capture, while the client obtains a cross-process file descriptor transmitted by the server through a first channel. This cross-process file descriptor identifies the read end of a second channel pre-built by the server. When the client responds to a subtitle launch command and notifies the server to start capture through the first channel, the client only needs to directly read the audio data stream from the read end of the second channel based on the cross-process file descriptor and perform parsing and recognition, without needing any audio recording permissions or initiating a permission request process. This directly overcomes the defect caused by the Android system permission mechanism, which requires third-party clients to rely on the MediaProjection API and trigger a forced user confirmation pop-up. In far-field interaction scenarios such as television, when the client application launches subtitles, the user no longer needs to perform a cumbersome pop-up confirmation operation via remote control, completely eliminating interruptions to the user's viewing continuity and achieving seamless interaction that is ready to use upon startup.

[0018] Based on this, the embodiments of this application provide a subtitle generation method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the subtitle generation method of this application.

[0019] In this embodiment, the subtitle generation method is applied to the client, and the subtitle generation method includes steps S10~S30: Step S10: Obtain the cross-process file descriptor transmitted by the server through the first channel between the server and the client. The cross-process file descriptor is used to characterize the read end of the second channel between the server and the client. The second channel is a pipe established by the server. It should be noted that the client can be an application module deployed on the Android system to provide real-time subtitle display functionality to end users. This client runs as a regular application and does not possess the system-level permissions required to record internal system audio. Its core responsibility is to interact with the server and receive data through a pre-established channel. Specifically, the client initiates a binding request to the server to establish a first channel, receives a cross-process file descriptor from the server through this first channel, and establishes a connection with a second channel based on this cross-process file descriptor. Upon receiving a subtitle start command triggered by the user, the client sends an audio capture command to the server through the first channel to initiate the audio data acquisition process. Subsequently, the client directly reads the audio data stream written by the server from the reader of the second channel, parses and recognizes the audio data stream, and finally presents the recognition result as subtitle text data on the user interface. The client's design avoids involvement in low-level audio hardware operations and permission requests, thus avoiding interactive interference caused by permission pop-ups.

[0020] The server can be an application module deployed within the Android system, running as a system-level component. This server possesses preset system permissions, particularly the ability to call the underlying audio recording interface to capture internal system audio output. The core function of the server is to act as an audio data acquisition source and distribution hub. During the initialization phase, it is responsible for building a second channel and encapsulating the read end of this channel as a cross-process file descriptor. This cross-process file descriptor is then passed to the bound client through the first channel. Upon receiving the audio capture command sent by the client through the first channel, the server, based on its preset system permissions, begins to capture the system's audio data stream in real time and continuously writes the captured audio data to the write end of the second channel, allowing the client in another process to obtain the data from the read end of the second channel. The existence of the server completely separates the high-privilege audio acquisition operation from the low-privilege subtitle application logic, achieving seamless recording and permission circumvention.

[0021] The first channel refers to a bidirectional communication link established between the client and server processes for transmitting control commands and lightweight control data. Within the Android system framework, this channel can be implemented through the bindService mechanism and the AIDL (Android Interface Definition Language) interface. The purpose of the first channel is to carry out control signaling interactions between processes. The client initiates a service binding request through this channel to obtain the server's proxy interface and then calls methods exposed by the server (such as sending audio capture commands and obtaining cross-process file descriptors). The first channel transmits small amounts of data and has high real-time requirements; its design focuses on ensuring reliable delivery of control commands and synchronous responses to interface calls, laying the foundation for the control layer of subsequent large-scale audio data transmission.

[0022] A cross-process file descriptor (CFD) is an inter-process communication handle used to securely and efficiently transfer access permissions to the same kernel object between a client and a server. In this embodiment, the descriptor specifically refers to a ParcelFileDescriptor object encapsulated by the server's self-created second channel read end. Through the Binder driver, this object can be transparently transferred from the server process to the client process, granting the client a valid read file descriptor pointing to the same pipe buffer in the kernel. Essentially, this CFD represents the sharing of file descriptor entries in the kernel space between processes. It allows the client to directly read audio data channels created by the server without requiring any audio permissions, making it a crucial token for achieving zero-copy or low-overhead cross-process data transfer.

[0023] The second channel is a unidirectional inter-process communication pipe actively built by the server for transmitting large-scale audio data streams. At its underlying implementation, this channel is based on Unix pipes or a similar kernel buffer mechanism. The server holds the file descriptor for its write end, while the file descriptor for the read end is encapsulated into a cross-process file descriptor and given to the client. The design goal of the second channel is to achieve high-throughput, low-latency transmission of audio data from the server to the client. The server continuously writes the captured audio data stream to the write end of the second channel, while the client reads from its hold on the read end in a blocking manner. Data flows directly in kernel space, without the need for multiple memory copies or additional application-layer protocol encapsulation, thus ensuring the real-time performance and integrity of the audio data stream.

[0024] In step S20, in response to the subtitle start command, an audio capture command is sent to the server through the first channel, and an audio data stream is read from the read end of the second channel according to the cross-process file descriptor, wherein the audio data stream is captured by the server based on the audio capture command and written to the write end of the second channel; It should be noted that the subtitle activation command is an internal event or signal triggered by user action and emitted from the client application layer. This command marks the beginning of the client's entry into subtitle working mode. For example, when the user clicks the "Enable Subtitles" button on the client interface or issues a corresponding command via remote control, this command is generated internally by the client. The main function of this command is to trigger the client to execute a series of predetermined action sequences, specifically including: sending an audio capture command to the server via the first channel to request the server to start acquiring audio, and activating the client's internal audio reading thread, preparing it to read data from the second channel. This command acts as a logical switch for the client to transition from standby to working state.

[0025] The audio capture command is an inter-process control message sent by the client to the server via the first channel. This command carries the client's explicit request that the server begin executing the audio acquisition task. Upon receiving the command, the server immediately invokes its pre-defined system permissions, activates the underlying audio recording interface, and begins capturing pulse-code modulation data from the system audio output stream. The audio capture command is the core link between the client's control logic and the server's execution logic. It ensures that audio data acquisition only begins when the client truly needs it, avoiding meaningless server-side idleness and resource waste.

[0026] Pre-defined system permissions are a set of special permissions pre-granted to the server in its application installation package or system signature. These permissions exceed the scope that ordinary third-party applications can request. These permissions at least include the ability to capture internal system audio output, such as the CAPTURE_AUDIO_OUTPUT permission or similar system-level recording permissions in Android. Since the server is a pre-installed or system-signed component, it can statically hold these permissions and directly call APIs protected by these permissions (such as AudioRecord) at runtime, without needing to dynamically request user authorization at runtime like a regular client. Pre-defined system permissions are the compliance basis for the server to silently capture audio and bypass the MediaProjection permission pop-up.

[0027] Step S30: Analyze and identify the audio data stream to obtain subtitle text data.

[0028] It's important to note that the subtitle text data is the visual text information presented to the user as the final output by the client. This data is generated by the client's internal speech recognition engine through parsing, recognizing, and converting the audio data stream read from the second channel. The audio data stream is essentially raw pulse-code modulation sampled data, which, after front-end processing by the speech recognition model (such as endpoint detection and feature extraction) and computation by the core decoder, is ultimately converted into a text sequence corresponding to the audio content. The subtitle text data is then rendered by the client onto a specific area of ​​the display device, presented synchronously with the playing video or audio content, thus achieving real-time subtitle functionality.

[0029] Understandably, since Android 10, the system has mandated that third-party client applications must display a user confirmation dialog box when recording internal audio via the MediaProjection API. This has resulted in users having to perform a cumbersome confirmation operation with the remote control every time they use the subtitle function in the specific scenario of a TV, which seriously disrupts the continuity of viewing. Therefore, this embodiment constructs a separate architecture for permission layering and server-side capture. The server with preset system permissions pre-establishes a second channel and transmits its cross-process file descriptor to the client through the first channel. When the client responds to the subtitle start command and sends an audio capture command through the first channel, the client only needs to read the audio data stream captured and written by the server directly from the read end of the second channel based on the cross-process file descriptor for parsing and recognition. This completely avoids the forced interaction process where the client must trigger a system pop-up and wait for user confirmation to request audio recording permissions, and completely eliminates the confirmation step under remote control operation. This realizes seamless start and continuous playback of the subtitle function in the TV scenario. At the same time, since the server captures pure internal system audio rather than environmental noise, it also provides a high-quality data source for subsequent speech recognition. Ultimately, this achieves the effects of simplifying user operation, improving interaction fluency, and ensuring the quality of subtitle generation.

[0030] For example, the client is a regular subtitle application running on an Android TV system, and the server is a background service component pre-installed in the system image and holding the CAPTURE_AUDIO_OUTPUT system-level permission. The client first establishes a first AIDL-based channel with the server through the bindService mechanism and calls the interface methods defined in this channel to obtain a cross-process file descriptor (ParcelFileDescriptor) pre-created by the server and its read end encapsulated. This cross-process file descriptor uniquely identifies the read end of the second channel between the server and the client. When the user triggers the subtitle start command on the TV using the remote control, the client generates a corresponding event and then sends an audio capture command to the server through the first channel. Upon receiving the command, the server immediately uses its pre-defined system permissions to call the AudioRecord interface to begin capturing the system's internal audio output stream and writes the captured PCM data to the write end of the second channel in real time. Meanwhile, the client constructs a file input stream based on the acquired cross-process file descriptor, and reads the audio data stream from the second channel in a blocking manner by repeatedly calling the read() method. The read audio data is then sent to the locally integrated speech recognition engine (such as Vosk or SherpaOnnx) for parsing and recognition, and finally generates subtitle text data and displays it on the TV screen in real time. Thus, clean system audio capture and subtitle generation are achieved without the client requesting any recording permissions or any system pop-up interference.

[0031] This embodiment provides a subtitle generation method. By constructing an architecture that decouples hierarchical permissions and cross-process communication, a server with preset system permissions is responsible for audio capture. The client obtains a cross-process file descriptor transmitted by the server through a first channel. This cross-process file descriptor identifies the read end of a second channel pre-built by the server. When the client responds to the subtitle start command and notifies the server to start capture through the first channel, the client only needs to directly read the audio data stream from the read end of the second channel based on the cross-process file descriptor and perform parsing and recognition. It does not need to have any audio recording permissions or initiate a permission request process. This directly overcomes the defect that the Android system permission mechanism requires third-party clients to rely on the MediaProjection API and trigger a forced user confirmation pop-up. In far-field interaction scenarios such as TV, when the client application starts subtitles, the user no longer needs to perform a cumbersome pop-up confirmation operation through the remote control, completely eliminating the interruption to the user's viewing continuity and realizing a seamless interaction that can be used right out of the box.

[0032] In one feasible implementation, step S20, which involves reading the audio data stream from the read end of the second channel based on the cross-process file descriptor, may include steps S21-S22: Step S21: Construct a file input stream based on the cross-process file descriptor to establish a connection between the reader and the client of the second channel; It's important to note that the file input stream is a standard input stream object constructed by the client based on the acquired cross-process file descriptor. The purpose of this file input stream is to encapsulate the kernel-level pipe read end, represented by the cross-process file descriptor, into a high-level interface that client applications can directly call to read byte data. Specifically, after the client obtains the cross-process file descriptor (i.e., the ParcelFileDescriptor object) from the server through the first channel, it calls its getFileDescriptor() method to obtain the underlying native file descriptor and instantiates a FileInputStream using this as a parameter. This construction process logically establishes the formal connection between the second channel and the client: the physical entity of the second channel is a pipe buffer in the kernel, whose read end was originally referenced by the cross-process file descriptor. The establishment of the file input stream allows the client to continuously extract audio data from the read end of this buffer using standard, blocking read() operations. File input streams transform the underlying handles of cross-process communication into a streaming programming model known to the application layer. This allows client developers to avoid worrying about the specific implementation details of inter-process data transmission and simply obtain byte data from the stream in a loop, just like reading a local file. This enables efficient and convenient reliable reception of audio data streams from the server to the client.

[0033] Step S22: The file input stream is read in a blocking manner in a loop to obtain the audio data stream.

[0034] Understandably, directly manipulating the underlying file descriptor for data reading involves complex system calls and is difficult to manage thread resources efficiently, easily leading to CPU (Central Processing Unit) idling or data loss. Therefore, this implementation further encapsulates the kernel-level pipe read end into a standard input stream object by constructing a file input stream from the cross-process file descriptor, thereby establishing an application-layer-friendly and automatically buffered connection between the second channel and the client. Subsequently, by cyclically reading the file input stream in a blocking manner, the client thread automatically enters a waiting state without consuming CPU resources when no audio data arrives, and is immediately awakened and data is retrieved when data is written. This mechanism completely avoids the complex state maintenance and resource scheduling problems required for manual polling or non-blocking reading. Ultimately, while ensuring real-time, complete, and lossless transmission of audio data streams, it greatly simplifies the development complexity of client code and significantly reduces the overall resource consumption of the system.

[0035] For example, when a client (such as a subtitle application on an Android TV) obtains a cross-process file descriptor from the server through the first channel, it does not directly manipulate the underlying handle. Instead, it calls `new FileInputStream(pfd.getFileDescriptor())` to construct a standard Java file input stream object, thus encapsulating the kernel pipe reader into an application-layer readable streaming interface. Subsequently, the client executes an infinite loop on the file input stream in a dedicated background recognition thread (such as `RecognizerThread`), and calls the `fis.read(buffer, 0, buffer.length)` method within the loop body for blocking reading. During this process, while the server has not yet written audio data to the second channel, the `read()` method keeps the thread in a suspended waiting state, without consuming CPU time slices. Once the server writes the captured PCM (Pulse Code Modulation) data to the write end of the second channel, the data in the kernel buffer becomes immediately available. At this point, the read() call will be immediately woken up and return the number of bytes actually read, filling the data into the pre-allocated byte buffer. This enables real-time, zero-polling transmission of audio data streams from the server to the client, ensuring that the client's speech recognition engine can obtain continuous and lossless audio data for subsequent processing.

[0036] In one feasible implementation, step S30 may include steps S31-S32: Step S31: Parse the audio data stream to obtain the audio data to be recognized; It should be noted that the audio data to be recognized is the pure audio content separated from the raw audio data stream read by the client from the second channel's reader after preliminary parsing, before undergoing the format conversion required for speech recognition. The data obtained by the client from the second channel through the file input stream is a pulse-code modulation stream containing basic attributes such as sampling rate and number of channels, but without any feature extraction or compression. This data constitutes the audio data to be recognized; it is the intermediate medium connecting the audio acquisition stage and the speech recognition stage, and its integrity and fidelity directly determine the accuracy of the subsequent recognition results.

[0037] Step S32: Based on the data format of the preset speech recognition model, the audio data to be recognized is converted into a new format, and the converted audio data is input into the preset speech recognition model to obtain subtitle text data.

[0038] It should be noted that the preset speech recognition model is a deep learning or statistical learning model pre-integrated locally on the client, trained on specific corpora, and optimized for the target application scenario. Examples include lightweight offline speech recognition engines like Vosk or SherpaOnnx. This model serves as the core algorithm component for the client-side audio-to-text conversion. Internally, it encapsulates an acoustic model, a language model, and a decoder, capable of decoding the input audio feature sequence into the corresponding text sequence. Its local deployment on the client, rather than in the cloud, ensures the real-time nature and privacy security of the subtitle generation.

[0039] The data format here specifically refers to the specific parameter specifications and encoding methods required by the preset speech recognition model for the input audio, including but not limited to the underlying technical specifications such as sampling rate, bit depth, number of channels, and the arrangement structure of audio data. Since different speech recognition models may have strict restrictions on the format of the input audio, the client needs to perform preprocessing operations such as resampling, quantization, or channel conversion on the audio data to be recognized according to the data format required by the preset speech recognition model to ensure that the audio data can be correctly received and recognized by the model.

[0040] Understandably, the raw audio data stream read directly from the second channel may not meet the specific requirements of the client's locally integrated preset speech recognition model for the input data format, such as sampling rate, bit depth, and number of channels. Directly inputting this data into the model would lead to recognition failure or a significant drop in accuracy. Therefore, this implementation further parses the audio data stream to extract clean audio data to be recognized, preprocesses the audio data to be recognized based on the data format required by the preset speech recognition model, and finally inputs the format-adapted audio data into the preset speech recognition model for decoding. This avoids recognition errors or compatibility issues caused by mismatched audio data formats, ensuring that the audio data can be correctly received and processed by the model. This significantly improves the accuracy and robustness of subtitle text data generation, enabling the entire subtitle generation system to adapt to various different speech recognition engines and audio source parameters.

[0041] For example, when the client reads the raw audio data stream from the second channel via a file input stream, the parsing module first processes the data stream, extracting continuous, clean audio data to be recognized—the original PCM sampling sequence—according to a predefined audio encapsulation format. Subsequently, the client obtains the specific data format parameters required by its locally integrated preset speech recognition model (e.g., the Vosk offline Chinese speech recognition engine). These parameters can be, but are not limited to, 16kHz sampling rate, 16-bit quantization depth, and mono PCM data. Based on this, the audio data to be recognized is format-adapted: for example, if the original audio has a 44.1kHz sampling rate, it is downsampled to 16kHz using a resampling algorithm; if it is stereo, it is merged into mono, ensuring the data is arranged in little-endian order. Finally, the format-converted audio data is input in blocks into the preset speech recognition model. The model's internal acoustic model, language model, and decoder work together to recognize and output the corresponding text sequence frame by frame, ultimately generating subtitle text data synchronized with the audio content for the client to render and display.

[0042] In one feasible implementation, the subtitle generation method may further include step S40: In step S40, in response to the subtitle stop command, an audio stop capture command is sent to the server through the first channel to cause the server to stop capturing the audio data stream and close the second channel.

[0043] It should be noted that the subtitle stop command is an internal event or signal actively triggered by the user through the client interface, marking the switch of the real-time subtitle function from working state to standby or off state. In a television scenario, this command can be generated by the user pressing the subtitle stop button on the remote control, exiting the current video application, or selecting the "Turn off subtitles" option in the subtitle function menu. The generation of the subtitle stop command is the logical starting point for the client to initiate the resource release process. Its function is to trigger the client to execute a series of orderly termination operations: First, the client sends an audio stop capture command to the server through the established first channel, notifying the server to terminate audio acquisition; second, the client internally stops the reading loop from the second channel, closes the relevant file input stream and recognition thread; finally, the client cleans up the subtitle text displayed on the UI (User Interface) and releases all temporarily occupied memory resources. The design of this command ensures that the subtitle function's closing behavior is controllable and complete, avoiding resource leaks or process blocking caused by abnormal termination.

[0044] The audio stop capture command is an inter-process control message sent by the client to the server through the first channel. It carries the client's explicit request for the server to immediately terminate the audio acquisition operation and release related hardware resources. This command is a cross-process response extension of the client's user-defined stop command, essentially a remote method called via the AIDL interface. Upon receiving this command, the server immediately stops calling the underlying audio recording interfaces, such as the `stop()` and `release()` methods of `AudioRecord`, terminates the capture of internal audio output, flushes any remaining data written to the second channel's write end, and actively closes the write end file descriptor for that channel. The execution of the audio stop capture command marks the termination of the entire audio data production chain. Together with the client's read stop behavior, it constitutes a complete two-way close handshake protocol, ensuring the orderly release of system audio resources and the proper destruction of the cross-process communication channel.

[0045] Understandably, if the client can only respond to the subtitle start command to start the audio capture and recognition process, and lacks a corresponding shutdown mechanism, the server will continue to occupy system audio hardware resources and write data to the second channel even when the user no longer needs the subtitle function. The client will also continuously perform invalid reads, resulting in problems such as CPU polling idle, memory occupation not being released, and power wastage. Therefore, this implementation further introduces a subtitle stop command triggered by user operation. After receiving the command, the client immediately sends an audio stop capture command to the server through the first channel, thereby driving the server to orderly terminate the acquisition of audio data stream and close the writing end of the second channel. This technical solution completely avoids system resource leakage and invalid power consumption caused by incomplete function shutdown, achieves precise synchronization between the audio acquisition life cycle and user intent, ensures that system resources can be recovered in a timely manner after the subtitle function is exited, and significantly improves the overall energy efficiency and long-term operational stability of the device.

[0046] For example, the client is a subtitle application running on an Android TV system, whose user interface provides a remote-controlled close button. When the user presses this button, the client internally generates a subtitle stop command. This command triggers the controller component in the client responsible for communicating with the server to call the server-defined `stopAudioCapture()` method through a pre-established first channel. This method call constitutes the cross-process transmission of the audio stop capture command. To ensure timely server response, this call is typically asynchronous or one-way to avoid blocking the client thread. Upon receiving the command, the server immediately stops audio capture in the underlying `AudioRecord`, closes the write-end file descriptor of the second channel, and releases related hardware resources. Simultaneously, the client synchronously performs local cleanup operations after sending the command, including closing the file input stream constructed by the cross-process file descriptor, terminating the background recognition thread, and clearing the subtitle display area on the screen, thereby completely ending the subtitle generation process and achieving precise synchronization between the user's intention to close and the release of system resources.

[0047] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 Before step S10, the subtitle generation method may also include step S01: Step S01: Establish the first channel according to the target service package identifier in the preset binding strategy list; The preset binding strategy list includes multiple candidate service package identifiers, and the target service package identifier is the candidate service package identifier among the candidate service package identifiers that can establish a first channel with the client.

[0048] It's important to note that the preset binding strategy list refers to a set of ordered binding rules pre-configured within the client to guide the client in selecting and connecting to the server. Essentially, this list is a priority queue containing multiple candidate service package identifiers, addressing compatibility issues arising from potential differences in server package names across different device manufacturers or system versions. In practice, the preset binding strategy list is stored in the client's configuration file as an array or set. Each entry in the list corresponds to a candidate service package identifier, and the order of the entries represents the priority order of the binding attempts. When the client needs to establish its first channel with the server, it will attempt to bind to each candidate service package sequentially according to the order defined in the preset binding strategy list, until a successful binding or all candidates have been traversed. This strategy list enables the client to dynamically adapt to various system environments, achieving a reliable connection to the server without requiring customized modifications for different devices.

[0049] The target service package identifier is the identifier of the specific service package that the client successfully binds to and establishes the first channel under the guidance of the preset binding strategy list. This identifier is usually the package name used to uniquely identify an application in the Android system, such as "com.mk.tv.systemserver" or "com.mk.tv.smartkey". Determining the target service package identifier is the result of a dynamic selection process: the client attempts to bind to the highest-priority candidate service package identifier according to the priority order in the preset binding strategy list. If the binding is successful, that candidate service package identifier becomes the target service package identifier for this connection; if the binding fails, it continues to try the next lower-priority candidate service package identifier until the first service package that can be successfully bound is found. The determination of the target service package identifier marks the successful establishment of the first channel between the client and the server, laying the foundation for all subsequent inter-process communication (such as obtaining inter-process file descriptors, sending audio capture commands, etc.).

[0050] Candidate service package identifiers are the identities of various service packages that a client might attempt to bind to, contained in a pre-defined binding strategy list. Each candidate service package identifier is a specific Android application package name, representing a server component that may be deployed in different locations on the system but all possess the same core functionality (i.e., the ability to capture system audio and provide a second channel). The set of candidate service package identifiers is pre-defined based on actual research of various device manufacturers and system versions; for example, the system service package identifier "com.mk.tv.systemserver" and the smart button service package identifier "com.mk.tv.smartkey" are both listed as candidates. There are clear priority differences among these candidate identifiers. The highest priority identifier (such as the system service package identifier) ​​is usually the standard configuration of the device, while the lower priority identifiers (such as the smart button service package identifier) ​​serve as alternatives for fallback compatibility when the standard configuration is unavailable. Candidate service package identifiers constitute a potential target pool of servers that the client can connect to, and the client determines the final available target service package by traversing this target pool.

[0051] Understandably, in the complex Android system environment, different device manufacturers or different system versions may deploy servers with preset system permissions under different package names. If the client only binds to a single package name, the server will not be found on some devices and will completely fail. Therefore, this embodiment adopts a preset binding strategy list containing multiple candidate service package identifiers and sorted by priority in the client. This allows the client to dynamically attempt to bind according to the target service package identifier in the list, thereby effectively avoiding the binding failure problem caused by the server package name not being fixed or the manufacturer's customization. This achieves adaptive connection and wide compatibility of the client on different brands and versions of Android devices, ensuring that the subtitle function can reliably establish the first channel and obtain audio data in various system environments.

[0052] In the specific implementation process, the candidate service package identifier includes the system service package identifier and the smart button service package identifier. Step S01 may include steps S011 to S014: Step S011: Use the system service package identifier as the target service package identifier and bind the system service package corresponding to the system service package identifier; It should be noted that the system service package identifier is a specific string pre-configured in the client's default binding policy list, used to uniquely identify the core system service application pre-installed in the device system, such as "com.mk.tv.systemserver". This identifier, as the highest priority candidate service package identifier, represents the device manufacturer's default intended location within the standard system service components where the server with audio capture permissions will be deployed. The client will first use this identifier to attempt to bind to the service, establishing the first channel for subsequent command interaction and data transmission.

[0053] The system service package is a pre-installed system-level application that runs in the background of the Android device and has preset system permissions (such as CAPTURE_AUDIO_OUTPUT). As the highest priority server implementation carrier in this embodiment, the service package's core responsibilities are to create a second channel, pass cross-process file descriptors to the client through the first channel, and capture the internal audio stream of the system using its system permissions after receiving the audio capture command and write it to the second channel. It is the source provider of audio data in the entire subtitle generation system.

[0054] Step S012: If the client and system service package are successfully bound, the first channel is established. Step S013: If the binding of the client and system service package fails, update the target service package identifier to the smart button service package identifier and bind the smart button service package corresponding to the smart button service package identifier. It should be noted that the smart button service package identifier is an alternative service package identifier with lower priority than the system service package identifier in the preset binding strategy list, such as "com.mk.tv.smartkey". This identifier is used as a fallback target when the system service package binding fails. It represents an alternative solution that the server may deploy in a system application on the device that specifically handles the smart button function of the remote control, ensuring that the client still has the opportunity to establish a connection with the server when the primary target is unavailable.

[0055] The smart button service package is a pre-installed system-level application originally used in the device to handle special button functions of the remote control (such as voice search, shortcut menus, etc.). This service package is reused as the second-priority server implementation carrier in this technical solution. When the system service package is unavailable or fails to bind, if the client can successfully bind to the smart button service package, it can also use its preset system permissions to complete the establishment of the second channel, the transfer of cross-process file descriptors, and the capture and writing of audio data. This achieves downgraded compatibility of server functions and ensures the availability of the subtitle generation system under different device configurations.

[0056] Step S014: If the client and smart button service package are successfully matched, the first channel is established.

[0057] Understandably, when clients actually execute the preset binding strategy list, they may still be unable to cope with the differentiated scenarios where different device manufacturers deploy servers in different locations due to the lack of a clear binding target. Therefore, this implementation further uses system service package identifiers and smart button service package identifiers as specific candidate service package identifiers, and explicitly sets the priority of system service package identifiers to be higher than that of smart button service package identifiers. This allows clients to prioritize binding the most standard and commonly used system service package, and only when this attempt fails will they downgrade to binding the alternative smart button service package. This effectively avoids binding failures or resource waste caused by unclear or disordered binding targets. It achieves the technical effect of successfully establishing the first channel with the highest efficiency and optimal path in various customized device environments, ensuring fast connection under mainstream configurations and providing a complete degradation compatibility mechanism.

[0058] For example, the client's internal preset binding strategy list explicitly includes two candidate service package identifiers. The system service package identifier "com.mk.tv.systemserver" is configured with the highest priority, and the smart key service package identifier "com.mk.tv.smartkey" is configured with the second highest priority. When the client needs to establish the first channel with the server, it first uses the system service package identifier as the target service package identifier and attempts to bind the corresponding system service package through bindService. If the service package exists in the system and the binding is successful, the client successfully establishes the first channel based on AIDL with the system service package, and all subsequent command interactions are completed through this channel. If the binding fails due to the removal of the system service package, signature mismatch, or other reasons, the client automatically switches the target service package identifier to the smart key service package identifier and attempts to bind the corresponding smart key service package. After successful binding, the first channel is established. Through this clearly prioritized binding mechanism, the client can find the actually available server on Android devices of different manufacturers and versions with the optimal path, ensuring reliable connection and normal startup of the subtitle generation function.

[0059] In this embodiment, by pre-setting a list of multiple candidate service package identifiers and sorting them by priority within the client, the client can dynamically determine the target service package identifier based on the list and attempt to bind in sequence. This completely avoids the binding failure problem caused by the server package name not being fixed or the manufacturer being customized. It realizes the adaptive connection and wide compatibility of the client on different brands and models of Android devices, ensuring that the subtitle function can reliably establish the first channel and obtain audio data in various system environments.

[0060] Based on the first and / or second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to that in embodiments one and two above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 The subtitle generation method is applied on the server side and includes steps A10 to A30: Step A10: Establish a pipe between the client and the server to obtain the second channel, and obtain the cross-process file descriptor corresponding to the read end of the second channel; Step A20: Transmit the cross-process file descriptor to the client through the first channel between the client and the server; In step A30, when the client responds to the subtitle start command and sends an audio capture command through the first channel, the audio data stream is captured based on the audio capture command and written to the write end of the second channel, so that the client can read the audio data stream from the read end of the second channel according to the cross-process file descriptor, and parse and recognize the audio data stream to obtain the subtitle text data.

[0061] For example, the server is a system-level background service component pre-installed in the Android TV system image, possessing preset system permissions such as CAPTURE_AUDIO_OUTPUT. During the initialization phase, the server creates a Unix pipe as a second channel by calling the underlying pipe() system call, obtains the read and write end file descriptors of this pipe, and encapsulates the read end as a cross-process file descriptor object. When the client establishes a first channel with the server through the AIDL binding mechanism, the server returns the cross-process file descriptor to the client through the getAudioCapturePipe() interface method exposed in the first channel. Subsequently, when the server receives an audio capture command sent by the client through the first channel, it immediately instantiates an AudioRecord object using its preset system permissions and sets it to capture the system's internal audio output mode, such as filtering the output device through AudioManager's getDevices(), and starts a recording thread to read the PCM data output by the audio hardware in a loop. In each round of reading, the server writes the acquired PCM data block to the file descriptor of the write end of the second channel through FileOutputStream, thereby continuously pushing the audio data stream to the kernel pipe buffer. Meanwhile, the client uses the previously acquired cross-process file descriptor to read data from the second channel reader and performs speech recognition, ultimately generating subtitle text for display. Throughout the entire process, the server acts as the producer of audio data, with its control command interaction and audio data transmission completely separated from the client, ensuring access control isolation and highly efficient cross-process communication.

[0062] In this embodiment, an architecture decoupled from hierarchical permissions and cross-process communication is constructed. The server with preset system permissions is responsible for audio capture, while the client obtains the cross-process file descriptor transmitted by the server through the first channel. This cross-process file descriptor identifies the read end of the second channel pre-built by the server. When the client responds to the subtitle start command and notifies the server to start capture through the first channel, the client only needs to directly read the audio data stream from the read end of the second channel based on the cross-process file descriptor and perform parsing and recognition, without having any audio recording permissions or initiating a permission request process. This directly overcomes the defect that the Android system permission mechanism requires third-party clients to rely on the MediaProjection API and trigger a forced user confirmation pop-up. In far-field interaction scenarios such as TV, when the client application starts the subtitle, the user no longer needs to perform a cumbersome pop-up confirmation operation through the remote control, completely eliminating the interruption to the user's viewing continuity and realizing a seamless interaction that can be used right out of the box.

[0063] In one feasible implementation, step A30, which involves capturing an audio data stream based on an audio capture command and writing the audio data stream to the write end of the second channel, may include steps A31-A32: Step A31: Upon receiving an audio capture command, the audio recording interface is called based on preset system permissions to poll and obtain the pulse code modulation data of the audio input hardware. It's important to note that the audio recording interface is a standardized application programming interface (API) that the server, with its pre-defined system permissions, can call to retrieve audio data from the system's underlying layers. In Android, this interface can be represented by the AudioRecord class or a similar system-level recording API. It allows the server to directly access the audio hardware abstraction layer (HHB) and capture internal system audio output. Calling this interface does not require any user interaction or confirmation, as the server itself has been granted the necessary system permissions. Its core function is to act as a bridge between the server and the underlying audio hardware, enabling the server to continuously read data from the audio input hardware buffer using a polling method.

[0064] Audio input hardware refers to the physical components or virtual devices within a device responsible for converting analog audio signals into digital signals. In this embodiment, this hardware may refer to a mixer output or loopback device capable of capturing internal system audio output, rather than an external physical microphone. The audio input hardware continuously generates a raw audio data stream at a fixed sampling rate and bit depth, and temporarily stores it in a circular buffer in the kernel space, waiting for upper-layer applications to read it through the audio recording interface. The server's polling operation is precisely the reading behavior of this hardware buffer.

[0065] Pulse code modulation (PCM) data refers to the raw digital audio data format generated after audio input hardware samples, quantizes, and encodes analog audio signals. Essentially, this data is a continuous, uncompressed sequence of sample points, with each sample point represented by a fixed-bit (e.g., 16-bit) binary number indicating its audio amplitude value at that moment. As the most fundamental raw data format in the audio field, PCM data preserves complete audio information and serves as the raw input for subsequent speech recognition processing. The server polls for this formatted data through the audio recording interface and directly writes it to the write end of the second channel for the client to read and recognize.

[0066] Step A32: The pulse code modulation data is written as an audio data stream to the write end of the second channel.

[0067] It is understandable that in practical applications, the server may fail to coordinate efficiently with the underlying audio hardware due to the use of inappropriate capture methods, resulting in problems such as data acquisition delays, frame drops, or format incompatibility. Therefore, this implementation adopts a technical solution that, upon receiving an audio capture command, calls an audio recording interface (such as AudioRecord) based on preset system permissions to poll and obtain the pulse code modulation data in the audio input hardware buffer, and directly writes the raw PCM data as an audio data stream to the second channel write end. This avoids the problems of poor real-time performance and data distortion that may be caused by unclear capture methods. It enables the server to continuously acquire clean raw audio data from the hardware level in a standard, efficient, and low-latency manner, and ensures that the data format written to the second channel is uniform and the timing is accurate, providing a stable and reliable high-quality audio source for the client's subsequent speech recognition.

[0068] For example, the server is a background service component pre-installed in the Android TV system, holding the preset system permission CAPTURE_AUDIO_OUTPUT. When the server receives the audio capture command sent by the client through the first channel, it immediately starts a dedicated audio acquisition thread. In this thread, an AudioRecord object is instantiated based on the preset system permission as the audio recording interface. The setAudioSource and setAudioFormat methods of AudioRecord are used to configure the audio source as a loopback device capturing the internal audio output of the system, while setting the sampling rate to 16kHz and the number of channels to mono. Subsequently, the server calls the AudioRecord.startRecording() method to start recording and continuously reads raw audio data from the kernel buffer of the audio input hardware in a polling manner using the read(byte[], int, int) method in a loop. After each successful read, the obtained byte array is the PCM data, and the server immediately writes this data block to the write-end file descriptor of the second channel via FileOutputStream. This round of reading and writing will continue until the server receives the audio stop capture command sent by the client through the first channel. At this time, the server calls AudioRecord.stop() and AudioRecord.release() to release the audio recording interface resources and closes the write end of the second channel, thus fully realizing audio hardware data capture and cross-process transmission based on preset system permissions.

[0069] For example, to help understand the implementation flow of the subtitle generation method obtained by combining Embodiment 1 and Embodiment 2 above, please refer to... Figure 4 , Figure 4 A system architecture diagram of a subtitle generation method is provided, specifically: The server, as a pre-installed component with preset system permissions, exposes its service capabilities through the Android Interface Definition Language Interface (AIDL interface) and establishes two core channels with the client: The first channel transmits control commands, specifically initiated by the client's service binding control, enabling service discovery, binding, and acquisition of cross-process file descriptors. The second channel (a Unix pipe) transmits large-scale audio data streams, with the server's audio acquisition function writing captured internal system audio data to the write end of this channel. After obtaining the cross-process file descriptor representing the read end of the second channel, the client reads the audio data stream from the read end of the channel in a blocking manner through audio reception and parsing. The data is then sent to the client-side speech recognition system for audio parsing and recognition, generating subtitle text. Finally, the subtitles are displayed in real-time on the user interface through a subtitle display interaction. This architecture, through the separation of the first and second channels, decouples the control flow from the data flow. Furthermore, by isolating the server's system permissions from the client's ordinary application attributes, it fundamentally avoids permission pop-up interference, ensuring smooth startup and high-quality operation of the subtitle function in television scenarios.

[0070] Further, please refer to Figure 5 , Figure 5 A schematic diagram of the audio data flow in a subtitle generation method is provided, specifically: After responding to the audio capture command sent by the client, the server starts recording by calling the Android system's audio recording interface (i.e., calling the start recording function). At this time, the audio data source points to the system mixer / loopback device, which represents the audio input hardware that can only be accessed with preset system permissions. The audio data first enters the system's underlying buffer. Then, in the application layer's While loop, the server continuously calls the audio recording and reading function to poll and read the raw audio data from this buffer. This data is the PCM data. The acquired audio data then enters the data splitting stage: one path optionally saves the recording file for debugging or archiving, while the main path transmits it to the client through the second channel established in this application. After receiving the audio data, the client sends the data to the speech recognition model in its application layer's speech recognition thread, that is, it calls the preset speech recognition model for parsing and recognition, and finally generates subtitle text. The entire process, through the server's system-level permission acquisition, the cross-process transmission of the second channel, and the client's end-side recognition, achieves efficient real-time subtitle generation without permission pop-up interference.

[0071] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the subtitle generation method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0072] This application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the subtitle generation method in Embodiment 1 above.

[0073] The following is for reference. Figure 6 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of this application. The electronic devices in these embodiments may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0074] like Figure 6 As shown, the electronic device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. The communication device 1009 allows the electronic device to communicate wirelessly or wiredly with other devices to exchange data. Although the diagrams show electronic devices with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented alternatively.

[0075] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0076] The electronic device provided in this application, employing the subtitle generation method described in the above embodiments, can solve the technical problem of the cumbersome interactive process for subtitle generation using third-party client applications. Compared with the prior art, the beneficial effects of the electronic device provided in this application are the same as those of the subtitle generation method provided in the above embodiments, and other technical features of the electronic device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0077] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0078] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0079] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the subtitle generation method in the above embodiments.

[0080] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0081] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0082] The aforementioned computer-readable storage medium carries one or more programs that, when executed by an electronic device, cause the electronic device to: acquire a cross-process file descriptor transmitted by the server via a first channel between the server and the client, wherein the cross-process file descriptor is used to characterize the read end of a second channel between the server and the client, the second channel being a pipe established by the server; in response to a subtitle start command, send an audio capture command to the server via the first channel, and read an audio data stream from the read end of the second channel according to the cross-process file descriptor, wherein the audio data stream is captured by the server based on the audio capture command and written to the write end of the second channel; and parse and identify the audio data stream to obtain subtitle text data. Alternatively, A pipeline is established between the client and the server to obtain a second channel, and the cross-process file descriptor corresponding to the read end of the second channel is obtained. The cross-process file descriptor is transmitted to the client through the first channel between the client and the server. When the client responds to the subtitle start command and sends an audio capture command through the first channel, the audio data stream is captured based on the audio capture command, and the audio data stream is written to the write end of the second channel, so that the client can read the audio data stream from the read end of the second channel according to the cross-process file descriptor, and parse and recognize the audio data stream to obtain the subtitle text data.

[0083] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0084] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0085] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0086] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described subtitle generation method. This solves the technical problem of the cumbersome interactive process for subtitle generation using third-party client applications. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the subtitle generation method provided in the above embodiments, and will not be repeated here.

[0087] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for generating subtitles, characterized in that, Applied to the client, the subtitle generation method includes: The cross-process file descriptor transmitted by the server is obtained through the first channel between the server and the client, wherein the cross-process file descriptor is used to characterize the read end of the second channel between the server and the client, and the second channel is a pipe established by the server. In response to the subtitle start command, an audio capture command is sent to the server through the first channel, and an audio data stream is read from the read end of the second channel according to the cross-process file descriptor, wherein the audio data stream is captured by the server based on the audio capture command and written to the write end of the second channel; The audio data stream is parsed and identified to obtain subtitle text data.

2. The subtitle generation method as described in claim 1, characterized in that, The step of reading the audio data stream from the read end of the second channel according to the cross-process file descriptor includes: A file input stream is constructed based on the cross-process file descriptor to establish a connection between the read end of the second channel and the client; The file input stream is read in a blocking manner in a loop to obtain an audio data stream.

3. The subtitle generation method as described in claim 1, characterized in that, Before the step of obtaining the cross-process file descriptor transmitted by the server through the first channel between the server and the client, the method further includes: The first channel is established according to the target service package identifier in the preset binding strategy list; The preset binding strategy list includes multiple candidate service package identifiers, and the target service package identifier is the candidate service package identifier among the candidate service package identifiers that corresponds to the service package that can establish the first channel with the client.

4. The subtitle generation method as described in claim 3, characterized in that, The candidate service package identifier includes a system service package identifier and a smart button service package identifier. The step of establishing the first channel based on the target service package identifier in the preset binding strategy list includes: Use the system service package identifier as the target service package identifier, and bind the system service package corresponding to the system service package identifier; If the client and the system service package are successfully bound, the first channel is established. If the binding between the client and the system service package fails, the target service package identifier is updated to the smart button service package identifier, and the smart button service package corresponding to the smart button service package identifier is bound. If the client and the smart button service package are successfully connected, the first channel is established.

5. The subtitle generation method as described in claim 1, characterized in that, The step of parsing and recognizing the audio data stream to obtain subtitle text data includes: The audio data stream is parsed to obtain the audio data to be identified; The audio data to be recognized is converted based on the data format of the preset speech recognition model, and the converted audio data is input into the preset speech recognition model to obtain subtitle text data.

6. The subtitle generation method as described in claim 1, characterized in that, The subtitle generation method also includes: In response to the subtitle stop command, an audio stop capture command is sent to the server through the first channel, so that the server stops capturing the audio data stream and closes the second channel.

7. A method for generating subtitles, characterized in that, Applied to the server side, the subtitle generation method includes: Establish a pipe between the client and the server to obtain a second channel, and obtain the cross-process file descriptor corresponding to the read end of the second channel; The cross-process file descriptor is transmitted to the client through the first channel between the client and the server. When the client responds to the subtitle start command and sends an audio capture command through the first channel, the audio data stream is captured based on the audio capture command, and the audio data stream is written to the write end of the second channel, so that the client reads the audio data stream from the read end of the second channel according to the cross-process file descriptor, and parses and identifies the audio data stream to obtain subtitle text data.

8. The subtitle generation method as described in claim 7, characterized in that, The step of capturing an audio data stream based on the audio capture command and writing the audio data stream to the write end of the second channel includes: Upon receiving the audio capture command, the audio recording interface is invoked based on preset system permissions to poll and obtain the pulse code modulation data of the audio input hardware; The pulse code modulation data is written as the audio data stream to the write end of the second channel.

9. An electronic device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the subtitle generation method as described in any one of claims 1 to 8.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the subtitle generation method as described in any one of claims 1 to 8.

Images