Display apparatus and display method for subtitle translation

Abstract

The application discloses a display device and a display method for subtitle translation. The display device monitors network state parameters in real time. The display device can obtain audio data through an audio serial bus interface and query a target transmission rate matching the network state parameters in response to a translation instruction in a target language. The display device sends the audio data and the target language to a server at the target transmission rate based on a communication device, so that the server converts first text corresponding to the audio data into second text corresponding to the target language based on a machine translation model. The second text returned by the server is received and displayed on a playing picture in a preset form. The application combines network state fluctuations to realize subtitle translation of audio data, avoids transmission packet loss and translation delay, ensures real-time and integrity of subtitle translation, adapts to different language background user needs, and optimizes user viewing experience.

Description

Technical Field

[0001] This application relates to the field of display device technology, and in particular to a display device and a method for displaying translated subtitles. Background Technology

[0002] Display devices are intelligent devices capable of presenting a user interface and supporting user interaction. Taking smart TVs as an example, smart TVs are television products based on Internet application technology, equipped with open operating systems and chips, possessing open application platforms, enabling two-way human-computer interaction, and integrating multiple functions such as audio-visual, entertainment, and data to meet diverse and personalized user needs.

[0003] Display devices are used in various scenarios such as homes, offices, and public displays. They can play various video content such as movies, news, educational videos, and live conferences. These videos are equipped with corresponding subtitles, which serve as a supplement to the video content and are used to present the audio information in the video, making it easier for viewers to understand the video content.

[0004] Because different videos are originally in different languages, and viewers have varying language comprehension abilities, the original subtitles provided with the videos cannot meet the needs of all viewers. Viewers may find it difficult to accurately understand subtitles in languages ​​they are not familiar with, which affects their understanding of the overall video content and results in poor subtitle display. Summary of the Invention

[0005] This application provides a display device and a method for displaying subtitle translations to solve the problem of poor subtitle translation display quality.

[0006] In a first aspect, this application provides a display device, comprising: The monitor is configured to display the playback screen of video data. An audio output device is provided with an audio serial bus interface and is connected to a controller through the audio serial bus interface; the audio output device is configured to play the sound of audio data and to transmit the audio data back to the controller through the audio serial bus interface; Communication devices that establish communication connections with the server; The controller is configured to monitor network status parameters between the communication device and the server, and the controller is further configured to: In response to a translation command for the target language, the audio data is acquired through the audio serial bus interface, and the appropriate target transmission rate is determined based on the network status parameters. Based on the communication device, the audio data and the target language are sent to the server at the target transmission rate, so that the server converts the first text corresponding to the audio data into the second text corresponding to the target language based on a machine translation model. Receive the second text returned by the server; The second text is displayed on top of the playback screen in a preset format.

[0007] The above technical solution has the following beneficial effects or advantages: The display device acquires audio data played by the audio output device through the audio serial bus interface, eliminating the need for additional voice acquisition equipment and reducing hardware costs. Simultaneously, by monitoring network status parameters to determine the target transmission rate, it achieves adaptive audio data transmission, ensuring real-time translation under different network environments. The translated text is then overlaid on the playback screen as subtitles, enabling synchronized display of voice translation and video content without manual user operation or external translation devices, thus enhancing user convenience in scenarios such as watching videos or making video calls.

[0008] In some embodiments of this application, the controller is specifically configured to send the audio data and the target language to the server according to the target transmission rate, as follows: The audio data is compressed and encoded to obtain encoded data; According to the target transmission rate, the encoded data is processed into packets to obtain encoded data packets; The encoded data packet and the target language are sent to the server.

[0009] The above technical solution has the following beneficial effects or advantages: Display devices can reduce the amount of audio data transmitted and lower network bandwidth usage by compressing and encoding audio data. By segmenting the encoded data into multiple data packets through packet processing, the reliability of data transmission can be improved, and lost data packets can be quickly retransmitted when the network is unstable, ensuring the continuity and stability of translation services.

[0010] In some embodiments of this application, the controller is configured to monitor network status parameters between the communication device and the server, specifically as follows: The upload bandwidth parameters, network latency parameters, data packet loss rate parameters, and network jitter parameters of the communication device are collected at a preset period. Smoothing filtering is performed on the upload bandwidth parameter, network latency parameter, data packet loss rate parameter, and network jitter parameter to obtain smoothing parameters; The smoothing parameters are stored in the corresponding smoothing window, which is used to store a preset number of parameters. The average value of the parameters in the smoothing window is calculated to obtain the network state parameters.

[0011] The above technical solution has the following beneficial effects or advantages: The display device can reflect the true state of the current network by collecting and smoothing network parameters in real time, avoiding misjudgment of transmission rate caused by instantaneous network fluctuations. By storing historical parameters and calculating the average value through a smoothing window, the impact of sudden fluctuations in network parameters can be eliminated, improving the accuracy and stability of network status assessment.

[0012] In some embodiments of this application, the controller is specifically configured to determine the appropriate target transmission rate based on the network state parameters as follows: Read the rate matching table, which includes the mapping relationship between network level and transmission rate; The upload bandwidth parameter, network latency parameter, data packet loss rate parameter, and network jitter parameter in the network status parameters are compared with their corresponding reference thresholds to obtain the detection results; The detection results are weighted to obtain the target network level; The target transmission rate is determined by querying the rate matching table for transmission rates that are mapped to the target network level.

[0013] The above technical solution has the following beneficial effects or advantages: The display device maps network level to transmission rate through a rate matching table. The target network level is determined based on a comprehensive evaluation of multi-dimensional network parameters, which can accurately reflect the network quality status. By weighted calculation, the importance of different network parameters is comprehensively considered to ensure that the selected target transmission rate can make full use of network bandwidth and guarantee the real-time performance and reliability of data transmission.

[0014] In some embodiments of this application, a memory is also included, and after the controller receives the second text returned by the server, it is further configured to: The second text is cached in the memory to form cached text; If the target network level is lower than a preset network level threshold, the cached text is read from the memory; The cached text is displayed on top of the playback screen in a preset format.

[0015] The above technical solution has the following beneficial effects or advantages: The display device caches the translated text in its memory and reads and displays the translated text from the local cache when the network quality is poor. This avoids the loss of translation content due to network delays or interruptions, ensuring the continuity and integrity of the translation service. At the same time, it determines whether to enable the caching display strategy by setting a preset network level threshold. When the network is good, it directly displays the real-time translation results, and automatically switches to the caching display mode when the network is poor. This achieves adaptive adjustment of the translation display strategy and improves the user experience in different network environments.

[0016] In some embodiments of this application, the controller executes the overlay display of the second text on top of the playback screen in a preset manner, specifically configured as follows: Obtain the background brightness of the playback screen; The display transparency of the second text is determined based on the background brightness; The display graphics of the second text are rendered according to the stated display transparency; The displayed graphics are overlaid on top of the playback screen.

[0017] The above technical solution has the following beneficial effects or advantages: The display device adaptively adjusts the transparency of the translated text based on the background brightness of the playback screen. When the screen is dark, the text transparency is increased to avoid obscuring important content, and when the screen is bright, the text transparency is decreased to ensure that the text is clear and readable. This achieves precise adaptation between the translated subtitle display effect and the video content, improving the user's visual comfort when watching the video and the readability of the translated content.

[0018] In some embodiments of this application, before the controller executes the display of the second text superimposed on the playback screen in a preset manner, it is further configured to: Perform image recognition on the playback screen; If a face area and / or the original subtitle area are detected in the playback screen, the display position of the second text is determined to be a blank area outside the face area and / or the original subtitle area, so that the second text is superimposed on the blank area of ​​the playback screen in a preset form.

[0019] The above technical solution has the following beneficial effects or advantages: The display device automatically detects the face area and the original subtitle area in the video, and displays the translated text in the blank area outside the aforementioned areas. This avoids the translated subtitles from obscuring important content in the video, ensuring the integrity and watchability of the video content. At the same time, by intelligently positioning blank areas for subtitle display, it can improve the user's experience when watching foreign language videos.

[0020] In some embodiments of this application, the controller is further configured to: Monitor the playback progress of the audio data; Generate timestamp information synchronized with the playback progress; The timestamp information is added to the audio data so that the timestamp information is synchronously sent to the server through the audio data; Receive the second text containing the timestamp information returned by the server; The display time of the second text is determined based on the timestamp information so that the display time of the second text is synchronized with the playback time of the audio data.

[0021] The above technical solution has the following beneficial effects or advantages: The display device monitors the audio playback progress and generates synchronization timestamp information, which is then sent to the server along with the audio data. This allows the server to perform timeline alignment of the translated text based on the timestamp information. After receiving the translated text containing the timestamp information, the display device precisely controls the display time of the translated text according to the timestamp information, achieving accurate synchronization between the translated subtitles and the audio playback. This avoids time discrepancies between the translated content and the playback screen and audio content, improving the accuracy of the translation service and the user experience.

[0022] In some embodiments of this application, a sound acquisition device is also included, wherein the controller is configured to: Control the sound acquisition device to collect external audio data from the external environment; The external audio data is subjected to noise reduction processing, and the external audio data is sent to the server according to the target transmission rate, so that the server converts the third text corresponding to the audio data into the fourth text corresponding to the target language and the fifth text corresponding to the language of the audio data based on the machine translation model; Receive the fourth text and the fifth text returned by the server; The fourth and fifth texts are displayed on top of the playback screen in a preset format.

[0023] The above technical solution has the following beneficial effects or advantages: The display device collects audio data from the external environment, such as user voice conversations and ambient sounds, through a sound acquisition device, and sends the external audio data to the server for translation processing. It can simultaneously display the target language translated text and the original language recognized text, realizing bilingual subtitle display function; through noise reduction processing, it improves the acquisition quality of external audio data, reduces the impact of environmental noise on translation accuracy, and expands the application scenarios of translation services, such as video calls, foreign language learning, and cross-language communication.

[0024] Secondly, this application also provides a method for displaying translated subtitles, applied to a display device. The display device establishes a communication connection with a server. The audio output device of the display device is configured with an audio serial bus interface and is connected to a controller through the audio serial bus interface. The audio output device is configured to play the sound of audio data and to transmit the audio data back to the controller through the audio serial bus interface. The controller is configured to monitor the network status parameters of the communication device and the server. The method includes: In response to a translation command for the target language, the audio data is acquired through the audio serial bus interface, and the appropriate target transmission rate is determined based on the network status parameters. Based on the communication device, the audio data and the target language are sent to the server at the target transmission rate, so that the server converts the first text corresponding to the audio data into the second text corresponding to the target language based on a machine translation model. Receive the second text returned by the server; The second text is displayed on top of the playback screen in a preset format.

[0025] The above technical solution has the following beneficial effects or advantages: The method acquires audio data through an audio serial bus interface, monitors network status parameters, determines the target transmission rate, compresses and encodes audio data and transmits it in packets, processes the translation in the cloud, and receives and overlays the translated text, thus realizing end-to-end real-time voice translation functionality for television devices without the need for additional hardware, reducing user costs. Through adaptive transmission rate adjustment and cached display strategies, it ensures the real-time performance and continuity of translation under different network environments. Furthermore, through intelligent subtitle display position selection and transparency adjustment, it achieves harmonious coexistence between translated subtitles and video content, enhancing the user's viewing experience. Attached Figure Description

[0026] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram illustrating the operation scenarios of a display device provided in some embodiments of this application; Figure 2 This is a schematic diagram of the hardware configuration of a display device provided in some embodiments of this application; Figure 3 This is a schematic diagram of the software configuration of a display device provided in some embodiments of this application; Figure 4 A flowchart illustrating the monitoring of network status parameters provided in some embodiments of this application; Figure 5 A flowchart illustrating a method for displaying translated subtitles provided in some embodiments of this application; Figure 6 A schematic diagram illustrating the cache display process provided for some embodiments of this application; Figure 7 A flowchart illustrating the process of determining the subtitle display position provided for some embodiments of this application; Figure 8 A schematic diagram illustrating the rendering process of subtitles provided for some embodiments of this application; Figure 9 A schematic diagram of the subtitle translation architecture provided for some embodiments of this application; Figure 10 Interactive diagrams illustrating the subtitle translation and display method provided in some embodiments of this application; Figure 11 Display effect diagrams of the second text provided in some embodiments of this application; Figure 12 The diagram shows the display effect of the settings control provided in some embodiments of this application. Detailed Implementation

[0028] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0029] In this embodiment, the display device 200 generally refers to a device with screen display and data processing capabilities. For example, the display device 200 includes, but is not limited to, smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, etc.

[0030] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device provided in some embodiments of this application. For example... Figure 1 As shown, a user can operate the display device 200 via touch operation, a mobile terminal 300, and a control device 100. The control device 100 receives user input commands and converts them into control commands that the display device 200 can recognize and respond to. For example, the control device 100 can be a remote control, a stylus, a gamepad, etc.

[0031] The mobile terminal 300 can function as a control device for human-computer interaction between the user and the display device 200. It can also function as a communication device for establishing a communication connection with the display device 200 and exchanging data. In some embodiments, the mobile terminal 300 can have software applications installed on it and communicate with the display device 200 via network communication protocols to achieve one-to-one control and data communication. Furthermore, it can transmit audio and video content displayed on the mobile terminal 300 to the display device 200 for synchronized display.

[0032] In some embodiments, the mobile terminal 300 or other electronic devices may also simulate the functions of the control device 100 by running an application that controls the display device 200.

[0033] like Figure 1 The diagram also shows that the display device 200 communicates with the server 400 via various communication methods. This allows the display device 200 to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0034] Display device 200 can provide broadcast television reception function, and can also be equipped with intelligent network television function that provides computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0035] Figure 2 Provided for some embodiments of this application Figure 1 Hardware configuration block diagram of display device 200.

[0036] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface 280.

[0037] In some embodiments, detector 230 is used to acquire signals from the external environment or to interact with the outside world. For example, detector 230 includes a light receiver, a sensor for acquiring ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to acquire external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds.

[0038] In some embodiments, the display 260 includes display function components for presenting images and driving components for driving image display. The display 260 is used to receive and display image signals output from the controller 250. For example, the display 260 can be used to display video content, image content, menu control interface components, and user control UI interfaces, etc.

[0039] In some embodiments, the communication device 220 is a component used to communicate with external devices or the server 400 according to various communication protocol types. The display device 200 may have multiple communication devices 220 depending on the supported communication methods. For example, when the display device 200 supports wireless network communication, it may have a communication device 220 with WiFi functionality. When the display device 200 supports Bluetooth connectivity, it needs to have a communication device 220 with Bluetooth functionality.

[0040] The communication device 220 enables the display device 200 to communicate with external devices or the server 400 via wireless or wired connections. Wired connections utilize data cables, interfaces, or other components to connect the display device 200 to external devices. Wireless connections utilize wireless signals or wireless networks. The display device 200 can directly establish a connection with external devices or indirectly through gateways, routers, or other connection devices.

[0041] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display device 200.

[0042] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0043] In some embodiments, a user can input user commands through a graphical user interface (GUI) displayed on a display 260, and the user input interface 280 receives the user input commands through the graphical user interface (GUI).

[0044] In some embodiments, the audio output device 270 can be a built-in speaker of the display device 200 or an external audio output device connected to the display device 200. For the external audio output device connected to the display device 200, the display device 200 may also be provided with an external audio output terminal, through which the audio output device can be connected to the display device 200 to output sound from the display device 200.

[0045] In some embodiments, the user input interface 280 can be used to receive instructions from user input.

[0046] To enable user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program used to manage and control the hardware and software resources of the display device 200. The operating system can control the display device to provide a user interface; for example, the operating system can directly control the display device to provide a user interface, or it can provide a user interface by running an application. The operating system also allows users to interact with the display device 200.

[0047] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system that is deeply customized based on a specific operating platform, or an independent operating system specifically developed for display devices.

[0048] An operating system can be divided into different modules or levels based on the functions it implements, for example... Figure 3 As shown, in some embodiments, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the System Library layer, and the Kernel layer.

[0049] In some embodiments, the application layer provides services and interfaces for applications, enabling the display device 200 to run applications and interact with the user based on the applications. The application layer may contain at least one application, which may be a built-in Windows program, system settings program, or clock program of the operating system; or it may be an application developed by a third-party developer. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0050] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0051] like Figure 3 As shown, Figure 3 The diagram below illustrates the software configuration of a display device according to some embodiments of this application. In some embodiments, the system of the display device 200 can be divided into three layers, from top to bottom: the application layer, the middleware layer, and the hardware layer.

[0052] The application layer mainly includes TV applications and application frameworks. The applications are mainly browser-based applications, such as HTML5 apps, and native apps. An application framework is a complete program model that has all the basic functions required by standard application software, such as file access, data exchange, and the user interface for these functions (toolbar, status bar, menu, dialog box).

[0053] Native apps can support online or offline access, push notifications, or access to local resources.

[0054] The middleware layer includes various television protocols, multimedia protocols, and system components. Middleware can use the basic services (functions) provided by system software to connect different parts of application systems or different applications on the network, achieving resource sharing and function sharing.

[0055] The hardware layer mainly includes the HAL interface, hardware, and drivers. The HAL interface is a unified interface for all TV chips, with the specific logic implemented by each chip. The drivers mainly include: audio drivers, display drivers, Bluetooth drivers, camera drivers, Wi-Fi drivers, USB drivers, HDMI drivers, sensor drivers (such as fingerprint sensors, temperature sensors, pressure sensors, etc.), and power drivers.

[0056] It should be noted that the above examples are merely a simple division of operating system functions and do not limit the specific form of the operating system of the display device 200 in this application embodiment. Depending on the function of the display device, the type of operating system, and other factors, the number of levels and the specific level type of the operating system may be expressed in other forms.

[0057] Display device 200 can adapt to viewing needs in different scenarios. For example, in a home setting, display device 200 is used by family members to watch movies and entertainment content, enhancing the family leisure experience; in an office setting, display device 200 is used to play meeting materials, training videos, project reports, and other content, assisting in office communication and knowledge transfer; in a public display setting, display device 200 can be used to play promotional videos, public service advertisements, information announcements, etc., to achieve widespread dissemination of information.

[0058] Based on the aforementioned diverse application needs, the types of video content that the display device 200 can play are varied, including movies, news, educational videos, live conferences, and other types of content. To facilitate viewers' accurate understanding of the audio information in the videos, these videos are usually equipped with corresponding original subtitles. As an important supplement to the video content, the original subtitles convert the audio information in the video into text, reducing the difficulty of understanding for viewers and improving the convenience of video viewing.

[0059] However, due to the wide range of video content sources and the significant differences in the original languages ​​of various videos, including Chinese, English, and French, viewers' language backgrounds and comprehension abilities vary greatly. Some viewers are only familiar with their native language and have limited understanding of other languages. In this context, the original subtitles provided with the videos often fail to meet the needs of all viewers, making it difficult for them to accurately interpret subtitles in languages ​​they are not familiar with, thus impacting the viewing experience and information acquisition efficiency.

[0060] In some embodiments, the display device 200 achieves real-time translation and display of original subtitles through audio acquisition, cloud translation, and screen display. However, the audio acquisition stage requires additional dedicated sound acquisition hardware, increasing the hardware cost and structural complexity of the display device 200. Furthermore, this method is highly dependent on network conditions; when network conditions are poor, data transmission is prone to delays and packet loss, leading to stuttering, lag, or missing translation content in the subtitle translation, thus affecting the real-time performance and display effect of the subtitle translation.

[0061] To address the aforementioned issues, this application provides a display device 200. During video playback, the display device 200 can acquire audio data via the audio serial bus (Inter-IC Sound, I2S) interface of the audio output device 270, and send the audio data to the server 400 (cloud server) for speech recognition and translation processing. The translation results are then overlaid on the playback screen as subtitles, thus achieving real-time speech translation functionality.

[0062] In some embodiments, the display device 200 includes a display 260, an audio output device 270, a communication device 220, and a controller 250. The display 260 is configured to display a video data playback screen; the audio output device 270 is equipped with an audio serial bus interface (I2S interface) and is connected to the controller 250 via the audio serial bus interface; the audio output device 270 is configured to play the sound of the audio data and to transmit the audio data back to the controller 250 via the audio serial bus interface; the communication device 220 is configured to establish a network connection with the server 400 to enable data uploading and downloading.

[0063] In some embodiments, the controller 250 of the display device 200 is configured to monitor network status parameters between the communication device 220 and the server 400. These network status parameters are indicators for evaluating the current network quality and can affect the real-time performance and reliability of audio data transmission.

[0064] Figure 4 This is a flowchart illustrating the monitoring of network status parameters provided in some embodiments of this application. For example... Figure 4 As shown, in some embodiments, the controller 250 of the display device 200 is configured to perform the following steps: S401. Collect the upload bandwidth parameters, network latency parameters, data packet loss rate parameters, and network jitter parameters of the communication device 220 according to the preset cycle.

[0065] The preset period can be set according to actual needs, such as 100ms, 200ms, 500ms, etc. The upload bandwidth parameter represents the maximum upload rate available in the current network environment, and the unit can be Kbps (kilobits per second) or Mbps (megabits per second). The controller 250 can calculate the actual upload bandwidth by sending test data packets and measuring the transmission time, or directly read the bandwidth parameter from the driver interface of the communication device 220.

[0066] The network latency parameter represents the time delay in which data is sent from the display device 200 to the server 400 and a response is received, measured in milliseconds (ms). The controller 250 can obtain the network latency parameter by sending heartbeat packets and measuring the round-trip time (RTT).

[0067] The packet loss rate parameter represents the proportion of data packets lost during data transmission, expressed as a percentage. Controller 250 can calculate the packet loss rate by counting the difference between the number of data packets sent and the number of acknowledgment data packets received.

[0068] Network jitter represents the degree of fluctuation in network latency, i.e., the magnitude of change in network latency parameters. Higher network jitter indicates a more unstable network and poorer real-time data transmission. Controller 250 can obtain network jitter parameters by calculating the standard deviation or variance of network latency parameters over multiple consecutive calculations.

[0069] S402. Perform smoothing filtering on the upload bandwidth parameter, network latency parameter, data packet loss rate parameter, and network jitter parameter to obtain smoothing parameters.

[0070] To prevent instantaneous fluctuations in network parameters from affecting the accuracy of network status assessment, the controller 250 performs smoothing filtering on upload bandwidth, network latency, packet loss rate, and network jitter parameters. The size of the smoothing window can be set according to actual needs, such as 5, 10, or 20 windows. A larger smoothing window provides better smoothing but a slower response to changes in network status; a smaller smoothing window provides a faster response to changes in network status but a worse smoothing effect.

[0071] S403. Store the smoothing parameters in the corresponding smoothing window.

[0072] Each network parameter corresponds to an independent smoothing window, which stores a preset number of parameters, such as 3 or 5. For example, the upload bandwidth parameter corresponds to the first smoothing window, the network latency parameter corresponds to the second smoothing window, the data packet loss rate parameter corresponds to the third smoothing window, and the network jitter parameter corresponds to the fourth smoothing window. Each smoothing window independently maintains its own historical parameter values ​​and average value calculations.

[0073] S404. Calculate the average value of the parameters in the smoothing window to obtain the network state parameters.

[0074] The display device 200 performs average calculations based on the aforementioned smoothing window to complete the smoothing filtering process of the parameters, enabling the display device 200 to obtain stable and reliable network status parameters and avoid misjudgments caused by instantaneous network fluctuations.

[0075] Figure 5 This is a flowchart illustrating a method for displaying translated subtitles provided in some embodiments of this application. In some embodiments, such as... Figure 5 As shown, controller 250 is configured to perform the following steps: S501, in response to the translation command for the target language, acquires audio data through the audio serial bus interface and determines the appropriate target transmission rate based on network status parameters.

[0076] Translation commands can be triggered by the user or automatically by the system.

[0077] In some embodiments, user-initiated translation methods include the user inputting translation commands and selecting a target language, such as English, Japanese, or French, via control device 100, mobile terminal 300, or touchscreen. Alternatively, when display device 200 is playing video data, the user can click a translation button on the video playback screen or input translation commands via voice.

[0078] In some embodiments, for automatic system triggering, the display device 200 automatically triggers a translation command when it detects that the audio language of the currently playing video content differs from the language set in the system. Alternatively, the display device 200 automatically triggers a translation command when it detects that the user has enabled the subtitle translation function.

[0079] In some embodiments, when the controller 250 acquires audio data through the audio serial interface, the controller 250 sends an audio data acquisition request to the audio output device 270 so that the audio output device 270 can transmit the currently playing digital audio data back to the controller 250 through the I2S interface.

[0080] For example, after receiving an audio data acquisition request from the controller 250, the digital amplifier chip of the audio output device 270 switches the operating mode of the I2S interface to the feedback mode, and transmits the received digital audio signal back to the controller 250 through the I2S interface. The controller 250 receives the audio data through the I2S interface and stores it in a buffer for subsequent processing.

[0081] In some embodiments, after acquiring audio data, the controller 250 performs compression encoding on the audio data to obtain encoded data. The compression encoding can employ audio compression algorithms such as Opus (Opus Interactive Audio Codec), AAC (Advanced Audio Coding), MP3, etc., to reduce the amount of audio data transmitted and reduce network bandwidth usage.

[0082] The display device 200 also responds to the translation command by determining the appropriate target transmission rate based on network status parameters, i.e., the transmission rate that matches the current network status.

[0083] In some embodiments, the display device 200 reads a rate matching table. This rate matching table includes a mapping relationship between network level and transmission rate, and can be pre-stored in memory or dynamically generated and updated by the controller 250. The upload bandwidth parameter, network latency parameter, data packet loss rate parameter, and network jitter parameter from the network status parameters are compared with their corresponding reference thresholds to obtain detection results. A weighted calculation is then performed on the detection results to obtain the target network level. Finally, the transmission rate that has a mapping relationship with the target network level is queried in the rate matching table to determine the target transmission rate.

[0084] For example, as shown in Table 1, Table 1 is a rate matching table.

[0085]

[0086] Table 1 As shown in Table 1, network quality is divided into five levels: Excellent, Good, Average, Poor, and Very Poor. Each level corresponds to a different range of network parameter thresholds and a target transmission rate. The higher the network quality, the better the network quality, and the higher the transmission rate it can support; the lower the network quality, the worse the network quality, and the lower the transmission rate needs to be to ensure the reliability of data transmission.

[0087] The controller 250 compares the upload bandwidth parameter, network latency parameter, data packet loss rate parameter, and network jitter parameter from the network status parameters with their corresponding reference thresholds to obtain the detection results. The detection results can be expressed as a Boolean value (satisfied or not satisfied) or a score (0-100 points). For example, for the upload bandwidth parameter, if the current upload bandwidth is 3Mbps, it is compared with the upload bandwidth threshold of the "good" level (≥2Mbps), and the result is satisfied; it is compared with the upload bandwidth threshold of the "excellent" level (≥5Mbps), and the result is not satisfied.

[0088] The controller 250 performs a weighted calculation on the detection results. The weighted calculation can assign different weight coefficients to each network parameter to reflect the importance of each parameter to the network quality assessment. For example, the upload bandwidth parameter has a weight of 0.3, the network latency parameter has a weight of 0.25, the data packet loss rate parameter has a weight of 0.25, and the network jitter parameter has a weight of 0.2.

[0089] The controller 250 calculates a comprehensive score based on the detection results and weighting coefficients of each parameter. For example, for a certain network condition, if the upload bandwidth parameter meets the "Excellent" level (5 points), the network latency parameter meets the "Good" level (4 points), the data packet loss rate parameter meets the "Average" level (3 points), and the network jitter parameter meets the "Good" level (4 points), then the comprehensive score is: 5×0.3+4×0.25+3×0.25+4×0.2=4.05 points.

[0090] The controller 250 determines the target network level based on the comprehensive score. For example, a comprehensive score ≥ 4.5 is "Excellent", 4.5 > ≥ 3.5 is "Good", 3.5 > ≥ 2.5 is "Average", 2.5 > ≥ 1.5 is "Poor", and a comprehensive score < 1.5 is "Very Poor".

[0091] After determining the target network level, the controller 250 queries the rate matching table for transmission rates that are mapped to the target network level to determine the target transmission rate. For example, if the target network level is "Good", the target transmission rate is 64Kbps; if the target network level is "Average", the target transmission rate is 32Kbps.

[0092] It should be noted that the specific values ​​in the above rate matching table are merely illustrative and do not constitute a limitation.

[0093] Through the aforementioned network status monitoring and transmission rate matching mechanism, the display device 200 can adaptively adjust the audio data transmission rate, increasing the transmission rate to ensure real-time translation when the network quality is good, and reducing the transmission rate to ensure transmission reliability when the network quality is poor, thereby optimizing the user's translation experience in different network environments.

[0094] S502. Based on the communication device, the audio data and target language are sent to the server according to the target transmission rate, so that the server can convert the first text corresponding to the audio data into the second text corresponding to the target language based on the machine translation model.

[0095] The display device 200 can transmit audio data and target language to the server 400 for processing according to the target transmission speed. Data transmission is achieved using the communication device 220. The communication device 220 supports multiple communication protocols, such as Wi-Fi, Bluetooth, and cellular mobile networks (e.g., 4G, 5G), to ensure stable connection and data interaction with the server 400 in different network environments.

[0096] In some embodiments, the controller 250 of the display device 200 performs compression encoding on the audio data to obtain encoded data. Then, according to the target transmission rate, the encoded data is processed into packets to obtain encoded data packets. The packet processing divides the encoded data into multiple fixed-size data packets, the size of which can be determined based on the target transmission rate and the network's Maximum Transmission Unit (MTU). The encoded data packets, along with the target language, are then sent to the server 400.

[0097] For example, if the target transmission rate is 64Kbps, the size of each data packet can be set to 1024 bytes; if the target transmission rate is 16Kbps, the size of each data packet can be set to 512 bytes. The smaller the data packet, the stronger the resistance to packet loss.

[0098] In some embodiments, each encoded data packet may include a header and a data body. The header contains information such as the data packet sequence number, timestamp, and checksum, which are used for data packet reordering, time synchronization, and error detection; the data body contains the actual encoded audio data.

[0099] In some embodiments, the target language data information can be included in the packet header or sent as a separate packet. After receiving the packet, the server 400 parses the target language data information and selects the corresponding machine translation model for translation processing based on the target language.

[0100] After receiving the data packet, server 400 converts the text corresponding to the audio data (i.e., the first text) into text corresponding to the target language (i.e., the second text) based on machine translation. In some embodiments, server 400 first performs Automatic Speech Recognition (ASR) processing on the audio data to convert the audio data into text data (the first text); then, it performs Machine Translation (MT) processing on the first text to translate the first text into the second text in the target language.

[0101] In some embodiments, speech recognition can employ end-to-end speech recognition models based on deep neural networks, such as DeepSpeech (deep speech-to-text model) and Wav2Vec (waveform-to-vector model), to achieve high-precision speech-to-text conversion. Machine translation can employ Transformer-based neural machine translation models to achieve high-quality cross-language translation.

[0102] In some embodiments, after completing the translation process, the server 400 returns the resulting second text to the display device 200. The second text may include text content in the target language, timestamp information, and confidence level information.

[0103] The timestamp information is used to identify the audio time point corresponding to the second text, so as to ensure the synchronization between the translated subtitles and the audio playback. The timestamp information can be added by the display device 200 when sending audio data, or it can be automatically generated by the server 400 based on the duration of the audio data.

[0104] The confidence level is used to indicate the reliability of the translation result, and its value ranges from 0 to 1. The higher the confidence level, the more reliable the translation result. The display device 200 can decide whether to display the translated text or adjust the display style of the translated text based on the confidence level.

[0105] For example, when the confidence level is lower than the confidence level threshold, the display device 200 can display the corresponding translated text in gray font or mark a prompt symbol next to the text to remind the user that there may be uncertainty in this part of the translation result; while when the confidence level is higher than the preset threshold, it is displayed clearly in normal font to ensure that the user obtains accurate information.

[0106] In some embodiments, the second text returned by the server 400 may also include language identification information to clearly indicate the target language to which the second text belongs, so that the display device 200 can call the corresponding font rendering method or character encoding rule according to the language identification information to ensure the correct presentation of different language texts on the playback screen.

[0107] S503, Receive the second text returned by the server.

[0108] Server 400 returns a second text to display device 200, which then receives the second text returned by server 400 for subsequent display processing.

[0109] Figure 6 This is a schematic diagram illustrating the cache display process provided in some embodiments of this application. For example... Figure 6 As shown, in some embodiments, after the display device 200 receives the second text returned by the server 400, it further performs the following steps: S601. Cache the second text to the memory to form cached text.

[0110] The display device 200 caches the second text in its memory to form cached text. The cached text can be arranged in timestamp order to form a corresponding translation text queue.

[0111] S602. If the target network level is lower than the preset network level threshold, read the cached text from the memory.

[0112] When the currently determined network level is lower than the preset network level threshold, such as "poor" or "very poor", it indicates that the current network quality is poor and translation delays or interruptions are likely to occur. At this time, the controller 250 reads the cached text from the memory.

[0113] S603. Display cached text overlaid on top of the playback screen according to a preset format.

[0114] After the controller 250 of the display device 200 reads the cached text, it overlays the cached text onto the playback screen according to a preset format. Through the cached display strategy, the display device 200 can achieve adaptive adjustment of the translation display, providing real-time translation when the network is good and cached translation when the network is poor, ensuring the continuity and stability of the translation service.

[0115] If the target network level is higher than or equal to the preset network level threshold, such as "excellent", "good" or "average", it means that the current network quality is good and the translation delay is small. At this time, the controller 250 can directly display the second text received in real time, i.e., step S504, without having to read it from the cache.

[0116] S504. Display the second text overlaid on the playback screen according to the preset format.

[0117] The display device 200 overlays the second text onto the playback screen in a preset format, wherein the preset format may include parameters such as display position, display style, display transparency, and display duration.

[0118] In some embodiments, the display position of the second text can be a fixed position on the screen, such as a fixed position at the bottom, top, middle, or sides of the screen.

[0119] To reduce the potential interference caused by the display of the second text, such as Figure 7 As shown, in some embodiments, the controller 250 of the display device 200 further performs the following steps: S701, Perform image recognition on the playback screen.

[0120] The controller 250 of the display device 200 performs image recognition on the playback screen to identify whether at least one of the following exists in the playback screen: a face region or an original subtitle region. The image recognition can use face detection algorithms (such as Haar feature classifiers or deep learning face detection models) to identify face regions and text detection algorithms (such as OCR text detection) to identify original subtitle regions.

[0121] S702. When a face area and / or original subtitle area are detected in the playback screen, the display position of the second text is determined to be a blank area outside the face area and / or original subtitle area, so as to display the second text superimposed on the blank area of ​​the playback screen in a preset form.

[0122] If the controller 250 detects at least one of the following in the playback screen: a face area or the original subtitle area, it determines the display position of the second text as a blank area outside the face area and the original subtitle area. The blank area refers to an area in the playback screen that does not contain important content (such as people or text), such as the sky or background wall.

[0123] In this embodiment of the disclosure, subtitles are displayed by intelligently locating blank areas, which can avoid the translated subtitles from obscuring important content in the video and ensure the integrity and watchability of the video content.

[0124] To improve the display of the second text, such as Figure 8 As shown, in some embodiments, the controller 250 of the display device 200 further performs the following steps: S801, Get the background brightness of the playback screen.

[0125] Background brightness can be obtained by calculating the average brightness value of the playback screen or by analyzing its histogram. For example, when the overall playback screen is dark, the average brightness value is lower, and the proportion of pixels in the low-brightness area of ​​the histogram is larger; while in a bright scene, the average brightness value is higher, and the high-brightness area of ​​the histogram is more concentrated. Background brightness obtained in these ways can accurately reflect the overall brightness of the playback screen.

[0126] S802. Determine the display transparency of the second text based on the background brightness.

[0127] The display transparency of the second text is determined based on the obtained background brightness. The display transparency is used to adjust the visibility of the translated text, and the value ranges from 0 to 100%. The higher the transparency, the more transparent the text and the less it obscures the background; the lower the transparency, the less transparent the text and the clearer and more readable the text.

[0128] The controller 250 of the display device 200 adaptively adjusts the display transparency according to the background brightness. For example, when the background brightness is high (such as in a bright scene), the display transparency is reduced (e.g., set to 30%) to improve the contrast and readability of the text; when the background brightness is low (such as in a dim scene), the display transparency is increased (e.g., set to 70%) to reduce obstruction of video content.

[0129] S803. Render the display graphics of the second text according to the display transparency.

[0130] The controller 250 of the display device 200 renders the display graphic corresponding to the second text according to the determined display transparency. During the rendering process, the controller 250 generates a text bitmap based on the content, font, font size, color, and other attributes of the second text; and performs alpha blending on the text bitmap according to the display transparency to obtain the display graphic with the corresponding transparency.

[0131] S804: Display the graphic overlaid on top of the playback screen.

[0132] After rendering the display graphics, the display device 200 overlays the display graphics onto a designated position on the playback screen for display on the monitor 260.

[0133] In this embodiment of the disclosure, by adjusting the transparency, the display device 200 can achieve intelligent adaptation between the translated subtitle display effect and the video content, thereby improving the visual comfort of users when watching videos and the readability of the translated content.

[0134] In some embodiments, the controller 250 also monitors the playback progress of the audio data and generates timestamp information synchronized with the playback progress. The playback progress can be obtained from the current playback time of the audio player, in seconds or milliseconds.

[0135] The timestamp information is used to identify the time point of the audio data. It can be included in the audio data packet and sent to server 400, or it can be sent as a separate time synchronization packet. After receiving the audio data containing the timestamp information, server 400 performs timeline alignment processing on the translation result according to the timestamp information to ensure that the timestamp of the second text (translated text) is consistent with the timestamp of the audio data.

[0136] The controller 250 receives a second text containing timestamp information returned by the cloud server and determines the display time of the second text based on the timestamp information. For example, if the timestamp of the second text is 00:01:30.000 (i.e., 1 minute and 30 seconds), the controller 250 displays the current text when the audio plays for 1 minute and 30 seconds.

[0137] In this embodiment of the disclosure, the display device 200 can achieve precise synchronization between translated subtitles and audio playback through a timestamp synchronization mechanism, avoiding time discrepancies between the translated content and the playback screen and audio content, thereby improving the accuracy of the translation service and the user experience.

[0138] In some embodiments, the display device 200 further includes a sound acquisition unit, which may be a microphone array, for acquiring external audio data of the external environment. The external audio data may include user voice conversations, ambient sounds, etc.

[0139] The controller 250 controls the sound acquisition unit to collect external audio data and performs noise reduction processing on the external audio data. The noise reduction processing can employ noise suppression algorithms, such as spectral subtraction, Wiener filtering, and deep learning noise reduction models, to reduce the impact of environmental noise on audio quality.

[0140] The controller 250 sends the external audio data to the server 400 at the target transmission rate, so that the server 400 can convert the third text corresponding to the external audio data into the fourth text corresponding to the target language and the fifth text corresponding to the language of the external audio data based on the machine translation model.

[0141] The third text is the speech recognition result of the external audio data (original language), the fourth text is the translation result of the third text (target language), and the fifth text is the original language recognition result of the third text (used to display bilingual subtitles).

[0142] The controller 250 receives the fourth and fifth texts returned by the cloud server and displays them superimposed on the playback screen according to a preset format. For example, the fifth text (original language) can be displayed at the top of the screen, and the fourth text (target language) can be displayed at the bottom of the screen, realizing the bilingual subtitle display function.

[0143] With external audio acquisition and translation functions, the display device 200 can expand the application scenarios of translation services, such as video calls, foreign language learning, and cross-language communication. Users can have real-time conversations with foreign friends through the display device 200, and the display device 200 will automatically translate the voices of both parties and display them as bilingual subtitles, thereby improving the convenience of cross-language communication.

[0144] Figure 9 This is a schematic diagram illustrating the architecture of subtitle translation provided in some embodiments of this application. For example... Figure 9 As shown, display device 200 establishes a communication connection with server 400. The speaker (audio output device 270) of display device 200 can be used to play audio data and transmit audio data back to controller 250. Controller 250 transmits the received audio data and the target language to be translated to server 400. The audio data is first noise-reduced and preprocessed before being transmitted to server 400. Server 400 translates the original text (first text) of the audio data according to the target language to obtain translated text (second text). The translated text is then displayed according to a preset format to realize the subtitle translation display process. During the transmission of audio data, display device 200 also adaptively adjusts the audio data transmission rate according to network status parameters.

[0145] In some embodiments, during the process of caching the second text, the display device 200 also adaptively adjusts the caching strategy of the second text based on network status parameters.

[0146] For example, when the network is in good condition, the display device 200 can adopt a real-time caching strategy, that is, after the server 400 generates the second text, it is immediately transmitted to the local machine for caching and display to ensure the timeliness of the subtitles; while when the network is in poor condition, the display device 200 can switch to a strategy that combines pre-caching and dynamic adjustment, pre-caching a certain length of the second text segment, and dynamically adjusting the length and update frequency of the cached segment according to the network recovery status, so as to avoid the subtitle display being interrupted or stuttered due to network fluctuations, thereby maintaining a relatively smooth subtitle translation experience in different network environments.

[0147] Figure 10 Interactive diagrams illustrating subtitle translation and display methods provided in some embodiments of this application. For example... Figure 10 As shown, the display device 200 establishes a communication connection with the server 400 through the communication device 220. The controller 250 monitors the network status parameters of the communication device 220 and the server 400 in real time to monitor the network quality. After receiving a playback command for video data, the controller 250 plays and displays the video data, causing the monitor 260 to display the video playback screen.

[0148] At this point, the user inputs a translation command in the target language. The controller 250 responds to the translation command by sending audio output back to the controller 250 via the I2S interface of the audio output device 270. This audio data can include audio data from the video data or external audio data collected by a sound acquisition device; it is the audio data that the audio output device 270 wants to play, and it is then sent back to the controller 250 via the I2S interface. The controller 250 then determines the target transmission rate according to network status parameters and sends the audio data to the server 400 via the communication device 220. The server 400 translates the audio data into the corresponding translated text (second text) in the target language. The translated text is then returned to the controller 250 via the communication device 220. The controller 250 displays the translated text overlaid on the playback screen according to a preset format, allowing the display 260 to display the translated subtitles in real time.

[0149] For example, the display effect of the second text is as follows: Figure 11 As shown, the second text is displayed as a semi-transparent text box overlaid on top of the playback screen.

[0150] In some embodiments, after displaying the second text overlaid on top of the playback screen, the display device 200 also displays subtitle translation settings controls. Users can use these controls to switch the target language to change the language type of the translated text. For example, as... Figure 12 As shown, the setting control 1201 is displayed in the area above the second text for the user to switch the display.

[0151] In some embodiments, the settings control can also allow users to adjust the display parameters of the second text, such as the transparency of the text box, font size, font color, and the display position of the text in the playback screen. For example, users can click the settings control to bring up the parameter adjustment menu, drag the transparency slider to adjust the transparency of the semi-transparent text box from the default 50% to 30% to reduce obstruction of the main content of the playback screen; or select a larger font size so that the translated text can be clearly read even when viewed from a distance, improving the readability of the text.

[0152] In addition, in some embodiments, users can also change the display area of ​​the second text by dragging the text box to select a preset position (such as the bottom center, top center, left or right side of the screen) to adapt to different users' viewing habits and the layout requirements of the screen content.

[0153] Based on the aforementioned display device 200, this application embodiment also provides a subtitle translation display method applied to the display device 200. The display device 200 establishes a communication connection with the server 400. The audio output device 270 of the display device 200 is equipped with an audio serial bus interface and is connected to the controller 250 through the audio serial bus interface. The audio output device 270 is configured to play the sound of audio data and to transmit the audio data back to the controller 250 through the audio serial bus interface. The controller 250 is configured to monitor the network status parameters between the communication device 220 and the server 400. Figure 5 As shown, the method includes the following steps: S501, in response to the translation command for the target language, acquires audio data through the audio serial bus interface and determines the appropriate target transmission rate based on network status parameters.

[0154] S502. Based on the communication device, the audio data and target language are sent to the server according to the target transmission rate, so that the server can convert the first text corresponding to the audio data into the second text corresponding to the target language based on the machine translation model.

[0155] S503, Receive the second text returned by the server.

[0156] S504. Display the second text overlaid on the playback screen according to the preset format.

[0157] The above method acquires audio data through an audio serial bus interface, monitors network status parameters, determines the target transmission rate, compresses and encodes audio data and transmits it in packets, processes the translation in the cloud, and receives and overlays the translated text. This enables real-time end-to-end voice translation functionality on the display device, eliminating the need for additional hardware and reducing user costs. Adaptive transmission rate adjustment and cached display strategies ensure the real-time performance and continuity of translation under different network environments. Intelligent subtitle display position selection and transparency adjustment achieve harmonious coexistence between translated subtitles and video content, enhancing the user experience when watching foreign language videos.

[0158] Similar parts between the embodiments provided in this application can be referred to mutually. The specific implementation methods provided above are only a few examples under the overall concept of this application and do not constitute a limitation on the scope of protection of this application. For those skilled in the art, any other implementation methods extended from the solution of this application without creative effort shall fall within the scope of protection of this application.

Claims

1. A display device, characterized in that, include: The monitor is configured to display the playback screen of video data. An audio output device is provided with an audio serial bus interface and is connected to a controller through the audio serial bus interface; the audio output device is configured to play the sound of audio data and to transmit the audio data back to the controller through the audio serial bus interface; Communication devices that establish communication connections with the server; The controller is configured to monitor network status parameters between the communication device and the server, and the controller is further configured to: In response to a translation command for the target language, the audio data is acquired through the audio serial bus interface, and the appropriate target transmission rate is determined based on the network status parameters. Based on the communication device, the audio data and the target language are sent to the server at the target transmission rate, so that the server converts the first text corresponding to the audio data into the second text corresponding to the target language based on a machine translation model. Receive the second text returned by the server; The second text is displayed on top of the playback screen in a preset format.

2. The display device according to claim 1, characterized in that, The controller is configured to send the audio data and the target language to the server according to the target transmission rate, specifically as follows: The audio data is compressed and encoded to obtain encoded data; According to the target transmission rate, the encoded data is processed into packets to obtain encoded data packets; The encoded data packet and the target language are sent to the server.

3. The display device according to claim 1, characterized in that, The controller is configured to monitor the network status parameters between the communication device and the server, specifically as follows: The upload bandwidth parameters, network latency parameters, data packet loss rate parameters, and network jitter parameters of the communication device are collected at a preset period. Smoothing filtering is performed on the upload bandwidth parameter, network latency parameter, data packet loss rate parameter, and network jitter parameter to obtain smoothing parameters; The smoothing parameters are stored in the corresponding smoothing window, which is used to store a preset number of parameters. The average value of the parameters in the smoothing window is calculated to obtain the network state parameters.

4. The display device according to claim 3, characterized in that, The controller is configured to determine the appropriate target transmission rate based on the network state parameters, specifically as follows: Read the rate matching table, which includes the mapping relationship between network level and transmission rate; The upload bandwidth parameter, network latency parameter, data packet loss rate parameter, and network jitter parameter in the network status parameters are compared with their corresponding reference thresholds to obtain the detection results; The detection results are weighted to obtain the target network level; The target transmission rate is determined by querying the rate matching table for transmission rates that are mapped to the target network level.

5. The display device according to claim 4, characterized in that, It also includes a memory, and after the controller receives the second text returned by the server, it is further configured to: The second text is cached in the memory to form cached text; If the target network level is lower than a preset network level threshold, the cached text is read from the memory; The cached text is displayed on top of the playback screen in a preset format.

6. The display device according to claim 1, characterized in that, The controller is configured to overlay the second text onto the playback screen in a preset manner, specifically as follows: Obtain the background brightness of the playback screen; The display transparency of the second text is determined based on the background brightness; The display graphics of the second text are rendered according to the stated display transparency; The displayed graphics are overlaid on top of the playback screen.

7. The display device according to claim 1, characterized in that, Before the controller executes the preset method of overlaying the second text on top of the playback screen, it is further configured to: Perform image recognition on the playback screen; If a face area and / or the original subtitle area are detected in the playback screen, the display position of the second text is determined to be a blank area outside the face area and / or the original subtitle area, so that the second text is superimposed on the blank area of ​​the playback screen in a preset form.

8. The display device according to claim 1, characterized in that, The controller is also configured to: Monitor the playback progress of the audio data; Generate timestamp information synchronized with the playback progress; The timestamp information is added to the audio data so that the timestamp information is synchronously sent to the server through the audio data; Receive the second text containing the timestamp information returned by the server; The display time of the second text is determined based on the timestamp information so that the display time of the second text is synchronized with the playback time of the audio data.

9. The display device according to claim 1, characterized in that, It also includes a sound acquisition unit, and the controller is configured to: Control the sound acquisition device to collect external audio data from the external environment; The external audio data is subjected to noise reduction processing, and the external audio data is sent to the server according to the target transmission rate, so that the server converts the third text corresponding to the audio data into the fourth text corresponding to the target language and the fifth text corresponding to the language of the audio data based on the machine translation model; Receive the fourth text and the fifth text returned by the server; The fourth and fifth texts are displayed on top of the playback screen in a preset format.

10. A method for displaying translated subtitles, characterized in that, This system is applied to display devices, which establish a communication connection with a server. The audio output device of the display device is equipped with an audio serial bus interface and is connected to a controller through the audio serial bus interface. The audio output device is configured to play audio data and transmit the audio data back to the controller through the audio serial bus interface. The controller is configured to monitor network status parameters between the communication device and the server; the method includes: In response to a translation command for the target language, the audio data is acquired through the audio serial bus interface, and the appropriate target transmission rate is determined based on the network status parameters. Based on the communication device, the audio data and the target language are sent to the server at the target transmission rate, so that the server converts the first text corresponding to the audio data into the second text corresponding to the target language based on a machine translation model. Receive the second text returned by the server; The second text is displayed on top of the playback screen in a preset format.

Images