Data processing method and device, computer equipment and computer readable storage medium

Abstract

Embodiments of the present application provide a data processing method and device, computer equipment and computer readable storage medium, which are used for synchronizing delay processing of video and audio, thereby reducing sound delay. The method comprises: obtaining video delay data and a mode identifier, the mode identifier being used to represent a content type corresponding to playing content of a display device; generating a first instruction based on the video delay data and the identifier; and delivering the first instruction to an external audio device, so that the external audio device determines a delay processing strategy of audio processing based on the first instruction.

Description

Technical Field

[0001] This application relates to the field of smart home appliances, specifically to a data processing method, apparatus, computer equipment, and computer-readable storage medium. Background Technology

[0002] With advancements in consumer electronics technology, users' demands for home entertainment experiences are constantly increasing. They are no longer satisfied with the built-in speakers of televisions, but instead seek high-quality, immersive sound and flexible home layouts from independent amplifiers, soundbars, or wireless audio systems. High-Definition Multimedia Interface (HDMI) technology, especially its supported Audio Return Channel (ARC) / eEnhanced Audio Return Channel (ARC), has become the industry standard for connecting televisions to external audio systems. This technology not only simplifies wiring but also transmits multi-channel high-definition audio formats such as Dolby Atmos, greatly enhancing the user's listening experience.

[0003] In current technical solutions, when a TV receives a signal from a game console or other signal source via the HDMI interface, it can automatically switch to "Game Mode" using mechanisms such as Auto Low Latency Mode (ALLM). In this mode, the TV disables or bypasses complex video post-processing functions such as Motion Estimation (MEMC) and dynamic contrast to minimize video signal processing latency and ensure the immediacy of the image display. Simultaneously, the audio signal is transmitted back to an external audio system via the HDMI ARC / eARC link for decoding and playback. To ensure stable audio playback and sound quality, the external audio system typically performs necessary buffering, decoding, and sound field processing on the received audio stream.

[0004] The key flaw in existing technologies lies in the fact that low-latency optimization control is primarily limited to the video link, while the audio link lacks effective synchronous low-latency control methods. Specifically, low-latency commands such as ALLM are typically executed at the TV device end, and there is no standardized mechanism to transmit them to the downstream external audio system via the HDMI ARC / eARC link. Therefore, when the TV enables ultra-low latency video mode for game visuals, the external audio system is still executing its regular audio process with higher processing latency. This leads to significant audio-visual asynchrony (audio-visual delay), with sound feedback noticeably lagging behind visual actions. In scenarios such as esports, where real-time feedback is extremely crucial, this latency severely impacts player judgment and actions, significantly impairing the immersive gaming experience. Summary of the Invention

[0005] This application provides a data processing method, apparatus, computer device, and computer-readable storage medium for synchronizing video and audio latency processing, thereby reducing sound latency.

[0006] The technical solution adopted by this invention to solve the problem is as follows: Firstly, this application provides a data processing method, including: Acquire video latency data and a mode identifier, which indicates the content type corresponding to the playback content on the display device; A first instruction is generated based on the video delay data and the pattern identifier; The first instruction is passed to the external audio device so that the external audio device can determine the audio processing delay strategy based on the first instruction.

[0007] In some embodiments of this application, obtaining video latency data and pattern identifiers includes: Read the log file to obtain the video latency data, which describes the internal processing latency of the video playback pipeline; Detect the automatic low-latency mode signal or the mode enable signal to obtain the mode identifier.

[0008] In some embodiments of this application, the acquisition of video delay data and pattern identifiers includes: Read the log file to obtain the video latency data, which describes the internal processing latency of the video playback pipeline; The content type of the content currently being played on the display device is detected to obtain the mode identifier.

[0009] In some embodiments of this application, the first instruction also includes lighting control information; The method also includes: The first instruction is sent to the smart lighting device so that the smart lighting device responds to the first instruction to control the lighting.

[0010] In some embodiments of this application, the first instruction further includes tactile control information; The method also includes: The first instruction is sent to the haptic feedback device so that the haptic feedback device responds to the first instruction to perform haptic control.

[0011] In some embodiments of this application, the method further includes: Generate a second instruction, which includes indication information for instructing the display device to restore the standard mode; The second instruction is sent to the external audio device so that the external audio device responds to the second instruction and resumes the standard audio processing mode.

[0012] In some embodiments of this application, the method further includes: the delayed processing strategy includes at least one of the following: Adjust the buffer length for audio data; Turn off sound effects processing; The delay compensation value is determined based on the video delay data and the audio processing data of the external audio device, and the delay is adjusted based on the delay compensation value.

[0013] Secondly, this application provides a data processing apparatus, comprising: The acquisition module is used to acquire video latency data and a mode identifier, which indicates the content type corresponding to the playback content of the display device. The processing module is used to generate a first instruction based on the video delay data and the pattern identifier; The transceiver module is used to transmit the first instruction to the external audio device so that the external audio device can determine the audio processing delay strategy based on the first instruction.

[0014] Thirdly, this application also provides a computer device, which includes: One or more processors; Memory; and One or more applications, wherein the applications are stored in memory and configured to be executed by a processor to implement the data processing method of any of the first aspects.

[0015] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to perform the steps of the data processing method of any one of the first aspects.

[0016] The beneficial effects of this invention are as follows: By acquiring precise delay data and mode identifiers of the internal video path in real time from the display device, and constructing control commands accordingly to send to the external audio device, the audio device can dynamically and accurately adjust its audio output delay, thereby achieving adaptive synchronization between the video stream and the audio stream. This solves the core pain point of audio-visual synchronization issues in different application scenarios, avoids cumbersome manual settings, and significantly improves the automation level of the entire audio-visual system and the user's immersive experience. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of an exemplary application scenario provided by an embodiment of the present invention; Figure 2 This is a schematic diagram of an embodiment of the data processing method provided by the present invention; Figure 3 This is a schematic diagram of the architecture of the data processing method provided in an embodiment of the present invention; Figure 4 This is a schematic block diagram of one embodiment of the data processing apparatus provided in this invention; Figure 5 This is a schematic diagram of an embodiment of the computer device provided in this invention. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] In the description of this application, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first," "second," "third," etc., may explicitly or implicitly include one or more features.

[0021] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0022] It should be noted that since the method in this application embodiment is executed in a computer device, the processing objects of each computer device exist in the form of data or information, such as time, which is essentially time information. It is understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data that exist so that the computer device can process them. Specific details will not be elaborated here.

[0023] With advancements in consumer electronics technology, users' demands for home entertainment experiences are constantly increasing. They are no longer satisfied with the built-in speakers of televisions, but instead seek high-quality, immersive sound and flexible home layouts from independent amplifiers, soundbars, or wireless audio systems. High-Definition Multimedia Interface (HDMI) technology, especially its supported Audio Return Channel (ARC) / eEnhanced Audio Return Channel (ARC), has become the industry standard for connecting televisions to external audio systems. This technology not only simplifies wiring but also transmits multi-channel high-definition audio formats such as Dolby Atmos, greatly enhancing the user's listening experience.

[0024] In current technical solutions, when a TV receives a signal from a game console or other signal source via the HDMI interface, it can automatically switch to "Game Mode" using mechanisms such as Auto Low Latency Mode (ALLM). In this mode, the TV disables or bypasses complex video post-processing functions such as Motion Estimation (MEMC) and dynamic contrast to minimize video signal processing latency and ensure the immediacy of the image display. Simultaneously, the audio signal is transmitted back to an external audio system via the HDMI ARC / eARC link for decoding and playback. To ensure stable audio playback and sound quality, the external audio system typically performs necessary buffering, decoding, and sound field processing on the received audio stream.

[0025] The key flaw in existing technologies lies in the fact that low-latency optimization control is primarily limited to the video link, while the audio link lacks effective synchronous low-latency control methods. Specifically, low-latency commands such as ALLM are typically executed at the TV device end, and there is no standardized mechanism to transmit them to the downstream external audio system via the HDMI ARC / eARC link. Therefore, when the TV enables ultra-low latency video mode for game visuals, the external audio system is still executing its regular audio process with higher processing latency. This leads to significant audio-visual asynchrony (audio-visual delay), with sound feedback noticeably lagging behind visual actions. In scenarios such as esports, where real-time feedback is extremely crucial, this latency severely impacts player judgment and actions, significantly impairing the immersive gaming experience.

[0026] To address this technical problem, this application provides the following technical solution: acquiring video latency data and a mode identifier, whereby the mode identifier represents the content type corresponding to the playback content on the display device; generating a first instruction based on the video latency data and the mode identifier; and transmitting the first instruction to an external audio device, enabling the external audio device to determine a latency processing strategy for audio processing based on the first instruction. Beneficial effects: By acquiring precise latency data and a mode identifier of its internal video path in real time from the display device, and constructing control instructions accordingly to send to the external audio device, the audio device can dynamically and precisely adjust its audio output latency, thereby achieving adaptive synchronization between the video stream and the audio stream. This solves the core pain point of audio-visual synchronization issues in different application scenarios, avoids cumbersome manual settings, and significantly improves the automation level of the entire audio-visual system and the user's immersive experience.

[0027] This application provides a data processing method, apparatus, computer device, and computer-readable storage medium for synchronizing video and audio latency processing, thereby significantly reducing sound latency. The electronic device provided in this application can be implemented as various types of user terminals or as a server.

[0028] Electronic devices can significantly reduce sound latency by running the data processing method provided in the embodiments of this application to synchronize video and audio latency processing.

[0029] The above methods can be applied to many smart home appliances, such as smart TVs, smart screens, etc.

[0030] In one exemplary scenario, this data processing method can be applied to a smart TV's video playback scenario. For instance, in a smart TV video playback scenario, a user connects a game console to the smart TV via an HDMI interface, and the smart TV is then connected to an external audio device (such as a soundbar) via an eARC interface. When the user starts the game, the game console sends a low-latency flag (such as an ALLM signal) to the smart TV. Upon receiving this flag, the smart TV's control unit immediately switches the video processing to an extremely low-latency "game mode" and obtains the video path latency data at this time, for example, 12 milliseconds, by monitoring its internal video processing pipeline in real time. Subsequently, the smart TV's control unit constructs a latency control command containing the 12-millisecond latency value and the low-latency flag, and sends it to the external audio device via the eARC channel. After receiving the command, the audio device's internal digital signal processor (DSP) can switch to an audio mode optimized for games (such as adjusting the audio buffer length to the buffer length corresponding to the game mode), and can also adjust the output latency of the audio stream to 12 milliseconds. The final result is that when an explosion occurs on the game screen, the visual effects seen by the user are perfectly synchronized with the sound of the explosion heard from the audio device.

[0031] It should be understood that the above is only an exemplary application scenario of the data processing method, and there are many other possible application scenarios, which are not limited here.

[0032] The data processing method provided in this application embodiment is applied to, for example, Figure 1 The system architecture diagram shown is for your reference. Figure 1To support a data processing method, the terminal device 100 connects to the server 300 via network 200, and the server 300 connects to the database 400. Network 200 can be a wide area network (WAN), a local area network (LAN), or a combination of both. The client used to implement the data processing scheme is deployed on the terminal device 100, or it can run on the terminal device 100 as a standalone application. The specific form of the client is not limited here.

[0033] The server 300 involved in this application can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms.

[0034] Terminal equipment 100, also known as user equipment (UE), mobile station (MS), mobile terminal (MT), customer premises equipment (CPE), etc., can be a device that includes both receiving and transmitting hardware, that is, a device with receiving and transmitting hardware capable of performing bidirectional communication on a bidirectional communication link. Such equipment can include cellular or other communication devices with single-line displays, multi-line displays, or no multi-line displays. Examples include handheld devices with wireless connectivity, vehicle-mounted devices, machine-type communication (MTC) terminals, etc. Currently, terminal devices 100 can include: mobile phones, tablets, laptops, PDAs, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving vehicles, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, or wireless terminals in smart homes, etc. For example, wireless terminals in self-driving vehicles can be drones, helicopters, or airplanes. For example, wireless terminals in vehicle-to-everything (V2X) systems can be in-vehicle equipment, vehicle-mounted equipment, in-vehicle modules, vehicles, or ships, etc. Wireless terminals in industrial control can be cameras, robots, or robotic arms, etc. Wireless terminals in smart homes can be televisions, air conditioners, robot vacuums, speakers, or set-top boxes, etc.

[0035] It should be noted that the terminal device 100 may be a device or apparatus with a chip, or a device or apparatus with integrated circuitry, or a chip, module, or control unit in the device or apparatus shown above; this application does not impose any specific limitations. The solution provided in this application can be implemented by the terminal device 100 and the server 300 working together.

[0036] In short, a database can be viewed as an electronic filing cabinet—a place to store electronic files, where users can perform operations such as adding, querying, updating, and deleting data. A "database" is a collection of data stored together in a certain way, shared by multiple users, with minimal redundancy, and independent of application programs. A Database Management System (DBMS) is a computer software system designed to manage databases, generally possessing basic functions such as storage, retrieval, security, and backup. DBMSs can be classified according to the database model they support, such as relational or Extensible Markup Language (XML); or according to the type of computer they support, such as server clusters or mobile phones; or according to the query language used, such as Structured Query Language (SQL) or XQuery; or according to performance priorities, such as maximum scale or maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMSs can cross categories, for example, supporting multiple query languages ​​simultaneously. In this application, database 400 can be used to store data such as video data and audio data.

[0037] Those skilled in the art will understand that Figure 1 The system architecture diagram shown is one possible system architecture for this application and does not constitute a limitation on the system architecture of this application. Other system architectures may include more advanced architectures. Figure 1 The number of more or fewer terminal devices or servers shown, for example Figure 1 The diagram shows one server. It is understood that the system architecture may also include one or more other terminal devices or servers, which are not limited here.

[0038] It should be noted that, Figure 1 The system architecture shown is an example. The servers and scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of servers and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0039] like Figure 2 The diagram shown is a flowchart of an embodiment of the data processing method in this application. The following description, using a display device as the execution subject, details the data processing method, which may include the following steps 201-204: 201. Obtain video delay data and mode identifier, which is used to indicate the content type corresponding to the playback content of the display device.

[0040] During video playback, the display device's built-in control unit continuously monitors content information through multiple methods in parallel and determines the final mode identifier based on preset priority rules. Once the mode identifier is determined or changes, the control unit immediately queries the video processing pipeline to obtain video latency data matching the current mode.

[0041] The display device can determine the content type of the currently playing content through image recognition or signal indication from an external signal source, thereby determining the current mode identifier; simultaneously, it extracts internal log files to obtain video latency data from the display device's video processing pipeline. Its execution flow can be as follows: Read the log file to obtain the video latency data, which describes the internal processing latency of the video playback pipeline; detect the automatic low latency mode signal or the mode enable signal to obtain the mode identifier.

[0042] And / or, Read the log file to obtain the video latency data, which describes the internal processing latency of the video playback pipeline; detect the content type of the content currently being played on the display device to obtain the mode identifier.

[0043] It should be understood that when the display device detects the content type of the content currently being played, it may use at least one of the image recognition model, audio recognition model, and text recognition model to identify the multimodal content of the content being played, without any specific limitation here.

[0044] This mode activation signal indicates the various preset playback modes on the display device. These include movie mode, game mode, music mode, live streaming mode, etc. Users can manually select or voice-select which playback mode to activate, or the display device can automatically match the playback mode based on the content being played; specific options are not limited here.

[0045] The video latency data is a core metric used to quantify the time cost incurred by the display device in optimizing image quality. It represents the total time from when the video signal enters the display device to when the corresponding image is finally displayed on the screen. Its fundamental purpose is to provide a precise time reference for subsequent collaborative operations such as audio-visual synchronization and optical-visual synchronization. This data is typically a value expressed in milliseconds. It is the sum of the delays of multiple processing modules in the display device's internal video pipeline, including but not limited to: video decoding, image scaling, dynamic contrast enhancement, HDR tone mapping, motion estimation (MEMC), and noise reduction. This data is dynamic and changes with the display mode and processing algorithm. In a specific implementation, this data may be a 32-bit unsigned integer variable, such as u32VideoPathDelay, calculated and maintained in real-time by the display device's system-on-a-chip (SoC) firmware. When the display device is in cinema mode, this value may be 75 (ms) due to the activation of complex image enhancement algorithms; when switching to game mode, this value may drop sharply to 12 (ms) in pursuit of the ultimate response.

[0046] A mode identifier is a semantic label used by a display device to understand "what content is currently playing." It acts as a trigger, the starting point for the system's contextual and adaptive experience optimization decisions, determining the video processing strategy, audio effect mode, and ambient lighting scheme to be adopted. This identifier is an enumeration value or status flag representing a specific content type. Its sources are diverse, including direct instructions from external signal sources (such as HDMI ALLM signals), metadata obtained from parsing media streams, and conclusions derived by the display device through internal algorithm analysis. The mode identifier can be represented in various forms. For example, it can be a Boolean value: isGameMode = TRUE. Or, it can be an enumeration string: ContentType = "MOVIE". Or, it can be a scene code output by an artificial intelligence (AI) engine: AI_Scene_ID = 0x03 (where 0x03 is predefined as a sports event).

[0047] Content type is a classification of the content being played, directly related to the user's expected experience. For example, users expect movies to be immersive, games to be highly responsive, and sporting events to feel like they are happening right before their eyes. In this embodiment, the content type is the specific category pointed to by the pattern identifier, which is a limited set of scenarios preset by the system. Common categories include: movies, TV series, games, sporting events, news, concerts, etc. In an exemplary scheme, the pattern identifier ALLM=ON can be used to indicate that the corresponding content type is "games". The pattern identifier ContentType="MOVIE" can be used to indicate that the corresponding content type is "movies". The AI ​​recognition model inside the display device maps the detected "large areas of green grass and scoreboard" scene features to the content type "sporting events".

[0048] Based on the above description, the process by which the display device obtains the video latency data and mode identifier is explained below: When determining the mode identifier based on priority, the display device's implementation logic can be as follows: external signal detection is set to the highest priority. That is, the display device monitors input ports such as HDMI in real time, checking for authoritative status signals, such as ALLM (Auto Low Latency Mode) signals. When an external signal is detected, the mode identifier can be directly determined based on that signal. For example, if the external device is detected as a device used to run a game program, the current mode identifier can be directly considered to be game mode.

[0049] Media metadata parsing is set to the second highest priority. This means that metadata in the currently playing streaming media or broadcast signal is parsed only when there are no strong external instructions. Some content providers embed content type tags (such as Content-Type: movie) in the data stream.

[0050] Image recognition results are set to general priority. If neither of the above two methods yields valid information, the built-in AI visual analysis engine is activated. This engine extracts video frames, analyzes their visual features (such as frame rate, aspect ratio, user interface elements, scene composition, etc.), compares them with a pre-trained model, and outputs the most likely content type.

[0051] Application / input source heuristics are set to the lowest priority. This means the system can make auxiliary judgments based on the currently active application (such as Netflix, YouTube) or custom tags of the input source (such as "Blu-ray player", "PC").

[0052] After determining the method for identifying the mode identifier, the display device can obtain the mode identifier in the following ways: In one exemplary scenario, the display device can bind an interrupt service routine to the HDMI controller, which immediately updates the system state variable to "game mode" when a change in the ALLM flag is detected.

[0053] If there is no interruption, the display device can periodically call functions used for media metadata parsing (such as the parse_stream_metadata() function), and update system state variables if a valid type is returned.

[0054] If neither of the above two methods works, the AI ​​task built into the display device will be triggered, and its output (such as SCENE_SPORTS) will be used to update the system state variables.

[0055] After obtaining the mode identifier, the display device can acquire the video latency data corresponding to the mode identifier in the following way: When system state variables change, the system triggers a callback function (such as the `update_video_latency()` function). Based on this function, a query request is sent to the firmware of the video processing SoC, such as `u32GetVideoPathDelay()`. Internally, the firmware reads the status registers of each hardware module (such as MEMC and Scaler), which store their respective processing delays. The firmware accumulates these values ​​and returns the final total delay value, such as 75 (ms), through the query interface.

[0056] An example application scenario is as follows: A user is watching a science fiction movie on a smart TV. The TV determines the content type as "movie" through image recognition, and the video processing pipeline is fully activated to present the best picture quality. The calculated video latency is 80ms.

[0057] Then, the user picks up the game controller, and the TV automatically switches to the HDMI 2 input connected to the game console. The user then starts a fast-paced racing game. The game console sends an ALLM signal to the TV via the HDMI cable. The TV's control unit immediately captures this signal and forces the mode identifier to "Game," prioritizing it over the previous image recognition results. Upon receiving the "Game" mode identifier, the smart TV's control unit immediately commands the video processing pipeline to disable all unnecessary and time-consuming image processing functions (such as MEMC dynamic compensation). Under the new streamlined processing path, the control unit re-queries and retrieves the video latency data, which has decreased from 80ms to 11ms.

[0058] At this point, the display device has successfully acquired two key pieces of information: the mode identifier is "Game," and the video latency data is "11ms." These two pieces of information will serve as the basis for the next step of constructing control commands and directing external audio devices to make synchronization adjustments.

[0059] 202. Generate the first instruction based on the video delay data and the mode identifier.

[0060] After the display device has successfully acquired the video delay data and the mode identifier, it generates the first instruction based on the video delay data and the mode identifier. The first instruction can be used to instruct the external audio device of the display device to perform corresponding operations.

[0061] It should be understood that the external audio device (such as a soundbar) can be connected to the display device via an HDMI ARC / eARC interface, and the firmware of the external audio device supports parsing and executing the first instruction in this embodiment (such as the defined Custom Consumer Electronics Control (CEC) instruction).

[0062] At the same time, the manufacturers of the display device and the external audio device need to agree in advance on the format of the customized CEC command, such as agreeing on the vendor ID.

[0063] When constructing the first instruction, its execution flow can be as follows: Assume that the instruction conforms to the HDMI-CEC protocol.<Vendor Command> (Manufacturer-specific instructions, opcode 0x89) framework.

[0064] At this point, a complete instruction frame contains a header and a data block.

[0065] Set a message header, which can include the identifiers of the initiator and the target. For example, if the initiator is a television device, its logical address is 0 (TV); and the target is an audio system, its logical address is 5 (Audio System). In this case, the message header byte can be set to 0x05.

[0066] Construct a data block, which can include information such as opcode and vendor identifier. For example, the opcode can be the vendor-specific instruction opcode "0x89"; the vendor identifier (Vendor ID) can be set to a 24-bit (3-byte) IEEE registered vendor ID immediately following the opcode. This is key to distinguishing customized instructions from different brands. For example, suppose the vendor ID is "0x1A2B3C".

[0067] Following the vendor ID, a custom payload field can be used to define custom data for transmitting video latency data and mode identification. In one exemplary scenario, this custom payload field could be designed as follows: Custom function code (1 byte): Used to identify the specific function of this instruction for future expansion. For example, "0x10" can be defined as "audio-video synchronization low-latency control".

[0068] Mode Flag (Flag, 1 byte): Used to enable or disable audio low-latency mode. For example, 0x01 is used to request that audio low-latency mode be enabled; 0x00 is used to request that audio low-latency mode be disabled (restore standard mode). It should be understood that this mode flag can be set one-to-one according to the mode identifier.

[0069] Video latency value (Value, 1 byte): Represents the actual processing latency of the current TV video link, in milliseconds (ms). One byte can represent 0-255ms, which is sufficient to cover most scenarios. For example, if the latency of the TV in game mode is 8ms, then this byte value is 0x08.

[0070] Based on the above description, an exemplary structure of the first instruction can be represented as follows: Assuming the TV (manufacturer ID 0x1A2B3C) enters game mode and the measured video latency is 8ms, the soundbar needs to be notified to enable low-latency mode. The complete first instruction can be constructed as follows (in hexadecimal): "[Header] [Opcode][Vendor ID (3 bytes)] [Custom Opcode][Flag] [Value]0x05, 0x89, 0x1A, 0x2B, 0x3C, 0x10, 0x01, 0x08".

[0071] 203. Pass the first instruction to the external audio device.

[0072] The display device will transmit the first command generated to the external audio device via the HDMI ARC / eARC interface.

[0073] 204. The external audio device determines the audio processing delay strategy based on the first instruction.

[0074] After receiving the first instruction, the external audio device parses the first instruction to obtain the video delay data and the mode identifier in the first instruction; finally, it determines the audio processing delay strategy based on the video delay data and the mode identifier.

[0075] It should be understood that this delay processing strategy refers to a methodology for achieving dynamic and precise timing compensation in external audio devices. The core objective of this strategy is to proactively and intelligently align the output timing of the audio stream with the dynamically changing presentation timing of the video stream, determined by the display device. This eliminates audio-visual asynchrony under any content mode and processing load, ensuring a high degree of consistency and immersion in the end-user's sensory experience. Based on this description, the delay processing strategy may include at least one of the following: adjusting the buffer length of the audio data; disabling audio effects processing; determining a delay compensation value based on the video delay data and the audio processing data of the external audio device; and adjusting the delay based on the delay compensation value.

[0076] The following describes the processing of the external audio device using the first instruction constructed in step 202: First, the external audio device checks the target address in the message header of the first instruction. Finding the target address to be 0x05 (its own address), it continues processing the message.

[0077] The processor of the external audio device reads the data block and begins parsing: Operation code identification: Operation code 0x89 was detected, confirming it as a manufacturer-specific instruction.

[0078] Verify Vendor ID: Check if the next 3 bytes are the pre-agreed vendor ID (0x1A2B3C).

[0079] If the ID matches, it means that this is a pre-agreed instruction from the manufacturer, so continue parsing.

[0080] If the ID does not match, this instruction will be ignored to prevent accidental operation.

[0081] Analyze custom load: The custom function code read is 0x10, which is confirmed to be the "Audio / Video Synchronization Low Latency Control" instruction.

[0082] The read mode flag is 0x01, indicating that low-latency audio mode needs to be enabled.

[0083] The video latency value is 0x08, indicating that the video latency on the TV is 8 milliseconds.

[0084] This external audio device is based on performing audio mode switching and synchronization.

[0085] After the external audio device successfully parses the command, its digital signal processor (DSP) or main controller immediately executes the corresponding operation: Switch audio processing mode (based on mode flag): Based on the received 0x01 flag, the external audio device immediately switches to the preset "low latency audio configuration".

[0086] This configuration will bypass or disable time-consuming audio processing algorithms, such as complex virtual surround sound or sound field expansion algorithms, room acoustic correction based on microphone measurements, multi-band graphic equalizers, etc.

[0087] Then the size of the internal processing buffer is reduced (i.e., the buffer length of the audio data is adjusted) to sacrifice some stability for ultimate speed.

[0088] It can also perform precise audio-visual synchronization (based on value) based on the video latency data. That is, the external audio device uses the received video latency value to perform precise audio-visual synchronization (Lip-Sync) compensation. For example, the external audio device will apply a small delay of 8ms to its own audio processing link. This ensures that even after both devices enter low-latency mode, the sound and picture remain perfectly aligned, avoiding the "negative latency" phenomenon where the sound is faster than the picture.

[0089] The implementation process can be as follows: Stage 1, enter low-latency audio mode; Stage 2, perform dynamic audio-visual synchronization.

[0090] Mode flag: 0x01 (enabled); Video link latency value (T_video): For example: 8ms.

[0091] Phase 1: Enter low-latency audio mode.

[0092] The goal of this phase is to rapidly reduce the inherent latency of the audio processing chain to its physical limits. This operation is accomplished by the audio system's firmware, which controls its internal digital signal processor (DSP) and data paths.

[0093] First, the external audio device dynamically adjusts the audio processing buffer.

[0094] The audio firmware first locates the main audio input buffer. The main function of this buffer is to absorb clock jitter in the data stream from the upstream (TV eARC transmitter) and prevent audio underflow, i.e., popping or muting, caused by temporary data interruption.

[0095] Calculating the new size: In normal mode, to ensure absolute stability, the buffer might be set to a large value, such as 512 samples (approximately 10.7ms latency at a 48kHz sampling rate). Upon receiving a low-latency command, the firmware will modify the target size of this buffer to an aggressive minimum value based on the preset "Game Mode" configuration file. Example: Dynamically adjust the buffer size from 512 samples to 128 samples.

[0096] Adjustment: The firmware writes new register values ​​to the hardware controller responsible for managing direct memory access or the audio interface to change the buffer watermark, or total size. The adjusted 128-sample buffer introduces only about 2.7ms of latency at a 48kHz sampling rate, significantly reducing buffer latency.

[0097] Then, the external audio device reconfigures the DSP processing pipeline.

[0098] Loading Low Latency Configuration: The audio firmware sends a command to the DSP to load a predefined "Low Latency Processing Graph".

[0099] Bypass Time-Consuming Algorithm: This configuration bypasses typical time-consuming DSP modules via a software switch or hardware multiplexer. These time-consuming DSP modules include: Spatial audio processing: Turn off algorithms such as Virtual Surround and Dolby Atmos Height Virtualization. Or switch to the simplest mode.

[0100] Room Correction: Disables equalization algorithms based on finite impulse response or infinite impulse response filters.

[0101] Dynamic Range Compression (DRC) and Night Mode: Turn off these modules that require pre-analysis of the signal.

[0102] Multi-band Graphic Equalizer (Graphic EQ): Bypasses complex EQ curves defined by the user.

[0103] Confirm Minimum Latency: After completing the above steps, the processing latency of the external audio device itself is minimized. This inherent minimum latency value, T_audio_min, is a known design parameter (e.g., measured to be 4ms), and it encompasses the entire time from HDMI receiver decoding, audio interface transmission, minimal buffering, to minimal DSP processing, and finally to D / A conversion and amplification.

[0104] Phase Two: Dynamic Audio-Visual Synchronization. This involves delay compensation when the audio latency has already reached its minimum in the firmware, in order to achieve accurate audio-visual synchronization.

[0105] First, the external audio device calculates the target synchronization delay. That is, the external audio device performs a logical judgment based on the read video delay data and the minimum delay determined in Phase 1, and calculates the delay compensation value. The logical judgment can be as follows: When the video delay is greater than or equal to the minimum audio delay, additional delay compensation is calculated to match the total audio delay with the video delay. In this case, the delay compensation value is equal to the difference between the video delay data and the minimum audio delay. For example, if the video delay is 8ms and the minimum audio delay is 4ms, the delay compensation value is 8-4=4ms.

[0106] When the video latency is less than the minimum audio latency, perfect synchronization is impossible because the audio cannot be faster than the minimum audio latency. Since the goal of audio latency processing is to minimize latency as much as possible, no additional latency is added.

[0107] After calculating the delay compensation value, it can be applied to achieve synchronization. The external audio device can write the calculated delay compensation value to a processing module in the DSP (e.g., an audio-visual synchronization delay module). This processing module can be configured as a small, precisely controllable first-in-first-out buffer. When the delay compensation value is 4ms, the DSP is instructed to add a 4ms delay.

[0108] Optionally, when the display device returns to standard mode, the display device may generate a second instruction to instruct the external audio device to return to standard audio processing mode. That is, the second instruction is generated, which includes indication information for instructing the display device to return to standard mode; the second instruction is sent to the external audio device so that the external audio device responds to the second instruction and returns to standard audio processing mode.

[0109] The following example illustrates this. When the game ends and the TV exits game mode, the TV will reconstruct and send a second instruction similar to the first instruction, wherein the mode flag bit of the second instruction is set to 0x00.

[0110] Example instruction: 0x05, 0x89, 0x1A, 0x2B, 0x3C, 0x10, 0x00, 0xXX (the delay value 0xXX can be the default value or the delay value in standard TV mode).

[0111] Upon receiving this instruction, the soundbar will perform the opposite operation: re-enable all advanced audio processing functions, restore the normal buffering strategy and default audio-visual synchronization settings, thereby returning to high-quality mode.

[0112] That is, when the instruction to "exit low-latency mode" (Flag=0x00) is received, the entire process will be executed in reverse: The delay compensation value has been reset to zero.

[0113] The DSP reloads the regular high-quality processing diagram, and all bypassed modules are re-enabled.

[0114] The size of the audio input buffer was restored to a safe, normal value (e.g., 512 samples).

[0115] This allows the display device to smoothly switch from low-latency gaming mode back to high-quality audio-visual mode.

[0116] Optionally, to ensure precise synchronization between the lighting effect and the screen image, this embodiment can also automatically match different immersive ambient lights for different playback modes based on mode identifiers, realizing intelligent switching of lighting strategies. The first instruction also includes lighting control information. Under this scheme, the display device will also send the first instruction to the intelligent lighting device, enabling the intelligent lighting device to control the lighting based on the lighting control information and the video delay data.

[0117] In an exemplary application scenario, after determining the content type (such as movie mode) based on a mode identifier, the TV's control unit generates corresponding lighting control information. This information defines a lighting scene (e.g., parameters such as color, brightness, and dynamic effects) that matches the content type. Subsequently, the control unit encapsulates this lighting control information along with acquired video latency data (e.g., 80 milliseconds) in a first instruction and sends this instruction to the smart lighting devices throughout the house via its built-in wireless communication module (such as Wi-Fi, Zigbee, or Bluetooth). The microcontroller unit (MCU) of the smart lighting device receives and parses the instruction, extracting the lighting control information and the video latency data. The MCU does not immediately execute the lighting control information; instead, it uses the video latency data (80 milliseconds) as a scheduling parameter to start an internal timer. Only after the timer expires does the MCU trigger its lighting drive circuit to execute the lighting changes defined by the lighting control information. This ensures that the timing of changes in ambient light effects is precisely aligned with the timing of the corresponding frame displayed on the screen, achieving high-precision visual-light synchronization.

[0118] Optionally, to ensure precise synchronization between haptic feedback and the screen display, and based on mode identifiers, different haptic feedback operations can be automatically matched for different modes, achieving intelligent switching of haptic feedback strategies. The first instruction also includes haptic control information. In this scheme, the display device will also send the first instruction to the haptic feedback device, enabling the haptic feedback device to perform haptic control based on the haptic control information and the video delay data.

[0119] In an exemplary application scenario, after recognizing a preset tactile event (such as an explosion scene) in the playback content, the TV's control unit generates corresponding tactile control information. This information defines the waveform, intensity, and duration of the tactile feedback. Subsequently, the control unit encapsulates this tactile control information along with acquired video latency data (e.g., 65 milliseconds) in a first instruction and sends it to the tactile feedback device (such as a tactile vest or gaming chair) via a low-latency wireless communication protocol (such as Bluetooth Low Energy). Upon receiving and parsing the instruction, the device's microcontroller unit (MCU) does not immediately trigger tactile feedback. Instead, it uses the video latency data (65 milliseconds) as a scheduling parameter for delayed execution, starting an internal high-precision timer. Only after the timer expires does the MCU output a drive signal to its internal tactile actuator (such as a linear resonant actuator) based on the pre-stored tactile control information, producing a precisely synchronized vibration effect. This process ensures perfect temporal synchronization between the user's tactile sensation and the visual event on the screen, greatly enhancing the realism of the immersive experience.

[0120] Based on the above description, the following will be used as... Figure 3 The schematic diagram shown illustrates the data processing method of this application.

[0121] like Figure 3 The diagram illustrates the collaborative workflow between audio / video and HDMI devices in low-latency gaming mode. The video service proactively collects video latency data from the underlying video hardware abstraction layer and video driver, and sends this data back to the upper-layer audio server to provide accurate timing reference for the low-latency strategy of the audio service and ensure audio-video synchronization.

[0122] Upon receiving the trigger signal for the game's low-latency mode, the audio service sends the "game low-latency status" and "video latency data" to the HDMI service via the CEC command, and simultaneously sends audio policy instructions to the audio policy manager. The audio policy manager forwards the corresponding low-latency audio policy to the audio hardware abstraction layer. The audio hardware abstraction layer further passes the instructions to the audio driver, which controls the audio DSP processor to enable low-latency audio processing logic. The driver also manages the device connection status to ensure the audio output link is in a low-latency ready state.

[0123] During the transmission of CEC commands, after receiving the CEC command from the audio service, the HDMI service passes the command to the HDMI driver through the HDMI Hardware Abstraction Layer. The HDMI driver then sends the low-latency configuration information to the external soundbar via the HDMI CEC protocol. After the soundbar completes the low-latency mode configuration, it sends a "game low-latency mode is enabled" status back to the HDMI driver via the CEC protocol. This status is then relayed upstream through the HDMI Hardware Abstraction Layer and the HDMI service, completing the entire status confirmation loop.

[0124] The entire process revolves around a low-latency experience. Through end-to-end collaboration between the Java framework layer, native framework layer, hardware abstraction layer and kernel driver, combined with the HDMI CEC protocol, it enables interaction with external devices, ultimately allowing audio, video and external speakers to enter low-latency mode simultaneously, meeting the smooth interaction requirements in gaming scenarios.

[0125] To better implement the data processing method in the embodiments of this application, based on the data processing method, the embodiments of this application also provide a data processing apparatus, such as... Figure 4 As shown, the data processing device 400 includes: The acquisition module 401 is used to acquire video delay data and a mode identifier, the mode identifier being used to indicate the content type corresponding to the playback content of the display device; Processing module 402 is used to generate a first instruction based on the video delay data and the mode identifier; The transceiver module 403 is used to transmit the first instruction to the external audio device so that the external audio device can determine the audio processing delay strategy based on the first instruction.

[0126] In this embodiment, the display device acquires precise delay data and mode identifiers of its internal video path in real time, and constructs control commands based on this data to send to the external audio device. This enables the audio device to dynamically and precisely adjust its audio output delay, thereby achieving adaptive synchronization between the video and audio streams. This solves the core pain point of audio-visual synchronization issues in different application scenarios, avoids tedious manual settings, and significantly improves the automation level of the entire audio-visual system and the user's immersive experience.

[0127] In some embodiments of this application, the acquisition module 401 is specifically used for: Read the log file to obtain the video latency data, which describes the internal processing latency of the video playback pipeline; Detect the automatic low-latency mode signal or the mode enable signal to obtain the mode identifier.

[0128] In this embodiment, log parsing, a non-intrusive method, is employed to accurately quantify the actual latency of each processing module (such as PQ quality enhancement) in the video pipeline with high compatibility and low cost, without modifying existing hardware or core firmware. Secondly, authoritative status indicators from the signal source ensure the immediacy and absolute reliability of low-latency mode switching, avoiding potential misjudgments in image recognition or user forgetting to manually set parameters. This efficient, reliable, and easily deployable approach provides precise and dynamic data input and clear scene triggers for subsequent audio-visual synchronization control, significantly improving the automation level of the entire audio-visual system and the final user experience.

[0129] In some embodiments of this application, the acquisition module 401 is specifically used for: Read the log file to obtain the video latency data, which describes the internal processing latency of the video playback pipeline; The content type of the content currently being played on the display device is detected to obtain the mode identifier.

[0130] In this embodiment, log parsing, a non-intrusive method, is employed to accurately quantify the actual latency of each processing module (such as PQ quality enhancement) in the video pipeline with high compatibility and low cost, without modifying existing hardware or core firmware. Secondly, the display device intelligently identifies the content type and sends commands to the external audio device accordingly to automatically switch to the preset optimal audio processing mode. This provides precise and dynamic data input and clear scene triggers for subsequent audio-visual synchronization control in an efficient, reliable, and easily deployable manner, thereby significantly improving the automation level of the entire audio-visual system and the final user experience.

[0131] In some embodiments of this application, the first instruction further includes lighting control information; The transceiver module 403 is used to send the first instruction to the smart lighting device so that the smart lighting device responds to the first instruction to control the lighting.

[0132] In this embodiment, precise video latency data within the display device is used to perform timing correction on lighting commands, ensuring accurate synchronization between the lighting effect and the screen image. Simultaneously, based on mode identifiers, different immersive ambient lights can be automatically matched to different playback modes, achieving intelligent switching of lighting strategies. This solution extends the audiovisual experience to the physical environment, achieving a high degree of unity between multiple senses in time and context, greatly enhancing the user's immersion.

[0133] In some embodiments of this application, the first instruction further includes tactile control information; The transceiver module 403 is used to send the first instruction to the haptic feedback device so that the haptic feedback device responds to the first instruction to perform haptic control.

[0134] In this embodiment, precise video latency data within the display device is used to perform timing correction on the haptic feedback control, ensuring accurate synchronization between the haptic feedback and the screen display. Furthermore, based on mode identifiers, different haptic feedback operations can be automatically matched to different modes, achieving intelligent switching of haptic feedback strategies. This solution extends the audiovisual experience to the physical environment, achieving a high degree of temporal and contextual unity across multiple senses, greatly enhancing user immersion.

[0135] In some embodiments of this application, the processing module 402 is configured to generate a second instruction, the second instruction including instruction information for instructing the display device to restore the standard mode; The transceiver module 403 is used to send the second instruction to the external audio device so that the external audio device responds to the second instruction and resumes the standard audio processing mode.

[0136] In this embodiment, by actively sending a recovery command, automatic and bidirectional synchronization between the audio processing mode and the video content mode is achieved. This ensures that when users switch between different application scenarios, they can seamlessly switch from game sound effects that pursue ultimate response speed to cinema-grade sound effects that pursue immersion and high fidelity. This constitutes a complete automated control closed loop, which greatly improves the intelligence level of the system and the consistency of the user experience, and avoids the need for any manual intervention.

[0137] In some embodiments of this application, the delayed processing strategy includes at least one of the following: Adjust the buffer length for audio data; Turn off sound effects processing; The delay compensation value is determined based on the video delay data and the audio processing data of the external audio device, and the delay is adjusted based on the delay compensation value.

[0138] In this application embodiment, a variety of delay processing strategies are provided to achieve precise adjustment of the audio delay processing.

[0139] This application also provides a computer device that integrates any of the data processing apparatuses provided in this application, the computer device comprising: One or more processors; Memory; and One or more applications, wherein the applications are stored in memory and configured to be executed by a processor from the steps of the data processing method in any of the embodiments described above.

[0140] This application also provides a computer device that integrates any of the data processing apparatuses provided in this application. For example... Figure 5 As shown, it illustrates a structural schematic diagram of the computer device involved in the embodiments of this application, specifically: The computer device may include components such as a processor 501 with one or more processing cores, a memory 502 with one or more computer-readable storage media, a power supply 503, and an input unit 504. Those skilled in the art will understand that... Figure 5 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 501 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 502, and by calling data stored in the memory 502, it performs various functions of the computer device and processes data, thereby providing overall monitoring of the computer device. Optionally, the processor 501 may include one or more processing cores; preferably, the processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 501.

[0141] The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.

[0142] The computer equipment also includes a power supply 503 that supplies power to the various components. Preferably, the power supply 503 can be logically connected to the processor 501 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 503 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0143] The computer device may also include an input unit 504, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0144] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 501 in the computer device loads the executable files corresponding to the processes of one or more application programs into the memory 502 according to the following instructions, and the processor 501 runs the application programs stored in the memory 502 to realize various functions, as follows: Acquire video latency data and a mode identifier, which indicates the content type corresponding to the playback content on the display device; A first instruction is generated based on the video delay data and the pattern identifier; The first instruction is passed to the external audio device so that the external audio device can determine the audio processing delay strategy based on the first instruction.

[0145] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0146] Therefore, embodiments of this application provide a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any of the data processing methods provided in embodiments of this application. For example, the computer program loaded by the processor can execute the following steps: Acquire video latency data and a mode identifier, which indicates the content type corresponding to the playback content on the display device; A first instruction is generated based on the video delay data and the pattern identifier; The first instruction is passed to the external audio device so that the external audio device can determine the audio processing delay strategy based on the first instruction.

[0147] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed descriptions of other embodiments above, which will not be repeated here.

[0148] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.

[0149] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0150] The data processing method, apparatus, computer device, and computer-readable storage medium provided in the embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A data processing method, characterized in that, include: Acquire video latency data and a mode identifier, wherein the mode identifier is used to indicate the content type corresponding to the playback content of the display device; A first instruction is generated based on the video delay data and the pattern identifier; The first instruction is passed to the external audio device so that the external audio device determines a delay processing strategy for audio processing based on the first instruction.

2. The method according to claim 1, characterized in that, Obtaining video latency data and mode identifiers includes: Read the log file to obtain the video latency data, which is used to describe the internal processing latency of the video playback pipeline; The automatic low-latency mode signal or the mode enable signal is detected to obtain the mode identifier.

3. The method according to claim 1, characterized in that, The acquisition of video delay data and mode identifier includes: Read the log file to obtain the video latency data, which is used to describe the internal processing latency of the video playback pipeline; The content type of the content currently being played on the display device is detected to obtain the mode identifier.

4. The method according to claim 1, characterized in that, The first instruction also includes lighting control information; The method further includes: The first instruction is sent to the smart lighting device so that the smart lighting device responds to the first instruction to control the lighting.

5. The method according to claim 1, characterized in that, The first instruction also includes tactile control information; The method further includes: The first instruction is sent to the haptic feedback device so that the haptic feedback device responds to the first instruction to perform haptic control.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Generate a second instruction, the second instruction including instruction information for instructing the display device to restore the standard mode; The second instruction is sent to the external audio device so that the external audio device responds to the second instruction and resumes the standard audio processing mode.

7. The method according to any one of claims 1 to 5, characterized in that, The delay processing strategy includes at least one of the following: Adjust the buffer length for audio data; Turn off sound effects processing; A delay compensation value is determined based on the video delay data and the audio processing data of the external audio device, and delay adjustment is performed based on the delay compensation value.

8. A data processing apparatus, characterized in that, include: The acquisition module is used to acquire video delay data and a mode identifier, wherein the mode identifier is used to indicate the content type corresponding to the playback content of the display device. The processing module is used to generate a first instruction based on the video delay data and the mode identifier; The transceiver module is used to transmit the first instruction to an external audio device, so that the external audio device determines a delay processing strategy for audio processing based on the first instruction.

9. A computer device, characterized in that, The computer device includes: One or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program, which is loaded by a processor to perform the steps of the method according to any one of claims 1 to 7.

Images