Display device and multi-path screenshot method

Abstract

The application discloses a display device and a multi-path screenshot method. The display device is configured with at least two physical hardware screenshot resources and is respectively bound to a first logical path and a second logical path, which are used to collect data of different display layers. In response to a screenshot request sent by a first screenshot session, the display device subscribes to a target logical path in the first logical path and the second logical path for the first screenshot session according to a display layer acted on by the screenshot request. If the target logical path is subscribed by at least one second screenshot session, request parameters of the first screenshot session and the second screenshot session are aggregated to generate collection parameters. Then, based on a hardware screenshot resource corresponding to the target logical path, screenshot data is collected according to the collection parameters. The screenshot data is processed for the screenshot request and is distributed to the first screenshot session. The application can break through the limitation of the hardware screenshot resource, improve the screenshot concurrency capability of the display device and the resource utilization rate.

Description

Technical Field

[0001] This application relates to the field of display device technology, and in particular to a display device and a method for multi-channel screenshotting. Background Technology

[0002] Display devices are intelligent devices capable of presenting a user interface and supporting user interaction. Taking smart TVs as an example, smart TVs are television products based on Internet application technology, equipped with open operating systems and chips, possessing open application platforms, enabling two-way human-computer interaction, and integrating multiple functions such as audio-visual, entertainment, and data to meet diverse and personalized user needs.

[0003] When performing the screenshot function, the hardware screenshot resources configured on the display device are allocated in an exclusive manner. Each screenshot request in each screenshot session occupies a separate physical hardware screenshot path. There is a one-to-one binding relationship between the hardware screenshot resources and the screenshot session. Each screenshot session independently occupies hardware resources for independent acquisition and data processing.

[0004] However, the exclusive allocation method of the above-mentioned hardware screenshot resources, due to the limited number of physical hardware paths, cannot simultaneously support multiple concurrent screenshot services with multiple parameters under the condition of a small number of hardware screenshot resources (i.e., it cannot handle multiple screenshot sessions with different parameter requirements at the same time), which leads to a decrease in the screenshot capability of the display device and affects the user experience. Summary of the Invention

[0005] This application provides a display device and a multi-channel screenshot method to solve the problem of low screenshot capability in display devices due to hardware limitations.

[0006] In a first aspect, this application provides a display device, comprising: The monitor is configured to display the user interface; The controller is configured with at least two physical hardware screenshot resources, and binds one physical hardware screenshot resource as a first logical path and the other physical hardware screenshot resource as a second logical path; the first logical path and the second logical path are used to collect data from different display layers; The controller is configured to: In response to a screenshot request sent by a first screenshot session, a target logical path is subscribed to and adapted for the first screenshot session according to the display layer on which the screenshot request is applied. The target logical path is at least one of the first logical path and the second logical path. If the target logical path is subscribed to by at least one second screenshot session, the request parameters corresponding to the first screenshot session and the second screenshot session are aggregated to generate the collection parameters of the target logical path; Based on the hardware screenshot resources corresponding to the target logical path, screenshot data is collected according to the collection parameters; The screenshot data is processed to adapt to the screenshot request, and the processed screenshot data is distributed to the first screenshot session.

[0007] The above technical solution has the following beneficial effects or advantages: The limited physical hardware screenshot resources are abstracted into two globally shared logical paths. For the newly added first screenshot session, the processing logic of dynamic subscription, unified collection of multiple session requests, and on-demand adaptive distribution is implemented. This breaks the limitation of one-to-one exclusive use of hardware screenshot resources. Multiple screenshot sessions can run concurrently by relying on two fixed physical hardware resources, thereby improving the device's screenshot concurrency capability and hardware resource utilization.

[0008] In some embodiments of this application, the acquisition parameters include acquisition resolution, acquisition frame rate, and acquisition data format; the controller executes the aggregation of request parameters corresponding to the first screenshot session and the second screenshot session to generate the acquisition parameters of the target logical path, specifically configured as follows: Obtain the request parameters of the first screenshot session and the request parameters of the second screenshot session, wherein the request parameters include the output resolution and the screenshot frame rate; The maximum width and maximum height of the output resolution are used as the acquisition resolution of the target logical path, and the maximum screenshot frame rate is used as the acquisition frame rate of the target logical path. Set the data acquisition format of the target logical path to a preset intermediate format.

[0009] The above technical solution has the following beneficial effects or advantages: By aggregating the resolution and frame rate parameters of all subscription sessions and taking the maximum value as the unified hardware acquisition specification, and at the same time fixing the standard intermediate format for acquisition, it can ensure that the acquired raw screenshot data can meet the highest requirements of all concurrent screenshot sessions, and can also improve the problem of frequent hardware reconfiguration caused by the difference in parameters between multiple sessions.

[0010] In some embodiments of this application, the display layer of the user interface includes a first layer and a second layer, the screenshot request carries a screenshot type identifier, and the screenshot type identifier includes at least one of a first layer identifier, a second layer identifier, and a composite identifier; the controller executes the target logical path adapted to the first screenshot session by subscribing to the display layer affected by the screenshot request, specifically configured as follows: In response to the screenshot request carrying the first layer identifier, the first screenshot session is subscribed to the first logical path; In response to the screenshot request carrying the second layer identifier, the first screenshot session is subscribed to the second logical path; In response to the screenshot request carrying the composite identifier, the first screenshot session is simultaneously subscribed to the first logical path and the second logical path.

[0011] The above technical solution has the following beneficial effects or advantages: Based on the screenshot type identifier, different layer acquisition requirements are matched to realize differentiated channel subscription for single-layer screenshots and dual-layer composite screenshots. It can adapt to the business needs of different screenshot sessions, avoid invalid layer acquisition and redundant data calculation, and improve the reusability accuracy and processing efficiency of hardware channels.

[0012] In some embodiments of this application, the controller performs processing to adapt the screenshot data to the screenshot request, specifically configured as follows: Based on the target resolution requested in the screenshot request, the screenshot data is scaled to obtain intermediate data; If the screenshot request carries the synthesis identifier, the intermediate data corresponding to the first logical path and the intermediate data corresponding to the second logical path are fused together; then, according to the target data format requested by the screenshot request, the fused intermediate data is converted to a new format to complete the adaptation to the screenshot request. If the screenshot request does not carry the composite identifier, the intermediate data is converted according to the target data format requested in the screenshot request to complete the adaptation to the screenshot request.

[0013] The above technical solution has the following beneficial effects or advantages: For single-layer and composite layer screenshot requirements, corresponding fusion and conversion logics are set up to unify the processing paradigm of multiple types of screenshot requests, ensuring that screenshot requests from different businesses can be adapted, thereby improving the system's processing stability and versatility.

[0014] In some embodiments of this application, the controller performs scaling processing on the screenshot data, specifically configured as follows: If the target resolution is consistent with the acquisition parameters of the target logical path, the resolution of the screenshot data is maintained and used as the intermediate data; If the target resolution is inconsistent with the acquisition parameters of the target logical path, the resolution of the screenshot data is scaled to the target resolution and used as the intermediate data.

[0015] The above technical solution has the following beneficial effects or advantages: The processing mechanism adopts an on-demand scaling mechanism, which only performs scaling calculations when there is a difference between the target resolution of the session and the hardware acquisition resolution. When the parameters are consistent, the original data is directly reused, reducing redundant calculations, reducing the computing power consumption of the display device, and thus improving the screenshot processing speed.

[0016] In some embodiments of this application, the controller performs format conversion processing on the intermediate data after fusion processing, or performs format conversion processing on the intermediate data, specifically configured as follows: If the target data format is consistent with the acquisition parameters of the target logical path, the data format of the intermediate data is retained; If the target data format is inconsistent with the acquisition parameters of the target logical path, the data format of the intermediate data will be converted to the target data format.

[0017] The above technical solution has the following beneficial effects or advantages: Based on format matching, on-demand format conversion is implemented, avoiding meaningless format conversion operations, further reducing the computational overhead of the screenshot adaptation process, and shortening the screenshot response latency.

[0018] In some embodiments of this application, the controller is further configured to: Maintain a corresponding frame rate counter for the first screenshot session; When the target logic path outputs a frame of screenshot data, the frame rate counter of the first screenshot session is incremented; When the controller distributes the processed screenshot data to the first screenshot session, it is also configured to: When the frame rate counter reaches the preset frame rate threshold of the first screenshot session, a frame of screenshot data collected by the target logical path is distributed to the first screenshot session.

[0019] The above technical solution has the following beneficial effects or advantages: Configure an independent frame rate counting mechanism for each screenshot session. By unifying the hardware frame rate, different frame rate distribution effects can be achieved for different sessions. This eliminates the need for repeated hardware acquisition to meet the different frame rate requirements of multiple sessions, thereby maximizing the reuse of hardware acquisition capabilities.

[0020] In some embodiments of this application, the controller performs processing to adapt the screenshot data to the screenshot request and distributes the processed screenshot data to the first screenshot session, specifically configured as follows: If the target resolution and target data format requested in the screenshot request are consistent with the acquisition parameters of the target logical path, and the number of the target logical path is one, the screenshot data acquired by the target logical path is written to the shared memory pool through direct memory access. The memory address is passed to the first screenshot session based on the shared memory pool in order to distribute the screenshot data to the first screenshot session.

[0021] The above technical solution has the following beneficial effects or advantages: In single-path screenshot scenarios with parameter matching, the zero-copy method of direct memory access writing and shared memory address distribution can avoid multiple data copies, reduce data transmission latency, and thus improve screenshot efficiency.

[0022] In some embodiments of this application, the controller is further configured to: Maintain the subscription reference count for the target logical path; In response to the subscription reference count changing from a first value to a second value, the hardware screenshot resource corresponding to the target logical path is activated to begin collection; In response to the subscription reference count changing from the second value to the first value, the hardware screenshot resource corresponding to the target logical path is closed to stop the collection.

[0023] The above technical solution has the following beneficial effects or advantages: By dynamically managing the start and stop status of hardware screenshot resources through reference counting, the system can start capturing data when there is a subscription demand and shut it down when there is no subscription demand. This avoids long-term idle operation of the hardware, reduces the overall power consumption of the device, and improves the rationality of resource scheduling.

[0024] Secondly, this application also provides a multi-channel screenshot method applied to a display device, wherein the display device is configured with at least two physical hardware screenshot resources, and one physical hardware screenshot resource is bound as a first logical path, and the other physical hardware screenshot resource is bound as a second logical path; the first logical path and the second logical path are used to collect data from different display layers; the method includes: In response to a screenshot request sent by a first screenshot session, a target logical path is subscribed to and adapted for the first screenshot session according to the display layer on which the screenshot request is applied. The target logical path is at least one of the first logical path and the second logical path. If the target logical path is subscribed to by at least one second screenshot session, the request parameters corresponding to the first screenshot session and the second screenshot session are aggregated to generate the collection parameters of the target logical path; Based on the hardware screenshot resources corresponding to the target logical path, screenshot data is collected according to the collection parameters; The screenshot data is processed to adapt to the screenshot request, and the processed screenshot data is distributed to the first screenshot session.

[0025] The above technical solution has the following beneficial effects or advantages: The method abstracts limited physical hardware screenshot resources into two globally shared logical paths. For the newly added first screenshot session, it implements processing logic such as dynamic subscription, unified collection of multiple session requests, and on-demand adaptive distribution. This breaks the limitation of one-to-one exclusive use of hardware screenshot resources and enables the concurrent operation of multiple screenshot sessions with limited hardware resources, thereby improving the device's screenshot concurrency capability and hardware resource utilization. Attached Figure Description

[0026] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram illustrating the operation scenarios of a display device provided in some embodiments of this application; Figure 2 This is a schematic diagram of the hardware configuration of a display device provided in some embodiments of this application; Figure 3 This is a schematic diagram of the software configuration of a display device provided in some embodiments of this application; Figure 4 A flowchart illustrating a multi-channel screenshot method provided in some embodiments of this application; Figure 5 A flowchart illustrating the target logical path for subscription adaptation provided in some embodiments of this application; Figure 6 This is a schematic diagram illustrating the process of generating acquisition parameters provided in some embodiments of this application; Figure 7 A schematic diagram illustrating the screenshot data adaptation process provided in some embodiments of this application; Figure 8 This is a schematic flowchart of the scaling process provided in some embodiments of this application; Figure 9 A schematic diagram illustrating the format conversion process provided in some embodiments of this application; Figure 10 A schematic diagram of the subscription citation counting process provided for some embodiments of this application; Figure 11 A schematic diagram of the architecture of multi-channel screenshots provided in some embodiments of this application; Figure 12 Interactive diagrams of multi-channel screenshot methods provided in some embodiments of this application. Detailed Implementation

[0028] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0029] In this application embodiment, display device 200 generally refers to a device with screen display and data processing capabilities. For example, display device 200 includes, but is not limited to, smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, etc.

[0030] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device provided in some embodiments of this application. For example... Figure 1 As shown, users can operate the display device 200 via touch operation, mobile terminal 300, and control device 100. For example, control device 100 can be a remote control, stylus, gamepad, etc.

[0031] The mobile terminal 300 can function as a control device for human-computer interaction between the user and the display device 200. It can also function as a communication device for establishing a communication connection with the display device 200 and exchanging data. In some embodiments, the mobile terminal 300 can have software applications installed on it and communicate with the display device 200 via network communication protocols to achieve one-to-one control and data communication. Furthermore, it can transmit audio and video content displayed on the mobile terminal 300 to the display device 200 for synchronized display.

[0032] like Figure 1 The diagram also shows that the display device 200 communicates with the server 400 via various communication methods. This allows the display device 200 to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0033] Display device 200 can provide broadcast television reception function, and can also be equipped with intelligent network television function that provides computer support, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0034] Figure 2 Provided for some embodiments of this application Figure 1 Hardware configuration block diagram of display device 200.

[0035] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface 280.

[0036] In some embodiments, detector 230 is used to collect signals from the external environment or signals interacting with the outside world. For example, detector 230 may include millimeter-wave radar, which can be used to detect whether a user is present within a preset range. Detector 230 may also include a voice acquisition unit to collect voice commands input by the user.

[0037] In some embodiments, the display 260 includes display function components for presenting images and driving components for driving image display. The display 260 is used to receive and display image signals output from the controller 250. For example, the display 260 can be used to display video content, image content, menu control interface components, and user control UI interfaces, etc.

[0038] In some embodiments, the communication device 220 is a component used to communicate with external devices or the server 400 according to various communication protocol types. The display device 200 may have multiple communication devices 220 depending on the supported communication methods. For example, when the display device 200 supports wireless network communication, it may have a communication device 220 with WiFi functionality. When the display device 200 supports Bluetooth connectivity, it needs to have a communication device 220 with Bluetooth functionality.

[0039] The communication device 220 enables the display device 200 to communicate with external devices or the server 400 via wireless or wired connections. Wired connections utilize data cables, interfaces, or other components to connect the display device 200 to external devices. Wireless connections utilize wireless signals or wireless networks. The display device 200 can directly establish a connection with external devices or indirectly through gateways, routers, or other connection devices.

[0040] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, a power processor, and a hardware screen capture unit, and a first to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display device 200.

[0041] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0042] In some embodiments, a user can input user commands through a graphical user interface (GUI) displayed on a display 260, and the user input interface receives user input commands through the graphical user interface (GUI).

[0043] In some embodiments, the audio output device 270 can be a built-in speaker of the display device 200 or an external audio output device connected to the display device 200. For the external audio output device connected to the display device 200, the display device 200 may also be provided with an external audio output terminal, through which the audio output device can be connected to the display device 200 to output sound from the display device 200.

[0044] In some embodiments, the user input interface 280 can be used to receive instructions input by a user. The user input interface 280 may include at least one of a microphone, touchpad, sensor, remote control, etc. The display device 200 can then receive user-input instructions based on the user input interface 280 to perform interactive functions with the user.

[0045] In some embodiments, to enable user interaction, the display device 200 may run an operating system. An operating system is a computer program that manages and controls the hardware and software resources of the display device 200. The operating system can control the display device to provide a user interface; for example, the operating system can directly control the display device to provide a user interface, or it can provide a user interface by running applications. The operating system also allows users to interact with the display device 200.

[0046] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system that is deeply customized based on a specific operating platform, or an independent operating system specifically developed for display devices.

[0047] An operating system can be divided into different modules or levels based on the functions it implements, for example... Figure 3 As shown, in some embodiments, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the System Library layer, and the Kernel layer.

[0048] In some embodiments, the application layer provides services and interfaces for applications, enabling the display device 200 to run applications and interact with the user based on the applications. The application layer may contain at least one application, which may be a built-in Windows program, system settings program, or clock program of the operating system; or it may be an application developed by a third-party developer. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0049] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0050] like Figure 3 As shown, the application framework layer in this embodiment includes a View System, Managers, Content Providers, and a Screen Capture Manager. The View System designs and implements the application's interface and interactions, and includes lists, grids, text boxes, and buttons. The Managers include at least one of the following modules: an Activity Manager for interacting with all running activities in the system; a Location Manager for providing system services or applications with access to system location services; a Package Manager for retrieving various information related to application packages currently installed on the device; a Notification Manager for controlling the display and clearing of notification messages; and a Window Manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface. The Screen Capture Manager is the core framework layer component of this application, responsible for receiving screenshot requests from the application layer, managing the entire lifecycle of screenshot sessions, coordinating the subscription allocation and parameter aggregation of logical paths, and scheduling the start and stop of hardware screenshot resources.

[0051] In some embodiments, the Activity Manager manages the lifecycle of individual applications and common navigation and back functions, such as controlling application exit, opening, and back actions. The Window Manager manages all window programs, such as obtaining the screen size, determining if a status bar is present, locking the screen, capturing the screen, and controlling changes to the display window, such as shrinking the display window, shaking the display, or distorting the display.

[0052] In some embodiments, the system runtime library layer can provide support for the framework layer. When the framework layer is used, the operating system runs the instruction library contained in the system runtime library layer, such as the C / C++ instruction library, to implement the functions to be performed by the framework layer.

[0053] In some embodiments, the kernel layer is a functional layer situated between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. For example, ... Figure 3 As shown, hardware drivers can be configured in the kernel layer. The kernel layer can contain at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0054] In some embodiments, the display device 200 may support screenshot functions for different applications. For example, in a live streaming scenario, the live streaming application needs to continuously take high-frame-rate screenshots of the On-Screen Display (OSD) layer to obtain the UI screen; in a video recording scenario, the video recording module needs to take independent screenshots of the Video layer to obtain the video content; in an image recognition scenario, the AI ​​recognition engine needs to simultaneously obtain a complete screenshot of the OSD and Video combined as input data.

[0055] To implement the screenshot function, in some embodiments, the controller 250 of the display device 200 integrates a dedicated hardware screenshot module, which includes at least two independent physical hardware screenshot resources. Each physical hardware screenshot resource corresponds to an independent layer acquisition circuit, including a screenshot trigger controller, an image acquisition buffer (FrameBuffer Capture Unit), and a data transmission bus interface. The physical hardware screenshot resources are independent of each other and can acquire different display layer data in parallel within different clock cycles.

[0056] In some embodiments, the controller 250 includes a System on Chip (SOC), which integrates a Graphics Processing Unit (GPU) for display layer compositing and rendering, a Direct Memory Access (DMA) controller for high-speed data transfer, and a hardware screenshot module. Both the GPU and the DMA controller can access the same physical shared memory (i.e., a shared memory pool). The controller 250 can be configured to configure the GPU to perform scaling, blending, and format conversion processing of screenshot data, or it can be configured to configure the DMA controller to directly write screenshot data to the shared memory pool, thereby bypassing the GPU's image processing and reducing the transmission latency of screenshot data.

[0057] In some embodiments, the GPU of the display device 200 renders and composites the data of each layer according to the layer hierarchy and transparency attribute, generating the final composite frame data, which is then output to the display 260 for display by the display driver. For example, the OSD layer is located on top of the layer stack, has a transparency attribute (Alpha channel), and is superimposed on the Video layer to ultimately form the user interface seen by the user. The hardware screenshot module can independently capture the raw acquisition data of each layer before the layers enter the GPU compositing pipeline, or it can capture the frame buffer data after the GPU has completed compositing, to meet the screenshot requirements of different business scenarios.

[0058] In some embodiments, the physical hardware screenshot resource operates via register configuration. The upper-layer driver configures the physical hardware screenshot resource by writing to specified memory-mapped I / O registers, specifying the capture resolution (width and height), capture frame rate, and capture data format (e.g., ABGR8888, NV12, RGB888, etc.), and controls the start and stop of the physical hardware screenshot resource via an enable register. After the physical hardware screenshot resource starts, it automatically completes the capture of one frame of layer data every capture frame cycle (determined by the capture frame rate), writes the captured data to the specified image capture buffer, and simultaneously generates a capture completion interrupt to the CPU. Upon receiving the interrupt, the kernel-level hardware screenshot driver reads the screenshot data from the image capture buffer and notifies the upper-layer screenshot manager for subsequent processing.

[0059] However, screenshot requests from multiple sessions may run concurrently at the same time. In related technologies, the number of physical hardware screenshot resources configured on the display device 200 is very limited, and they are allocated exclusively: each screenshot request from a single session occupies a separate physical hardware screenshot path, and there is a one-to-one binding relationship between hardware screenshot resources and screenshot sessions. This exclusive allocation method directly limits the number of concurrent screenshot sessions that can be supported due to the limited number of physical hardware screenshot paths. When multiple business scenarios concurrently request screenshots, screenshot requests exceeding the hardware path limit will not be responded to, severely limiting the screenshot concurrency capability of the display device 200 and affecting the user experience.

[0060] To address the aforementioned issues, this application provides a display device 200 that abstracts limited physical hardware screenshot resources into a shareable logical path layer. Through a dynamic subscription mechanism, multiple screenshot sessions are allowed to reuse the same logical path. Combined with multi-session parameter aggregation and on-demand adaptive distribution, this enables support for concurrent screenshot sessions exceeding the number of hardware paths without increasing hardware screenshot resources, thereby improving the screenshot concurrency capability and hardware resource utilization of the display device 200.

[0061] In some embodiments, the display device 200 includes at least a display 260 and a controller 250. The display 260 is configured to display a user interface, and the controller 250 is configured with at least two physical hardware screenshot resources. One physical hardware screenshot resource is bound to a first logical path, and the other physical hardware screenshot resource is bound to a second logical path. The first and second logical paths are used to acquire data from different display layers.

[0062] In some implementations, of the at least two physical hardware screenshot resources, the hardware screenshot resource of the first logical path is dedicated to acquiring OSD layer data. The OSD layer carries user interface elements, including application interfaces, system menus, subtitle overlays, etc.; the hardware screenshot resource of the second logical path is dedicated to acquiring Video layer data. The Video layer carries decoded video frame data, including live video streams, local video playback content, etc. The two physical hardware screenshot resources are connected to different display layer output buses, and can independently acquire data from both layers within the same frame period, achieving parallel acquisition capability of the OSD layer and the Video layer.

[0063] In other implementations, the hardware screenshot resources of the first logical path are dedicated to acquiring Video layer data, and the hardware screenshot resources of the second logical path are dedicated to acquiring OSD layer data.

[0064] For ease of explanation, the following embodiments of this application are described using the example of hardware screenshot resources of the first logical path being dedicated to collecting OSD layer data and hardware screenshot resources of the second logical path being dedicated to collecting Video layer data, but this is not intended to limit the scope of the embodiments.

[0065] In some embodiments, during the operating system initialization phase, the controller 250 abstracts physical hardware screenshot resources into logical paths through a logical path registration and binding process, allowing for reuse in subsequent multi-channel screenshot sessions. The controller 250 maintains a logical path configuration table, where each record includes a logical path identifier, the bound physical hardware screenshot resource number, current acquisition parameters (including acquisition resolution, acquisition frame rate, and acquisition data format), and subscription reference count. During system initialization, the controller 250 scans available physical hardware screenshot resources, sequentially registers and binds the first physical hardware screenshot resource as the first logical path (OSD path), registers and binds the second physical hardware screenshot resource as the second logical path (Video path), and initializes the subscription reference counts of both logical paths to 0, placing the corresponding physical hardware screenshot resources in a closed state.

[0066] In some embodiments, the logical path configuration table is stored in the memory of the controller 250 in key-value pairs. The key is the logical path identifier, and the value is the corresponding physical resource binding information and the current acquisition parameters. When multiple screenshot sessions concurrently access the logical path configuration table, the controller 250 ensures thread-safe access to the configuration table through a read-write lock mechanism. Multiple screenshot sessions can concurrently read the configuration table content, but updating the acquisition parameters requires an exclusive write lock. In this way, by using read-write locks, concurrency safety can be ensured while reducing the impact of lock contention on screenshot processing performance.

[0067] In some embodiments, the binding relationship between logical paths and physical hardware screenshot resources remains fixed during system operation and does not change due to the addition or removal of screenshot sessions. That is, the first logical path always corresponds to the first physical hardware screenshot resource (OSD acquisition), and the second logical path always corresponds to the second physical hardware screenshot resource (Video acquisition). In this way, the path switching overhead caused by dynamic allocation can be avoided by using a fixed binding method, simplifying the management logic of acquisition parameters and ensuring that OSD layer data and Video layer data are always acquired by fixed hardware resources.

[0068] Figure 4 This is a flowchart illustrating a multi-channel screenshot method provided in some embodiments of this application. Figure 4 As shown, in some embodiments, controller 250 is configured to perform the following steps: S401. In response to the screenshot request sent by the first screenshot session, subscribe to the target logical path adapted for the first screenshot session according to the display layer on which the screenshot request is applied.

[0069] The target logical path is at least one of the first logical path and the second logical path. The controller 250 can act on the display layer to be affected according to the screenshot request, such as the OSD layer, the Video layer, or the OSD layer + Video layer. For example, when the first screenshot session comes from a live streaming application and needs to capture the UI screen of the OSD layer, the controller subscribes to the first logical path for the first screenshot session; when the first screenshot session comes from a video recording module and needs to capture the video content of the Video layer, the controller subscribes to the second logical path for the first screenshot session; when the first screenshot session comes from an AI recognition engine and needs to obtain the complete screen image, the controller subscribes to both the first and second logical paths for the first screenshot session.

[0070] In some embodiments, the controller 250 first queries the logical path configuration table to obtain the current subscription reference count of the target logical path. If the subscription reference count is 0, it means that no other screenshot session is currently occupying the target logical path. The controller 250 then enables the physical hardware screenshot resource corresponding to the target logical path and increments the subscription reference count by 1 to complete the subscription process. If the subscription reference count is greater than 0, it means that the target logical path has been subscribed to by other screenshot sessions. The controller 250 then increments the subscription reference count by 1 to complete the subscription.

[0071] S402. If the target logical path is subscribed to by at least one second screenshot session, aggregate the request parameters corresponding to the first screenshot session and the second screenshot session to generate the collection parameters of the target logical path.

[0072] When multiple screenshot sessions subscribe to the same logical path simultaneously, the controller 250 performs collection parameter aggregation to determine the unified collection parameters for the current logical path. In other words, the same logical path outputs only one set of screenshot data collected according to the aggregated parameters. Multiple screenshot sessions subscribing to the same path can share the same collection result without triggering separate hardware collection processes, thereby saving hardware path resources and improving concurrency support capabilities.

[0073] After the controller 250 detects the screenshot request sent by the first screenshot session and subscribes to the adapted target logical path for the first screenshot session, it checks whether the target logical path is also subscribed to by other screenshot sessions. For ease of distinction, in this embodiment, other screenshot sessions that have subscribed to the target logical path are described as second screenshot sessions. The number of second screenshot sessions can be one or more. When there are other second screenshot sessions subscribed to the target logical path, the controller 250 aggregates the request parameters corresponding to the first screenshot session and the second screenshot session to generate the acquisition parameters of the target logical path.

[0074] In some embodiments, when aggregating the request parameters corresponding to the first screenshot session and the second screenshot session, the controller 250 can aggregate the parameters (i.e., request parameters) carried by the screenshot request in each screenshot session to generate corresponding acquisition parameters. Request parameters may include target resolution, screenshot frame rate, and target data format. During the aggregation of request parameters, the controller 250 selects the highest resolution among multiple screenshot sessions as the unified acquisition resolution of the target logical path; selects the highest screenshot frame rate among multiple screenshot sessions as the unified acquisition frame rate of the target logical path; and selects a preset intermediate format adapted to more screenshot sessions as the acquisition data format. The acquisition resolution, acquisition frame rate, and acquisition data format constitute the acquisition parameters of the target logical path.

[0075] In some embodiments, after the controller 250 aggregates and generates the acquisition parameters of the target logical path, the controller 250 updates the acquisition parameters recorded in the current target logical path configuration table, and then updates the working parameters of the physical hardware screenshot resource through register configuration.

[0076] S403. Based on the hardware screenshot resources corresponding to the target logical path, collect screenshot data according to the collection parameters.

[0077] The controller 250 aggregates the request parameters corresponding to the first and second screenshot sessions to obtain the acquisition parameters. Based on the hardware screenshot resources corresponding to the target logical path, it acquires the corresponding screenshot data according to the acquisition parameters. The acquired screenshot data is written to a preset shared memory buffer for subsequent processing.

[0078] S404. Adapt the screenshot data to the screenshot request and distribute the processed screenshot data to the first screenshot session.

[0079] When multiple screenshot sessions share the collection results of the same logical path, the original collected data needs to be adapted according to the request parameters of each screenshot session before being distributed to the corresponding screenshot session. That is, the controller 250 will adapt the screenshot data to the screenshot request sent by the first screenshot session, so that the adapted screenshot data meets the parameter requirements of the first screenshot session, and then distribute the processed screenshot data to the first screenshot session.

[0080] Similarly, in some embodiments, for other second screenshot sessions that subscribe to the same target logical path, the controller 250 will also perform adaptation processing according to the screenshot request of each second screenshot session, and then distribute the processed screenshot data to the corresponding second screenshot session.

[0081] In some embodiments, the process of adapting screenshot requests includes, but is not limited to, scaling, format conversion and frame distribution operations. Based on the original high-resolution, high-frame-rate screenshot data, screenshot data that meets the parameter requirements of different screenshot sessions can be generated, which can realize the sharing and reuse of hardware resources and ensure that each screenshot session can obtain screenshot data that meets its own needs.

[0082] For example, if the unified acquisition resolution of the target logical path is 1920×1080, and a screenshot session requests a resolution of 1280×720, the controller 250 will perform downsampling and scaling on the original 1920×1080 screenshot data, and then distribute the scaled 1280×720 data to this screenshot session. As another example, if the unified acquisition data format is NV12, and a screenshot session requests a format of RGB888, the controller 250 will perform format conversion on the original data, and then distribute the converted data to this screenshot session.

[0083] In some embodiments, if the first screenshot session terminates the screenshot request, the controller 250 responds to the stop request of the first screenshot session by decrementing the subscription reference count of the target logical path by 1. If the subscription reference count becomes 0 after decrementing by 1, it means that no screenshot session is currently subscribed to the target logical path. The controller 250 then shuts down the physical hardware screenshot resources corresponding to the target logical path, releases the hardware acquisition bandwidth, and reduces unnecessary power consumption.

[0084] This embodiment of the disclosure abstracts physical hardware screenshot resources into shareable logical paths, allowing multiple screenshot sessions to reuse the same logical path through a subscription mechanism. Only one hardware acquisition needs to be completed after aggregating parameters, and then each acquisition is adapted and distributed to meet the screenshot requirements of multiple sessions. With a fixed number of hardware screenshot paths, more concurrent screenshot sessions can be supported without adding new physical hardware resources. This improves the hardware resource utilization of the display device 200 without increasing hardware costs, while ensuring that each screenshot session obtains screenshot data that meets its own parameter requirements. This addresses the problem of the display device 200 being unable to support multiple concurrent screenshots due to insufficient hardware screenshot resources.

[0085] In some embodiments of step S401 above, the display layer of the user interface includes a first layer (OSD layer) and a second layer (Video layer). The screenshot request carries a screenshot type identifier, which includes at least one of the first layer identifier (OSD identifier), the second layer identifier (Video identifier), and the composite identifier (OSD+Video identifier).

[0086] The first layer identifier indicates that the screenshot request needs to collect the image data of the first layer, the second layer identifier indicates that the screenshot request needs to collect the image data of the second layer, and the composite identifier indicates that the screenshot request needs to collect the image data of the first layer and the second layer simultaneously, and then composite the two layers of data into a complete screen image.

[0087] Figure 5 This is a flowchart illustrating the target logical path for subscription adaptation provided in some embodiments of this application. For example... Figure 5 As shown, in some embodiments, the controller 250 subscribes to the target logical path adapted for the first screenshot session based on the display layer affected by the screenshot request. The specific execution steps are as follows: S501. In response to the screenshot request carrying the first layer identifier, subscribe the first screenshot session to the first logical path.

[0088] When a screenshot request carries a first layer identifier (i.e., the screenshot content only includes the OSD layer), the controller 250 subscribes to the first screenshot session to the first logical path. The physical hardware screenshot resource corresponding to the first logical path is responsible for collecting OSD layer data, and subsequent screenshot data processing only adapts and distributes the OSD layer data.

[0089] S502, In response to the screenshot request carrying the second layer identifier, subscribe the first screenshot session to the second logical path.

[0090] When a screenshot request carries a second layer identifier (i.e., the screenshot content only contains the Video layer), the controller 250 subscribes the first screenshot session to the second logical path. The physical hardware screenshot resource corresponding to the second logical path is responsible for collecting Video layer data, and subsequent screenshot data processing only adapts and distributes the Video layer data.

[0091] S503. In response to the screenshot request carrying a composite identifier, the first screenshot session is simultaneously subscribed to the first logical path and the second logical path.

[0092] When a screenshot request carries a compositing identifier (i.e., the screenshot content must simultaneously contain composite data of the OSD layer and the Video layer), the controller 250 subscribes to both the first logical path and the second logical path for the first screenshot session. The controller 250 collects OSD layer data and Video layer data from the first logical path and the second logical path respectively, and then performs fusion processing to generate composite screenshot data.

[0093] This embodiment of the disclosure matches different layer acquisition requirements according to the screenshot type identifier, realizes differentiated channel subscription for single-layer screenshots and dual-layer composite screenshots, can adapt to the business needs of different screenshot sessions, and improve the reusability accuracy and processing efficiency of hardware channels.

[0094] For step S402, Figure 6 This is a schematic diagram illustrating the process of generating acquisition parameters provided in some embodiments of this application. For example... Figure 6 As shown, in some embodiments, the controller 250 aggregates the request parameters of the first screenshot session and the second screenshot session, and the specific execution steps are as follows: S601. Obtain the request parameters of the first screenshot session and the request parameters of the second screenshot session.

[0095] The request parameters include, but are not limited to, output resolution and screenshot frame rate. In some implementations, the controller 250 can obtain the corresponding request parameters from the screenshot requests of the first screenshot session and the second screenshot session.

[0096] S602. The maximum width and maximum height of the output resolution are used as the acquisition resolution of the target logical path, and the maximum screenshot frame rate is used as the acquisition frame rate of the target logical path.

[0097] The controller 250 compares the output resolution width of all screenshot session requests and selects the largest width value as the unified acquisition resolution width. Then, it compares the output resolution height of all screenshot session requests and selects the largest height value as the unified acquisition resolution height. This yields the maximum acquisition resolution for screenshot sessions that are compatible with all subscribed target logical paths, satisfying the resolution requirements of all screenshot sessions. Similarly, it compares the screenshot frame rate of all screenshot session requests and selects the highest frame rate as the unified acquisition frame rate for the target logical path, ensuring that sessions with high frame rate requirements can also obtain sufficiently smooth screenshot data.

[0098] S603. Set the data acquisition format of the target logical path to a preset intermediate format.

[0099] The controller 250 sets the preset intermediate format, which is adapted to the needs of the most screenshot sessions, as the data acquisition format of the target logical path. The preset intermediate format can take into account the computational complexity and universality of subsequent format conversion, reducing the computational overhead of performing format conversion separately for each screenshot session.

[0100] In this embodiment of the disclosure, by aggregating the request parameters of multiple screenshot sessions under the same target logical path, the maximum value of each parameter is taken to generate a unified collection parameter. Only one data collection is needed to meet the output requirements of multiple screenshot sessions, avoiding repeated collection that occupies unnecessary hardware bandwidth, thereby reducing the computing power consumption of the display device 200 in the scenario of multiple screenshot sessions.

[0101] For the process of aggregating multiple screenshot session request parameters, in some embodiments, the controller 250 traverses all screenshot sessions (including the first screenshot session and the second screenshot session) of the subscribed target logical path and obtains the output resolution (width W) of each screenshot session. i Height H i ) and screenshot frame rate (FPS) i And aggregated according to the following rules: acquisition resolution width W cap =max(W1, W2, ..., W n ), acquisition resolution height H cap =max(H1, H2, ..., H n ), capture frame rate (FPS) cap =max(FPS1, FPS2, ..., FPS) n The data collected is in a fixed intermediate format (e.g., RGBA8888).

[0102] For example, taking the concurrent subscription of three screenshot sessions to the second logical path (Video path) as an example, the request parameters for the first screenshot session are 1920×1080, 30fps, NV12; the request parameters for the second screenshot session are 1280×720, 60fps, NV12; and the request parameters for the third screenshot session are 3840×2160, 25fps, ABGR8888. The controller 250 aggregates the request parameters of the three screenshot sessions, setting the acquisition resolution width to max(1920, 1280, 3840) = 3840, the height to max(1080, 720, 2160) = 2160, the frame rate to max(30, 60, 25) = 60fps, and the acquisition data format to a unified ABGR8888. Therefore, the unified acquisition parameters for the second logical path are 3840×2160, 60fps, ABGR8888. Controller 250 reconfigures the second physical hardware screenshot resource according to this parameter.

[0103] In some embodiments, when a screenshot session is cancelled, the controller 250 needs to perform reverse aggregation (re-aggregation) on the acquisition parameters of the target logical path to avoid unnecessary waste of hardware resources caused by the acquisition parameters remaining in historical parameters for a long time. That is, when a screenshot session is cancelled, the controller 250 re-aggregates the request parameters of each screenshot session that is still subscribed to the target logical path to obtain new aggregation values ​​(resolution, frame rate). If the new aggregation value is lower than the current acquisition parameters, the controller 250 updates the acquisition parameters of the logical path to the new aggregation value.

[0104] For example, in the above three-way concurrent example, if the third screenshot session (3840×2160, 25fps) with the highest request parameters is logged out, the controller 250 will re-aggregate the acquisition parameters of the second logical path to 1920×1080, 60fps, ABGR8888, in order to reduce the acquisition specifications of the physical hardware screenshot resources and save system resources.

[0105] For step S404, Figure 7 This is a schematic diagram illustrating the screenshot data adaptation process provided in some embodiments of this application. For example... Figure 7 As shown, in some embodiments, the controller 250 performs multi-stage adaptation processing on the acquired screenshot data to meet the output parameter requirements of different screenshot sessions. The screenshot data is the raw data acquired by the target logical path (uniformly in a preset intermediate format, such as ABGR8888), which undergoes scaling, fusion (enabled as needed), and format conversion processing to finally output the final data conforming to the screenshot session. That is, the controller 250 is configured to perform the following steps: S701. Based on the target resolution requested in the screenshot request, perform scaling processing on the screenshot data to obtain intermediate data.

[0106] The controller 250 scales the original captured screenshot data according to the target resolution requested by the screenshot session. If the target resolution is smaller than the capture resolution, downsampling scaling is performed; if the target resolution is larger than the capture resolution, upsampling interpolation is performed to obtain intermediate data that meets the target resolution size, so as to match the resolution requirements of the screenshot session.

[0107] In some embodiments, the controller 250 invokes the GPU's bilinear interpolation or bicubic interpolation algorithm to perform scaling processing on the acquired screenshot data. The bilinear interpolation algorithm is fast and suitable for applications where screenshot quality requirements are not high; the bicubic interpolation algorithm is suitable for applications where screenshot quality requirements are high. The controller 250 can dynamically select the scaling algorithm type based on the priority of the screenshot session or user settings.

[0108] S702. If the screenshot request carries a compositing identifier, perform fusion processing on the intermediate data corresponding to the first logical path and the intermediate data corresponding to the second logical path; then, according to the target data format requested by the screenshot request, perform format conversion processing on the fused intermediate data to complete the processing to adapt to the screenshot request.

[0109] When the layer affected by the screenshot request includes both an OSD layer and a Video layer, the controller 250, after obtaining the intermediate data of the OSD layer in the first logical path and the intermediate data of the Video layer in the second logical path, performs pixel-level fusion of the two intermediate data according to the layer display level to obtain composite intermediate data that contains the content of both layers. Then, the composite intermediate data is converted into the target data format required by the screenshot request to generate screenshot data that meets the requirements of composite screenshot.

[0110] In some embodiments, the controller 250 invokes the GPU to perform alpha fusion processing on the intermediate data of the OSD layer corresponding to the first logical path and the intermediate data of the Video layer corresponding to the second logical path. The resulting composite image after fusion processing can retain the background content of the Video layer and the overlay content of the OSD layer, maintaining consistency with the actual screen content displayed by the display device 200.

[0111] S703. If the screenshot request does not carry a composite identifier, perform format conversion processing on the intermediate data according to the target data format requested in the screenshot request to complete the processing to adapt to the screenshot request.

[0112] When the layer affected by the screenshot request is only the OSD layer or the Video layer, the controller 250 does not need to perform cross-path layer fusion. It only performs format conversion processing on the intermediate data output by the target logical path, and can directly output single-layer screenshot data that meets the parameter requirements, thereby reducing unnecessary computational overhead and improving the output efficiency of single-layer screenshots.

[0113] In some embodiments, the controller 250 can perform format conversion processing via the GPU for the above steps S702 and S703, and then distribute the converted screenshot data to the corresponding screenshot session.

[0114] In this embodiment of the disclosure, the controller 250 performs scaling, fusion, and format conversion on the screenshot data for adaptation processing. The adaptation processing can be uniformly executed on the GPU. By utilizing the parallel computing capabilities of the GPU, the processing latency of a single frame of screenshot data can be reduced, and the system throughput can be improved.

[0115] Figure 8 This is a schematic flowchart illustrating the scaling process provided in some embodiments of this application. For example... Figure 8 As shown, in some embodiments, the controller 250 performs on-demand scaling on the screenshot data, specifically by executing the following steps: S801. If the target resolution is consistent with the acquisition parameters of the target logical path, maintain the resolution of the screenshot data as intermediate data.

[0116] The controller 250 obtains the target resolution requested by the screenshot request of the first screenshot session and compares it with the acquisition resolution in the current acquisition parameters of the target logical path. If the two are consistent, there is no need to perform scaling operations. The screenshot data acquired by the target logical path is directly used as intermediate data, and the scaling processing step is skipped to reduce redundant calculations.

[0117] S802. If the target resolution is inconsistent with the acquisition parameters of the target logical path, the resolution of the screenshot data is scaled to the target resolution and used as intermediate data.

[0118] When the target resolution differs from the acquisition resolution of the target logical path, the controller 250 invokes the GPU to perform scaling processing on the screenshot data, scaling the resolution of the screenshot data to the target resolution. The scaling result is used as intermediate data for subsequent fusion processing or format conversion processing. Adopting a scaling-then-fusion processing order can reduce the amount of pixel computation during subsequent fusion processing and optimize GPU performance.

[0119] In some embodiments, when the same logical path is subscribed to by multiple screenshot sessions and the target resolution of each screenshot session is different, the controller 250 performs scaling processing independently for each screenshot session, and the scaling operations of each session are independent of each other and do not affect each other.

[0120] In this embodiment, an on-demand scaling processing mechanism is adopted. Scaling calculations are only performed when there is a difference between the target resolution of the session and the hardware acquisition resolution. When the parameters are consistent, the original data is directly reused, reducing redundant calculations and lowering the computing power consumption of the display device 200, thereby improving the screenshot processing speed.

[0121] Figure 9 This is a schematic diagram illustrating the format conversion process provided in some embodiments of this application. For example... Figure 9 As shown, in some embodiments, the controller 250 performs format conversion processing on the intermediate data after fusion processing, or performs format conversion processing on the intermediate data, specifically performing the following steps: S901. If the target data format is consistent with the acquisition parameters of the target logical path, retain the data format of the intermediate data.

[0122] The controller 250 obtains the target data format requested by the screenshot request of the first screenshot session and compares it with the data format of the current acquisition parameters of the target logical path. If the two are consistent, there is no need to perform a format conversion operation, and the data format of the intermediate data is directly retained, skipping the format conversion step to avoid meaningless format conversion operations.

[0123] S902. If the target data format is inconsistent with the acquisition parameters of the target logical path, convert the data format of the intermediate data to the target data format.

[0124] When the target data format is inconsistent with the data format acquired by the target logical path, the controller 250 calls the GPU to perform format conversion processing on the intermediate data, converting the data format of the intermediate data into the target data format to meet the format requirements of the first screenshot session.

[0125] In some embodiments, the controller 250 is further configured with a format conversion capability matrix, which records the conversion paths (e.g., conversion latency and GPU computational load) between all source and target formats supported by the GPU. When the target data format of a screenshot session does not have a direct conversion path in the format conversion capability matrix, the controller 250 searches for the shortest conversion path (e.g., RGBA8888→RGB888→YUV444→NV12) and achieves the target data format through multi-step format conversion, thereby improving the success rate of format conversion processing and avoiding screenshot request failure due to the inability to convert directly.

[0126] In this embodiment, on-demand format conversion is achieved based on format matching, avoiding meaningless format conversion operations, further reducing the computational overhead of the screenshot adaptation process, and shortening the screenshot response latency.

[0127] In some embodiments, to achieve differentiated frame rate distribution across multiple screenshot sessions, the controller 250 also maintains an independent frame rate counter for each screenshot session. Specifically, a corresponding frame rate counter is maintained for the first screenshot session. The frame rate counter for the first screenshot session is incremented each time a frame of screenshot data is output from the target logic path. When the frame rate counter reaches a preset frame rate threshold for the first screenshot session, a frame of screenshot data collected by the target logic path is distributed to the first screenshot session.

[0128] In some embodiments, when the first screenshot session is created, the controller 250 initializes an independent frame rate counter for the first screenshot session, with an initial value of 0. The frame rate counter records the number of frames output by the target logical path since the last screenshot data was distributed to the first screenshot session. The frame rate counters for each screenshot session are completely independent and do not interfere with each other, ensuring that the frame rate distribution of multiple screenshot sessions does not affect each other.

[0129] Whenever the target logical path collects and outputs a frame of screenshot data, the controller 250 increments the frame rate counters of all screenshot sessions subscribed to the target logical path by 1, and checks whether the frame rate counter of each screenshot session has reached its corresponding preset frame rate threshold. When the frame rate counter of a screenshot session reaches its preset frame rate threshold, the controller 250 distributes the currently collected frame of screenshot data from the target logical path to this screenshot session after adaptation processing, ensuring that the frame rate distribution of each screenshot session can be performed according to its respective target frame rate.

[0130] In this embodiment, an independent frame rate counting mechanism is configured for a single screenshot session. By unifying the hardware frame rate, a differentiated frame rate distribution effect is achieved for different sessions. This eliminates the need for repeated hardware acquisition to meet the different frame rate requirements of multiple sessions, thereby maximizing the reuse of hardware acquisition capabilities.

[0131] To further reduce the latency and CPU overhead of screenshot data distribution, in some embodiments, the controller 250 employs a zero-copy transfer mechanism based on direct memory access (DMA) in single-path screenshot scenarios with matching parameters. If the target resolution, target data format, and acquisition parameters requested in the screenshot request are consistent with those of the target logical path, and the number of target logical paths is one, the screenshot data acquired by the target logical path is written to the shared memory pool via direct memory access. Then, the memory address is passed to the first screenshot session based on the shared memory pool to distribute the screenshot data to the first screenshot session.

[0132] In some embodiments, the shared memory pool of the controller 250 consists of a contiguous physical memory region, visible to both the kernel space of the controller 250 and the user space of the application process via memory mapping (mmap). The shared memory pool is divided into multiple fixed-size memory slots per frame, each large enough to hold a screenshot data frame of maximum size (e.g., 3840×2160×4 bytes). The controller 250 maintains the allocation and reclamation of memory slots using a ring buffer management strategy, sequentially allocating memory slots to store newly acquired screenshot frame data and automatically reclaiming the corresponding memory slots after older frame data is consumed by the application.

[0133] In some embodiments, when the controller 250 determines that it has entered the pass-through mode (i.e., the target resolution, target data format and acquisition parameters are consistent, and the number of target logical paths is one), the controller 250 configures the DMA controller to use the acquisition buffer address of the target logical path as the DMA transfer source address and the memory slot address allocated in the shared memory pool as the DMA transfer target address. After the DMA transfer is triggered, the screenshot data is directly moved from the acquisition buffer to the shared memory pool by the DMA controller. The entire process does not go through the CPU or the GPU, avoiding multiple data copies. After the DMA transfer is completed, the hardware automatically generates an interrupt notification to the controller 250, which can reduce the CPU load of the screenshot data transfer.

[0134] In some embodiments, after the DMA transfer is completed, the controller 250 transmits the memory slot address (virtual address) corresponding to the screenshot data in the shared memory pool to the application process where the first screenshot session resides via an inter-process communication mechanism (IPC, such as the Android Binder mechanism or Unix Socket). The application process can directly read the screenshot data in the shared memory pool through this virtual address without performing a data copy operation, thus achieving end-to-end zero-copy distribution.

[0135] To prevent data races, in some embodiments, the controller 250 uses a reference counting mechanism to manage the reference state of each memory slot: the memory slot will not be reclaimed while it is being referenced by the application; when the application finishes reading, it explicitly releases the reference by calling the screenshot service interface, and the controller 250 decrements the reference count of the memory slot by 1. Only after the reference count reaches zero can the memory slot be reclaimed and reused.

[0136] In this embodiment of the disclosure, in a single-path screenshot scenario with matching parameters, a zero-copy method using DMA direct writing and shared memory address distribution is adopted to eliminate redundant copy operations on the path from the acquisition of screenshot data to the application, thereby reducing the distribution latency of screenshot data and improving screenshot efficiency.

[0137] To achieve dynamic start / stop management of hardware screenshot resources and reduce overall device power consumption, such as... Figure 10 As shown, Figure 10 The diagram below illustrates the subscription reference counting process for some embodiments of this application. In some embodiments, the controller 250 manages the physical hardware screenshot resources corresponding to the logical path through the subscription reference counting mechanism, specifically executing the following steps: S1001, Maintain the subscription reference count of the target logical path.

[0138] The controller 250 maintains a subscription reference counter for each logical path to record the number of screenshot sessions currently subscribed to the logical path. Initially, the subscription reference counts for both logical paths are at their first value (e.g., 0), and the corresponding physical hardware screenshot resources are in a closed state.

[0139] S1002. In response to the subscription reference count changing from the first value to the second value, start the hardware screenshot resource corresponding to the target logical path to begin collection.

[0140] When the subscription reference count of the target logical path changes from the first value (e.g., 0) to the second value (e.g., 1), that is, when the target logical path changes from an unsubscribed state to being subscribed for the first time, the controller 250 immediately sends a start command to the corresponding physical hardware screenshot resource. After the physical hardware screenshot resource completes power-on initialization, it starts collecting screenshot data according to the currently aggregated collection parameters.

[0141] In some embodiments, the startup process of the physical hardware screenshot resource includes power-on initialization, register configuration (acquisition resolution, frame rate, data format), and data acquisition enable. During the startup of the physical hardware screenshot resource, the controller 250 caches screenshot requests that arrive during the startup period and begins processing them only after the hardware acquisition is ready, ensuring that the screenshot session does not lose the first frame of screenshot data due to hardware startup delay.

[0142] S1003. In response to the subscription reference count changing from the second value to the first value, close the hardware screenshot resource corresponding to the target logical path to stop the collection.

[0143] When the subscription reference count of the target logical path drops from the second value (e.g., 1) back to the first value (e.g., 0), that is, when the last subscriber of the target logical path cancels, the controller 250 immediately sends a shutdown command to the corresponding physical hardware screenshot resource. The physical hardware screenshot resource stops capturing screenshot data and enters a low-power sleep state until it is subscribed to again.

[0144] In some embodiments, when the subscription reference count of the target logical path changes to a second value (e.g., 1) or higher (i.e., the number of subscriptions changes from n to n±1, where n≥1), the physical hardware screenshot resource does not perform start / stop operation, but only updates the acquisition parameters (through parameter aggregation or reverse aggregation). The controller 250 performs dynamic acquisition parameter reconfiguration on the physical hardware screenshot resource (i.e., updates the resolution, frame rate, and other parameters in the hardware register without interrupting the current acquisition).

[0145] This embodiment of the disclosure precisely controls the start / stop status of physical hardware screenshot resources through reference counting, enabling data acquisition to start when there is subscription demand and to shut down and sleep when there is no subscription demand. This avoids long-term idle operation of the hardware, reduces the overall power consumption of the display device 200, and improves the rationality of resource scheduling. Furthermore, in scenarios where the number of subscriptions changes but does not return to zero, the acquisition specifications are updated through dynamic parameter reconfiguration rather than hardware restart, reducing the gaps in screenshot data frames caused by hardware restarts and ensuring the continuity of screenshot data.

[0146] Based on the above embodiments, the controller 250 can manage the entire lifecycle of each screenshot session—from creation and subscription to update and cancellation—through the screenshot session lifecycle management module. In some embodiments, the screenshot session lifecycle includes the following four stages: The first phase, the session creation phase: When the application calls the screenshot service interface provided by the display device 200, the screenshot session management module of the controller 250 creates a corresponding screenshot session for the application and assigns a unique session identifier (Session ID) to the screenshot session. The screenshot session contains information such as the request parameters of the screenshot request (screenshot type identifier, target output resolution, target screenshot frame rate, target data format), target logical path subscription relationship, independent frame rate counter, and screenshot data distribution buffer.

[0147] The second phase, the logical path subscription phase: After the screenshot session is created, the controller 250 subscribes to the appropriate target logical path for the screenshot session based on the screenshot type identifier, and updates the subscription reference count and collection parameters (execution parameter aggregation) of the target logical path. If the subscription reference count of the target logical path changes from 0 to 1, the controller 250 starts the corresponding physical hardware screenshot resource and begins collecting screenshot data.

[0148] The third stage is the data distribution stage. After the screenshot session enters an active state, when the target logic path outputs a frame of screenshot data, the controller 250 updates the frame rate counter of the screenshot session. When the preset frame rate threshold is reached, the screenshot data is adapted (scaled, blended, format converted) and then distributed to the data distribution buffer corresponding to the screenshot session. The application can read the screenshot data from the data distribution buffer.

[0149] The fourth stage, the session cancellation stage: When the application no longer needs the screenshot data, it calls the screenshot service interface to cancel the corresponding screenshot session. Controller 250 removes the screenshot session from the target logical path's subscription list and decrements the target logical path's subscription reference count by 1. If the reference count drops to 0, controller 250 shuts down the physical hardware screenshot resources corresponding to the target logical path and stops data acquisition. Finally, controller 250 destroys the screenshot session object, releasing the session identifier and related memory resources.

[0150] In some embodiments, when the controller 250 of the display device 200 is configured with three or more hardware screenshot resources, it can support the simultaneous activation of three independent hardware screenshot paths: one path for capturing data from the OSD layer, one path for capturing data from the Video layer, and another path for jointly capturing data from both the OSD and Video layers. This reduces the number of fusion processing steps, lowers the additional computational overhead of fusion calculations, and further improves screenshot processing efficiency.

[0151] Based on the above embodiments, Figure 11 This application provides a schematic diagram of the architecture for multi-channel screenshots in some embodiments, such as... Figure 11As shown, in some embodiments, the architecture of multi-channel screenshots is divided into an application layer, a framework layer, a chip manufacturer's framework layer, a driver layer, and a hardware layer from top to bottom. The application layer handles various screenshot services such as single screenshots, live streaming, video recording, and AI image recognition, and is implemented through the screencontrol SDK (Software Development Kit) and AIDL (Android Interface Definition Language). The Android Interface Definition Language (API) sends a screenshot request carrying the screenshot type identifier, target resolution, target frame rate, and target data format to the CaptureManager-service in the framework layer via cross-process communication. The CaptureManager-service integrates screenshot session lifecycle management, logical path subscription and reference counting management, multi-session parameter aggregation, and frame rate control and distribution modules. It is responsible for maintaining the logical path configuration table and dynamically subscribing to the first logical path (OSD path) and the second logical path (Video path) that correspond to the physical hardware screenshot resources for the screenshot session. The chip manufacturer's framework layer interfaces with the underlying driver through the vsscapture-service (video screenshot service) and HAL (Hardware Abstraction Layer), and is configured with a shared memory pool that uses a circular buffer to support zero-copy data transfer. The driver layer includes the hardware screenshot driver, shader, driver, and DMA driver, which are responsible for the register configuration and interrupt handling of the hardware screenshot module, scaling, blending, format conversion operations, and DMA direct memory access control on the GPU side, respectively. The hardware layer includes the GPU and DIP (Digital Image Processing Layer). The processor (digital image processing unit), DMA controller, and hardware screenshot module integrating at least two independent physical hardware screenshot resources can acquire OSD layer and Video layer data in parallel. The screenshot data can be directly written to the shared memory pool through the DMA controller, and then adapted and processed as needed by the screenshot manager service. It is then distributed to the corresponding screenshot session with zero copy by passing memory address, thus fully realizing the multi-channel screenshot described in the embodiments of this application.

[0152] Figure 12 Interactive diagrams illustrating the multi-channel screenshot method provided in some embodiments of this application. For example... Figure 12As shown, in some embodiments, a user triggers a screenshot through an application. The application sends a request to the screenshot manager service in the framework layer to create a screenshot session, carrying the screenshot type, target resolution, target frame rate, and target data format parameters of this screenshot request. After the screenshot manager service completes the screenshot session creation, it determines the logical path to be subscribed to based on the screenshot type and sends a path subscription request to the video screenshot service in the chip manufacturer's framework layer to subscribe to the appropriate logical path for the screenshot session. The video screenshot service updates the subscription reference count of the corresponding logical path. If the subscription reference count changes from 0 to 1, it sends a hardware startup command to the driver layer. After the driver layer completes the initialization configuration of the hardware screenshot resources, it returns a startup completion response to the video screenshot service. After the video screenshot service updates the acquisition parameters, it returns a subscription completion response to the screenshot manager service. The screenshot manager service then begins to adapt the acquired screenshot data according to the target frame rate and distributes the processed screenshot data to the application until the application initiates a screenshot session cancellation request.

[0153] After receiving the deregistration request, the screenshot manager service initiates a path deregistration request to the video screenshot service. The video screenshot service updates the subscription reference count of the corresponding logical path. If the reference count drops to 0, it sends a hardware shutdown command to the driver layer. The driver layer controls the hardware screenshot resource to enter a low-power sleep state. After that, the video screenshot service returns a deregistration completion response to the screenshot manager service. The screenshot manager service destroys the screenshot session, completing the entire lifecycle of the screenshot session.

[0154] Based on the aforementioned display device 200, some embodiments of this application also provide a multi-channel screenshot method applied to the display device 200. The display device 200 is configured with at least two physical hardware screenshot resources, and one physical hardware screenshot resource is bound as a first logical path, and the other physical hardware screenshot resource is bound as a second logical path. The first logical path and the second logical path are used to collect data from different display layers. Figure 4 As shown, the method includes the following steps: S401. In response to the screenshot request sent by the first screenshot session, according to the display layer on which the screenshot request is applied, subscribe to the target logical path adapted for the first screenshot session, wherein the target logical path is at least one of the first logical path and the second logical path.

[0155] S402. If the target logical path is subscribed to by at least one second screenshot session, aggregate the request parameters corresponding to the first screenshot session and the second screenshot session to generate the collection parameters of the target logical path.

[0156] S403. Based on the hardware screenshot resources corresponding to the target logical path, collect screenshot data according to the collection parameters.

[0157] S404. Adapt the screenshot data to the screenshot request and distribute the processed screenshot data to the first screenshot session.

[0158] Based on the above-mentioned multi-channel screenshot method, the limitation of one-to-one exclusive use of hardware screenshot resources in display device 200 can be broken. It can support the concurrent operation of multiple screenshot sessions with two fixed physical hardware screenshot resources. Through dynamic subscription, parameter aggregation and on-demand adaptation processing logic, the concurrent screenshot capability and hardware resource utilization of display device can be improved, the overall power consumption of device can be reduced, and the screenshot processing efficiency and user experience can be improved.

[0159] It should be noted that the multi-channel screenshot method described in this application embodiment can adopt the same principle and implementation as the above-mentioned display device 200 embodiment, and will not be described again here.

[0160] Similar parts between the embodiments provided in this application can be referred to mutually. The specific implementation methods provided above are only a few examples under the overall concept of this application and do not constitute a limitation on the scope of protection of this application. For those skilled in the art, any other implementation methods extended from the solution of this application without creative effort shall fall within the scope of protection of this application.

Claims

1. A display device, characterized in that, include: The monitor is configured to display the user interface; The controller is configured with at least two physical hardware screenshot resources, and binds one physical hardware screenshot resource as a first logical path and the other physical hardware screenshot resource as a second logical path; the first logical path and the second logical path are used to collect data from different display layers; The controller is configured to: In response to a screenshot request sent by a first screenshot session, a target logical path is subscribed to and adapted for the first screenshot session according to the display layer on which the screenshot request is applied. The target logical path is at least one of the first logical path and the second logical path. If the target logical path is subscribed to by at least one second screenshot session, the request parameters corresponding to the first screenshot session and the second screenshot session are aggregated to generate the collection parameters of the target logical path; Based on the hardware screenshot resources corresponding to the target logical path, screenshot data is collected according to the collection parameters; The screenshot data is processed to adapt to the screenshot request, and the processed screenshot data is distributed to the first screenshot session.

2. The display device according to claim 1, characterized in that, The acquisition parameters include acquisition resolution, acquisition frame rate, and acquisition data format; the controller aggregates the request parameters corresponding to the first screenshot session and the second screenshot session to generate the acquisition parameters for the target logical path, specifically configured as follows: Obtain the request parameters of the first screenshot session and the request parameters of the second screenshot session, wherein the request parameters include the output resolution and the screenshot frame rate; The maximum width and maximum height of the output resolution are used as the acquisition resolution of the target logical path, and the maximum screenshot frame rate is used as the acquisition frame rate of the target logical path. Set the data acquisition format of the target logical path to a preset intermediate format.

3. The display device according to claim 1, characterized in that, The user interface display layer includes a first layer and a second layer. The screenshot request carries a screenshot type identifier, which includes at least one of a first layer identifier, a second layer identifier, and a composite identifier. The controller executes the target logical path adapted to the first screenshot session by subscribing to the display layer affected by the screenshot request, specifically configured as follows: In response to the screenshot request carrying the first layer identifier, the first screenshot session is subscribed to the first logical path; In response to the screenshot request carrying the second layer identifier, the first screenshot session is subscribed to the second logical path; In response to the screenshot request carrying the composite identifier, the first screenshot session is simultaneously subscribed to the first logical path and the second logical path.

4. The display device according to claim 3, characterized in that, The controller performs the process of adapting the screenshot data to the screenshot request, specifically configured as follows: Based on the target resolution requested in the screenshot request, the screenshot data is scaled to obtain intermediate data; If the screenshot request carries the synthesis identifier, the intermediate data corresponding to the first logical path and the intermediate data corresponding to the second logical path are fused together. Then, according to the target data format requested by the screenshot request, the intermediate data after fusion is converted to adapt to the screenshot request. If the screenshot request does not carry the composite identifier, the intermediate data is converted according to the target data format requested in the screenshot request to complete the adaptation to the screenshot request.

5. The display device according to claim 4, characterized in that, The controller performs scaling processing on the screenshot data, specifically configured as follows: If the target resolution is consistent with the acquisition parameters of the target logical path, the resolution of the screenshot data is maintained and used as the intermediate data; If the target resolution is inconsistent with the acquisition parameters of the target logical path, the resolution of the screenshot data is scaled to the target resolution and used as the intermediate data.

6. The display device according to claim 4, characterized in that, The controller performs format conversion processing on the intermediate data after fusion processing, or performs format conversion processing on the intermediate data, specifically configured as follows: If the target data format is consistent with the acquisition parameters of the target logical path, the data format of the intermediate data is retained; If the target data format is inconsistent with the acquisition parameters of the target logical path, the data format of the intermediate data will be converted to the target data format.

7. The display device according to claim 1, characterized in that, The controller is also configured to: Maintain a corresponding frame rate counter for the first screenshot session; When the target logic path outputs a frame of screenshot data, the frame rate counter of the first screenshot session is incremented; When the controller distributes the processed screenshot data to the first screenshot session, it is also configured to: When the frame rate counter reaches the preset frame rate threshold of the first screenshot session, a frame of screenshot data collected by the target logical path is distributed to the first screenshot session.

8. The display device according to any one of claims 1-7, characterized in that, The controller performs processing to adapt the screenshot data to the screenshot request, and distributes the processed screenshot data to the first screenshot session, specifically configured as follows: If the target resolution and target data format requested in the screenshot request are consistent with the acquisition parameters of the target logical path, and the number of the target logical path is one, the screenshot data acquired by the target logical path is written to the shared memory pool through direct memory access. The memory address is passed to the first screenshot session based on the shared memory pool in order to distribute the screenshot data to the first screenshot session.

9. The display device according to claim 1, characterized in that, The controller is also configured to: Maintain the subscription reference count for the target logical path; In response to the subscription reference count changing from a first value to a second value, the hardware screenshot resource corresponding to the target logical path is activated to begin collection; In response to the subscription reference count changing from the second value to the first value, the hardware screenshot resource corresponding to the target logical path is closed to stop the collection.

10. A multi-channel screenshot method, characterized in that, Applied to a display device, the display device is configured with at least two physical hardware screenshot resources, and one physical hardware screenshot resource is bound as a first logical path, and the other physical hardware screenshot resource is bound as a second logical path; The first logical path and the second logical path are used to collect data from different display layers; the method includes: In response to a screenshot request sent by a first screenshot session, a target logical path is subscribed to and adapted for the first screenshot session according to the display layer on which the screenshot request is applied. The target logical path is at least one of the first logical path and the second logical path. If the target logical path is subscribed to by at least one second screenshot session, the request parameters corresponding to the first screenshot session and the second screenshot session are aggregated to generate the collection parameters of the target logical path; Based on the hardware screenshot resources corresponding to the target logical path, screenshot data is collected according to the collection parameters; The screenshot data is processed to adapt to the screenshot request, and the processed screenshot data is distributed to the first screenshot session.

Images