A method for independent floating layer superimposed display and interactive control of multi-split screen video

Abstract

The application provides a kind of independent floating layer superimposed display and interactive control method of multi-screen video, it is related to image communication and video interaction field, including creating main window module, and the matrix module being set in main window module;Several independent video matrix unit modules are set in matrix module, and video matrix unit module creates video display module;When video matrix unit module receives preset play instruction, create floating layer module on video matrix unit module, create interactive control and superposition drawing component inside floating layer module;The screen position and size information of video display module are obtained, and the position and size of floating layer module are aligned with video display module respectively;After floating layer module receives user input event, it is judged whether the event trigger point falls in interactive control area;If yes, event is handled by floating layer module;If not, event is converted into event under parent control coordinate system and is forwarded to corresponding video matrix unit module.

Description

Technical Field

[0001] This invention relates to the field of image communication and video interaction, specifically a method for independent overlay display and interactive control of multi-screen video. Background Technology

[0002] Video surveillance platforms are a typical application in the field of image and video communication, typically comprising client proxy gateways, device access gateways, streaming media gateways, storage modules, and client applications. Industry users generally require client-side capabilities such as simultaneous display of multiple video streams, video walls, split-screen, rotation, and playback. Audio and video data are commonly transmitted using image transmission protocols such as RTSP, RTMP, HTTP-FLV, HLS, and WebSocket.

[0003] As the terminal carrier of digital image communication, video surveillance clients generally support functions such as multi-screen video preview and playback. They require overlaying interactive controls such as PTZ control, recording, and split-screen closure, as well as watermarks and status prompts, onto the video screen. Currently, video rendering mostly uses native windows or hardware acceleration paths, which inherently conflict with UI controls such as Qt in terms of window hierarchy, transparency compositing, and mouse event capture, thus limiting the interactive experience of image communication terminals.

[0004] Existing overlay solutions mainly fall into three categories: First, directly nesting UI controls within the parent window of the video area, which easily leads to problems such as controls being obscured by the video, flickering, asynchronous refresh, and click failure, requiring a large amount of adaptation code and resulting in high maintenance costs; Second, using full-window mouse penetration or a strongly placed transparent overlay, which easily leads to image communication terminal adaptation defects such as unresponsive controls, focus snatching, and poor compatibility; Third, overlaying OSD / controls within the rendering pipeline, which has a high degree of coupling with specific players, making it difficult to implement complex interactive controls and resulting in high expansion costs.

[0005] In summary, existing technologies struggle to balance overlay stability, interaction consistency, and cross-environment compatibility in image communication scenarios, and cannot meet the requirements for dynamic adaptation and reliable operation in multi-screen video communication scenarios. Summary of the Invention

[0006] To address the aforementioned technical problems, the present invention aims to provide a method for independent overlay display and interactive control of multi-screen video, which solves the problem that existing technologies are unable to simultaneously ensure overlay stability, interactive consistency, and cross-environment compatibility in image communication scenarios, and cannot meet the requirements for dynamic adaptation and reliable operation in multi-screen video communication scenarios.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for independent overlay display and interactive control of multi-screen video, the method comprising: Start the video monitoring client, create a main window module, and set a matrix module within the main window module; the matrix module contains several independent video matrix unit modules, and a video display module is created within the video matrix unit module. The video display module is used to decode and render the video image. When the video matrix unit module receives a preset playback command, a transparent background, borderless floating layer module is created above the video matrix unit module, and interactive controls and overlay drawing components are created inside the floating layer module. In response to a preset trigger event, obtain the screen position and size information of the corresponding video display module, and synchronize and align the position and size of the overlay module with the video display module. After receiving a user input event, the overlay module determines whether the event trigger point falls within the interactive control area. If so, the overlay module handles the event; otherwise, the event is converted to an event in the parent control's coordinate system and forwarded to the corresponding video matrix unit module. The display and hiding of the overlay module are controlled according to the video playback status, and its key display status is saved when the overlay module is hidden and restored when it is displayed.

[0008] Preferably, the specific process for creating the floating layer module is as follows: When a video matrix unit module receives a video playback command, it checks whether the current matrix unit module has created a floating layer module. If so, it directly uses the corresponding floating layer module. Conversely, an independent overlay module is created within the corresponding video matrix unit module, and the segment index, window flag, parent control, and playback mode parameters are passed in. Based on the created overlay module, configure the corresponding overlay window core properties and the corresponding internal child controls; Establish signal connections between the floating layer module and its corresponding matrix unit module, and send the operation signals of the interactive controls back to the video matrix unit module to execute the corresponding business logic.

[0009] Preferably, synchronizing the position of the overlay module with the video display module includes: When a preset position trigger event is detected, the matrix unit module obtains the local logical coordinates of the video display module within the parent control; the preset position trigger event includes: the video display module moving, the main window module moving, the completion of switching between full-screen and non-full-screen states, and the window resuming display; The video matrix unit module passes the local logical coordinates within the parent control to the floating layer module; the floating layer module stores the local logical coordinates in its own internal storage variable and triggers the position update state method to execute the coordinate transformation strategy and position synchronization operation; The coordinate transformation strategy is as follows: the floating layer module uses the matrix unit module as the reference coordinate system and calls the global coordinate transformation interface provided by the graphical interface framework to convert the local logical coordinates into screen global coordinates. The graphical interface framework automatically handles the differences in the coordinate reference offset and display resolution scaling of each screen in a multi-monitor scenario. Based on the global screen coordinates, the overlay module is moved to the global screen coordinates so that the position of the overlay module is aligned with the position of the video display module. Specifically, by pre-installing an event filter on the main window module to monitor the movement of the main window module, when the movement of the main window module is detected, the above-mentioned floating layer module position synchronization alignment operation is forcibly triggered.

[0010] Preferably, synchronizing the size of the overlay module with the video display module includes: When a preset size trigger event is detected, the matrix unit module uses a dual-path approach to capture the size change of the video display module, obtains the latest width and height values ​​of the video display module, and transmits the width and height values ​​to the floating layer module; the preset size trigger event includes changes in the control size of the video display module, grid deformation caused by switching the matrix split-screen layout, or grid size changes caused by switching between full-screen and non-full-screen. The floating layer module stores the received width and height values ​​into its own internal storage variables and triggers the size update state method to execute subsequent size synchronization operations. The size update state method first obtains the current size of the parent control as a fallback value. When the passed width or height value is less than or equal to 0, it uses the size of the parent control as the valid size and finally calls the fixed size method to fix the size of the floating layer module itself to the valid size. Based on its adjusted size, an absolute coordinate positioning strategy is used to trigger the rearrangement of the sub-controls inside the overlay module. The position update of all sub-controls is completed through a dual trigger path, and a differentiated anchoring strategy is adopted according to the different types of sub-controls to ensure that the visible area of ​​the overlay module and the video display module always remains completely overlapped.

[0011] Preferably, the matrix unit module employs a dual-path approach to capture the size changes of the video display module, including: When the size of the video display module changes, it transmits a size change signal. During the construction phase, the matrix unit module binds the signal to the corresponding response function. Inside the response function, the position update operation and size update operation of the floating layer module are triggered in sequence. The size update operation reads the latest width and height of the video display module and passes them to the floating layer module. The matrix unit module rewrites its own frame size change event. Whenever the matrix unit module is resized due to external layout adjustments, the event handler recalculates and sets the size and position of the video display module according to the current scaling strategy, and then triggers the size update operation of the overlay module again to synchronize the changes to the overlay module.

[0012] Preferably, after receiving a user input event, the floating layer module determines whether the event trigger point falls within the interactive control area, including: The floating layer module receives user input events, including mouse press events, mouse move events, mouse release events, scroll wheel events, and keyboard events. The overlay module calls its own sub-control hit detection method, and based on the sub-control hit detection method and the user input event, determines whether the event trigger point of the user input event falls on a certain sub-control within the interactive control area inside the overlay module; If so, the overlay module directly processes the corresponding sub-control based on the user input event; Conversely, the coordinates of the user input event are converted to coordinates in the coordinate system of the parent control, and a new input event of the same type as the user input event is constructed. The new input event is forwarded to the matrix unit module using a synchronous event sending method. Differentiated pass-through processing logic is used for different types of user input events. A preset hit determination strategy is used for boundary scenarios. The matrix unit module executes the corresponding video operation, which includes split-screen selection operation, split-screen drag operation, screen frame selection operation, and screen zoom operation.

[0013] Preferably, the display and hiding of the overlay module are controlled according to the video playback status, and its key display state is saved when the overlay module is hidden, and the key display state is restored when the display is restored, including: The matrix unit module uses dual gating logic to determine the visibility state of the floating layer module and monitors the video playback status in real time. When the video enters the effective playback state, it notifies the floating layer module to enter the working state; when the video exits the effective playback state, it notifies the floating layer module to hide it according to the preset hiding strategy. The overlay module employs differentiated display-hiding strategies for different types of internal interactive controls. The display and hiding of some controls are driven by the mouse position in real time, while others adopt a delayed hiding strategy. When the overlay module needs to be hidden, the overlay module actively records the logical visibility state of the internal interactive controls, without relying on the display state of the controls themselves for judgment; after the recording is completed, the interactive controls and the overlay module itself are hidden in sequence. When it is necessary to restore the display, a delay waiting mechanism is adopted to ensure that the interface layout is stable before performing the restoration operation. The floating layer module is displayed first, and position alignment and size alignment are forcibly performed. Then, the display state of the interactive control in the floating layer module is restored according to the recorded logical visibility state.

[0014] Preferably, the method further includes: When a user triggers a full-screen operation, the system enters full-screen mode. At this time, the video matrix unit module is marked as being in full-screen state, and if the overlay module has not yet been created, it is created immediately. The system sets the top-level properties for each window in a strict bottom-up order, sequentially completing the top-level placement of the matrix main window, the overlay window, and the independent child control windows inside the overlay, ensuring that the overlay is always displayed in front of the video screen. After the top placement is completed, the position and size of the overlay are forcibly synchronized. After the window flag is changed, the window is rebuilt and the layer is fine-tuned to ensure that the window properties take effect immediately and eliminates layer anomalies. When the user triggers the exit from fullscreen, the video matrix unit module cancels the fullscreen mark; it removes the system-level top-of-the-line attribute of each window in a strict top-down order, sequentially removing the top-of-the-line attribute of independent child control windows inside the floating layer, removing the top-of-the-line attribute of the floating layer window, and removing the top-of-the-line attribute of the matrix main window, restoring the window hierarchy relationship in normal mode; after the top-of-the-line removal is completed, the position and size of the floating layer are resynchronized. When switching between full-screen mode and non-full-screen mode, if a preset floating layer instability event is detected, the floating layer reconstruction mechanism is triggered. The preset floating layer instability events include changes in window flags and changes in parent-child relationships.

[0015] Preferably, the floating layer reconstruction mechanism includes: Perform pre-validation; if the overlay module does not exist, is not visible, or is not playing, return directly. Read the key overlay status of the current floating layer module, which includes at least the three-dimensional box selection switch status and / or the video recording prompt text display status; A secure destruction process of hiding first and then destroying is adopted. The old floating layer module is first made invisible, the old floating layer module is destroyed by the delayed destruction method, and the pointer pointing to the old floating layer module is immediately set to null. The old floating layer module automatically performs a self-cleaning operation during destruction, releasing all occupied system resources and severing its association with external objects; Recall the floating layer creation function and create a new floating layer module using the exact same construction parameters as the old floating layer module; The key overlay state read is written back to the new overlay module, and the new overlay module is displayed; depending on whether the current mode is full screen, the corresponding alignment strategy is selected, and the position and size of the new overlay module are synchronized with the video display module.

[0016] Compared with the prior art, the beneficial effects of the present invention are: This invention abandons the existing method of overlaying ordinary controls and video images on the same layer. Instead, it uses an independent, transparent, borderless overlay to carry interactive controls and overlay content. The overlay is independent of the video rendering channel and is not affected by the native video window hierarchy or hardware-accelerated compositing logic. This fundamentally eliminates problems such as video images obscuring controls, control flickering and tearing, and asynchronous refresh, ensuring that overlay content such as PTZ buttons, control bars, and watermarks are always displayed stably and clearly, significantly improving display reliability.

[0017] Existing technologies require writing numerous adaptation patches for different system versions, graphics card drivers, and player types, and are prone to compatibility issues in scenarios such as multi-screen, DPI, and full-screen switching. This invention decouples the overlay logic from the underlying video decoding and rendering chain, implementing overlay interactions uniformly in an independent floating layer. It does not rely on specific player window composition rules, nor does it require special adaptation for platform differences. It maintains consistent performance across different operating systems, graphics card drivers, and playback component environments, effectively reducing adaptation workload and subsequent maintenance costs.

[0018] Existing technologies suffer from interactive defects such as controls being visible but unclickable, controls becoming ineffective due to overlay penetration, and top-mounted windows preempting focus. This invention employs an intelligent interaction mechanism that allows control areas to respond and blank areas to transmit events. The overlay does not rely on system-level penetration or top-mounted attributes, ensuring that interactive controls such as PTZ control, recording, and the close button respond normally to clicks, while also allowing underlying video operations such as drag-and-drop split-screen, scroll wheel zoom, and shortcut keys to exit full-screen mode to transmit events through the blank areas of the overlay. This completely resolves interaction conflicts, achieving a smooth experience where controls are clickable and underlying operations are unaffected.

[0019] Existing technologies are prone to issues such as overlay misalignment, control position offset, and state loss when the main window is moved, split-screen layout is switched, or full-screen / non-full-screen is switched. This invention addresses these issues through a real-time position and size synchronization mechanism. In dynamic scenarios such as video area movement, main window dragging, split-screen switching, and full-screen entry / exit, it forcibly synchronizes the coordinates and size of the overlay with the video area, ensuring they always perfectly overlap. Simultaneously, it supports saving key states when the overlay is hidden and accurately restoring them when restored, effectively avoiding misalignment, deformation, and state inconsistencies in dynamic scenarios, and improving operational continuity in multi-split-screen scenarios.

[0020] Existing technologies bind the overlay logic to a specific renderer, requiring modifications to the underlying rendering code to add new interactive controls and overlay styles, resulting in high expansion costs. This invention employs a modular, independent overlay design, centralizing all interactive controls and overlay drawing content within the overlay module, decoupling it from the core video playback logic. Subsequent additions of control buttons, watermark styles, and interactive functions only require expansion within the overlay module, without altering the core logic of video decoding, rendering, and matrix scheduling. This enhances reusability and significantly reduces feature iteration costs. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0022] Figure 1 This is a schematic diagram of a method for displaying and interactively controlling independent overlays of multi-screen videos. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] Example 1 like Figure 1 As shown in the figure, this embodiment discloses a method for independent overlay display and interactive control of multi-screen videos, the method including: Start the video monitoring client, create a main window module, and set a matrix module within the main window module; the matrix module contains several independent video matrix unit modules, and a video display module is created within the video matrix unit module. The video display module is used to decode and render the video image. In detail, the main window module (TopLevelWindow) carries the entire complete client interface, including menus, layout controls, video matrix areas, etc. It is responsible for listening to window movement and scaling operations, and listening to the window Move event through event filters as the trigger event source for updating the floating layer position.

[0026] The VideoMatrix module centrally manages the creation, destruction, and split-screen layout switching of multiple video matrix units, supporting common multi-split-screen modes such as 1 / 4 / 9 / 16. It issues play, stop, and full-screen switching commands to subordinate matrix units, enabling unified scheduling of multiple video streams.

[0027] The video matrix unit module (MatrixUnit) exists independently, grid by grid. It is the smallest business unit of the video matrix, with each video stream corresponding to an independent MatrixUnit instance. Internally, it integrates two core sub-modules: VideoPlay: The actual area carrying the video image, supporting various rendering methods such as native windows and hardware rendering surfaces. As the output carrier of the video image, it completes the decoding, rendering and real-time display of the video stream.

[0028] VideoOverlay: An independent window / interface component that overlays the corresponding video area, carrying all interactive operations and overlay drawing functions. It uses a transparent background, no border, and can be placed on top, internally containing a full set of interactive controls and overlay drawing content. Interactive controls: Bottom control bar (including function buttons for play, record, intercom, sound, 3D, PTZ, full screen, etc.), PTZ control controls, preset controls, close button, and prompt labels; Overlay drawing content: video status text, watermark logo, 3D selection rectangle and other visual information.

[0029] Key structural relationships: The video image is independently output and displayed by the VideoPlay module; all overlaid display content and interactive operations are uniformly implemented by the VideoOverlay module; the MatrixUnit module, as the core coordination unit, is responsible for accurately aligning the position and size of the VideoOverlay module to the actual display area of ​​the VideoPlay module's screen, ensuring that the two are completely overlapped.

[0030] When the video matrix unit module receives a preset playback command, a transparent background, borderless floating layer module is created above the video matrix unit module, and interactive controls and overlay drawing components are created inside the floating layer module. In this embodiment, the specific process for creating the floating layer module is as follows: When a video matrix unit module receives a video playback command, it checks whether the current matrix unit module has created a floating layer module. If so, it directly uses the corresponding floating layer module. Conversely, an independent overlay module is created within the corresponding video matrix unit module, and the segment index, window flag, parent control, and playback mode parameters are passed in. Based on the created overlay module, configure the corresponding overlay window core properties and the corresponding internal child controls; Establish signal connections between the floating layer module and its corresponding matrix unit module, and send the operation signals of the interactive controls back to the video matrix unit module to execute the corresponding business logic.

[0031] In detail, the timing of the overlay module creation is as follows: In each MatrixUnit unit, when the business needs to enable the overlay capability (e.g., entering video preview playback state, user actively triggering the display of toolbar / pan-tilt control, etc.), the overlay initialization operation is immediately executed: if the current unit has not created an overlay instance, the SetUpVideoOverlay(parent, isPlaybackMode) method is called to create the overlay; after creation, the position and size synchronization operation between the overlay and the video area is immediately executed.

[0032] Specifically, it was created as follows: Instantiate the overlay window, create independent VideoOverlay objects according to the video segments, and pass in the segment index, window flag, parent control and playback mode parameters; Configure core window properties, set a borderless style to prevent the system title bar from obscuring the video screen; set a transparent background to ensure the underlying video screen can be seen through normally, only displaying the control area; enable layered transparent compositing capability on the Windows platform to ensure the stability of overlay display; Create internal sub-controls, such as a close button, bottom control bar, PTZ control, preset position control, and prompt label, inside the floating layer. These sub-controls are hidden by default and displayed as needed according to business requirements. Establish a signal connection and uniformly transmit signals such as bottom bar button operations, close button clicks, PTZ control commands, and preset position queries back to MatrixUnit. MatrixUnit then interfaces with the upper-layer business logic to execute operations such as PTZ control, full-screen switching, and recording initiation.

[0033] The floating layer forms an independent overlay display layer, no longer relying on the hierarchical relationship between ordinary child controls and the video rendering surface, thus eliminating the problems of controls being obscured, screen flickering, and inconsistent layers from the root.

[0034] In response to a preset trigger event, obtain the screen position and size information of the corresponding video display module, and synchronize and align the position and size of the overlay module with the video display module. In this embodiment, synchronizing the position of the overlay module with the video display module includes: When a preset position trigger event is detected, the matrix unit module obtains the local logical coordinates of the video display module within the parent control; the preset position trigger event includes: the video display module moving, the main window module moving, the completion of switching between full-screen and non-full-screen states, and the window resuming display; The video matrix unit module passes the local logical coordinates within the parent control to the floating layer module; the floating layer module stores the local logical coordinates in its own internal storage variable and triggers the position update state method to execute the coordinate transformation strategy and position synchronization operation; The coordinate transformation strategy is as follows: the floating layer module uses the matrix unit module as the reference coordinate system and calls the global coordinate transformation interface provided by the graphical interface framework to convert the local logical coordinates into screen global coordinates. The graphical interface framework automatically handles the differences in the coordinate reference offset and display resolution scaling of each screen in a multi-monitor scenario. Based on the global screen coordinates, the overlay module is moved to the global screen coordinates so that the position of the overlay module is aligned with the position of the video display module. Specifically, an event filter is pre-installed on the main window module to monitor its movement. When movement is detected, the aforementioned floating layer module's position synchronization and alignment operation is forcibly triggered. The complete installation and processing logic of the event filter is as follows: During the construction phase of the floating layer module, the parent control is traced back to the top-level window and saved as an internal member variable. The floating layer module itself is then registered as the event filter for that top-level window. In the overridden event filtering method, the floating layer module's position update state method is triggered to re-execute coordinate transformation and position synchronization only when the monitored object is the saved top-level window and the event type is a movement event. All other events are allowed by calling the base class event filter. During the destruction phase of the floating layer module, the event filter for the top-level window is removed to prevent the risk of dangling pointers. The floating layer module does not debouncing or throttling the main window's movement events, nor does it introduce timers for delayed merging. It immediately executes the complete position update process each time a movement event triggered by the operating system arrives, relying on the underlying event loop mechanism of the graphical interface framework and the operating system's natural merging ability for window movement events to ensure synchronization performance.

[0035] Detailed Implementation method: MatrixUnit reads the local logical coordinates (x, y) of VideoPlay in the parent control (MatrixUnit), calls VideoOverlay.UpdatePosition(x, y) to store the coordinates into member variables (m_nX, m_nY), and then triggers UpdatePositionStatus to perform coordinate transformation and position synchronization. The coordinate transformation strategy is as follows: using MatrixUnit (i.e., the parentWidget of the overlay) as the reference coordinate system, the Qt framework's QWidget::mapToGlobal(QPoint(m_nX, m_nY)) interface is called to convert the local coordinates in the MatrixUnit coordinate system into screen global coordinates (Tx_, Ty_). The Qt framework automatically handles the coordinate base offset and DPI scaling differences of each screen in a multi-monitor scenario, and finally calls move(Tx_, Ty_) to accurately position the overlay to the target screen position. The multi-screen / boundary adaptation strategy is as follows: In the size alignment method Resize(w,h), the available geometric area width TotalScreenW and height CurrentScreenH of the main screen are obtained through QGuiApplication::primaryScreen(), and QGuiApplication::screens() is queried to determine whether it is a multi-screen environment. On Windows, `setFixedSize(w,h)` is called directly without any clipping. In Linux single-screen environments, if the converted global coordinates `Tx_` < 0, the effective width is clipped to `w - (0 - Tx_)`; if `Ty_` < 0, the effective height is clipped to `h - (0 - Ty_)`; if `Tx_ + w` ≥ `TotalScreenW`, the effective width is clipped to `w - (Tx_ + w - `TotalScreenW`); if `Ty_ + h` ≥ `CurrentScreenH`, the effective height is clipped to `h - (Ty_ + h - `CurrentScreenH`). In Linux multi-screen environments, no clipping is performed, and the behavior is consistent with Windows. Specifically, the key implementation is the complete logic of the main window event filter: During the construction phase of the overlay (NP_VideoOverlay) managed by each MatrixUnit, the parent control is traced upwards to the top-level window parent->window() and saved as a member variable m_pTopLevelWindow. Then, m_pTopLevelWindow->installEventFilter(this) is called to register the overlay itself as the event filter of the top-level window. In the overridden eventFilter(QObject*watched,QEvent*event) method, the event is triggered only when the watched object is equal to the saved top-level window and the event type is QEvent::Move. The event handling branch calls the base class QWidget::eventFilter to allow all other events to pass. The Move event handling branch directly calls the UpdatePositionStatus() method. This method uses the stored local coordinates (m_nX, m_nY) within the MatrixUnit to re-execute the MapToGlobal coordinate transformation, and applies the updated global coordinates (Tx_, Ty_) to the overlay window in real time through move(Tx_, Ty_), thus ensuring that the overlay always follows synchronously when the main window is dragged to any screen position by the user. When the overlay is destroyed, the event filter is removed by calling m_pTopLevelWindow->removeEventFilter(this) in the destructor function to prevent dangling pointers. Frequency control of floating layer position update: This solution does not debounce or throttle the Move event, and does not introduce timer delay merging. Instead, it immediately executes a complete position update process (MapToGlobal + move) when the QEvent::Move event triggered by the operating system arrives. It relies on the natural merging mechanism of Qt's underlying event loop and the operating system for window movement events to avoid introducing additional delays that may cause floating layer ghosting or misalignment.

[0036] Synchronizing and aligning the size of the overlay module with the video display module includes: When a preset size trigger event is detected, the matrix unit module uses a dual-path approach to capture the size change of the video display module, obtains the latest width and height values ​​of the video display module, and transmits the width and height values ​​to the floating layer module; the preset size trigger event includes changes in the control size of the video display module, grid deformation caused by switching the matrix split-screen layout, or grid size changes caused by switching between full-screen and non-full-screen. The floating layer module stores the received width and height values ​​into its own internal storage variables and triggers the size update state method to execute subsequent size synchronization operations. The size update state method first obtains the current size of the parent control as a fallback value. When the passed width or height value is less than or equal to 0, it uses the size of the parent control as the valid size and finally calls the fixed size method to fix the size of the floating layer module itself to the valid size. Based on its adjusted size, an absolute coordinate positioning strategy is used to trigger the rearrangement of the sub-controls inside the overlay module. The position update of all sub-controls is completed through a dual trigger path, and a differentiated anchoring strategy is adopted according to the different types of sub-controls to ensure that the visible area of ​​the overlay module and the video display module always remains completely overlapped.

[0037] The matrix unit module employs a dual-path capture method to capture the size changes of the video display module, including: When the size of the video display module changes, it transmits a size change signal. During the construction phase, the matrix unit module binds the signal to the corresponding response function. Inside the response function, the position update operation and size update operation of the floating layer module are triggered in sequence. The size update operation reads the latest width and height of the video display module and passes them to the floating layer module. The matrix unit module overrides its own frame size change event. Whenever the matrix unit module is resized due to external layout adjustments, the event handler recalculates and sets the size and position of the video display module according to the current scaling strategy, and then triggers the size update operation of the overlay module again, synchronizing the changes to the overlay module. Specifically, the dual-trigger path for updating the position of all child controls includes: after the overlay module's fixed-size method is executed, the graphical interface frame automatically triggers the overlay module's size change event. In this event handler, the position movement method or geometry setting method of each child control is called to complete the repositioning; when the overlay module moves due to global coordinate changes, the overlay module's position change event is triggered synchronously. In this event handler, the same coordinate recalculation is performed on all child controls except for tooltip-type child controls, ensuring that the overlay position change does not cause relative drift of child controls; after the child control layout is completed, the controls that need to be placed on top are placed on top to control the display hierarchy.

[0038] Different anchoring strategies are adopted according to the different types of child controls: Top-mounted control: anchored to the upper right corner of the floating layer module, using the first preset horizontal offset and the first preset vertical offset, and the top-mounted operation is performed after the layout is completed; Bottom function bar: Anchored to the bottom of the floating layer module and fully horizontally filled, always occupying a fixed-height horizontal strip at the very bottom of the floating layer module; Prompt-type sub-controls: These controls only participate in repositioning when they are in the display state. They are horizontally centered and anchored, with a second preset vertical offset from the top edge of the overlay module. These controls are not repositioned in the overlay module's position change event. Collapsible functional control: Anchors the lower right area of ​​the floating layer module, using the second preset horizontal offset, with the vertical coordinate as an adaptive variable, dynamically selecting the anchoring reference based on the visibility of the bottom function bar; refreshes its own size before each repositioning to ensure accurate coordinate calculation; Independent window control: As an independent top-level window, it is not affected by changes in the size of the overlay module. After converting the third preset offset in the coordinate system of the overlay module into global screen coordinates through the global coordinate transformation interface, it performs a position movement operation so that it follows the position of the overlay module as an independent window.

[0039] In detail, the method for monitoring size changes is as follows: a dual path of signal-driven and event-driven approaches is used to capture changes in the size of VideoPlayer.

[0040] Path 1 (Signal Driven): When VideoPlay changes its size, it emits the sgnResize() signal. During the construction phase, MatrixUnit binds this signal to the slot function OnVideoPlayResizeSlot() via connect. Inside the slot function, updateOverlayPosition() and updateOverlaySize() are called in sequence. The latest dimensions of m_pVideoPlay->width() and m_pVideoPlay->height() are read by updateOverlaySize() and passed to the overlay.

[0041] Path Two (Event-Driven): MatrixUnit overrides QFrame::resizeEvent(). Whenever MatrixUnit itself is resized due to layout changes, the event handler calls the DoSwitchScal() method. This method internally recalculates and sets the size and position of VideoPlay based on the current scaling strategy, and then calls updateOverlaySize() again to synchronize the changes to the overlay. These two paths form a double guarantee, ensuring that the overlay can promptly detect and respond to any size changes, whether they originate from VideoPlay itself or from external layout adjustments.

[0042] Implementation method: After MatrixUnit obtains the latest width and height (w, h) of VideoPlay via the aforementioned listening path, it calls VideoOverlay.UpdateSize(w, h) to store (w, h) into the overlay's member variable (m_nWidth, m_nHeight). Subsequently, it triggers the UpdateSizeStatus() method to perform size synchronization. UpdateSizeStatus() obtains the current size of the parent control as a fallback value (the parent control's size is used when the passed w or h ≤ 0), and finally calls the internal Resize(w, h) method to fix the overlay's own size using setFixedSize(w, h). After setFixedSize is completed, the Qt framework automatically triggers resizeEvent, in which the overlay performs absolute coordinate layout for each child control one by one.

[0043] The adaptive layout strategy for child controls within the overlay: This solution does not rely on Qt's layout managers (such as QGridLayout, QVBoxLayout, etc.), but instead adopts a purely manual absolute coordinate positioning strategy. Its core idea is to use the overlay's current width and height as the sole reference point. The position and size of each child control are calculated in real-time using a preset anchor offset formula, without saving any intermediate layout states. There are two triggering paths for layout updates: the primary path is `resizeEvent`, which is automatically triggered by Qt when `setFixedSize` changes the overlay's size. In this event handler, `move()` or `setGeometry()` is called on each child control to complete repositioning. The secondary path is `moveEvent`, which is triggered synchronously when the overlay window moves due to global coordinate changes. The same coordinate recalculation is performed on all child controls except for the tooltip label, ensuring that changes in the overlay's position do not cause relative drift in the child controls. After the child controls are laid out, `raise()` is called on controls that need to be placed on top (such as the close button) to control the Z-order.

[0044] The specific anchoring formula is as follows: Close button: Anchored to the top right corner of the floating layer, horizontal offset constant = 10px, vertical offset constant = 10px, x = width() - button->width() - 10, y = 10, raise() after layout; Bottom control bar (NP_UnitBottomCaptionBar): Anchored to the bottom of the floating layer and fully horizontally filled, x=0, y=height()-NP_UNITBOTTOMCAPTIONBAR_HEIGHT, width=width(), so that the control bar always occupies a fixed height horizontal strip at the bottom of the floating layer. Tip label (m_tipLabel, only participates in repositioning when isVisible() is true): horizontally centered and anchored, x=(width()-label->width()) / 2, y=50 (fixed 50px from the top edge), this control is not repositioned in moveEvent; PTZ gimbal control: Anchored to the lower right area of ​​the floating layer, x = width() - ptzWidget->width() - 10 (right-aligned, 10px from the right). The y-coordinate is an adaptive variable, dynamically selecting the anchor base by querying the visibility of the bottom control bar. When visible, y = height() - NP_UNITBOTTOMCAPTIONBAR_HEIGHT - ptzWidget->height() - 5 (5px above the bottom bar); when invisible, y = height() - ptzWidget->height() - 10 (anchored 10px above the bottom of the floating layer). Before each repositioning, ptzWidget->adjustSize() is called to refresh the control's size to ensure accurate coordinate calculation. The PresetControlToolWidget (an independent top-level window inherited from QDialog, unaffected by the setFixedSize of the floating layer): After converting the fixed offset in the floating layer's coordinate system to global screen coordinates using mapToGlobal(QPoint(10,36)), move() is called to make it an independent window that follows the floating layer's position. The offset is 10px to the right and 36px down relative to the top left corner of the floating layer.

[0045] After receiving a user input event, the overlay module determines whether the event trigger point falls within the interactive control area. If so, the overlay module handles the event; otherwise, the event is converted to an event in the parent control's coordinate system and forwarded to the corresponding video matrix unit module. In this embodiment, after receiving a user input event, the floating layer module determines whether the event trigger point falls within the interactive control area, including: The floating layer module receives user input events, including mouse press events, mouse move events, mouse release events, scroll wheel events, and keyboard events. The overlay module calls its own sub-control hit detection method, and based on the sub-control hit detection method and the user input event, determines whether the event trigger point of the user input event falls on a certain sub-control within the interactive control area inside the overlay module; If so, the overlay module directly processes the corresponding sub-control based on the user input event; Conversely, the coordinates of the user input event are converted to coordinates in the coordinate system of the parent control, and a new input event of the same type as the user input event is constructed. The new input event is forwarded to the matrix unit module using a synchronous event sending method. Differentiated pass-through processing logic is applied to different types of user input events, and a preset hit determination strategy is used for boundary scenarios. The matrix unit module then executes the corresponding video operations, including split-screen selection, split-screen dragging, screen selection, and screen zooming. The child control hit detection method adopts a pure rectangular geometry hit test mechanism, which receives a local coordinate relative to the floating window and returns the topmost visible direct child control at that coordinate position; if there is no visible child control at that position, it returns a null value.

[0046] In detail, this solution does not use system-level full-window mouse pass-through properties (such as Windows' WS_EX_TRANSPARENT), but instead uses a two-layer event dispatch mechanism to implement an intelligent event routing strategy that handles the event when it hits a control by floating over the layer and passes through the parent control when it hits a blank area.

[0047] Terminology Explanation: childAt(pos): A member method of the QWidget class in the Qt framework. It takes a QPoint with local coordinates relative to the current window and returns a pointer to the topmost direct child control at that coordinate position; if there are no visible child controls at that position, it returns nullptr.

[0048] QCoreApplication::sendEvent(receiver, event): The static synchronous event dispatch method of the Qt framework, which directly sends the event to the event handler function of the specified receiving object and immediately returns the processing result (boolean value); unlike the asynchronous queue delivery of postEvent, sendEvent guarantees that the event is processed synchronously at the calling point.

[0049] `mapFromGlobal` and `mapToGlobal` are coordinate transformation methods in the Qt framework. `mapToGlobal` converts a point in the current window's coordinate system to screen-global coordinates; `mapFromGlobal` converts screen-global coordinates to a point in the current window's coordinate system. Using them together allows for precise transformations across window coordinate systems.

[0050] The complete forwarding logic for various mouse events is as follows.

[0051] Mouse press event (mousePressEvent) forwarding process: Call childAt(e->pos()) to get the pointer to the direct child control at the current mouse local coordinates; If childAt returns nullptr (empty area), execute the following in sequence: coordinate transformation: parentWidget()->mapFromGlobal(mapToGlobal(e->pos())), which converts the local coordinates of the floating layer to parentPos in the MatrixUnit coordinate system after being transformed through the global coordinate system; Event Construction: A new QMouseEvent object, parentEvent, is constructed using the original event type e->type(), the transformed local coordinates parentPos, the original global coordinates e->globalPos(), the trigger button e->button(), all currently pressed buttons e->buttons(), and keyboard modifiers e->modifiers() as parameters; Synchronous Forwarding: QCoreApplication::sendEvent(parentWidget(),&parentEvent) is called, and the MatrixUnit responds to the press operation (typically used to start dragging or 3D selection); Returning early prevents the base class's mousePressEvent from being called again, thus avoiding the floating layer itself handling the event additionally. If childAt returns a non-null pointer (hitting a child control), QWidget::mousePressEvent(e) is called, and the Qt default mechanism dispatches the press event to the corresponding child control (such as a button press state change, PTZ arrow key start of continuous movement command, etc.).

[0052] Mouse release event (mouseReleaseEvent) forwarding process: The logic is completely symmetrical to `mousePressEvent`: `childAt` checks for null → coordinate transformation (same formula as above) → constructs a `QMouseEvent` containing `e->type()`, `parentPos`, `e->globalPos()`, `e->button()`, `e->buttons()`, and `e->modifiers()` → `sendEvent` synchronously forwards the event → returns early; if a child control is hit, the base class's release event is called. After receiving the release event, `MatrixUnit` completes operations such as determining the drag landing point and confirming the 3D selection rectangle.

[0053] Mouse move event (mouseMoveEvent) forwarding process: A key difference from press / release events is that when `childAt` returns `nullptr` and `parentWidget` exists, coordinate transformation and `sendEvent` pass-through are still performed. However, after pass-through, the function does not return prematurely; instead, `QWidget::mouseMoveEvent(e)` continues to execute. That is, while the movement event is passed through to `MatrixUnit` in an empty area, the floating layer itself maintains the base class handling of the movement event to support Qt's internal maintenance of mouse tracking (`setMouseTracking`) related states. The newly constructed `QMouseEvent` is passed `e->button()` and `e->buttons()` parameters to maintain the drag state (e.g., when dragging with the left mouse button pressed, `buttons` includes `Qt::LeftButton`), allowing `MatrixUnit` to distinguish between hover movement and drag movement based on the state of `buttons`.

[0054] Mouse wheel event (wheelEvent) forwarding process: The overridden `wheelEvent` does not perform `childAt` hit checks, and performs a complete pass-through for all wheel events without calling the base class `wheelEvent`. The pass-through steps are as follows: Coordinate transformation: `parentWidget()->mapFromGlobal(mapToGlobal(e->pos()))` obtains `parentPos`; Event construction: A new `QWheelEvent` object is constructed with `parentPos`, `e->globalPos()`, `e->pixelDelta()` (pixel-level wheel increment), `e->angleDelta()` (angle-level wheel increment, usually a multiple of ±120), `e->delta()` (wheel increment compatible with older versions of Qt), `e->orientation()` (wheel direction, vertical / horizontal), `e->buttons()`, and `e->modifiers()` as parameters; Synchronous forwarding: QCoreApplication::sendEvent(parentWidget,&newEvent), which in turn sends the sgnWheelEvent signal from MatrixUnit's wheelEvent to VideoPlay, allowing VideoPlay to adjust the electronic scaling of the video frame accordingly.

[0055] The second layer is the general event virtual function (event) forwarding process: In the overridden virtual function `event(QEvent*e)`, a unified `childAt` check is performed on the four types: `QEvent::MouseButtonPress`, `QEvent::MouseButtonRelease`, `QEvent::MouseMove`, and `QEvent::MouseButtonDblClick`. If a child control is hit, `QWidget::event(e)` is returned to enter the normal event handling chain. If an empty area is hit, `e->ignore()` is called to mark that the event is not accepted by the current window and `false` is returned, notifying the Qt event system to continue probing upwards along the parent control chain to handle the event. This layer serves as a backup guarantee for the first-level specific event handler, ensuring that even if the specific event virtual function is not overridden or is not fully processed, events in empty areas can still correctly penetrate the overlay.

[0056] Keyboard event (keyPressEvent) forwarding process: The overridden virtual function of keyPressEvent in the floating layer only performs a step-by-step forwarding of the Esc key (Qt::Key_Escape) up the parent widget chain: starting from parentWidget(), it iterates through each parent widget, constructing a QKeyEvent with the same parameters (QEvent::KeyPress, e->key(), e->modifiers(), e->text()), and calling QCoreApplication::sendEvent(parent, &newEvent) to send it synchronously; if a certain level's sendEvent returns true (indicating that the target has received and processed the Esc key), it immediately returns and stops traversing; if it reaches the top level and still has not processed, the key event is discarded. Other keys execute normally, with QWidget::keyPressEvent(e) handled by the base class. Event judgment logic for boundary cases: The childAt(pos) method relied upon by this solution employs a pure rectangular geometry hit test mechanism, rather than a pixel-level hit test. Therefore, a predetermined behavior strategy exists in the following boundary scenarios: In scenarios where child controls overlap: `childAt` returns the visible direct child control with the highest Z-order at the current coordinates. Because the overlay actively calls `raise()` on the close button in `resizeEvent` and `moveEvent` to increase its Z-order, it ensures the close button is always above all child controls, preventing it from being obscured by controls like the bottom control bar and causing it to become unclickable. The bottom control bar fills the entire row at the bottom of the overlay, the PTZ control is at the bottom right corner, and the preset control is offset at the top left corner. Under normal split-screen sizes, these three rectangular areas do not overlap. If overlap occurs in extremely small-sized scenarios, the control that performs the layout refresh later (or is raised later) receives higher hit priority.

[0057] In scenarios involving transparent areas of child controls: `childAt` uses the bounding box (QRect) of the control as its hit detection scope and does not check the transparency (alpha channel) of specific pixels within the control. Therefore, if the user hovers the mouse over or clicks on a transparent area within the rectangular region of a child control (such as the blank area without buttons at both ends of the bottom control bar), `childAt` will still return the pointer to that child control, and the event will not be propagated to the underlying MatrixUnit. This behavior contradicts the design intent of the bottom control bar. Figure 1 Therefore, when the bottom control bar fills the entire row, mouse movement within its control area will trigger the control bar's display hold logic, rather than accidentally penetrating into the video area.

[0058] Indirect child control scenarios: `childAt` only iterates through the list of direct child controls of the current overlay window, without recursively querying indirect child controls (i.e., child controls of child controls). For indirect child controls such as buttons inside the bottom control bar and arrow keys inside the PTZ control, their click events are automatically passed down to the corresponding direct child controls for processing through Qt's default event dispatch chain, without the need for explicit intervention in the overlay code.

[0059] In the case of invisible controls: childAt automatically excludes child controls where isVisible() is false. Therefore, when the bottom control bar is not displayed (the bottom bar is hidden), the mouse cursor will be considered to have hit a blank area in the bottom region and will be passed through to MatrixUnit, which is exactly the expected interactive behavior.

[0060] Disabled control scenario: The hit determination of `childAt` does not depend on the enabled / disabled state of the control. Even if a child control is disabled by `setEnabled(false)`, `childAt` will still return its pointer. In actual interaction, disabled controls cannot respond to mouse events due to Qt's own enabled check, but the event will not be passed through (because `childAt` returns a non-null value), and the event will be silently consumed within the overlay. In this solution, the enabled state of each control is controlled according to business logic (e.g., the button is grayed out when playback stops), without introducing additional mapping logic from enabled to pass-through.

[0061] Downstream processing chain after the pass-through event reaches MatrixUnit: Mouse / scroll wheel events transmitted from the overlay to the MatrixUnit are then relayed by the MatrixUnit to trigger specific video operations. The complete chain for each operation type is as follows.

[0062] Split-screen selection and drag preparation, left-click to press the pass-through link: The left-click event in the blank area of ​​the floating layer reaches MatrixUnit via sendEvent, triggering MatrixUnit::mousePressEvent. The function performs the following steps: Records event->pos() as the point coordinate m_pMousePress, serving as the reference point for subsequent drag calculations; converts the point coordinates to relative coordinates within VideoPlay (NP_VideoInside), and calls m_pVideoPlay->SetMouseRelease(videoInsidePos, isInsideVideoPlay) to synchronize the position information to the video playback component for setting the starting point of 3D selection operations; sets the drag flag m_bLButtonDown=true to indicate that it is currently in a drag-ready state; emits the LButtonClicked(GetIndex()) signal, which is forwarded to the upper-layer business module via NP_VideoMatrix, making the current split-screen cell selected (highlighted with a blue selection border); emits the LButtonClickedPoint(GetIndex(),QPoint(x,y)) and sgnMousePress(m_nIndex,x,y) signals to transmit precise click coordinates for use by upper-layer right-click menu positioning and other functions.

[0063] Video zooming, scroll wheel event pass-through link: After the floating layer `wheelEvent` passes `QWheelEvent` to `MatrixUnit`, it triggers `MatrixUnit::wheelEvent`. This function extracts the local coordinates `x / y`, the scroll wheel increment `delta` (`e->delta()`), and the current width and height of `MatrixUnit` as parameters from the event object, and emits the `sgnWheelEvent(x,y,delta,width,height)` signal. During the `MatrixUnit` construction phase (constructor), this signal is bound to the `NP_VideoInside::onWheelEvent(int,int,int,int,int)` slot function via `connect`. Within the slot function, `VideoInside` adjusts the display scaling of the video rendering surface according to the sign and absolute value of `delta`. A positive `delta` (scrolling the wheel up) zooms in, and a negative `delta` (scrolling the wheel down) zooms out, thus allowing the user to directly control the electronic scaling of the video using the scroll wheel in the floating layer's blank area.

[0064] Visual cleanup during drag-and-drop, mouse movement pass-through link: When a mouse movement event occurs in the blank area of ​​the overlay, it is passed through to the MatrixUnit, triggering MatrixUnit::mouseMoveEvent. The main action of this function is to check if the overlay is visible. If visible, it calls m_pVideoOverlay->hideCloseButton() and m_pVideoOverlay->hideBottomControlBar() to hide the close button and bottom control bar on the overlay. This prevents the overlay controls from obscuring the video frame when the user performs dragging or selection operations, ensuring a clean and unobstructed view of the dragging path. This event then continues to propagate upwards along the base class QFrame::mouseMoveEvent.

[0065] After dragging and dropping ends and the 3D selection rectangle is calculated, the downstream link is released when the mouse is released: MatrixUnit itself does not override mouseReleaseEvent (it is handled by default by the base class QFrame), but NP_VideoInside in MatrixUnit handles the release logic through its own mouse signal chain: the sgnMouseRelease(int,int) signal of NP_VideoInside is connected to the OnMouseReleaseSlot(int,int) slot of MatrixUnit during the construction phase. The slot function checks the m_bLButtonDown flag. If it is true, it calculates the rectangular region between the press point m_pMousePress and the release point m_pMouseRelease, including the rectangle length nLengthX / nLengthY and the midpoint coordinates nMidPointX / nMidPointY. It then determines the zoom direction bZoomFlag based on the relative positions of the press and release points (zooming in from left to right and zooming out from right to left). Finally, it emits the sgnDragZoom(m_nIndex,width(),height(),nMidPointX,nMidPointY,nLengthX,nLengthY,bZoomFlag) signal, which is then used by the upper-layer video matrix module to perform 3D electronic zoom operation based on these rectangular parameters.

[0066] In summary, the event chain from the overlay to MatrixUnit forms a closed loop: Press → Select the split-screen grid and mark the drag start point; Scroll wheel → Drive video image scaling; Move → Clear overlay controls to ensure unobstructed view; Release → Calculate the selected rectangle and perform 3D zoom. In each step, MatrixUnit uses a signal mechanism to propagate the operation intent upwards level by level, maintaining the decoupling between the overlay and video operation logic.

[0067] The display and hiding of the overlay module are controlled according to the video playback status, and its key display status is saved when the overlay module is hidden and restored when it is displayed.

[0068] In this embodiment, the display and hiding of the overlay module are controlled according to the video playback status, and its key display state is saved when the overlay module is hidden, and the key display state is restored when the display is restored, including: The matrix unit module uses dual gating logic to determine the visibility state of the floating layer module and monitors the video playback status in real time. When the video enters the effective playback state, it notifies the floating layer module to enter the working state; when the video exits the effective playback state, it notifies the floating layer module to hide it according to the preset hiding strategy. The overlay module employs differentiated display-hiding strategies for different types of internal interactive controls. The display and hiding of some controls are driven by the mouse position in real time, while others adopt a delayed hiding strategy. When the overlay module needs to be hidden, the overlay module actively records the logical visibility state of the internal interactive controls, without relying on the display state of the controls themselves for judgment; after the recording is completed, the interactive controls and the overlay module itself are hidden in sequence. When display needs to be restored, a delayed waiting mechanism is used to ensure the interface layout is stable before performing the restoration operation. The floating layer module is displayed first, and position and size alignment are forcibly performed. Then, the display state of the interactive controls in the floating layer module is restored according to the recorded logical visibility state. Among them, the matrix unit module uses dual gating logic to determine the visibility state of the floating layer module, including: the first primary gating: the matrix unit module maintains a video playback status flag, which is set by the external business layer after the video stream is ready and reset when the video stops or is abnormally disconnected; this flag applies to all paths that trigger the display of the floating layer, such as the mouse entering the video area, the mouse moving in the video area, and external requests to display the floating layer. When the video is not playing, all floating layer display requests are ignored. The second layer of gating is maintained: the overlay module defines an internal working status determination method. When any of the following conditions are met, such as the 3D selection rectangle being enabled, the recording prompt text being non-empty, or the watermark list being non-empty, the overlay is determined to be in a working state. When an external request is made to hide the overlay but the overlay is in a working state, the hiding request is suppressed, and the overlay remains displayed to ensure that key overlay information remains visible.

[0069] The differentiated display / hiding strategies for different types of internal interactive controls include: the bottom control bar and the close button do not use a delay timer and are driven entirely by the mouse position in real time; the matrix unit module determines in real time whether the mouse is in the bottom control bar area based on the mouse's coordinates relative to the video display module; if it is in the area, the bottom bar and the close button are displayed immediately, and if it leaves, they are hidden immediately; the prompt labels use a delayed hiding strategy, automatically hiding the prompt labels after a preset timeout period by triggering a timer once; the PTZ control controls are actively controlled by the user and are not affected by the real-time driving of the mouse position.

[0070] Detailed criteria for determining the effective playback status of a video: Internally, MatrixUnit maintains a boolean flag, m_bIsVideoPlaying, as the primary criterion for determining whether the overlay is visible or hidden. This flag is set by the external business layer when video streaming and decoding are ready, by calling SetVideoPlaying(true), and reset by calling SetVideoPlaying(false) when the video stops or experiences an abnormal interruption. Internally, SetVideoPlaying synchronously calls m_pVideoOverlay->setWorking(bPlaying) to pass the playback status to the overlay, which then updates the state of the play button (playing / paused) in the bottom control bar accordingly. This flag acts as the first gating mechanism, affecting the following critical paths: OnMouseMoveSlot (when the mouse moves within the video area): If m_bIsVideoPlaying is false and the overlay is already displayed, immediately call hideAllControls() and SetVisible(false) to force the overlay to be hidden, and return early; ShowBottomCaptionBar (external request to display the bottom bar): If the video is not playing, return directly without displaying it; enterEvent (mouse enters the MatrixUnit area): If the video is not playing, do not trigger the display of the overlay; setOverlayVisible / TrySetOverLayVisible: If a request to display is made but the video is not playing, ignore the request.

[0071] Among them, the hold / delay strategy hidden in the floating layer is as follows: The overlay defines an `IsWorking()` method to provide a second layer of display-keeping logic in addition to the primary criteria mentioned above. `IsWorking()` returns true if any of the following conditions are met: the 3D selection rectangle switch is enabled (`m_3DRectEnabled`), the recording prompt text is not empty (`m_strRecordText`), or the watermark list is not empty (`m_lstWatermark`). This logic operates on the `TrySetVisible(boolvisible)` method. When an external request to hide the overlay is made (`visible=false`) but `IsWorking()` returns true, the hiding request is suppressed, and the overlay remains visible to ensure that key overlay information such as recording status text and watermarks remain visible. Conversely, when `IsWorking()` returns false, the overlay is hidden according to normal logic. It's important to note that the display / hiding of the bottom control bar and close button in the overlay does not use a delayed timer; instead, it is entirely driven by the mouse position in real-time. The `OnMouseMoveSlot` method of `MatrixUnit` determines in real-time whether the mouse is within the bottom control bar area (rectangular area determination: y-coordinate between `videoHeight` - `NP_UNITBOTTOMCAPTIONBAR_HEIGHT` and `videoHeight`). If the mouse is within the area, the bottom bar and close button are immediately displayed; if it leaves, they are immediately hidden. The only exception to delayed hiding is the tooltip label (`m_tipLabel`): the `showTipLabel(text, timeout)` method accepts a timeout parameter in milliseconds. By triggering a timer `m_tipLabelTimer` once, `hideTipLabel()` is automatically called after `timeout` milliseconds to hide the label.

[0072] The trigger scenario for restoring the state when the display is restored is when the video grid is redisplayed (such as when a split-screen layout is switched back from hidden or a window is restored from minimized).

[0073] The specific implementation steps for processing and waiting for the interface layout to stabilize are as follows: After MatrixUnit's showEvent(QShowEvent*e) finishes displaying itself, it calls m_pVideoOverlay->restoreState(). Internally, the restoreState method processes the saved logical state flag m_bPTZVisibleBeforeHide in two ways: If the PTZ control was visible before being hidden, a single-delay timer is used to ensure the UI layout stabilizes: `QTimer::singleShot(100,this,lambda)` is called to postpone subsequent recovery operations by 100 milliseconds. During this 100-millisecond wait, the Qt event loop continues to process backlogged redraw events and layout calculations (such as `QGridLayout` resizing grids), and the final geometric parameters of the matrix grid are determined. The delayed callback function `lambda` executes sequentially: if the overlay is currently hidden, `QWidget::show()` is called to make it visible; `UpdatePositionStatus()` is called to force a recalculation of the global screen coordinates and move it to the correct position; `UpdateSizeStatus()` is called to force a recalculation and `setFixedSize` to align with the video area size; `showPTZControl()` is called to display the PTZ control and recalculate its anchor point position; `raise()` is called on the PTZ control to ensure its Z-order is at the top of the overlay. This 100ms delay value is an empirical parameter based on the Qt layout system, which ensures that the layout calculation has enough time to complete, but does not cause abrupt flickering that is perceptible to the user due to excessive waiting time.

[0074] If the PTZ control is not visible before hiding, check the working status of m_bWorking. Only call show() to restore the display of the overlay itself when the video is playing (m_bWorking is true), without restoring the state of the child controls.

[0075] Two smaller delay strategies are used for other time-sensitive scenarios: the overlay's own show() method, after calling QWidget::show(), delays for 20 milliseconds using QTimer::singleShot(20,this,...) before emitting the sgnOverlayShow signal, ensuring that the overlay window has fully entered the visible state before notifying the outside; when switching to fullscreen, the SetStaysOnTop method, after setting the top-pinning flag, delays for 50 milliseconds using QTimer::singleShot(50,this,...) to perform fine-tuning move and resize, avoiding jitter caused by the overlap of the top-pinning attribute taking effect and geometric changes.

[0076] In this embodiment, the method further includes: When a user triggers a full-screen operation, the system enters full-screen mode. At this time, the video matrix unit module is marked as being in full-screen state, and if the overlay module has not yet been created, it is created immediately. The system sets the top-level properties for each window in a strict bottom-up order, sequentially completing the top-level placement of the matrix main window, the overlay window, and the independent child control windows inside the overlay, ensuring that the overlay is always displayed in front of the video screen. After the top placement is completed, the position and size of the overlay are forcibly synchronized. After the window flag is changed, the window is rebuilt and the layer is fine-tuned to ensure that the window properties take effect immediately and eliminates layer anomalies. When the user triggers the exit from fullscreen, the video matrix unit module cancels the fullscreen mark; it removes the system-level top-of-the-line attribute of each window in a strict top-down order, sequentially removing the top-of-the-line attribute of independent child control windows inside the floating layer, removing the top-of-the-line attribute of the floating layer window, and removing the top-of-the-line attribute of the matrix main window, restoring the window hierarchy relationship in normal mode; after the top-of-the-line removal is completed, the position and size of the floating layer are resynchronized. When switching between full-screen mode and non-full-screen mode, if a preset floating layer instability event is detected, the floating layer reconstruction mechanism is triggered. The preset floating layer instability events include changes in window flags and changes in parent-child relationships.

[0077] The step of setting the system-level "top" attribute for each window in a strict bottom-up order includes: Step 1: Position the main window of the video matrix to the top, setting it as a borderless, full-screen window that fills the available geometric area of ​​the target screen. At this point, the main window of the matrix is ​​located at the top level of the operating system window hierarchy. Step 2: Traverse all matrix unit modules and trigger each matrix unit module to enter full-screen mode; after each matrix unit module enters full-screen mode, if the overlay module has not yet been created, it will be created immediately, and the overlay module will be notified to enter full-screen mode. Step 3: The floating layer window is pinned to the top. After entering full-screen mode, the floating layer module adds a system-level pinned attribute to itself. Since the pinned attribute of the floating layer is added after the pinned attribute of the matrix main window is set, the operating system window manager places the floating layer window with the pinned attribute set later on top of the matrix main window that is pinned first, according to the activation order of the pinned windows. Step 4: Position the independent child control windows within the overlay to the top. The overlay module sets the system-level "position to top" property for both the gimbal control and preset position controls, ensuring that the interactive controls are positioned higher than the overlay windows. The complete window hierarchy from bottom to top in full-screen mode is as follows: Screen Desktop Layer → Video Matrix Main Window (Full Screen, Top-of-the-Line, Borderless) → Matrix Unit Module (Fills the Video Matrix Main Window) → Video Display Module (Video Rendering Surface) → Floating Layer Module (Places at the Back, Covers the Video Rendering Surface) → PTZ Control Controls / Preset Position Controls (Independent Top-Level Window, Places at the Back, Above the Floating Layer Module).

[0078] Detailed, such as the hierarchy adjustment process for full-screen mode: Fullscreen operation is initiated by NP_VideoMatrix::showFullScreen(), which sets the window to top flag in a strict order to construct the correct Z-order hierarchy: The first step is to place the main window of the matrix on top: VideoMatrix calls setWindowFlags(Qt::Window|Qt::FramelessWindowHint|Qt::WindowStaysOnTopHint) to set itself as a borderless full-screen window on top, filling the available geometric area of ​​the target screen (obtained through QDesktopWidget::screenGeometry). At this time, VideoMatrix is ​​located at the top level of the operating system window Z-order. The second step is to trigger the floating layer to the top in a grid-by-grid manner: VideoMatrix traverses each MatrixUnit and calls EnterFullScreen(). After EnterFullScreen sets m_bIsFullScreen=true, if the floating layer has not been created yet, it immediately calls SetUpVideoOverlay to create it, and then calls m_pVideoOverlay->setFullScreenMode(true). The third step is to bring the overlay window to the top: After setting the overlay member flag `m_isFullScreen=true` within `setFullScreenMode(true)`, `SetStaysOnTop(true)` is called. This method adds a system-level top-of-the-line attribute to the overlay window using `setWindowFlags(windowFlags()|Qt::WindowStaysOnTopHint)`. Since the overlay's `WindowStaysOnTopHint` is added after the VideoMatrix's top-of-the-line flag is set, the operating system's window manager (on Windows) places the overlay window with the top-of-the-line attribute set later on top of the VideoMatrix, ensuring that the overlay is always displayed in front of the video screen, based on the activation order of the top-of-the-line windows. The fourth step is to bring the child control window to the top: setFullScreenMode continues to call m_ptzControlWidget->SetOnTop(true) and m_presetControlWidget->SetOnTop(true) to set the top attribute for the PTZ gimbal control and the preset position control (both of which are independent top-level windows or QDialogs) to ensure that the interactive control is higher than the floating window. Step 5, geometric alignment: After EnterFullScreen is placed on top, it calls updateOverlayPosition() and updateOverlaySize() in sequence to align the overlay to the VideoPlay area under the full screen size. Step 6: Fine-tuning the window flags: After modifying windowFlags, the SetStaysOnTop method calls SetVisible(true) to force the window to rebuild so that the flags take effect immediately. Then, it uses QTimer::singleShot(50,...) to delay for 50 milliseconds and execute move(Tx_+1,Ty_+1) and setFixedSize(m_nWidth-1,m_nHeight-1). By using a small position / size perturbation (1 pixel offset), it forces the Windows window manager to re-evaluate the window's Z-order and composition layer, eliminating possible layer freezing or flickering problems caused by changes to windowFlags.

[0079] The hierarchical recovery path after exiting fullscreen mode: NP_VideoMatrix::showNormal() restores in reverse order: VideoMatrix calls setWindowFlags(Qt::SubWindow) to remove its own WindowStaysOnTopHint, and restores the minimum / maximum size limits; it iterates through each MatrixUnit and calls ExitFullScreen(), which sets m_bIsFullScreen=false and then calls m_pVideoOverlay->setFullScreenMode(false); setFullScreenMode(false) internally calls SetStaysOnTop(false), and removes the top-of-the-line attribute of the overlay through setWindowFlags(windowFlags()&(~Qt::WindowStaysOnTopHint)), and the PTZ control and preset position control are removed from the top at the same time; finally, updateOverlayPosition() and updateOverlaySize() are executed to re-align to the grid layout size in normal mode, and update the state of the full-screen button icon in the bottom control bar (the icon switches from exiting full-screen to entering full-screen).

[0080] The complete Z-order hierarchy in full-screen mode (bottom-up): Screen desktop layer → NP_VideoMatrix (full screen, top-mounted, borderless) → NP_MatrixUnit (fills VideoMatrix) → NP_VideoInside (video rendering surface) → NP_VideoOverlay (backward, top-mounted, covers video surface) → PTZControlToolWidget / PresetControlToolWidget (independent top-level window, backward, top-mounted, located above the floating layer).

[0081] In this embodiment, the floating layer reconstruction mechanism includes: Perform pre-validation; if the overlay module does not exist, is not visible, or is not playing, return directly. Read the key overlay status of the current floating layer module, which includes at least the three-dimensional box selection switch status and / or the video recording prompt text display status; A secure destruction process of hiding first and then destroying is adopted. The old floating layer module is first made invisible, the old floating layer module is destroyed by the delayed destruction method, and the pointer pointing to the old floating layer module is immediately set to null. The old floating layer module automatically performs a self-cleaning operation during destruction, releasing all occupied system resources and severing its association with external objects; Recall the floating layer creation function and create a new floating layer module using the exact same construction parameters as the old floating layer module; The key overlay state read is written back to the new overlay module, and the new overlay module is displayed; depending on whether the current mode is full screen, the corresponding alignment strategy is selected, and the position and size of the new overlay module are synchronized with the video display module.

[0082] In detail, in some scenarios, switching to fullscreen mode can cause changes to window icons and parent-child relationships. Simply adjusting the position and size cannot guarantee stability. Therefore, a reconstruction strategy is adopted, which is fully implemented by the NP_MatrixUnit::ReDrawOverlay(boolisFullScreen,constQPoint&offset) method. The process is as follows: Pre-check: If the floating layer pointer is null, the floating layer is not visible, or the video is not playing, return directly to avoid empty operation or accidental destruction.

[0083] State extraction: The `m_pVideoOverlay->Get3DRectEnabled()` and `m_pVideoOverlay->GetRecordText()` methods are called to read the current 3D bounding box enable status and recording prompt text content of the old overlay, respectively, and save them to the local variables `is3DEnabled` and `recordText`. This step is completed before any destruction actions are performed to ensure the integrity and consistency of the state snapshot.

[0084] Hide before destroying: Call `m_pVideoOverlay->setVisible(false)` to make the old overlay invisible, preventing visual persistence or flickering during destruction. Then call `m_pVideoOverlay->deleteLater()` instead of directly deleting it. `deleteLater()` is a deferred destruction method provided by the Qt framework, which postpones the actual destruction of the object until the next event handling cycle after the current event loop returns. If there are still pending events pointing to the old overlay in the event queue (such as mouse movements, timer callbacks, etc.), Qt can safely discard or complete the processing of these events, avoiding crashes caused by event handling functions accessing already released memory due to direct deletion. Immediately after calling `deleteLater()`, set the `m_pVideoOverlay` pointer to NULL to prevent any subsequent access to this dangling pointer in the code.

[0085] Self-cleaning during destruction: When the actual destruction triggered by `deleteLater()` occurs, the destructor of `NP_VideoOverlay` automatically performs the following cleanup: it calls `m_pTopLevelWindow->removeEventFilter(this)` to remove the event filter installed on the top-level window during the construction phase, eliminating the risk of the top-level window sending Move events to the destroyed object; there is no need to explicitly delete child controls (close button, bottom control bar, PTZ control, preset controls, tooltip labels, timers) in the destructor body, because Qt's parent-child object tree mechanism automatically recursively deletes all child QObjects when the parent object is destroyed. Simultaneously, all signal-slot connections bound to the old overlay are automatically disconnected by the Qt metaobject system because the sender object is destroyed.

[0086] Recreation: Calling SetUpVideoOverlay(this, m_bIsPlaybackMode) creates a new overlay instance with the exact same constructor parameters. SetUpVideoOverlay internally re-executes window property configuration (no border, transparent background, Windows layered window flags, event filter installation) and completes the full set of signal-slot connection registration. The new overlay is in a blank initial state.

[0087] State write-back and display: Call m_pVideoOverlay->SetRecordText(recordText) and m_pVideoOverlay->Set3DRectEnabled(is3DEnabled) to write the overlay state saved in the previous step to the new overlay; then call m_pVideoOverlay->SetVisible(true) to make the new overlay visible.

[0088] Geometric alignment: Select the alignment strategy based on the isFullScreen parameter. In full-screen mode, call updateOverlayPosition(offset) to correct the position with the offset parameter. In non-full-screen mode, call updateOverlayPosition(). Finally, call updateOverlaySize() to complete the size alignment.

[0089] This reconstruction strategy uses deleteLater() to ensure the safe delayed destruction of old objects, complementing the direct delete in the MatrixUnit destructor (suitable for deterministic destruction at the end of the lifecycle), thus covering both runtime reconstruction and final destruction scenarios.

[0090] For example, the overlay interaction in the 4-screen split-screen real-time preview of the Windows client; Application scenario: Users can enable a 4-screen preview in the video surveillance client, with each cell playing a live video stream; hovering the mouse over a cell displays the bottom control bar, clicking PTZ opens the pan-tilt panel, and using the scroll wheel to zoom in and out of the blank area; after dragging the main window, users can perform full-screen / exit full-screen operations on a single cell.

[0091] Implementation steps: The client starts, creates the main window and video matrix area, and the matrix module generates 4 independent MatrixUnit units; Each MatrixUnit creates a VideoPlay video display module to start video streaming and playback; when the business triggers an overlay requirement, it calls SetUpVideoOverlay to create a floating layer for the corresponding segment, and configures transparent borderless and layered display attributes; MatrixUnit listens to VideoPlay's movement and size change events, and calls updateOverlayPosition and updateOverlaySize in real time to ensure that the overlay is in the same position and shape as the video frame; When the user moves the mouse over a segment, the overlay triggers a display event, showing the bottom control bar and the close button; clicking the PTZ button creates and displays the PTZ control controls in the overlay. When a user drags in the blank area of ​​the overlay, the overlay forwards the event to MatrixUnit to execute the segmented drag logic; when the user scrolls the wheel, the overlay forwards the wheel event to achieve screen zooming. When the user drags the main window, the main window's Move event triggers a forced update of the floating layer's position, resulting in no drifting or misalignment of the floating layer. When a user performs a single-pane fullscreen operation, EnterFullScreen initiates fullscreen mode and synchronizes the overlay. In abnormal scenarios, the overlay is automatically rebuilt and its state restored. Pressing Esc exits fullscreen mode; the key event is forwarded to the upper layer via the overlay, and the overlay is realigned after exiting. Overlay controls display stably without obstruction; the overlay remains unoffset after the main window is moved, split-screen is switched, or fullscreen is switched; controls are clickable, and dragging and scaling functions are retained in blank areas; PTZ, preset positions, and other controls can be shown, hidden, and restored to their original state as needed.

[0092] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0093] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0094] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more electronic devices to execute all or part of the steps of the methods described in the various embodiments of this application.

[0095] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0096] In the several embodiments provided in this application, it should be understood that the disclosed application can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of units or modules may be electrical or other forms.

[0097] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0098] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0099] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for independent overlay display and interactive control of multi-screen video, characterized in that, The method includes: Start the video monitoring client, create a main window module, and set a matrix module within the main window module; the matrix module contains several independent video matrix unit modules, and a video display module is created within the video matrix unit module. The video display module is used to decode and render the video image. When the video matrix unit module receives a preset playback command, a transparent background, borderless floating layer module is created above the video matrix unit module, and interactive controls and overlay drawing components are created inside the floating layer module. In response to a preset trigger event, obtain the screen position and size information of the corresponding video display module, and synchronize and align the position and size of the overlay module with the video display module. After receiving a user input event, the overlay module determines whether the event trigger point falls within the interactive control area. If so, the overlay module handles the event; otherwise, the event is converted to an event in the parent control's coordinate system and forwarded to the corresponding video matrix unit module. The display and hiding of the overlay module are controlled according to the video playback status, and its key display status is saved when the overlay module is hidden and restored when it is displayed. Synchronizing and aligning the position of the overlay module with the video display module includes: When a preset position trigger event is detected, the matrix unit module obtains the local logical coordinates of the video display module within the parent control; the preset position trigger event includes: the video display module moving, the main window module moving, the completion of switching between full-screen and non-full-screen states, and the window resuming display; The video matrix unit module passes the local logical coordinates within the parent control to the floating layer module; the floating layer module stores the local logical coordinates in its own internal storage variable and triggers the position update state method to execute the coordinate transformation strategy and position synchronization operation; The coordinate transformation strategy is as follows: the floating layer module uses the matrix unit module as the reference coordinate system and calls the global coordinate transformation interface provided by the graphical interface framework to convert the local logical coordinates into screen global coordinates. The graphical interface framework automatically handles the differences in the coordinate reference offset and display resolution scaling of each screen in a multi-monitor scenario. Based on the global screen coordinates, the overlay module is moved to the global screen coordinates so that the position of the overlay module is aligned with the position of the video display module. Specifically, by pre-installing an event filter on the main window module to monitor the movement of the main window module, when the movement of the main window module is detected, the above-mentioned floating layer module position synchronization alignment operation is forcibly triggered.

2. The method for independent overlay display and interactive control of multi-screen video according to claim 1, characterized in that, The specific process for creating the floating layer module is as follows: When a video matrix unit module receives a video playback command, it checks whether the current matrix unit module has created a floating layer module. If so, it directly uses the corresponding floating layer module. Conversely, an independent overlay module is created within the corresponding video matrix unit module, and the segment index, window flag, parent control, and playback mode parameters are passed in. Based on the created overlay module, configure the corresponding overlay window core properties and the corresponding internal child controls; Establish signal connections between the floating layer module and its corresponding matrix unit module, and send the operation signals of the interactive controls back to the video matrix unit module to execute the corresponding business logic.

3. The method for independent overlay display and interactive control of multi-screen video according to claim 2, characterized in that, Synchronizing and aligning the size of the overlay module with the video display module includes: When a preset size trigger event is detected, the matrix unit module uses a dual-path approach to capture the size change of the video display module, obtains the latest width and height values ​​of the video display module, and transmits the width and height values ​​to the floating layer module; the preset size trigger event includes changes in the control size of the video display module, grid deformation caused by switching the matrix split-screen layout, or grid size changes caused by switching between full-screen and non-full-screen. The floating layer module stores the received width and height values ​​into its own internal storage variables and triggers the size update state method to execute subsequent size synchronization operations. The size update state method first obtains the current size of the parent control as a fallback value. When the passed width or height value is less than or equal to 0, it uses the size of the parent control as the valid size and finally calls the fixed size method to fix the size of the floating layer module itself to the valid size. Based on its adjusted size, an absolute coordinate positioning strategy is used to trigger the rearrangement of the sub-controls inside the overlay module. The position update of all sub-controls is completed through a dual trigger path, and a differentiated anchoring strategy is adopted according to the different types of sub-controls to ensure that the visible area of ​​the overlay module and the video display module always remains completely overlapped.

4. The method for independent overlay display and interactive control of multi-screen video according to claim 3, characterized in that, The matrix unit module employs a dual-path capture method to capture the size changes of the video display module, including: When the size of the video display module changes, it transmits a size change signal. During the construction phase, the matrix unit module binds the signal to the corresponding response function. Inside the response function, the position update operation and size update operation of the floating layer module are triggered in sequence. The size update operation reads the latest width and height of the video display module and passes them to the floating layer module. The matrix unit module rewrites its own frame size change event. Whenever the matrix unit module is resized due to external layout adjustments, the event handler recalculates and sets the size and position of the video display module according to the current scaling strategy, and then triggers the size update operation of the overlay module again to synchronize the changes to the overlay module.

5. The method for independent overlay display and interactive control of multi-screen video according to claim 4, characterized in that, After receiving a user input event, the overlay module determines whether the event trigger point falls within the interactive control area, including: The floating layer module receives user input events, including mouse press events, mouse move events, mouse release events, scroll wheel events, and keyboard events. The overlay module calls its own sub-control hit detection method, and based on the sub-control hit detection method and the user input event, determines whether the event trigger point of the user input event falls on a certain sub-control within the interactive control area inside the overlay module; If so, the overlay module directly processes the corresponding sub-control based on the user input event; Conversely, the coordinates of the user input event are converted to coordinates in the coordinate system of the parent control, and a new input event of the same type as the user input event is constructed. The new input event is forwarded to the matrix unit module using a synchronous event sending method. Differentiated pass-through processing logic is used for different types of user input events. A preset hit determination strategy is used for boundary scenarios. The matrix unit module executes the corresponding video operation, which includes split-screen selection operation, split-screen drag operation, screen frame selection operation, and screen zoom operation.

6. The method for independent overlay display and interactive control of multi-screen video according to claim 5, characterized in that, The display and hiding of the overlay module are controlled according to the video playback status, and its key display state is saved when the overlay module is hidden, and the key display state is restored when the display is restored, including: The matrix unit module uses dual gating logic to determine the visibility state of the floating layer module and monitors the video playback status in real time. When the video enters the effective playback state, it notifies the floating layer module to enter the working state; when the video exits the effective playback state, it notifies the floating layer module to hide it according to the preset hiding strategy. The overlay module employs differentiated display-hiding strategies for different types of internal interactive controls. The display and hiding of some controls are driven by the mouse position in real time, while others adopt a delayed hiding strategy. When the overlay module needs to be hidden, the overlay module actively records the logical visibility state of the internal interactive controls, without relying on the display state of the controls themselves for judgment; after the recording is completed, the interactive controls and the overlay module itself are hidden in sequence. When it is necessary to restore the display, a delay waiting mechanism is adopted to ensure that the interface layout is stable before performing the restoration operation. The floating layer module is displayed first, and position alignment and size alignment are forcibly performed. Then, the display state of the interactive control in the floating layer module is restored according to the recorded logical visibility state.

7. The method for independent overlay display and interactive control of multi-screen video according to claim 6, characterized in that, The method further includes: When a user triggers a full-screen operation, the system enters full-screen mode. At this time, the video matrix unit module is marked as being in full-screen state, and if the overlay module has not yet been created, it is created immediately. The system sets the top-level properties for each window in a strict bottom-up order, sequentially completing the top-level placement of the matrix main window, the overlay window, and the independent child control windows inside the overlay, ensuring that the overlay is always displayed in front of the video screen. After the top placement is completed, the position and size of the overlay are forcibly synchronized. After the window flag is changed, the window is rebuilt and the layer is fine-tuned to ensure that the window properties take effect immediately and eliminates layer anomalies. When the user triggers the exit from fullscreen, the video matrix unit module cancels the fullscreen mark; it removes the system-level top-of-the-line attribute of each window in a strict top-down order, sequentially removing the top-of-the-line attribute of independent child control windows inside the floating layer, removing the top-of-the-line attribute of the floating layer window, and removing the top-of-the-line attribute of the matrix main window, restoring the window hierarchy relationship in normal mode; after the top-of-the-line removal is completed, the position and size of the floating layer are resynchronized. When switching between full-screen mode and non-full-screen mode, if a preset floating layer instability event is detected, the floating layer reconstruction mechanism is triggered. The preset floating layer instability events include changes in window flags and changes in parent-child relationships.

8. The method for independent overlay display and interactive control of multi-screen video according to claim 7, characterized in that, The floating layer reconstruction mechanism includes: Perform pre-validation; if the overlay module does not exist, is not visible, or is not playing, return directly. Read the key overlay status of the current floating layer module, which includes at least the three-dimensional box selection switch status and / or the video recording prompt text display status; A secure destruction process of hiding first and then destroying is adopted. The old floating layer module is first made invisible, the old floating layer module is destroyed by the delayed destruction method, and the pointer pointing to the old floating layer module is immediately set to null. The old floating layer module automatically performs a self-cleaning operation during destruction, releasing all occupied system resources and severing its association with external objects; Recall the floating layer creation function and create a new floating layer module using the exact same construction parameters as the old floating layer module; The key overlay state read is written back to the new overlay module, and the new overlay module is displayed; depending on whether the current mode is full screen, the corresponding alignment strategy is selected, and the position and size of the new overlay module are synchronized with the video display module.

Images