Cross-browser window adapting and cutting method for Web browser plug-in of domestic system

Abstract

The invention discloses a cross-browser window adaptation and cutting method for a Web browser plug-in of a domestic system, and belongs to the technical field of security and protection video playing. The method aims at solving the problems that plug-ins are instable in cross-browser embedding and dynamic display crosses the boundary in a localized system. According to the method, firstly, through multi-dimensional browser environment recognition and dynamic parent window binding, it is ensured that a plug-in accurately belongs to multiple browsers; and then, on the basis of visual area boundary parameters dynamically issued by the browser, performing successive dependent collaborative clipping on the plug-in main window and the floating layer window on the plug-in main window: firstly performing first clipping on the boundary-crossing main window, and then performing secondary constraint clipping on the floating layer window according to a clipping result. According to the method, stable embedding of the plug-in in a multi-browser environment and accurate display control under dynamic interaction are achieved, and visual errors and interface shielding are effectively eliminated.

Description

A method for cross-browser window adaptation and trimming of domestically developed system web browser plugins Technical Field

[0001] This invention belongs to the field of security video playback technology in software development, specifically involving a method for cross-browser window adaptation and trimming of a domestically produced system web browser plugin. Background Technology

[0002] As domestic operating systems (such as Kylin) are increasingly used in various fields, "playing front-end camera videos through a web browser on a domestic operating system" has become an important technical application scenario in the video surveillance industry, and the need for adaptation and optimization of related technologies is becoming increasingly prominent.

[0003] In this application scenario, there are clear functional requirements and interface specifications: When a user is watching a video, if they need to interact with the video, they need to perform interactive actions on the video interface. At this time, feedback information needs to be provided to the user. Therefore, a floating window (a transparent window) needs to be added above the video interface as a prompt window, and text or graphics need to be rendered on the floating window. When the display area of ​​the plugin exceeds the browser's specified area, the excess part needs to be made invisible, otherwise it will cover the browser title or be displayed on an interface outside the browser.

[0004] Currently, there are mature solutions available for the above requirements in Windows systems, including: 1. In Windows systems, first create a Windows system window and use it as a child window of the browser. The video playback window is then created as a Qt window on top of this Windows system window. This Qt window can be adjusted synchronously as the Windows system window moves or changes size. However, for the requirement of adapting to domestic systems, it is necessary to directly create a child window of the Qt browser to ensure compatibility with Windows and Kylin systems, and the implementation method is simpler.

[0005] 2. In Windows systems, drawing and text creation can be performed through the Windows-specific GDI interface, allowing text, graphics, and video to be displayed in the same window; while in Kylin systems, a floating window needs to be added to render text or graphics.

[0006] 3. On Windows systems, only plugins that exceed the allowed range are trimmed; overlays are not trimmed.

[0007] However, the mature solutions in the Windows system mentioned above cannot be applied to domestic operating systems (such as Kylin system). There are three main reasons for this: 1. Windows system has a unique interface for text drawing, and no corresponding interface has been found on the domestic platform. Specifically, Windows system can call the GDI interface to draw directly in the window, but the domestic platform does not have this interface, so the existing solution does not support it and a new implementation solution needs to be found.

[0008] 2. In Windows systems, the video playback window of the plugin does not visually exceed the browser's boundaries. However, this issue exists in domestic operating systems. In Windows systems, the actual size of the 360 ​​browser matches the visible area, while in domestic operating systems, the actual size does not match the visible area. This causes the child window to appear to exceed the visible area when the browser is the parent window and the plugin is the child window, even though it is still within the browser. This creates a visual error, leading users to mistakenly believe that the child window has moved out of the browser. Therefore, the video playback window of the plugin needs to be cropped in domestic operating systems.

[0009] 3. In Windows systems, rendered text does not exceed the browser's boundaries, but this issue exists in domestically developed systems. This is because in Windows systems, text and video are displayed together in the same window. When a plugin moves outside the browser, the Windows window mechanism makes the portion of the plugin that exceeds the boundaries invisible. However, in domestically developed systems, plugins need to use the Qt overlay mechanism to draw text, which means that the portion of the Qt overlay window that exceeds the boundaries cannot be hidden in the above scenario. Therefore, the portion exceeding the boundaries needs to be trimmed in domestically developed systems.

[0010] Furthermore, the domestic browser ecosystem is diverse (e.g., 360, Qi An Xin, Firefox, etc.), and different browsers have different window management mechanisms, process models, window hierarchy adaptation logic, and interaction methods with the operating system. Directly applying the simple logic of "creating browser child windows" under Windows will cause the plugin window to fail to correctly "fit" or "follow" its expected parent window (i.e., a specific browser tab) in domestic systems. Specifically, the plugin window may display independently of the browser, not move with the browser window, or be offset to an unexpected area (e.g., obscuring the browser navigation bar, tabs, or other interface elements). This cross-browser compatibility issue makes it extremely difficult to achieve stable and consistent plugin window embedding and management in a domestic environment. In addition, in dynamic interaction scenarios, such as when a user scrolls the browser page, the relative position of the plugin changes in real time. If only static initial positions and clipping rules are relied upon, the plugin or its floating tooltips can easily move out of the browser's visible area or obscure the browser's navigation bar, tabs, and other key interface elements, affecting normal user operation and interface appearance. Existing solutions lack the ability to accurately perceive and adapt to this dynamic process in real time.

[0011] Therefore, in order to adapt to domestic operating systems, meet the requirements of "floating window prompts and out-of-range cropping when Web browsers call plugins to play front-end camera videos", and solve the resulting problems of precise cross-browser window positioning and fitting, as well as real-time collaborative cropping of plugins and floating windows during dynamic interaction, it is urgent to provide a technical solution that is compatible with domestic operating systems and multiple mainstream browsers, and can systematically solve the problems of dynamic positioning of the parent window and differentiated real-time cropping of plugins and floating windows. Summary of the Invention

[0012] In view of the shortcomings of existing technologies, this invention provides a method for cross-browser window adaptation and trimming of web browser plugins for domestic systems. It solves the problems of redundancy in plugin and browser window hierarchy caused by the need to create an additional Windows system window for Windows systems, the need for special handling of position and size, and incompatibility with domestic systems (such as Kylin). It also addresses the incompatibility of the Windows-specific GDI interface with Kylin systems, preventing domestic systems from rendering text or graphic prompts on top of video playback. Furthermore, it resolves the issue of plugin window "fitting" and "following" failures across browsers due to the variety of browsers and window management mechanisms in the domestic ecosystem. Finally, it addresses the problem of visual offset errors when plugins move as child windows due to inconsistencies between the actual size and visible area of ​​browsers (such as 360 Browser) in domestic systems. Finally, it addresses the inability of existing solutions to perceive and process changes in the position of plugins and their overlay windows in real time during dynamic interactive scenarios such as user scrolling, causing them to move out of the visible area or obscure browser interface elements. This document addresses several issues: First, it resolves the problem that when plugins in domestically developed systems use Qt's overlay mechanism to draw text, the portion of the overlay window exceeding the browser's view cannot be automatically hidden. Second, it addresses the issue that Windows systems only trim plugins, not overlays, failing to meet the requirement of trimming both plugins and overlays in domestically developed systems. Third, it optimizes the window relationship between plugins and the browser by directly creating Qt browser child windows to simplify the structure, streamline code, and ensure compatibility with both Windows and Kylin systems. Fourth, it enables graphical or text prompts to be displayed on top of videos in transparent overlay windows on Kylin systems, meeting user feedback needs. Fifth, it achieves precise dynamic positioning and stable alignment of plugin windows in various domestically developed browser environments. Sixth, it eliminates visual offset errors for plugins, ensuring plugin display conforms to the browser's visible range. Seventh, it enables collaborative real-time trimming of the plugin's main window and overlay window in dynamic scenarios, effectively avoiding any obstruction and ensuring a clean interface and smooth operation. Finally, it unifies the trimming logic across both systems, improving interface standardization and user experience.

[0013] To achieve the above objectives, the technical solution adopted by this invention is as follows: a method for cross-browser window adaptation and trimming of web browser plugins for domestically developed systems. This method addresses the problems of plugins failing to stably embed across browsers and displaying out of bounds during dynamic interactions under domestically developed operating systems. It ensures the accurate placement of the plugin window across multiple browsers through multi-dimensional browser environment identification and dynamic parent window binding; and performs differentiated collaborative trimming (which can also be understood as secondary collaborative trimming based on sequential dependencies) on the plugin's main window and its upper-level floating windows based on the browser's dynamically issued visible area boundaries. The plugin, based on its communication connection with the browser, performs multi-dimensional browser environment identification, and... Accordingly, the correct browser tab window is dynamically bound from the system window as the parent window of its main window; the browser dynamically sends parameter commands containing the browser's visible area boundary to the plugin based on its own visible area state; the plugin monitors the relative positional relationship between its main window and the boundary accordingly; when the display area of ​​the main window exceeds the boundary, the plugin performs an initial cropping of the main window based on the boundary parameters to obtain its effective display area; subsequently, based on the effective display area of ​​the main window obtained after the initial cropping, the plugin performs a secondary constraint cropping of the overlay window above the main window used to display overlay information, removing the portion of the overlay window that exceeds the effective display area of ​​the main window.

[0014] Furthermore, the multi-dimensional browser environment identification and dynamic parent window binding includes: the browser sending a location command to the plugin via a script; the plugin obtaining the process identifier of the corresponding browser process based on the current communication connection with the browser, and querying the process name according to the process identifier; determining the browser type and its window title rules by comparing the process name with pre-configured browser process information; subsequently, the plugin matching the received browser tab title in system windows that conform to the window title rules to dynamically bind the current browser tab window as the parent window of the plugin's main window, wherein the pre-configured browser process information is stored in the plugin's configuration file.

[0015] Furthermore, the browser sends a `getbrowser` command to the plugin via JavaScript. The plugin finds the communication port currently connected to the browser, obtains the corresponding process ID (PID) through that port, queries the process name based on the PID, compares it with the process name in the configuration file, and determines the browser type and browser title format. The browser then sends an `Open` command to the plugin via JavaScript, which contains the title of the current browser tab. The plugin uses the tab title transmitted in the `Open` command to locate the parent window within the associated windows of the determined browser type. If a matching window is located, the browser is successfully identified as the parent window of the plugin's main window; otherwise, the parent window location fails, the plugin terminates and exits.

[0016] Furthermore, the browser dynamically sends parameter commands containing the browser's visible area boundaries to the plugin based on its own visible area state. This includes: the browser sending window parameter commands containing dynamic boundary parameters to the plugin via a script. The dynamic boundary parameters include at least: the expected position and size of the plugin's main window in the browser, and field information representing the left, top, right, and bottom boundaries of the browser's current visible area.

[0017] Furthermore, the plugin monitors the relative position of its main window to the boundary, including: the plugin adjusts the display state of the plugin's main window according to the dynamic boundary parameters, and monitors the offset of the plugin's main window relative to the browser's visible area boundary in real time.

[0018] Furthermore, the initial cropping of the main window includes: when the display area of ​​the main window is detected to exceed the boundary, the plugin calculates the excess offset of each boundary of the main window based on the field information in the boundary parameters, and crops the main window based on the excess offset.

[0019] Furthermore, the calculation of the excess offset specifically involves: when the values ​​left, top, right, and bottom representing the left, top, right, and bottom boundaries in the field information are negative, it is determined that the corresponding edge is out of bounds, and the clipping offset or the clipped size is calculated based on the negative value; where, if left < 0, then the left offset offsetX = 0 - left; if top < 0, then the top offset offsetY = 0 - top; if right < 0, then the clipped width offsetW = w + right; if bottom < 0, then the clipped height offsetH = h + bottom; where w and h are the original width and height of the plugin's main window, respectively.

[0020] Furthermore, after cropping the main window, its effective display area is obtained. The coordinates of the upper left corner, the width, and the height of the effective display area are determined by the calculated cropping offsets offsetX and offsetY, and the cropped dimensions offsetW and offsetH.

[0021] Furthermore, the secondary constraint clipping of the overlay window includes: the plugin obtaining the screen coordinates of the overlay window that is superimposed on the video window, and obtaining the coordinate range of the effective display area of ​​the main window on the screen after the first clipping; comparing the screen coordinates of the overlay window with the screen coordinate range of the effective display area of ​​the main window, calculating the portion of the overlay window that exceeds the effective display area of ​​the main window, and clipping the portion that exceeds the limit.

[0022] Further, the calculation of the exceeded part includes the following steps: (a) Obtain the coordinates (Sx, Sy) from the upper left corner of the browser window to the upper left corner of the screen, and multiply Sx and Sy by the screen DPI respectively to obtain the real distances (i.e., the real screen coordinates (X, Y)); (b) Obtain the relative coordinates (GX, GY, GW, GH) of the effective display area of the main window within the browser, where GX and GY are the relative coordinates of the upper left corner, and GW and GH are the width and height; (c) Obtain the screen coordinates (UX, UY) and its size (UW, UH) of the floating window; (d) Determine the exceeded part of each boundary of the floating window through the following judgment and calculation: If UX < GX + X, the left boundary exceeds, and the left clipping offset offsetX = GX + X - UX; If UY < GY + Y, the upper boundary exceeds, and the upper clipping offset offsetY = GY + Y - UY; If UX + UW > GX + X + GW, the right boundary exceeds, and the clipped width offsetW = (GX + X + GW) - UX; If UY + UH > GY + Y + GH, the lower boundary exceeds, and the clipped height offsetH = (GY + Y + GH) - UY; (e) Clip the floating window according to the calculated offsetX, offsetY, offsetW, and offsetH.

[0023] Further, it also includes a communication establishment step: The plugin starts and reads the network port in the configuration file, creates a WebSocket server for listening; The Web control end of the browser发起连接请求,验证通过后与插件建立双向通信链路。

[0024] Further, it also includes the step of constructing a four - level window system: Create a plugin main window with the browser tab page window as the parent window, a service container window with the plugin main window as the parent window, a video window with the service container window as the parent window, and a transparent floating window that overlaps with the video window and has no parent window for displaying overlay prompt information.

[0025] The present invention adopts the above technical solutions and has the following specific beneficial effects: Abandon the redundant design of adding additional system windows in the Windows system, innovatively construct a four - level window architecture, and clarify the window subordination relationship. By directly establishing the parent - child window binding between the plugin and the browser, there is no need to maintain the synchronization logic of redundant windows, which not only simplifies the code structure, reduces the cross - system development and maintenance costs, but also achieves seamless compatibility between the Windows and Kylin dual systems and avoids duplicate development.

[0026] Note: In the translation of , the part "发起连接请求" should be a more accurate expression in English according to the context, but the original text seems to be incomplete here. It might be something like "initiates a connection request". The above translation is based on the best understanding of the overall context.Based on pre-configured browser compatibility information, combined with communication connections, process characteristics, and tab identifiers, the system dynamically and accurately locates and binds the correct browser tab as the parent window in a multi-browser environment. This mechanism effectively solves problems such as plugin position offset, loss of control, and inability to follow movement caused by differences in window management mechanisms of different browsers, ensuring that the plugin can be stably embedded and accurately fitted in real time in various mainstream browsers (such as 360, Qi An Xin, and Firefox) within the domestic ecosystem.

[0027] To address the technical limitations of Windows' proprietary GDI interface being incompatible with the Kylin system, a transparent overlay design independent of the video window is adopted, on which text, graphics, and other prompts are flexibly rendered. This overlay window can adjust its position and size synchronously with the video window, meeting the dynamic operation feedback needs of different monitoring scenarios such as real-time preview and video playback, while ensuring consistency in functionality and user experience between Windows and Kylin systems.

[0028] To address the discrepancy between the actual browser size and the visible area in domestically developed systems, as well as the issue of display boundaries exceeding limits during dynamic interactions, a real-time clipping mechanism is implemented for the plugin's main window based on dynamically issued visual boundary parameters from the browser, eliminating visual errors. Furthermore, by combining screen coordinate conversion and real-world distance calibration, a secondary constraint clipping is applied to the overlay window, strictly limiting its display area to the effective visible area of ​​the plugin's main window. This sequential collaborative clipping mechanism fundamentally prevents the plugin's main window or overlay prompts from obscuring browser navigation bars, tabs, and other interface elements during scrolling and other interactions, ensuring accurate and standardized interface display. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 is a structural diagram of the four-level window system of the present invention; Figure 2 is a schematic diagram of the normal display position of the plugin of the present invention; Figure 3 is a schematic diagram of the abnormal display position of the first type of browser plugin of the present invention; Figure 4 is a schematic diagram of the abnormal display position of the second type of browser plugin of the present invention; Figure 5 is a schematic diagram of the abnormal display position of the third type of browser plugin of the present invention; Figure 6 is a schematic diagram of the abnormal display position of the fourth type of browser plugin of the present invention; Figure 7 is a schematic diagram of the plugin of the present invention extending beyond the browser area; Figure 8 is a schematic diagram of the plugin of the present invention extending beyond the browser area; Figure 9 is a clipping schematic diagram of the main window (window A) of the plugin of the present invention; Figure 10 is a schematic diagram of the plugin of the present invention not extending beyond the browser area; Figure 11 is a schematic diagram of the plugin of the present invention not extending beyond the browser area; Figure 12 is a plugin diagram of the present invention normally displaying identification information; Figure 13 is a schematic diagram of the browser tabs obscured by the identification information of the present invention; Figure 14 is a clipping schematic diagram of the overlay window (window D) of the present invention; Figure 15 is a schematic diagram of the overlay image after clipping. Detailed Implementation

[0031] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0032] Figure 1 shows a schematic diagram of the four-level window system in this embodiment. It intuitively presents the hierarchical inheritance relationship of the browser window, the plugin main window (window A), the business container window (window B), and the video window (window C), as well as the overlapping layout of the transparent overlay window (window D) and the video window, clarifying the subordinate relationship and functional carrying logic of each window.

[0033] Under a domestically developed operating system (such as Kylin), the web browser communicates with plugins that listen on preset ports in a configuration file via JavaScript. The overall program consists of three parts: the browser (application layer), the JavaScript (link layer), and the plugins (business implementation layer). The functions and relationships of each window are as follows: Plugin main window (Window A): The parent window is set to the web browser; its core function is to receive commands transmitted by the JavaScript and adjust its position and size. Business container window (Window B): The parent window is set to the plugin main window; its core function is to host and manage the video window. Video window (Window C): The parent window is set to the business container window; it fills the business container window according to preset rules; its core function is to receive and render the video stream from the front-end camera. Overlay window (Window D): The parent window is set to none; it is configured with a transparent background; its core function is to draw text or graphic prompts on it. Its initial position and size are consistent with the video window, and it adjusts synchronously with changes in the position or size of the video window to create an overlay display effect.

[0034] Create a plugin main window whose parent window is the browser tab window, a business container window whose parent window is the plugin main window, a video window whose parent window is the business container window, and a transparent overlay window that is superimposed on the video window and has no parent window, used to display superimposed prompt information.

[0035] The specific inheritance relationships of each window are as follows: the parent window of the video window (Window C) is the business container window (Window B), the parent window of the business container window (Window B) is the plugin main window (Window A), and the parent window of the plugin main window (Window A) is the browser; the overlay window (Window D) has no parent window.

[0036] In this embodiment, the plugin starts and reads the network port in the configuration file, creating a WebSocket server to listen for connections. The browser's web control initiates a connection request, and after successful verification, establishes a bidirectional communication link with the plugin. The plugin reads the preset control port in the configuration file, initializes a server instance based on the WebSocket protocol, and binds it to the port. The browser initiates a connection request through its HTML web control. The plugin verifies the validity of the connection. If the verification is successful, a bidirectional communication channel is established for transmitting location commands, parameter commands, and video playback requests. If the verification fails or the connection times out, communication is terminated.

[0037] The specific process is as follows: The plugin creates a WebSocket server and listens on the control port to receive links from HTML. This listening port is configured in the configuration file. After the HTML (web control server) starts, it attempts to establish a connection with the plugin; if the connection is successful, communication occurs; if the connection fails, communication is terminated. This implementation provides a method for cross-browser window adaptation and trimming of web browser plugins in domestic systems. The method is used to solve the problems of plugins being unable to be stably embedded across browsers under domestic operating systems and displaying out of bounds in dynamic interactions. Through multi-dimensional browser environment identification and dynamic parent window binding, it ensures the accurate ownership of the plugin window in various browsers; and based on the visible area boundaries dynamically issued by the browser, it performs differentiated collaborative trimming of the plugin's main window and its upper-level floating windows based on their sequential dependencies; the plugin performs multi-dimensional... The system identifies the browser environment and dynamically binds the correct browser tab window from the system window as the parent window of its main window. Based on its own visible area state, the browser dynamically sends parameter commands containing the browser's visible area boundaries to the plugin. The plugin then monitors the relative position of its main window to the boundaries. When the main window's display area exceeds the boundaries, the plugin performs an initial cropping of the main window based on the boundary parameters to obtain its effective display area. Subsequently, based on the effective display area of ​​the main window obtained after the initial cropping, the plugin performs a secondary constraint cropping of the overlay window above the main window used to display superimposed information, removing the portion of the overlay window that exceeds the effective display area of ​​the main window.

[0038] As one implementation method, the multi-dimensional browser environment identification and dynamic parent window binding in this embodiment includes: the browser sending a location command to the plugin via a script; the plugin obtaining the process identifier of the corresponding browser process based on the current communication connection with the browser, and querying the process name according to the process identifier; determining the browser type and its window title rules by comparing the process name with pre-configured browser process information; subsequently, the plugin matching the received browser tab title in system windows that conform to the window title rules, so as to dynamically bind the current browser tab window as the parent window of the plugin's main window.

[0039] Preferably, the browser sends a `getbrowser` command to the plugin via JavaScript. The plugin finds the communication port currently connected to the browser, obtains the corresponding process PID through the port, queries the process name based on the PID, compares it with the process name in the configuration file, and determines the browser type and browser title format. The browser sends an `Open` command to the plugin via JavaScript, the `Open` command containing the title of the current browser tab. The plugin uses the tab title transmitted in the `Open` command to locate the parent window in the associated windows of the determined browser type. If a matching window is located, the browser is successfully determined as the parent window of the plugin's main window; if no matching window is located, the parent window location fails, the plugin terminates and exits.

[0040] In this implementation, the browser sends the `getbrowser` command to the plugin via JavaScript. The plugin then searches for the port it's currently connected to, finds the process ID (PID) through the port, and then finds the process name through the PID. It then compares the process name with the process name in the `browser.conf` configuration file to determine the corresponding browser type and title format. This step is to confirm the browser type and, subsequently, to locate the parent window.

[0041] The browser sends an Open command to the plugin via JavaScript, containing the title of this tab. The plugin uses the received tab title to locate the parent window. If successful, the parent window is located successfully. If unsuccessful, the parent window cannot be located successfully.

[0042] Figure 2 shows the normal display position of the plugin; the plugin is exactly in the expected display position indicated by the red border. Figures 3, 4, 5, and 6 show the abnormal positions of the plugin. In Figure 3, the red border is the expected display position in the first browser, but the plugin has moved upwards and deviated from the expected display position. In Figure 4, in a computer desktop, the red border is the expected display position in the second browser, but the plugin keeps shifting to the left and even moves out of the browser boundary. In Figure 5, the red border is the expected display position in the third browser, but the plugin has moved upwards and deviated from the expected display position. In Figure 6, the red border is the expected display position in the fourth browser, but the plugin has moved upwards and deviated from the expected display position. As one implementation method, after the plugin is successfully bound to the dynamic parent window through multi-dimensional browser environment recognition, it is necessary to further adjust the size of the plugin itself and its expected display position in the corresponding browser according to various browsers.

[0043] In this embodiment, the browser dynamically sends parameter commands containing the browser's visible area boundary to the plugin based on its own visible area state. This includes: the browser sending window parameter commands containing dynamic boundary parameters to the plugin via a script. The dynamic boundary parameters include at least: the expected position and size of the plugin's main window in the browser, and field information representing the current visible area boundary of the browser.

[0044] In this embodiment, the specific implementation of determining the size, position, and displayable area of ​​the plugin includes: the browser sends a position command to the plugin via a JavaScript script. The position command includes the following fields: x and y fields: used to determine the position of the plugin's main window relative to the browser; width and height fields: used to determine the size of the plugin's main window; left, top, right, and bottom fields: used to determine the displayable area of ​​the plugin's main window in the browser.

[0045] Next, the plugin plays the video. The browser obtains the playback address of the front-end camera through the back-end interface and sends a video playback request to the plugin. After receiving the video playback request, the plugin analyzes the IP address and port in the request address. Based on the analyzed IP address and port, the plugin establishes a network connection with the front-end camera. The plugin receives the video stream sent by the front-end camera, calls the video decoding library to decode the video stream, and renders the decoded video data to the video window to achieve video playback.

[0046] In this embodiment, the plugin monitors the relative position of its main window to the boundary, including: the plugin adjusts the display state of the plugin's main window according to the dynamic boundary parameters, and monitors the offset of the plugin's main window relative to the browser's visible area boundary in real time.

[0047] Preferably, in this embodiment, the initial cropping of the main window includes: when the display area of ​​the main window is detected to exceed the boundary, the plugin calculates the excess offset of each boundary of the main window based on the field information in the boundary parameters, and crops the main window based on the excess offset.

[0048] The browser sends window information to the plugin, adding the information (left, top, right, bottom) to the `position` command to specify the area to be clipped. The plugin's movement range is then limited to this area. If it exceeds this range, it will be clipped. If not clipped, the JavaScript will resend the `position` command when the user scrolls the browser. If the plugin exceeds the browser's normal display position, Figures 7, 8, and 9 will appear. Figure 7 shows the plugin's top exceeding the browser's range, obscuring the browser's title bar; Figure 8 shows the plugin's bottom exceeding the browser's range. In Figure 7, the value of the `top` field of the `position` command is the difference between the plugin's top and bottom values. In Figure 8, the value of the `bottom` field is the difference between the plugin's bottom and left values. Similarly, if the plugin exceeds the left or right of the normal browser's display position, the difference between the left and right values ​​can be obtained. The plugin can use the information from the four fields (top, bottom, left, right) to clip the area exceeding its normal display position. Figure 9 is an abstract diagram, where `tag1` and `tag2` refer to the names of the opened web pages. In this embodiment, the browser sends window information to the plugin, adding the necessary information (left, top, right, bottom) to the position command. The plugin's movement range is then limited to this area; if it exceeds this range, it will be cropped. The red area at this point represents the visible area of ​​the plugin, named the plugin's visible region. The green area will be cropped.

[0049] Before trimming, the offset must be calculated. Specifically, when the values ​​left, top, right, and bottom representing the left, top, right, and bottom boundaries in the field information are negative, it is determined that the corresponding edge is out of bounds, and the trimming offset or the trimmed size is calculated based on the negative value. Specifically, if left < 0, then the left offset offsetX = 0 - left; if top < 0, then the top offset offsetY = 0 - top; if right < 0, then the trimmed width offsetW = w + right; if bottom < 0, then the trimmed height offsetH = h + bottom; where w and h are the original width and height of the plugin's main window, respectively.

[0050] In this embodiment, after cropping the main window, its effective display area is obtained. The coordinates of the upper left corner, the width, and the height of the effective display area are determined by the calculated cropping offsets offsetX and offsetY, and the cropped dimensions offsetW and offsetH.

[0051] For the normal display after cropping, please refer to Figures 10 and 11. In Figure 10, the top of the plugin does not exceed the browser area; in Figure 11, the bottom of the plugin does not exceed the browser area.

[0052] As one implementation method, this embodiment includes real-time video recording and prompts. When a user records real-time video, a prompt needs to be displayed on the real-time video window. This requires adding a transparent window D (overlay) on top of the real-time video window, where the prompt information is displayed. This window needs to move with the video window. To maintain a visually appealing interface and avoid obstructing other browser tabs, the browser restricts plugin movement within a certain area.

[0053] Because the position and size of window D are determined based on window C. That is, window D overlaps window C. When playing a video, the user will perform some operations on the video, and at this time, some text (such as identification information) will be rendered on window D, as shown in Figure 12, which shows the plugin diagram of normally displaying identification information.

[0054] When a user scrolls the browser, the text drawn in window D may obscure the browser's tabs or navigation bar, causing malfunctions and an unsightly appearance. As shown in Figure 13, the signage displayed in the video recording window (in a talk show) obscures the browser's tabs. To address this, a recording prompt message is drawn on a floating window, including text or graphics. The floating window adjusts synchronously with the video window's position or size, maintaining its overlap with the video window. To ensure a visually appealing interface and prevent obscuring other browser tabs, the browser restricts the plugin's main window from moving within a preset area. Any portion of the plugin's main window exceeding this area is clipped, and the portion of the floating window extending beyond the main window's effective display area is removed.

[0055] As one implementation method, the secondary constraint clipping of the overlay window in this embodiment includes: the plugin obtaining the screen coordinates of the overlay window that is superimposed on the video window, and obtaining the coordinate range of the effective display area of ​​the main window on the screen after the first clipping; comparing the screen coordinates of the overlay window with the screen coordinate range of the effective display area of ​​the main window, calculating the portion of the overlay window that exceeds the effective display area of ​​the main window, and clipping the portion that exceeds the limit.

[0056] Preferably, in this embodiment, the calculation of the exceeded part includes the following steps. For convenient reading, refer to FIG. 14. (a) Obtain the coordinates (Sx, Sy) from the upper left corner of the browser window to the upper left corner of the screen, and multiply Sx and Sy by the screen DPI respectively to obtain the real distances (X, Y); (b) Obtain the relative coordinates (GX, GY, GW, GH) of the effective display area of the main window within the browser, where GX and GY are the relative coordinates of the upper left corner, and GW and GH are the width and height; (c) Obtain the screen coordinates (UX, UY) and its size (UW, UH) of the floating window; (d) Determine the exceeded parts of each boundary of the floating window through the following judgments and calculations: If UX < GX + X, the left boundary exceeds, and the left clipping offset offsetX = GX + X - UX; If UY < GY + Y, the upper boundary exceeds, and the upper clipping offset offsetY = GY + Y - UY; If UX + UW > GX + X + GW, the right boundary exceeds, and the clipped width offsetW = (GX + X + GW) - UX; If UY + UH > GY + Y + GH, the lower boundary exceeds, and the clipped height offsetH = (GY + Y + GH) - UY; (e) Clip the floating window according to the calculated offsetX, offsetY, offsetW, and offsetH.

[0057] As shown in FIG. 15, it is a schematic diagram of the floating window after clipping. It can be seen that as the scroll bar moves down, the plugin moves up, the floating window is clipped, the original (during intercom) text is clipped, and the browser title bar is not blocked.

[0058] In this embodiment, by constructing a three-layer interaction architecture of browser, JS script, and plugin and a four-level window system, the problems of floating window prompt and window clipping when playing security videos in a Web browser under a domestic operating system (such as Kylin system) are innovatively solved. On the one hand, the redundant window levels of the Windows system are abandoned to achieve compatible adaptation of the Windows and Kylin dual systems; on the other hand, the prompt requirements during video operation are met through a transparent floating window, and a dual协同裁剪机制 is designed for the plugin main window and the floating window to accurately process the area beyond the visible range of the browser, which not only ensures the interface standardization (avoiding blocking the browser title or external interface), but also eliminates the visual offset problem unique to domestic browsers, and finally realizes a consistent video playback and operation experience under the dual systems, providing efficient and standardized technical support for the application of domestic operating systems in the field of security videos.

[0059] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limitations of the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention. It should be noted that there is an unclear expression "双协同裁剪机制" in the original text. I have translated it as "dual协同裁剪机制" for now. You may need to clarify this term for a more accurate translation.

Claims

1. A method for cross-browser window adaptation and trimming of a domestically developed system web browser plugin, characterized in that: The method addresses the issues of stable cross-browser embedding and out-of-bounds display during dynamic interactions of plugins on domestic operating systems. It ensures accurate plugin window placement across multiple browsers through multi-dimensional browser environment identification and dynamic parent window binding. Based on the dynamically assigned visible area boundaries by the browser, it performs differentiated collaborative trimming on the plugin's main window and its upper-level overlay windows, prioritizing their respective dependencies. The plugin, based on its communication connection with the browser, performs multi-dimensional browser environment identification and dynamically binds the correct browser tab window from the system window as its parent window. The browser, according to its own visible area state, dynamically sends parameters containing the browser's visible area boundaries to the plugin. The plugin monitors the relative position of its main window to the boundary. When the display area of ​​the main window exceeds the boundary, the plugin performs an initial cropping of the main window based on the boundary parameters to obtain its effective display area. Based on the effective display area of ​​the main window obtained after the initial cropping, the plugin performs a secondary constraint cropping on the floating layer window above the main window used to display overlay information, removing the portion of the floating layer window that exceeds the effective display area of ​​the main window.

2. The method according to claim 1, characterized in that: The multi-dimensional browser environment identification and dynamic parent window binding includes: the browser sending a location command to the plugin via a script; the plugin obtaining the process identifier of the corresponding browser process based on the current communication connection with the browser, and querying the process name according to the process identifier; determining the browser type and its window title rules by comparing the process name with pre-configured browser process information; subsequently, the plugin matching the received browser tab title in system windows that conform to the window title rules to dynamically bind the current browser tab window as the parent window of the plugin's main window.

3. The method according to claim 2, characterized in that: The browser sends a `getbrowser` command to the plugin via JavaScript. The plugin finds the communication port currently connected to the browser, obtains the corresponding process ID (PID) through that port, queries the process name based on the PID, compares it with the process name in the configuration file, and determines the browser type and browser title format. The browser then sends an `Open` command to the plugin via JavaScript, which contains the title of the current browser tab. The plugin uses the tab title transmitted in the `Open` command to locate the parent window within the associated windows of the determined browser type. If a matching window is located, the browser is successfully identified as the parent window of the plugin's main window; otherwise, the parent window location fails, the plugin terminates and exits.

4. The method according to claim 1, characterized in that: The browser dynamically sends parameter commands containing the browser's visible area boundaries to the plugin based on its own visible area state. This includes: the browser sending window parameter commands containing dynamic boundary parameters to the plugin via a script. The dynamic boundary parameters include at least: the expected position and size of the plugin's main window in the browser, and field information representing the left, top, right, and bottom boundaries of the browser's current visible area.

5. The method according to claim 4, characterized in that: The plug-in monitors the relative position relationship between its main window and the boundary accordingly, including: the plug-in adjusts the display state of the plug-in main window according to the dynamic boundary parameters, and monitors the offset of the plug-in main window relative to the boundary of the browser visible area in real time.

6. The method according to claim 1, characterized in that: The first cropping of the main window includes: when it is monitored that the display area of the main window extends beyond the boundary, the plug-in calculates the offset of each boundary of the main window based on the field information in the boundary parameters, and crops the main window based on the offset.

7. The method according to claim 6, characterized in that: The specific calculation of the offset is as follows: when the values of left, top, right, and bottom representing the left, upper, right, and lower boundaries in the field information are negative, it is determined that the corresponding side is out of bounds, and the cropping offset or the cropped size is calculated according to the negative values respectively; where, if left < 0, the left offset offsetX = 0 - left; if top < 0, the upper offset offsetY = 0 - top; if right < 0, the cropped width offsetW = w + right; if bottom < 0, the cropped height offsetH = h + bottom; where w and h are the original width and height of the plug-in main window respectively.

8. The method according to claim 7, characterized in that: After cropping the main window, its effective display area is obtained, and the upper left corner coordinates, width, and height of the effective display area are determined by the calculated cropping offsets offsetX, offsetY and the cropped sizes offsetW, offsetH.

9. The method according to claim 1, characterized in that: The secondary constrained cropping of the floating window includes: the plug-in obtains the screen coordinates of the floating window that is superimposed and displayed with the video window, and obtains the coordinate range of the effective display area of the main window after the first cropping on the screen; compares the screen coordinates of the floating window with the coordinate range of the effective display area of the main window on the screen, calculates the excess part of the floating window relative to the effective display area of the main window, and crops the excess part.

10. The method according to claim 9, characterized in that: The calculation of the excess part includes the following steps: (a) Obtain the coordinates (Sx, Sy) from the upper left corner of the browser window to the upper left corner of the screen, and multiply Sx and Sy by the screen DPI respectively to obtain the real distances (X, Y); (b) Obtain the relative coordinates (GX, GY, GW, GH) of the effective display area of the main window in the browser, where GX and GY are the relative coordinates of the upper left corner, and GW and GH are the width and height; (c) Obtain the screen coordinates (UX, UY) and its size (UW, UH) of the floating window; (d) Determine the excess parts of each boundary of the floating window through the following judgment and calculation: If UX < GX + X, the left boundary is exceeded, and the left cropping offset offsetX = GX + X - UX; If UY < GY + Y, the upper boundary exceeds, and the upper clipping offset offsetY = GY + Y - UY; if UX + UW > GX + X + GW, the right boundary exceeds, and the clipped width offsetW = (GX + X + GW) - UX; if UY + UH > GY + Y + GH, the lower boundary exceeds, and the clipped height offsetH = (GY + Y + GH) - UY; (e) Clip the floating window according to the calculated offsetX, offsetY, offsetW, and offsetH.

Images