Display interface synchronization method and apparatus, device, storage medium, and program product

Abstract

The present disclosure relates to a display interface synchronization method, device, equipment, storage medium and program product, comprising: in response to the size of a first window of an application program running on a terminal being changed, obtaining the size of the first window after being changed; updating a kernel parameter related to the size of the first window based on the changed size, and adjusting the layout of first display content in a second window of a process corresponding to the application program based on the updated kernel parameter; determining the layout information of a panel interface of a target service called by the application program based on the changed size; and adjusting the layout of second display content in the panel interface based on the layout information and the size of the second display content in the panel interface. In this way, the synchronization between the size of the first window and the layout of the first display content and the layout of the second display content can be achieved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to display interface synchronization methods, apparatus, devices, storage media, and program products. Background Technology

[0002] In modern AI-driven terminals, it is necessary to maintain both the application's corresponding process and the highly dynamic AI real-time user interface (UI). However, in related technologies, application synchronization mechanisms typically only focus on the size synchronization between the physical window and the application's corresponding process, lacking awareness of the synchronization of "application-layer dynamic components (e.g., Rich Live UI)," resulting in a lack of UI synchronization awareness. Summary of the Invention

[0003] This disclosure provides display interface synchronization methods, apparatus, devices, storage media, and program products to at least solve the problem in the aforementioned related technologies where, once the application window is scaled, the UI panel may exceed the application window boundary or obscure the output of native commands.

[0004] According to a first aspect of the present disclosure, a display interface synchronization method is provided, applied to a control scheduler, comprising: in response to a change in the size of a first window of an application running on a terminal, obtaining the changed size of the first window; updating kernel parameters related to the size of the first window based on the changed size, and adjusting the layout of a first display content in a second window of a process corresponding to the application based on the updated kernel parameters; determining layout information of a panel interface of a target service called by the application based on the changed size; and adjusting the layout of the second display content in the panel interface based on the layout information and the size of the second display content in the panel interface, so as to achieve display interface synchronization between the first window, the second window, and the panel interface.

[0005] Optionally, the layout information of the panel interface includes a first horizontal width of the panel interface, and the size of the second displayed content is the number of words contained in the second displayed content; adjusting the layout of the second displayed content in the panel interface based on the layout information and the size of the second displayed content in the panel interface includes: calculating the ratio between the number of words and the first horizontal width as the number of rows corresponding to the second displayed content in the panel interface; and displaying the second displayed content in the panel interface according to the calculated number of rows.

[0006] Optionally, after displaying the second display content in the panel interface according to the calculated number of rows, the method further includes: for each row of content contained in the panel interface, determining whether the last word of the row is fully displayed; if it is determined that the last word of the row is not fully displayed, adjusting the last word to the beginning of the next row for display.

[0007] Optionally, the layout information of the panel interface includes a first horizontal width of the panel interface, and the changed size includes the changed horizontal width; the method further includes: calculating the distance between the panel interface and the vertical boundary of the first window whose size has been changed based on the first horizontal width and the changed horizontal width; and adjusting the display position of the panel interface within the first window whose size has been changed based on the distance.

[0008] Optionally, the layout information of the panel interface includes a first horizontal width of the panel interface; before adjusting the layout of the second display content in the panel interface based on the layout information and the size of the second display content in the panel interface, the method further includes: determining whether the first horizontal width is consistent with the cache width, wherein the cache width is the default horizontal width of the panel interface or the horizontal width of the panel interface calculated after the size of the first window was last changed, wherein adjusting the layout of the second display content in the panel interface based on the layout information and the size of the second display content in the panel interface includes: if the first horizontal width and the cache width are inconsistent, adjusting the layout of the second display content in the panel interface based on the layout information and the size of the second display content in the panel interface.

[0009] Optionally, the kernel parameters include a device file descriptor associated with the process; the step of updating the kernel parameters related to the size of the first window based on the changed size, and adjusting the layout of the first display content in the second window of the process corresponding to the application based on the updated kernel parameters, includes: writing the changed size to the device file descriptor; broadcasting a window change signal to the process; and, in response to the process receiving the window change signal, controlling the process to read the changed size from the device file descriptor, and adjusting the layout of the first display content based on the read changed size.

[0010] Optionally, in response to a change in the size of the first window of an application running on the terminal, obtaining the changed size of the first window includes: in response to a change in the size of the first window of an application running on the terminal, calling a function for reading the size of the first window of the application to obtain a response result, wherein the response result includes first information and second information, the first information indicating the size of the first window, and the second information indicating whether a calling error exists during the calling of the function; if the second information indicates that a calling error exists during the calling of the function, re-calling the function to obtain a response result until the second information indicates that no calling error exists during the calling of the function; if the second information indicates that no calling error exists during the calling of the function, obtaining the changed size based on the first information.

[0011] According to a second aspect of the present disclosure, a display interface synchronization device is provided. The display interface synchronization device includes a control scheduler, comprising: a size acquisition module configured to acquire the changed size of a first window of an application running on a terminal in response to a change in the size of the first window; a first layout adjustment module configured to update kernel parameters related to the size of the first window based on the changed size, and adjust the layout of a first display content in a second window of a process corresponding to the application based on the updated kernel parameters; a layout information determination module configured to determine the layout information of a panel interface of a target service called by the application based on the changed size; and a second layout adjustment module configured to adjust the layout of the second display content in the panel interface based on the layout information and the size of the second display content in the panel interface, so as to achieve display interface synchronization between the first window, the second window, and the panel interface.

[0012] Optionally, the layout information of the panel interface includes a first horizontal width of the panel interface, and the size of the second display content is the number of words contained in the second display content; the second layout adjustment module is configured to: calculate the ratio between the number of words and the first horizontal width as the number of rows corresponding to the second display content in the panel interface; and display the second display content in the panel interface according to the calculated number of rows.

[0013] Optionally, the device further includes: a judgment module configured to determine whether the last word of each line of content in at least one line of content contained in the panel interface is fully displayed; and a display adjustment module configured to adjust the last word to the beginning of the next line for display if it is determined that the last word of the line of content is not fully displayed.

[0014] Optionally, the layout information of the panel interface includes a first horizontal width of the panel interface, and the changed size includes the changed horizontal width; the device further includes: a distance calculation module, configured to calculate the distance between the panel interface and the vertical boundary of the first window whose size has been changed based on the first horizontal width and the changed horizontal width; and a position adjustment module, configured to adjust the display position of the panel interface within the first window whose size has been changed based on the distance.

[0015] Optionally, the layout information of the panel interface includes a first horizontal width of the panel interface; the device further includes: a consistency determination module configured to determine whether the first horizontal width is consistent with the cache width, wherein the cache width is the default horizontal width of the panel interface or the horizontal width of the panel interface calculated after the size of the first window was last changed, and the second layout adjustment module is configured to: adjust the layout of the second display content in the panel interface based on the layout information and the size of the second display content in the panel interface when the first horizontal width is inconsistent with the cache width.

[0016] Optionally, the kernel parameters include a device file descriptor associated with the process; the first layout adjustment module is configured to: write the changed size to the device file descriptor; broadcast a window change signal to the process; and in response to the process receiving the window change signal, control the process to read the changed size from the device file descriptor and adjust the layout of the first display content based on the read changed size.

[0017] Optionally, the size acquisition module is configured to: in response to a change in the size of the first window of an application running on the terminal, call a function for reading the size of the first window of the application to obtain a response result, wherein the response result includes first information and second information, the first information indicating the size of the first window, and the second information indicating whether a calling error exists during the calling of the function; if the second information indicates that a calling error exists during the calling of the function, re-call the function to obtain a response result until the second information indicates that no calling error exists during the calling of the function; if the second information indicates that no calling error exists during the calling of the function, acquire the changed size based on the first information.

[0018] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement a display interface synchronization method according to the present disclosure.

[0019] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform a display interface synchronization method according to the present disclosure.

[0020] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements a display interface synchronization method according to the present disclosure.

[0021] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects: In this disclosure, the layout of the first displayed content in the second window of the process corresponding to the application and the layout of the second displayed content in the panel interface of the target service called by the application can be dynamically adjusted according to the change in the size of the first window of the application running on the terminal, thereby achieving synchronization between the size of the first window, the layout of the first displayed content, and the layout of the second displayed content.

[0022] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0024] Figure 1 This is a schematic diagram illustrating a three-dimensional size synchronization architecture according to an exemplary embodiment of the present disclosure; Figure 2 This is a flowchart illustrating a display interface synchronization method according to an exemplary embodiment of the present disclosure; Figure 3 This is a flowchart illustrating a specific implementation of a display interface synchronization method according to an exemplary embodiment of the present disclosure; Figure 4 This is a block diagram illustrating a display interface synchronization device according to an exemplary embodiment of the present disclosure; Figure 5 This is a block diagram illustrating an electronic device according to exemplary embodiments of the present disclosure. Detailed Implementation

[0025] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0026] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following examples do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0027] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. Another example is "performing at least one of step one and step two", which means the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.

[0028] As mentioned earlier, when the application window is resized, the synchronization mechanisms in related technologies lack awareness of the synchronization of "application-layer dynamic components (e.g., AI-enhanced execution interfaces (AI Shell Rich Execution UI, Rich Live UI))," meaning there is a lack of UI synchronization awareness. Furthermore, this synchronization mechanism also suffers from the following problems: Subprocess redraw lag: Although the pseudo terminal (PTY) size has been updated, due to the non-real-time nature of signal transmission in the Linux kernel, the underlying text-based user interface (TUI) programs (such as Vim and Htop) often cannot respond to scaling within milliseconds, which can cause graphic tearing in the interface during scaling.

[0029] High-concurrency rendering conflicts: When the AI ​​is rapidly outputting inference streams and the user is simultaneously dragging the window, the lack of coordination between the native output buffer and the UI refresh frame can easily lead to overlapping content rendering.

[0030] To address the aforementioned problems in related technologies, the display interface synchronization method, apparatus, device, storage medium, and program product provided in this disclosure can dynamically adjust the layout of the first displayed content in the second window of the process corresponding to the application and the layout of the second displayed content in the panel interface of the target service called by the application based on the change in the size of the first window of the application running on the terminal, thereby achieving synchronization among the size of the first window, the layout of the first displayed content, and the layout of the second displayed content.

[0031] Figure 1 This is a schematic diagram illustrating a three-dimensional size synchronization architecture according to an exemplary embodiment of the present disclosure.

[0032] Reference Figure 1 This disclosure establishes a synchronization architecture centered on a control scheduler, which can be, but is not limited to, an "AIShell control scheduler". The AIShell control scheduler can be regarded as the "brain" of the entire architecture, and it is mainly responsible for capturing the host system's window change signal (SIGWINCH) and initiating the synchronization primitive sequence.

[0033] Furthermore, the synchronization architecture provided in this disclosure enables a three-way endpoint collaborative design, that is, the synchronization path can be abstracted into three core endpoints: endpoint A, endpoint B, and endpoint C. Endpoint A can be a physical window controller, which mainly serves as the source of size synchronization; endpoint B can be a pseudo-terminal slave device (Slave PTY), which mainly serves as the underlying execution end; endpoint C can be a target service engine, which, for example, can be, but is not limited to, a Rich Live UI engine, and mainly serves as the high-level rendering end. The following explanation will use a Rich Live UI engine as the target service engine as an example to illustrate the data processing process of the three-way size synchronization architecture. Unlike traditional bidirectional synchronization, this disclosure also incorporates the state of the UI rendering frame into the atomic sequence of size synchronization.

[0034] Reference Figure 1 Endpoint A, i.e. the input source, can correspond to a physical window on the terminal (Parent Terminal). In response to the user's zooming and dragging operations on the physical window, the physical window can provide original geometric changes, such as, but not limited to, row, column, and pixel coordinates.

[0035] Then, the AIShell control scheduler can listen for window change events, that is, it can capture the window change signal (SIGWINCH) of the physical window mentioned above.

[0036] Next, for the Slave PTY branch, the AIShell control scheduler can inject the scaled-down size of the physical window into the master file descriptor (Master FD) associated with the Slave PTY. Specifically, this can be done via kernel-level ioctl (TIOCSWINSZ), ensuring that the underlying Bash and all TUI child processes derived from it (e.g., Vim, Htop) receive the correct system-level window parameters.

[0037] It should be noted that this disclosure can also provide a signal compensation mechanism: the AIShell control scheduler can also send a forced refresh signal to the child process group to eliminate the redrawing time lag of the TUI program after the PTY update, thereby achieving visually immediate response. That is, this disclosure can also provide a dual signal broadcast mechanism: in addition to updating the kernel size table through the input / output control interface (ioctl) (e.g., window size control code (TIOCSWINSZ)), the AIShell control scheduler can also actively send a specific SIGWINCH to the child process group to forcibly wake up the TUI program's redrawing process, thereby significantly reducing perceived latency.

[0038] For the Rich Live UI branch, specifically the Rich UI component unique to AI Shell, the AI ​​Shell control scheduler can no longer passively wait for the UI framework's automatic layout. Instead, it can actively send "frame reflow" commands to the Rich Live UI renderer. This allows the Rich Live UI renderer to perform streaming layout calculations, dynamically adjusting panel margins and Markdown rendering wrap logic. For example, the Rich UI layer can introduce dynamic margin constraints based on real-time `console.size`, recalculating the panel container's `max_width` in real-time as the physical window scales, and then adjusting the layout of the content displayed within the panel container based on the calculated `max_width`.

[0039] Figure 2 This is a flowchart illustrating a display interface synchronization method according to an exemplary embodiment of the present disclosure, applied to a control scheduler, which may be, but is not limited to, an AIShell control scheduler.

[0040] Reference Figure 2In step 201, in response to a change in the size of the first window of the application running on the terminal, the changed size of the first window can be obtained. For example, when the user manually resizes the first window of the application, the size of the first window changes, and the resized size of the first window can be obtained. For instance, the horizontal width and vertical height of the resized first window can be obtained.

[0041] According to an exemplary embodiment of this disclosure, in response to a change in the size of a first window of an application running on a terminal, a function for reading the size of the first window of the application can be called to obtain a response result. This response result may include first information and second information; the first information may be used to indicate the size of the first window, and the second information may be used to indicate whether a calling error occurred during the function call.

[0042] If the second information indicates that there is a calling error during the function call, the function can be called again to obtain the response result until the second information indicates that there is no calling error during the function call; if the second information indicates that there is no calling error during the function call, the aforementioned changed size can be obtained based on the first information.

[0043] For example, in response to a change in the size of the first window of an application running in the terminal, the internal `_read_terminal_size()` function can be called to obtain a response result. This response result can contain stdout and stderr, where stdout can be used to indicate the size of the first window; stderr can be used to indicate whether there was a call error in the `read_terminal_size()` function. Additionally, the response result can also contain stdin, which is standard input and by default reads data from the keyboard.

[0044] Thus, this disclosure also provides a multi-source size verification mechanism. By polling and verifying stdout and stderr, that is, by polling os.get_terminal_size through the two file descriptors sys.stdout and sys.stderr, it can ensure the extraction of the reliable size after the first window has been changed, and can effectively eliminate the phenomenon of size reading failure due to pipe redirection or pseudo-terminal conflict of a single file descriptor.

[0045] In step 202, the kernel parameters related to the size of the first window can be updated based on the changed size, and the layout of the first displayed content in the second window of the process corresponding to the application can be adjusted based on the updated kernel parameters.

[0046] For example, the fcntl / ioctl system calls can be used to inject the changed size of the first window, i.e., the new winsize structure, into kernel parameters. For instance, the changed size of the first window can be written to the master device file descriptor associated with the Slave PTY, thus updating the window state at the kernel level.

[0047] According to an exemplary embodiment of this disclosure, the kernel parameters described above may include a device file descriptor associated with a process.

[0048] The changed size of the first window can be written to the device file descriptor. Then, a window change signal can be broadcast to a process, which can be the application's shell process and all TUI child processes derived from that shell process. Next, in response to the process receiving the window change signal, the process can read the changed size from the aforementioned device file descriptor and adjust the layout of the first displayed content based on the read changed size.

[0049] It should be noted that some TUI programs (e.g., less pagers or complex Vim plugins) exhibit lag in responding to kernel parameter changes. Therefore, in this disclosure, a window change signal (SIGWINCH) can be broadcast to the entire child process group using os.killpg. This "active wake-up" mechanism effectively reduces the perceived latency of rendering response. For example, by using os.killpg to broadcast (process.pid, signal.SIGWINCH) in shell_pty_executor.py, it can be ensured that not only the shell process, but also all its derived TUI child processes (e.g., pagers) synchronously receive the window change signal, i.e., the scaling instruction. After receiving the window change signal, the underlying child process can autonomously redraw its text cell layout according to the kernel's updated winsize.

[0050] In this way, by setting up an active signal wake-up mechanism, that is, by actively broadcasting SIGWINCH to the process group bound to PTY after physically injecting the size of the first window through ioctl, the scaling response speed of the TUI program can be improved to the level of the native Secure Shell Protocol (SSH), thereby effectively eliminating the display-perceptible latency.

[0051] In step 203, the layout information of the panel interface of the target service called by the application can be determined based on the changed size. For example, the target service may include, but is not limited to, artificial intelligence (AI) services, testing tool services, auxiliary tool services, etc., and this disclosure does not impose specific limitations on this. The following will use an AI service as an example to illustrate the implementation process of display interface synchronization. At this time, in response to the change in the size of the first window of the application running on the terminal, a refresh callback of the application layer Rich Engine can also be triggered synchronously. Specifically, the application layer Rich Engine can obtain the `console.size` at this time, that is, the changed size of the first window, and then calculate the layout information of the AI ​​service's panel interface based on this changed size. For example, the horizontal width (max_width) reserved for the AI ​​panel can be calculated to prevent UI panel overflow in large font sizes or narrow windows.

[0052] For example, when calculating the max_width reserved for the AI ​​panel, two scenarios can be distinguished: when the AI ​​needs to display all output content, the horizontal width of the AI ​​panel can be 0.8 of the horizontal width of the first window; when the AI ​​is in thinking state, the horizontal width of the AI ​​panel can be the horizontal width of the first window - 4.

[0053] It should be noted that the core parameters during size synchronization can exist in the form of a struct winsize extension and can be encapsulated in a synchronization frame object: @dataclass class TerminalSizeFrame: rows: int # Number of rows in the terminal (corresponding to ioctl WS_ROW) cols: int # Number of terminal columns (corresponding to ioctl WS_COL) ui_max_width: int # The adaptive width of the UI panel (cols - offset) timestamp: float # Synchronization timestamp.

[0054] In step 204, the layout of the second display content within the panel interface can be adjusted based on the layout information and the size of the second display content within the panel interface to achieve synchronization of the display interfaces between the first window, the second window, and the panel interface. That is, the high-level Rich UI can dynamically adjust the text wrap width of the Markdown within the panel interface and the panel container margins, i.e., the distance between the panel interface and the vertical boundary of the first window whose size has been changed, based on the calculated constraints.

[0055] According to an exemplary embodiment of this disclosure, the layout information of the panel interface may include a first horizontal width of the panel interface, and the size of the second display content may be the number of words contained in the second display content, that is, the number of characters contained in the second display content. For example, the first horizontal width may be 10, and the number of words contained in the second display content may be 30.

[0056] The ratio between the number of word elements and the first horizontal width can be calculated as the number of rows corresponding to the second displayed content within the panel interface. For example, 30 / 10 = 3, therefore, the number of rows corresponding to the second displayed content can be 3. Next, the second displayed content can be displayed within the panel interface according to the calculated number of rows. For example, the second displayed content can be displayed in 3 rows.

[0057] According to an exemplary embodiment of this disclosure, after displaying the second display content in the panel interface according to the calculated number of lines, for each line of content contained in the panel interface, it can be determined whether the last word of the line is fully displayed. If it is determined that the last word of the line is not fully displayed, the last word can be adjusted to the beginning of the next line for display.

[0058] For example, for the word "temperature," the first part "tempera" might appear at the end of a line, while the second part "ture" appears on the next line, resulting in poor readability. In this case, "tempera," which would normally appear at the end of a line, could be moved to the beginning of the next line. This ensures that each line in the panel displays complete words, improving readability.

[0059] According to an exemplary embodiment of this disclosure, the layout information of the panel interface may include a first horizontal width of the panel interface, and the changed size may include the changed horizontal width.

[0060] The distance between the panel interface and the vertical boundary of the first window whose size has been changed can also be calculated based on the first horizontal width and the changed horizontal width. Then, the display position of the panel interface within the first window whose size has been changed can be adjusted based on the calculated distance.

[0061] In this way, in addition to adjusting the layout of the content displayed in the AI ​​panel, the overall display position of the AI ​​panel within the application window can also be adjusted, which can effectively prevent the AI ​​panel from exceeding the boundary of the first window or obscuring other content within the first window due to scaling of the first window.

[0062] According to an exemplary embodiment of this disclosure, the layout information of the panel interface may include a first horizontal width of the panel interface.

[0063] It can also be determined whether the first horizontal width is consistent with the cache width, where the cache width can be the default horizontal width of the panel interface or the horizontal width of the panel interface calculated after the size of the first window was last changed. Then, if the first horizontal width and the cache width are inconsistent, the layout of the second display content in the panel interface can be adjusted based on the aforementioned layout information and the size of the second display content in the panel interface.

[0064] It should be noted that this disclosure provides a latency compensation buffer mechanism. In the UI refresh loop, by comparing the last_size cache, expensive rich redraws can be performed only when the geometric parameters actually change, thereby effectively avoiding unnecessary CPU consumption.

[0065] Furthermore, after the three-way state alignment is achieved, the aligned interface can be output to the user window, ensuring that the AI ​​inference animation and the command output in its background remain geometrically aligned even during high-speed scaling. This means that even during real-time scaling of the application's first window during extremely long inference flows, the system can still guarantee the integrity of Markdown rendering and panel coordinate alignment, improving the robustness of content display. Additionally, it ensures zero visual tearing, preventing the AI ​​inference information and TUI command content from overlapping when the application's first window is scaled.

[0066] Figure 3 This is a flowchart illustrating a specific implementation of a display interface synchronization method according to an exemplary embodiment of the present disclosure.

[0067] Reference Figure 3The display interface synchronization process specifically involves the operating system, AIShell control scheduler, pseudo-terminal slave devices (Slave PTYs), and AI-enhanced execution interface renderer. The operating system primarily monitors the scaling of application windows and notifies the AIShell control scheduler of this scaling. The AIShell control scheduler can be considered the control center of the entire display interface synchronization process; it mainly captures window change signals from the host system and controls the three ends to achieve interface synchronization. The pseudo-terminal slave device mainly acts as the low-level execution end, primarily responsible for redrawing the layout within the window of the application's corresponding process. The AI-enhanced execution interface renderer mainly acts as the high-level rendering end, primarily responsible for redrawing the layout of the AI ​​service's panel interface.

[0068] Reference Figure 3 In step 301, when the user manually resizes the first window of the application, the host operating system's window manager can send a window change signal (SIGWINCH) to the AIShell control scheduler.

[0069] In step 302, the AIShell control scheduler, in response to the received window change signal, can call the internal `_read_terminal_size()` function to obtain the response result, i.e., it can perform multi-FD combination verification. This response result can include stdout and stderr, where stdout can be used to indicate the size of the first window; stderr can be used to indicate whether there was a calling error during the call to the `read_terminal_size()` function.

[0070] In step 303a, after the AIShell control scheduler reads the changed size of the application's first window from Stdout, it can use the fcntl / ioctl system calls to inject the changed size of the first window into kernel parameters.

[0071] In step 303b, the AIShell control scheduler can broadcast a window change signal (SIGWINCH) to the child process group.

[0072] In step 303c, the AIShell control scheduler can trigger the _refresh_live_for_resize() logic, that is, the AIShell control scheduler can trigger the refresh callback of the application layer Rich Live UI engine.

[0073] In step 304, after the child process (e.g., Vim or Htop) receives the broadcast window change signal, it can autonomously redraw its geometric layout, i.e., the text cell layout, according to the changed size updated by the kernel.

[0074] In step 305, the Rich Live UI engine can calculate the horizontal width (max_width) reserved for the AI ​​panel, and then dynamically adjust the line break width of the Markdown text within the panel interface and the margins of the panel container based on the calculated max_width.

[0075] In step 306, after the three dimensions are aligned, the rendered lossless interface can be displayed to the user.

[0076] This approach can cover the entire closed-loop process of "physical size acquisition, kernel PTY parameter injection, and application layer UI layout recalculation," ensuring that the scaling states of the three remain consistent. This enables lossless visual synchronization in scenarios where high-concurrency AI inference streams and high-frequency physical scaling are intertwined.

[0077] Figure 4 This is a block diagram illustrating a display interface synchronization device 400 according to an exemplary embodiment of the present disclosure, which may include a control scheduler.

[0078] Reference Figure 4 The display interface synchronization device 400 may include a size acquisition module 401, a first layout adjustment module 402, a layout information determination module 403, and a second layout adjustment module 404.

[0079] The size acquisition module 401 responds to a change in the size of the first window of the application running on the terminal, and can acquire the changed size of the first window. For example, when the user manually resizes the first window of the application, the size of the first window changes, and the resized size of the first window can be acquired. For instance, the horizontal width and vertical height of the resized first window can be acquired.

[0080] According to an exemplary embodiment of this disclosure, in response to a change in the size of the first window of an application running on the terminal, the size acquisition module 401 may call a function for reading the size of the first window of the application to obtain a response result. This response result may include first information and second information; the first information may be used to indicate the size of the first window, and the second information may be used to indicate whether a calling error occurred during the function call.

[0081] If the second information indicates that there is a calling error during the function call, the size acquisition module 401 can call the function again to obtain the response result until the second information indicates that there is no calling error during the function call; if the second information indicates that there is no calling error during the function call, the size acquisition module 401 can obtain the changed size based on the first information.

[0082] The first layout adjustment module 402 can update kernel parameters related to the size of the first window based on the changed size, and can adjust the layout of the first displayed content in the second window of the process corresponding to the application based on the updated kernel parameters.

[0083] According to an exemplary embodiment of this disclosure, the kernel parameters described above may include a device file descriptor associated with a process.

[0084] The first layout adjustment module 402 can write the changed size of the first window to the device file descriptor. Then, the first layout adjustment module 402 can broadcast a window change signal to a process, which can be the Shell process corresponding to the application and all TUI child processes derived from the Shell process. Next, in response to the process receiving the window change signal, the first layout adjustment module 402 can control the process to read the changed size from the aforementioned device file descriptor and adjust the layout of the first displayed content based on the read changed size.

[0085] The layout information determination module 403 can determine the layout information of the panel interface of the target service called by the application based on the changed size.

[0086] The second layout adjustment module 404 can adjust the layout of the second display content in the panel interface based on the layout information and the size of the second display content in the panel interface, so as to achieve synchronization of the display interface between the first window, the second window and the panel interface.

[0087] According to an exemplary embodiment of this disclosure, the layout information of the panel interface may include a first horizontal width of the panel interface, and the size of the second display content may be the number of words contained in the second display content. For example, the first horizontal width may be 10, and the number of words contained in the second display content may be 30.

[0088] The second layout adjustment module 404 can calculate the ratio between the number of word elements and the first horizontal width as the number of rows corresponding to the second display content within the panel interface. For example, 30 / 10=3, therefore, the number of rows corresponding to the second display content can be 3. Next, the second layout adjustment module 404 can display the second display content within the panel interface according to the calculated number of rows. For example, the second display content can be displayed in 3 rows.

[0089] According to an exemplary embodiment of this disclosure, the above-described display interface synchronization device 400 may further include a judgment module and an adjustment display module.

[0090] After displaying the second content in the panel interface according to the calculated number of rows, the judgment module can further determine whether the last word of each line in the panel interface is fully displayed. If it is determined that the last word of the line is not fully displayed, the display adjustment module can adjust the last word to the beginning of the next line for display.

[0091] According to an exemplary embodiment of this disclosure, the display interface synchronization device 400 may further include a distance calculation module and a position adjustment module. The layout information of the panel interface may include a first horizontal width of the panel interface, and the changed size may include the changed horizontal width.

[0092] The distance calculation module can calculate the distance between the panel interface and the vertical boundary of the first window whose size has been changed, based on the first horizontal width and the changed horizontal width. Then, the position adjustment module can adjust the display position of the panel interface within the first window whose size has been changed based on the calculated distance.

[0093] According to an exemplary embodiment of this disclosure, the display interface synchronization device 400 may further include a consistency determination module. The layout information of the panel interface may include a first horizontal width of the panel interface.

[0094] The consistency determination module can determine whether the first horizontal width is consistent with the cache width, wherein the cache width can be the default horizontal width of the panel interface or the horizontal width of the panel interface calculated after the size of the first window was last changed. Then, if the first horizontal width and the cache width are inconsistent, the second layout adjustment module 404 can adjust the layout of the second display content in the panel interface based on the aforementioned layout information and the size of the second display content in the panel interface.

[0095] Figure 5 This is a block diagram illustrating an electronic device 500 according to an exemplary embodiment of the present disclosure.

[0096] Reference Figure 5 The electronic device 500 includes at least one memory 501 and at least one processor 502. The at least one memory 501 stores instructions that, when executed by the at least one processor 502, execute a display interface synchronization method according to an exemplary embodiment of the present disclosure.

[0097] As an example, electronic device 500 may be a PC, tablet, personal digital assistant, smartphone, or other device capable of executing the aforementioned instructions. Here, electronic device 500 is not necessarily a single electronic device, but may be a collection of any devices or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. Electronic device 500 may also be part of an integrated control system or system manager, or may be configured to interconnect with a portable electronic device locally or remotely (e.g., via wireless transmission) through an interface.

[0098] In electronic device 500, processor 502 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, processor may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, etc.

[0099] The processor 502 can execute instructions or code stored in the memory 501, which can also store data. Instructions and data can also be sent and received over a network via a network interface device, which can employ any known transmission protocol.

[0100] The memory 501 may be integrated with the processor 502, for example, by arranging RAM or flash memory within an integrated circuit microprocessor. Alternatively, the memory 501 may include a separate device, such as an external disk drive, a storage array, or other storage device usable by any database system. The memory 501 and the processor 502 may be operatively coupled, or may communicate with each other, for example, via I / O ports, network connections, etc., enabling the processor 502 to read files stored in the memory.

[0101] In addition, electronic device 500 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of electronic device 500 can be interconnected via a bus and / or network.

[0102] According to exemplary embodiments of this disclosure, a computer-readable storage medium may also be provided, which, when executed by a processor of an electronic device, enables the electronic device to perform the aforementioned display interface synchronization method. Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and to provide the computer program and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, servers, etc. Furthermore, in one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.

[0103] According to exemplary embodiments of the present disclosure, a computer program product may also be provided, including a computer program that, when executed by a processor, implements the display interface synchronization method according to the present disclosure.

[0104] According to the display interface synchronization method, apparatus, device, storage medium and program product disclosed herein, the layout of the first display content in the second window of the process corresponding to the application and the layout of the second display content in the panel interface of the target service called by the application can be dynamically adjusted according to the change in the size of the first window of the application running on the terminal. This can keep the scaling state of the three consistent, thereby achieving lossless visual synchronization in high-frequency physical scaling scenarios.

[0105] According to an exemplary embodiment of this disclosure, a multi-source size verification mechanism is also provided. By polling and verifying stdout and stderr, that is, by polling os.get_terminal_size through the two file descriptors sys.stdout and sys.stderr, it can ensure the extraction of the reliable size after the first window has been changed, and can effectively eliminate the phenomenon of size reading failure due to pipe redirection or pseudo-terminal conflict of a single file descriptor.

[0106] According to an exemplary embodiment of this disclosure, by setting an active signal wake-up mechanism, that is, by actively broadcasting SIGWINCH to the process group bound to PTY after physically injecting the size of the first window by ioctl, the scaling response speed of the TUI program can be improved to the native SSH level, thereby effectively eliminating the display perceived latency.

[0107] According to exemplary embodiments of this disclosure, it can be ensured that the content displayed in each row of the panel interface is a complete word, which can improve readability.

[0108] According to an exemplary embodiment of this disclosure, in addition to adjusting the layout of the content displayed in the AI ​​panel, the overall display position of the AI ​​panel in the application window can also be adjusted, which can effectively avoid the phenomenon that the AI ​​panel exceeds the boundary of the first window or obscures other content in the first window due to the scaling of the first window.

[0109] According to exemplary embodiments of this disclosure, a latency compensation buffer mechanism is also provided. In the UI refresh loop, by using a last_size cache comparison, expensive rich redraws can be performed only when geometric parameters actually change, thereby effectively avoiding unnecessary CPU consumption.

[0110] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0111] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for synchronizing a display interface, characterized in that, Applied to control schedulers, including: In response to a change in the size of the first window of an application running on the terminal, obtain the changed size of the first window; Based on the changed size, update the kernel parameters related to the size of the first window, and adjust the layout of the first displayed content in the second window of the process corresponding to the application based on the updated kernel parameters; Based on the changed size, determine the layout information of the panel interface of the target service called by the application; Based on the layout information and the size of the second display content within the panel interface, the layout of the second display content within the panel interface is adjusted to achieve display interface synchronization between the first window, the second window, and the panel interface.

2. The method as described in claim 1, characterized in that, The layout information of the panel interface includes the first horizontal width of the panel interface, and the size of the second display content is the number of words contained in the second display content; Adjusting the layout of the second displayed content within the panel interface based on the layout information and the size of the second displayed content within the panel interface includes: The ratio between the number of the word elements and the first horizontal width is used as the number of rows corresponding to the second displayed content in the panel interface; The second display content is displayed in the panel interface according to the calculated number of rows.

3. The method as described in claim 2, characterized in that, After displaying the second display content in the panel interface according to the calculated number of rows, the method further includes: For each line of content contained in at least one line of content within the panel interface, determine whether the last word of that line of content is displayed completely; If it is determined that the last word of the line is not fully displayed, the last word is moved to the beginning of the next line for display.

4. The method as described in claim 1, characterized in that, The layout information of the panel interface includes the first horizontal width of the panel interface, and the changed size includes the changed horizontal width. The method further includes: Based on the first horizontal width and the changed horizontal width, calculate the distance between the panel interface and the vertical boundary of the first window whose size has been changed; Based on the distance, adjust the display position of the panel interface within the first window whose size has been changed.

5. The method as described in claim 1, characterized in that, The layout information of the panel interface includes the first horizontal width of the panel interface; Before adjusting the layout of the second displayed content within the panel interface based on the layout information and the size of the second displayed content within the panel interface, the method further includes: Determine whether the first horizontal width is consistent with the cache width, wherein the cache width is either the default horizontal width of the panel interface or the horizontal width of the panel interface calculated after the last change in the size of the first window. The step of adjusting the layout of the second displayed content within the panel interface based on the layout information and the size of the second displayed content within the panel interface includes: If the first horizontal width is inconsistent with the cache width, the layout of the second display content in the panel interface is adjusted based on the layout information and the size of the second display content in the panel interface.

6. The method as described in claim 1, characterized in that, The kernel parameters include the device file descriptor associated with the process; The step of updating kernel parameters related to the size of the first window based on the changed size, and adjusting the layout of the first displayed content within the second window of the process corresponding to the application based on the updated kernel parameters, includes: Write the changed dimensions into the device file descriptor; Broadcast window change signal to the process; In response to the process receiving the window change signal, the process is controlled to read the changed size from the device file descriptor and adjust the layout of the first display content based on the read changed size.

7. The method as described in claim 1, characterized in that, The process of obtaining the changed size of the first window in response to a change in the size of an application running on the terminal includes: In response to a change in the size of the first window of an application running on the terminal, a function for reading the size of the first window of the application is called to obtain a response result, wherein the response result includes first information and second information, the first information being used to indicate the size of the first window, and the second information being used to indicate whether there was a calling error during the calling of the function; If the second information indicates that there is a calling error during the function call, the function is called again to obtain the response result until the second information indicates that there is no calling error during the function call; If the second information indicates that there is no calling error during the function call, the changed size is obtained based on the first information.

8. A display interface synchronization device, characterized in that, The display interface synchronization device includes a control scheduler, comprising: The size acquisition module is configured to acquire the size of the first window after the size of the first window of the application running on the terminal is changed. The first layout adjustment module is configured to update kernel parameters related to the size of the first window based on the changed size, and adjust the layout of the first display content in the second window of the process corresponding to the application based on the updated kernel parameters. The layout information determination module is configured to determine the layout information of the panel interface of the target service called by the application based on the changed size. The second layout adjustment module is configured to adjust the layout of the second display content in the panel interface based on the layout information and the size of the second display content in the panel interface, so as to achieve display interface synchronization between the first window, the second window and the panel interface.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the display interface synchronization method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the display interface synchronization method as described in any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the display interface synchronization method as described in any one of claims 1 to 7.

Images