Multi-screen continuous display interaction method, device and equipment of document view and medium

Abstract

The present application relates to a kind of multi-screen continuous display interaction method, device, equipment and medium of document view, involve terminal interaction field, this method includes: the number of display endpoint is obtained and arrangement direction, according to the number of display endpoint and arrangement direction, construct the virtual canvas covering all display endpoint;The document view of the preset document session is mapped to virtual canvas, and the viewport corresponding to each display endpoint in virtual canvas is controlled to present;Receive the input event of acting on any display endpoint, input event carries the input coordinate based on the coordinate system of this display endpoint itself;According to the position offset of the viewport corresponding to this display endpoint in virtual canvas, input coordinate is converted into the logic coordinate of document view;Logic coordinate is back to document view, to drive the editing, selection or scrolling state of the same document content.It is realized that single document view is continuously shown between multiple display endpoints and the effect of consistent interaction across screen.

Description

Technical Field

[0001] This invention relates to the field of terminal interaction technology, and in particular to a method, apparatus, device, and medium for multi-screen continuous display interaction of a document view. Background Technology

[0002] When opening office documents such as Word, PowerPoint, and Excel on existing dual-screen or multi-screen terminals, only one screen is typically used, leaving the other screens idle, or the same image is mirrored across multiple screens, without providing users with more reading and operational space. If the same file is opened on two separate screens, two independent document sessions are created. After scrolling, selecting, or modifying in one window, the other cannot automatically maintain the same position and editing state. In some cases, one window may have been modified while the other still displays the old content. Furthermore, the parallel operation of two document engines increases memory, CPU, and graphics rendering overhead. Even when a long screen is divided into upper and lower parts, the coordinates of a finger on the secondary screen are usually recalculated from the top of the primary screen. If these coordinates are not converted to the coordinates of the entire logical canvas, issues may arise such as clicking a row on the lower screen but selecting the corresponding position on the upper screen, or swiping the secondary screen not driving the same document. When the secondary screen is disconnected or fails to be created, the application may also leave behind double-height, black areas, or incorrect orientations, rendering the single screen unusable. Summary of the Invention

[0003] This invention provides a method for continuous multi-screen display and interaction of a document view, in order to solve the problem of how to achieve continuous display of document content and consistent cross-screen interaction with a single document view across multiple display endpoints on a terminal with multiple display endpoints.

[0004] In a first aspect, the present invention provides a multi-screen continuous display interaction method for a document view, the method comprising: Obtain the number and arrangement direction of the display endpoints, and construct a virtual canvas covering all display endpoints based on the number and arrangement direction of the display endpoints; The document view of the preset document session is mapped to the virtual canvas, and each display endpoint is controlled to present the viewport corresponding to it in the virtual canvas; wherein, the document session includes a document content and a document view, and the document view is the only view that carries the rendering and input processing of the document content; Receive an input event applied to any display endpoint, the input event carrying input coordinates based on the display endpoint's own coordinate system; Based on the position offset of the viewport corresponding to the display endpoint in the virtual canvas, the input coordinates are converted into logical coordinates of the document view; The logical coordinates are fed back to the document view to drive the editing, selection, or scrolling of the same document content.

[0005] Secondly, the present invention provides a multi-screen continuous display interactive device for a document view, comprising: The canvas construction module is used to obtain the number and arrangement direction of the display endpoints, and construct a virtual canvas covering all display endpoints based on the number and arrangement direction of the display endpoints; The viewport rendering module is used to map the document view of a preset document session to the virtual canvas and control each display endpoint to render the viewport corresponding to it in the virtual canvas; wherein, the document session includes a document content and a document view, and the document view is the only view that carries the rendering and input processing of the document content; An input receiving module is used to receive input events acting on any display endpoint, wherein the input event carries input coordinates based on the display endpoint's own coordinate system; The coordinate transformation module is used to convert the input coordinates into logical coordinates of the document view based on the position offset of the viewport corresponding to the display endpoint in the virtual canvas. The coordinate back-annotation module is used to back-annotate the logical coordinates to the document view to drive the editing, selection, or scrolling of the same document content.

[0006] Thirdly, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements the steps of the multi-screen continuous display interaction method for a document view as described in any embodiment of the first aspect.

[0007] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the multi-screen continuous display interaction method for a document view as described in any embodiment of the first aspect.

[0008] The technical solutions provided in the embodiments of the present invention have the following advantages compared with the prior art: By maintaining a single document session and uniformly mapping the viewports of each display endpoint with a virtual canvas covering all display endpoints, multiple display endpoints can continuously display different sections of the document while sharing the same document state, thus expanding the scope of visibility and operability and avoiding dual-session state drift and mirrored duplicate display.

[0009] By uniformly converting the input coordinates of each display endpoint, carrying its own coordinate system, into logical coordinates of the document view according to the position offset of its corresponding viewport in the virtual canvas and then back-annotating them, the consistency and accuracy of cross-screen interaction are ensured. This ensures that clicks, swipes, or edits on any screen affect the same document and the same set of cursors, selections, and scrolling states. In addition, when a display endpoint is removed, closed, or malfunctions, a unified backtracking path is used to restore a single screen, improving the stability of switching between dual screens and single screens. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart illustrating a multi-screen continuous display interaction method for a document view provided in an embodiment of the present invention; Figure 2 A structural block diagram of a multi-screen continuous display interactive device for a document view provided in an embodiment of the present invention; Figure 3 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the effect of a single document view being continuously displayed across multiple display endpoints, as provided in an embodiment of the present invention. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0014] Example Figure 1 This is a flowchart illustrating a multi-screen continuous display interaction method for a document view provided in an embodiment of the present invention. The present invention proposes a multi-screen continuous display interaction method for a document view; specifically, see [link to documentation]. Figure 1The multi-screen continuous display interaction method of this document view includes the following steps S101-S105.

[0015] S101, obtain the number and arrangement direction of the display endpoints, and construct a virtual canvas covering all display endpoints based on the number and arrangement direction of the display endpoints.

[0016] S102, map the document view of the preset document session to the virtual canvas, and control each display endpoint to present the viewport corresponding to it in the virtual canvas; wherein, the document session includes a document content and a document view, and the document view is the only view that carries the rendering and input processing of the document content.

[0017] S103, receive an input event acting on any display endpoint, the input event carrying input coordinates based on the display endpoint's own coordinate system.

[0018] S104, based on the position offset of the viewport corresponding to the display endpoint in the virtual canvas, the input coordinates are converted into logical coordinates of the document view.

[0019] S105, the logical coordinates are fed back to the document view to drive the editing, selection, or scrolling state of the same document content.

[0020] In this embodiment, a display endpoint refers to a display screen or display area on a terminal that can independently present images and receive input, such as a main screen, a secondary screen, or an external monitor; a virtual canvas refers to a virtual display area that is continuous in logical coordinates and covers all display endpoints. The document view is mapped to this canvas, and each display endpoint only presents its corresponding local segment in the canvas, i.e., the viewport; a document session refers to a running instance that carries an editable document, including a document content and a document view, which is the only view that carries the rendering and input processing of the document content; a self-coordinate system refers to a coordinate system established by a display endpoint with its own upper left corner as the origin; logical coordinates refer to the coordinates relative to the entire virtual canvas, i.e., the unified document view, and are the coordinate reference that each display endpoint input must be unified to before feedback.

[0021] In a specific embodiment, see Figure 4Taking a terminal with two screens as an example, both screens have a resolution of 1920×1280 and a refresh rate of 60Hz. The upper screen is the main screen, and the lower screen is the secondary screen. When a user opens a PPT document from the main screen, the system only starts one document engine session, maintains one editable document model, and one document view. The system first obtains that the number of display endpoints is 2 and the arrangement direction is vertical. Based on this, it constructs a virtual canvas covering both screens, with a logical size of 1920 width and 2560 height. The upper screen corresponds to Y∈[0, 1280), and the lower screen corresponds to Y∈[1280, 2560). Then, the document view is mapped to this virtual canvas, and the main screen is controlled to display the [0, 1280) segment of the canvas, and the secondary screen is controlled to display the [1280, 2560) segment of the canvas. This allows pages 4 and 5 of the PPT to be displayed on the upper screen, and pages 6 and 7 to be displayed on the lower screen. The end of a Word document can also be displayed on the lower screen, and adjacent rows in an Excel document can be displayed continuously across screens. When the secondary screen detects a touch event with physical coordinates (960, 1000) based on its own coordinate system, the system converts these input coordinates into logical coordinates (960, 2280) according to the secondary screen's viewport offset in the virtual canvas (Y-direction offset of 1280). This result is then fed back into the single document view, driving scrolling or selection of the same document. In three tests, this mapping consistently yielded logical Y=2280, demonstrating that secondary screen swiping accurately drives the same document. Therefore, scrolling, selection, or editing on either screen affects the same document state.

[0022] The implementation scheme achieves continuous multi-screen display using a single document session and a unified virtual canvas, and converts the inputs from each screen back to the same document coordinates. Multiple display endpoints continuously display different sections of the document while sharing the same document state. This expands the visual and operable range, ensures the consistency and accuracy of cross-screen interaction, and avoids dual-session state drift and mirrored repetitive display.

[0023] In one embodiment, the arrangement direction includes vertical and horizontal arrangement; the total size of the virtual canvas along the arrangement direction is determined by the sum of the viewport sizes of each display endpoint, each display endpoint corresponds to a continuous coordinate interval in the virtual canvas, and the viewports of each display endpoint are sequentially spliced ​​together along the arrangement direction.

[0024] In this embodiment, the arrangement direction refers to the splicing orientation of multiple display endpoints in physical space, including at least vertical and horizontal arrangement; the viewport size refers to the resolution size of a single display endpoint along its arrangement axis; and the continuous coordinate interval refers to a range of coordinates on the logical coordinate axis of the virtual canvas, which is occupied by each display endpoint in sequence, connected end to end and without overlap.

[0025] In a specific embodiment, the total size of the virtual canvas along the arrangement direction is determined by the sum of the viewport sizes of each display endpoint. Taking two vertically arranged 1920×1280 screens as an example, the total height of the virtual canvas is 1280+1280=2560, and the width is 1920; each display endpoint corresponds to a continuous coordinate interval in the virtual canvas: the main screen corresponds to Y∈[0, 1280), and the secondary screen corresponds to Y∈[1280, 2560). The two are sequentially adjacent and non-overlapping along the vertical direction. Taking two horizontally arranged 1920×1280 screens as an example, the total width of the virtual canvas is 1920+1920=3840, and the height is 1280; the left screen corresponds to X∈[0, 1920), and the right screen corresponds to X∈[1920, 3840). The two are sequentially adjacent and non-overlapping along the horizontal direction. Because each viewport is connected end-to-end on the logical coordinate axis, the document content can be arranged continuously after crossing the physical seams of the screen, with the content of the lower or right screen directly continuing from the content of the upper or left screen.

[0026] The implementation achieves a continuous logical canvas covering all endpoints by accumulating the sizes of each viewport according to the arrangement direction, and by sequentially splicing each viewport adjacent to each other along the arrangement direction. The document content can be naturally and continuously arranged across the screen, which ensures the integrity and continuity of multi-screen display from the mechanism, rather than each screen presenting an isolated picture or a repeated mirror.

[0027] In one embodiment, controlling each display endpoint to present its corresponding viewport in the virtual canvas includes extracting the coordinate range corresponding to the display endpoint from the virtual canvas and outputting it to the display endpoint by at least one of cropping, translating, or sharing a texture source rectangle.

[0028] In this embodiment, cropping refers to extracting a pixel region corresponding to the coordinate interval of a certain display endpoint from the virtual canvas and outputting it to the endpoint; translation refers to translating the entire virtual canvas so that the target coordinate interval falls into the display range of the endpoint before outputting it; sharing the texture source rectangle refers to selecting only the coordinate interval corresponding to the endpoint in the virtual canvas in the GPU texture as the source rectangle for drawing, so that multiple endpoints share different segments of the same texture.

[0029] In a specific embodiment, controlling each display endpoint to present its corresponding viewport in the virtual canvas can be achieved through at least one of clipping, translation, or sharing a texture source rectangle. For a secondary screen endpoint based on Presentation or Surface, the system extracts the coordinate range corresponding to the endpoint from the virtual canvas and outputs it: one implementation uses clipping, setting the source rectangle of the secondary screen viewport to the [1280, 2560) segment in the virtual canvas and then calling the clipping output of the drawing interface; another implementation uses a shared texture source rectangle, treating the entire virtual canvas as a single GPU texture, with the secondary screen using only the [1280, 2560) segment as the texture source rectangle, while the main screen uses the [0, 1280) segment as the source rectangle, sharing different segments of the same texture; yet another implementation uses translation, translating the virtual canvas by -1280 in the Y-axis direction and outputting the portion that falls into the secondary screen display area. Any of the above methods makes the secondary screen present a real segment adjacent to the logical canvas that is different from the "second document, second engine, or static mirror".

[0030] The implementation example extracts and outputs the coordinate ranges corresponding to each endpoint from the virtual canvas by at least one method such as cropping, translating, or sharing the texture source rectangle. The implementation method is flexible, which can adapt to different display endpoint types and ensure that the secondary screen presents the real adjacent segments in the logical canvas rather than a mirror or repeated image.

[0031] In one embodiment, when a change in display topology is detected, the display code is incremented, and the incremented display code is associated with the topology configuration obtained in the current detection; the viewport of each display endpoint is determined only when the display code is still the current value.

[0032] In this embodiment, the display code is a counter that increments each time the display topology is renegotiated, also known as displayGeneration; the display topology refers to the layout information such as the number, arrangement direction, size and parameters of each display endpoint; the topology configuration refers to the configuration result obtained from a display topology detection and used to determine the viewport configuration of each display endpoint.

[0033] In a specific embodiment, the system monitors the addition, removal, and parameter changes of display endpoints. Each time the display topology is renegotiated, the display code is incremented, and the incremented display code is associated with the topology configuration obtained in the current detection and recorded. Only when the display code is still the current value, i.e., not covered by subsequent detections, is the topology configuration obtained in the current detection used to determine the viewport of each display endpoint. If a callback carrying an old code, such as an expired secondary screen creation success notification, an old Presentation, or an old touch event, arrives after the generational advancement, it is directly discarded, and the expired display topology cannot be restored. For example, after the Nth round of detection of the secondary screen (code N) issues a creation request, the user's plug-and-play triggers the N+1th round of detection (code N+1). When the delayed "secondary screen creation successful" callback of the Nth round returns with code N, and the current code is already N+1, it is discarded if they do not match, and the N+1th round callback correctly determines the state, thus avoiding the old configuration from overwriting the new state.

[0034] The implementation effectively shields outdated events during the asynchronous topology detection process by displaying generational increments and a verification mechanism that "only adopts topology configuration when the generation number is still the current value," thus preventing old layouts from polluting the current state and improving stability in scenarios involving hot-swapping and changes in screen parameters.

[0035] In one embodiment, step S104 specifically includes: Perform an inverse transformation on the input coordinates to remove the scaling or rotation applied to the display endpoint, and then superimpose the position offset onto the inversely transformed input coordinates to obtain the logical coordinates; When arranged vertically, the position offset is the cumulative height of all display endpoints before the current display endpoint; when arranged horizontally, the position offset is the cumulative width of all display endpoints before the current display endpoint.

[0036] In this embodiment, transformation refers to the reverse restoration of input coordinates to eliminate the scaling or rotation applied to a certain display endpoint; position offset refers to the cumulative displacement of the viewport corresponding to the display endpoint relative to the origin of the virtual canvas; cumulative height refers to the sum of the heights of all display endpoints before the current endpoint when arranged vertically; and cumulative width refers to the sum of the widths of all display endpoints before the current endpoint when arranged horizontally.

[0037] In a specific embodiment, the input coordinates are converted into logical coordinates based on the position offset of the viewport corresponding to the display endpoint in the virtual canvas. This includes: first, performing an inverse transformation on the input coordinates to remove the scaling or rotation applied to the display endpoint; then, superimposing the position offset onto the inversely transformed input coordinates to obtain the logical coordinates. In a vertical arrangement, the position offset is the cumulative height of all display endpoints preceding the current display endpoint; in a horizontal arrangement, the position offset is the cumulative width of all display endpoints preceding the current display endpoint. Taking a vertical dual-screen setup as an example, with only the main screen preceding the secondary screen and a cumulative height of 1280, when the physical coordinates of the secondary screen are (960, 1000), they are first restored to standard screen coordinates through an inverse transformation. In this example, the secondary screen has not been scaled or rotated, so the inverse transformation still results in (960, 1000). Then, the position offset + 1280 is added to obtain the logical coordinates (960, 2280), which are then reflected back into the unique document view. In a horizontal dual-screen setup, the right screen's position offset is 1920 units of the main screen's width, and the right screen's physical X-coordinate is superimposed by 1920 before being reflected back into the viewport.

[0038] The implementation example achieves accurate restoration of the input of any display endpoint carrying its own coordinate system to the logical coordinates of the entire virtual canvas through a two-step transformation of "inverse transformation to remove the endpoint's own pose and superimposing the viewport position offset". This mechanism eliminates the misalignment of "clicking a row on the lower screen but selecting the same position on the upper screen", ensuring the consistency of cross-screen input.

[0039] In one embodiment, the inverse transformation is: based on the scaling ratio and rotation angle of the display endpoint, its physical pixel coordinates are restored to standard screen coordinates.

[0040] In this embodiment, scaling ratio refers to the factor by which a display endpoint magnifies or reduces the content it presents on the display scale; rotation angle refers to the rotation angle of the display endpoint relative to the standard display posture; physical pixel coordinates refer to the original input coordinates from the endpoint hardware without coordinate restoration; standard screen coordinates refer to the normalized display coordinates after removing the effects of scaling and rotation, serving as a unified intermediate coordinate before the overlay position offset.

[0041] In a specific embodiment, the inverse transformation is as follows: based on the scaling ratio and rotation angle of the display endpoint, its physical pixel coordinates are restored to standard screen coordinates. For example, if a display endpoint is set with a scaling ratio s and a rotation angle θ is applied to the screen, its physical pixel coordinates (x_phys, y_phys) are first restored to (x_phys / s, y_phys / s) according to the scaling ratio, and then rotated in the opposite direction according to the rotation angle θ, that is, rotated around the center of the endpoint by -θ, to obtain the standard screen coordinates (x', y'); then the standard screen coordinates are then entered into the aforementioned position offset superposition step. When the endpoint is not scaled or rotated, the inverse transformation is a unit transformation, and the physical pixel coordinates are equal to the standard screen coordinates. The present invention does not limit the specific values ​​of the scaling ratio and rotation angle, which are determined by the actual display posture of the endpoint.

[0042] The implementation example achieves the restoration of physical pixel coordinates to standard screen coordinates based on scaling ratio and rotation angle, enabling the inverse transformation to adapt to display endpoints with scaling or rotation postures, and providing a unified restoration basis for input conversion of more complex layouts such as different resolutions and rotations.

[0043] In one embodiment, the multi-screen continuous display interaction method for the document view further includes the step of: Maintain a single display state machine, which includes at least a single display state, a multi-screen ready state, a continuous interaction state, and a single-screen rollback state. When any display endpoint is removed, closed, the parameters do not meet the preset display conditions, or creation fails, the drawing listener for the document view is released and the display endpoint is closed, and the size and orientation of the document view are restored to the single-screen configuration.

[0044] In this embodiment, the display state machine refers to a state machine that uniformly manages the lifecycle of multi-screen displays, including at least: a single display state, i.e., only the main screen is working; a multi-screen preparation state, i.e., creating secondary screen endpoints and building canvases; a continuous interaction state, i.e., multi-screen continuous display and interaction in progress; and a single-screen rollback state, i.e., cleaning up and restoring a single screen after an anomaly. Render listening refers to a mechanism that listens for unique document view rendering notifications to dispatch updates to each endpoint within the refresh cycle.

[0045] In a specific embodiment, when the secondary screen is successfully created and the display code remains at its current value, the system transitions from the multi-screen preparation state to the continuous interaction state. In the continuous interaction state, each display endpoint presents the document using the aforementioned virtual canvas and viewport, and inputs are fed back using unified coordinates. When any display endpoint is removed, closed, its parameters do not meet the preset display conditions, or endpoint creation fails, the system releases the drawing monitoring of the document view and closes the display endpoint, restoring the document view's size and orientation to the single-screen configuration and returning to the single-display state. This cancels the double layout and avoids leaving black areas or incorrect orientations. In some preferred implementations, the drawing of the authoritative document view can also be monitored. At most one invalidation refresh is scheduled for each display endpoint in each refresh cycle, and no redrawing is performed when the screen is static. Each display endpoint shares the same document content status update to reduce redrawing overhead.

[0046] The implementation example demonstrates that by displaying a state machine and a unified rollback path, the multi-screen layout can be automatically revoked and a single screen restored when a secondary screen is removed, closed, or fails to be created, thus improving the stability of switching between dual screens and single screens. Combined with a preferred refresh cycle merging mechanism, it also reduces the overhead of repeated drawing and state update.

[0047] See Figure 2 This invention also provides a multi-screen continuous display interactive device 400 for a document view, which includes a canvas construction module 401, a viewport rendering module 402, an input receiving module 403, a coordinate transformation module 404, and a coordinate back annotation module 405.

[0048] The canvas construction module 401 is used to obtain the number and arrangement direction of the display endpoints, and construct a virtual canvas covering all display endpoints based on the number and arrangement direction of the display endpoints. The viewport rendering module 402 is used to map the document view of a preset document session to the virtual canvas, and control each display endpoint to render the viewport corresponding to it in the virtual canvas; wherein, the document session includes a document content and a document view, and the document view is the only view that carries the rendering and input processing of the document content; The input receiving module 403 is used to receive input events acting on any display endpoint, wherein the input event carries input coordinates based on the display endpoint's own coordinate system; The coordinate transformation module 404 is used to convert the input coordinates into logical coordinates of the document view based on the position offset of the viewport corresponding to the display endpoint in the virtual canvas. The coordinate back-annotation module 405 is used to back-annotate the logical coordinates to the document view to drive the editing, selection or scrolling state of the same document content.

[0049] In one embodiment, the arrangement direction includes vertical and horizontal arrangement; the total size of the virtual canvas along the arrangement direction is determined by the sum of the viewport sizes of each display endpoint, each display endpoint corresponds to a continuous coordinate interval in the virtual canvas, and the viewports of each display endpoint are sequentially spliced ​​together along the arrangement direction.

[0050] In one embodiment, controlling each display endpoint to present its corresponding viewport in the virtual canvas includes extracting the coordinate range corresponding to the display endpoint from the virtual canvas and outputting it to the display endpoint by at least one of cropping, translating, or sharing a texture source rectangle.

[0051] In one embodiment, when a change in display topology is detected, the display code is incremented, and the incremented display code is associated with the topology configuration obtained in the current detection; the viewport of each display endpoint is determined only when the display code is still the current value.

[0052] In one embodiment, the step of converting the input coordinates into logical coordinates of the document view based on the position offset of the viewport corresponding to the display endpoint in the virtual canvas specifically includes: Perform an inverse transformation on the input coordinates to remove the scaling or rotation applied to the display endpoint, and then superimpose the position offset onto the inversely transformed input coordinates to obtain the logical coordinates; When arranged vertically, the position offset is the cumulative height of all display endpoints before the current display endpoint; when arranged horizontally, the position offset is the cumulative width of all display endpoints before the current display endpoint.

[0053] In one embodiment, the inverse transformation is: based on the scaling ratio and rotation angle of the display endpoint, its physical pixel coordinates are restored to standard screen coordinates.

[0054] In one embodiment, a maintenance module is also included for maintaining a single display state machine, which includes at least a single display state, a multi-screen preparation state, a continuous interaction state, and a single-screen rollback state. When any display endpoint is removed, closed, the parameters do not meet the preset display conditions, or creation fails, the drawing listener of the document view is released and the display endpoint is closed, and the size and orientation of the document view are restored to the single-screen configuration.

[0055] like Figure 3 As shown, Figure 3This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device 500 can be a terminal or a server. The terminal can be an electronic device with communication functions, such as a smartphone, tablet, laptop, desktop computer, personal digital assistant, or wearable device. The server can be a standalone server or a server cluster composed of multiple servers.

[0056] The computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a non-volatile storage medium 503 and internal memory 504.

[0057] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, it causes the processor 502 to execute a multi-screen continuous display interactive method for a document view.

[0058] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0059] The internal memory 504 provides an environment for the execution of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a multi-screen continuous display interactive method for a document view.

[0060] The network interface 505 is used for network communication with other devices. Those skilled in the art will understand that the above structure is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 500 to which the present application is applied. A specific computer device 500 may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements.

[0061] It should be understood that in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0062] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program may be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0063] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program.

[0064] The storage medium is a physical, non-transient storage medium, such as a USB flash drive, external hard drive, read-only memory (ROM), magnetic disk, or optical disk, or any other physical storage medium capable of storing program code. The computer-readable storage medium can be non-volatile or volatile.

[0065] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0066] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0067] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0068] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0069] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0070] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Since these modifications and variations fall within the scope of the claims and their equivalents, this invention also intends to include these modifications and variations.

[0071] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for multi-screen continuous display interaction of a document view, characterized in that, The method includes the following steps: The number and arrangement direction of the display endpoints are obtained, and a virtual canvas covering all display endpoints is constructed based on the number and arrangement direction of the display endpoints; a display endpoint refers to a display screen or display area on the terminal that can independently display the screen and receive input. The document view of the preset document session is mapped to the virtual canvas, and each display endpoint is controlled to present the viewport corresponding to it in the virtual canvas; wherein, the document session includes a document content and a document view, and the document view is the only view that carries the rendering and input processing of the document content; Receive an input event applied to any display endpoint, the input event carrying input coordinates based on the display endpoint's own coordinate system; Based on the position offset of the viewport corresponding to the display endpoint in the virtual canvas, the input coordinates are converted into logical coordinates of the document view; The logical coordinates are fed back to the document view to drive the editing, selection, or scrolling of the same document content.

2. The method according to claim 1, characterized in that, The arrangement direction includes vertical and horizontal arrangement; the total size of the virtual canvas along the arrangement direction is determined by the sum of the viewport sizes of each display endpoint, each display endpoint corresponds to a continuous coordinate interval in the virtual canvas, and the viewports of each display endpoint are sequentially spliced ​​together along the arrangement direction.

3. The method according to claim 1, characterized in that, Controlling each display endpoint to present its corresponding viewport in the virtual canvas includes extracting the coordinate range corresponding to the display endpoint from the virtual canvas and outputting it to the display endpoint by at least one of cropping, translating, or sharing the texture source rectangle.

4. The method according to claim 1, characterized in that, When a change in display topology is detected, the display code is incremented, and the incremented display code is associated with the topology configuration obtained in this detection. The viewport of each display endpoint is determined only when the display code is still the current value.

5. The method according to claim 1, characterized in that, The step of converting the input coordinates into logical coordinates of the document view based on the position offset of the viewport corresponding to the display endpoint in the virtual canvas specifically includes: Perform an inverse transformation on the input coordinates to remove the scaling or rotation applied to the display endpoint, and then superimpose the position offset onto the inversely transformed input coordinates to obtain the logical coordinates; When arranged vertically, the position offset is the cumulative height of all display endpoints before the current display endpoint; when arranged horizontally, the position offset is the cumulative width of all display endpoints before the current display endpoint.

6. The method according to claim 5, characterized in that, The inverse transformation is: based on the scaling ratio and rotation angle of the display endpoint, its physical pixel coordinates are restored to standard screen coordinates.

7. The method according to claim 1, characterized in that, Also includes: Maintain a single display state machine, which includes at least a single display state, a multi-screen ready state, a continuous interaction state, and a single-screen rollback state. When any display endpoint is removed, closed, the parameters do not meet the preset display conditions, or creation fails, the drawing listener for the document view is released and the display endpoint is closed, and the size and orientation of the document view are restored to the single-screen configuration.

8. A multi-screen continuous display interactive device for document view, characterized in that, include: The canvas construction module is used to obtain the number and arrangement direction of the display endpoints, and construct a virtual canvas covering all display endpoints based on the number and arrangement direction of the display endpoints; A display endpoint refers to a screen or display area on a terminal that can independently display images and receive input. The viewport rendering module is used to map the document view of a preset document session to the virtual canvas and control each display endpoint to render the viewport corresponding to it in the virtual canvas; wherein, the document session includes a document content and a document view, and the document view is the only view that carries the rendering and input processing of the document content; An input receiving module is used to receive input events acting on any display endpoint, wherein the input event carries input coordinates based on the display endpoint's own coordinate system; The coordinate transformation module is used to convert the input coordinates into logical coordinates of the document view based on the position offset of the viewport corresponding to the display endpoint in the virtual canvas. The coordinate back-annotation module is used to back-annotate the logical coordinates to the document view to drive the editing, selection, or scrolling of the same document content.

9. A computer device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-7.

Images