Screen display control methods and storage media

Abstract

This application relates to a screen display control method and storage medium. The method includes: obtaining a layout description file of an application; the layout description file defines multiple original local regions of the application at a preset logical resolution and the scaling type of each original local region; creating an off-screen canvas adapted to the preset logical resolution and redirecting the original display content of the application to the off-screen canvas; the regions on the off-screen canvas correspond to the original local regions; creating a visible canvas whose size matches the physical size of the target screen; calculating the target display layout of the multiple original local regions on the visible canvas based on the scaling type and physical size; for each original local region, reading pixel data of the original display content from the corresponding region of the off-screen canvas and rendering the pixel data onto the visible canvas according to the target display layout. This application solves the problem of low screen display space utilization.

Description

Technical Field

[0001] This application relates to the field of computers, and in particular to screen display control methods and storage media. Background Technology

[0002] In the process of web front-end graphics rendering, many HTML5 Canvas applications or games are developed based on a fixed logical resolution (such as 800x600). These applications tightly couple all their drawing instructions and interaction logic to this fixed coordinate system, which makes them challenged when adapting to modern diverse mobile devices, tablets, and large-screen displays.

[0003] In related technologies, screen adaptation is generally achieved through proportional scaling. This method uses techniques such as CSSTransform to maintain the integrity of the image content and uses getBoundingClientRect to calibrate interaction coordinates, solving the fundamental problems of visual fidelity and accurate interaction. However, while proportional scaling ensures no visual distortion, when the aspect ratio of the target device's screen differs significantly from the application's original logical aspect ratio, it creates a large number of "black borders" (i.e., blank areas) on the sides or top and bottom of the screen. This results in a huge waste of screen display space, reducing the efficiency of information display and the user's visual immersion.

[0004] Currently, no effective solution has been proposed to address the issue of low screen display space utilization in related technologies. Summary of the Invention

[0005] This application provides a screen display control method and a storage medium to at least solve the problem of low screen display space utilization in related technologies.

[0006] In a first aspect, embodiments of this application provide a screen display control method, the method comprising:

[0007] Obtain the layout description file of the application; the layout description file defines multiple original local regions of the application at a preset logical resolution, and the scaling type of each original local region;

[0008] Create an off-screen canvas adapted to the preset logical resolution, and redirect the original display content of the application to the off-screen canvas; the area on the off-screen canvas corresponds to the original local area;

[0009] Create a visible canvas whose size matches the physical size of the target screen;

[0010] Based on the scaling type and the physical size, the target display layout of the plurality of original local regions on the visible canvas is calculated; for each of the original local regions, the pixel data of the original display content is read from the corresponding area of ​​the off-screen canvas, and the pixel data is rendered onto the visible canvas for display according to the target display layout.

[0011] In some embodiments, calculating the target display layout of the plurality of original local regions on the visible canvas based on the scaling type and the physical size includes:

[0012] A baseline scaling factor is calculated based on the comparison results between the physical dimensions and the logical dimensions of the multiple original local regions;

[0013] For the rigid local region with the rigid scaling type, a scaling calculation is performed based on the reference scaling factor, and the target display layout corresponding to the rigid local region on the visible canvas is determined.

[0014] For the flexible local area with the scaling type being flexible, based on the reference scaling factor and the physical size, the remaining display space on the visible canvas is filled by the rigid local area after scaling, and the target display layout corresponding to the flexible local area is determined.

[0015] In some embodiments, calculating the baseline scaling factor based on a comparison between the physical size and the logical sizes of the plurality of original local regions includes:

[0016] Based on the physical dimensions, and the comparison result between the physical dimensions and the logical dimensions of any rigid local region in the original local region, the reference scaling factor is calculated; or...

[0017] Based on the logical dimensions of each of the rigid local regions in the multiple original local regions, calculate the total logical spans occupied by the rigid local regions in the logical space along multiple different logical directions;

[0018] For each of the different logical directions, calculate the ratio between the physical dimension and the corresponding total logical span;

[0019] The minimum value among the plurality of said ratios is determined as the reference scaling factor.

[0020] In some embodiments, determining the remaining display space on the visible canvas, based on the reference scaling factor and the physical size, after scaling the rigid region, and determining the target display layout corresponding to the flexible local region, includes:

[0021] Based on the physical size occupied by each of the rigid local regions after scaling, calculated according to the reference scaling factor, the remaining unoccupied physical space on the visible canvas is calculated.

[0022] The remaining physical space is allocated to one or more of the flexible local regions to determine the target display layout corresponding to the flexible local regions.

[0023] In some embodiments, the scaling type further includes a constrained scaling type; the method further includes:

[0024] For the constrained local region whose scaling type is constrained scaling, the preset layout constraint dimension corresponding to the constrained scaling is determined, and scaling calculation is performed based on the physical size and the preset layout constraint dimension to determine the target display layout corresponding to the constrained local region on the visible canvas.

[0025] For the flexible local area, based on the reference scaling factor and the physical size, the remaining display space on the visible canvas is filled by the scaling of the rigid local area and the scaling of the constrained local area, and the target display layout corresponding to the flexible local area is determined.

[0026] In some embodiments, after rendering the pixel data onto the visible canvas for display, the method further includes:

[0027] In response to a user input operation on the visible canvas, obtain the physical coordinates corresponding to the user input operation;

[0028] Based on the target display layout, determine the original local area hit by the physical coordinates;

[0029] Based on the size ratio between the original local region hit and the target display layout, the physical coordinates are mapped to logical coordinates under the preset logical resolution;

[0030] The logical coordinates are provided to the application.

[0031] In some embodiments, mapping the physical coordinates to logical coordinates at the preset logical resolution based on the size ratio between the hit original local region and the target display layout includes:

[0032] Obtain the source rectangle of the hit original local region, and the target rectangle in the target display layout corresponding to the hit original local region;

[0033] Calculate the relative physical coordinates of the physical coordinates with respect to the target rectangle;

[0034] Based on the size ratio of the source rectangle and the target rectangle, linear interpolation is performed on the relative physical coordinates to obtain the relative logical coordinates of the logical coordinates relative to the source rectangle;

[0035] Based on the relative logical coordinates and the position of the source rectangle, the logical coordinates mapped to the preset logical resolution are determined.

[0036] In some embodiments, redirecting the original display content of the application to the off-screen canvas includes:

[0037] Intercept the application's request to obtain canvas elements, and return the off-screen canvas as the response result to the application; the original displayed content is then redirected to the off-screen canvas based on the response result; or,

[0038] When initializing the application, the off-screen canvas is passed to the application as a rendering parameter, and the original display content is redirected to the off-screen canvas based on the rendering parameter.

[0039] In some embodiments, the multiple original local regions defined in the layout description file are determined based on the screen semantic function attributes of the application, and the scaling type of the original local regions is determined based on the preset visual fidelity index of the application.

[0040] Secondly, embodiments of this application provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the screen display control method as described in the first aspect above.

[0041] Compared to related technologies, the screen display control method and storage medium provided in this application embodiment obtain the layout description file of the application; the layout description file defines multiple original local regions of the application at a preset logical resolution, and the scaling type of each original local region; an off-screen canvas adapted to the preset logical resolution is created, and the original display content of the application is redirected to the off-screen canvas; the regions on the off-screen canvas correspond to the original local regions; a visible canvas is created, the size of which matches the physical size of the target screen; based on the scaling type and physical size, the target display layout of multiple original local regions on the visible canvas is calculated; for each original local region, the pixel data of the original display content is read from the corresponding region of the off-screen canvas, and the pixel data is rendered onto the visible canvas for display according to the target display layout.

[0042] Based on this, it ensures that every physical pixel on the screen can be effectively utilized, completely eliminating the black border problem caused by traditional proportional scaling. This allows for the simultaneous maximization of screen space utilization and lossless display of core visual content without modifying the application source code. It breaks the technical limitations of traditional single proportional scaling solutions that can only take into account visual fidelity or pursue full-screen display, thus effectively solving the problem of low screen display space utilization.

[0043] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0044] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0045] Figure 1 This is a hardware structure block diagram of a terminal for a screen display control method according to an embodiment of this application;

[0046] Figure 2 This is a flowchart of a screen display control method according to an embodiment of this application;

[0047] Figure 3 This is a schematic diagram of an adapted system architecture according to an embodiment of this application;

[0048] Figure 4 This is a structural block diagram of a screen display control device according to an embodiment of this application. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application. Furthermore, it is understood that although the efforts made in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, modifications to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0050] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0051] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application means two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The terms “first,” “second,” “third,” etc., used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0052] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. Taking running on a terminal as an example, Figure 1 This is a hardware structure block diagram of a terminal for a screen display control method according to an embodiment of this application. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. Optionally, the terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1The different configurations shown.

[0053] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the screen display control method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0054] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0055] This embodiment provides a screen display control method. Figure 2 This is a flowchart of a screen display control method according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps:

[0056] Step S210: Obtain the layout description file of the application; the layout description file defines multiple original local regions of the application at a preset logical resolution, and the scaling type of each original local region.

[0057] The steps described above divide the application's original display screen into different parts by acquiring and parsing a separate external layout description file, typically in JSON format (a lightweight data exchange format). This layout description file explicitly specifies which logical regions (i.e., original local regions) the screen should be divided into under the application's inherent preset logical resolution coordinate system, and specifies the key attribute of scaling type for each region.

[0058] More specifically, in some embodiments, the multiple original local regions defined in the above layout description file are determined based on the application's screen semantic function attributes, and the scaling type of the original local regions is determined based on the application's preset visual fidelity index.

[0059] The specific implementation path is as follows: The multiple original local areas defined in the layout description file are not based on simple geometric divisions, but on logical deconstruction based on the semantic functional attributes of the application screen—that is, according to the actual function and importance of UI elements in the application (for example, user interaction controls such as buttons and sliders, information display elements such as data charts, brand identification areas such as logos, and decorative backgrounds are divided into different logical units according to their functional semantics); at the same time, the scaling type allocation of each area is not arbitrarily specified, but follows a set of preset visual fidelity indicators. This indicator establishes quantitative standards based on the functional sensitivity and deformation tolerance of the area content (for example, interactive elements need to maintain a rigid constraint of absolute deformation prohibition, data visualization elements need to maintain proportion to maintain the high fidelity requirement of information accuracy, while purely decorative backgrounds can accept low fidelity processing of stretching deformation). Through this dual decision-making mechanism that combines functional semantics and fidelity requirements, the adaptation strategy is elevated from the geometric level to the semantic understanding level, enabling the adaptation system to intelligently identify and differentiate different content in the screen. Ultimately, while achieving efficient use of screen space, it ensures that the visual performance and interactive experience of key functions and core content can achieve optimal fidelity on different devices.

[0060] Thus, by introducing an external layout description file, developers do not need to modify the application code. They only need to define the logical areas of the original Canvas using coordinates and dimensions in this file, and specify the scaling characteristics of each area, thereby providing configuration rule information for subsequent graphics processing operations. One method of area division can be a "nine-grid model"; for the divided nine-grid areas, the corresponding scaling types are usually divided into flexible and rigid. In addition, where the implementation allows, a constrained scaling type between these two can also be defined. The following is a code example for defining a nine-grid layout for an application with a logical resolution of 800×600:

[0061] {

[0062] "logicalSize": { "width": 800, "height": 600},

[0063] "zones": {

[0064] "topLeft": { "rect": [0, 0, 100, 50], "type": "rigid"},

[0065] "top": { "rect": [100,0, 600, 50],"type": "flexible"},

[0066] "topRight": { "rect": [700,0, 100, 50], "type": "rigid"},

[0067] "left": { "rect": [0, 50, 100, 500],"type":"flexible"},

[0068] "center": { "rect": [100,50, 600, 500], "type": "rigid"},

[0069] "right": { "rect": [700, 50, 100, 500], "type": "flexible"},

[0070] "bottomLeft": { "rect": [0, 550, 100, 50], "type": "rigid"},

[0071] "bottom": { "rect": [100, 550, 600, 50], "type": "flexible"},

[0072] "bottomRight": { "rect": [700, 550, 100, 50], "type": "rigid"}

[0073] }

[0074] }

[0075] For example, in the application scenario of a data visualization dashboard, the original local area containing the dashboard's background and border can be defined as a flexible area that can be stretched, while the core elements inside, such as charts, pointers, and scales, can be defined as a rigid area to ensure the accuracy of the data display. In the application scenario of a game with a complex UI, the game's dialog boxes and status bar backgrounds can be set to flexible, allowing them to stretch with the screen width, while the buttons and avatars on the dialog boxes are rigid and remain unchanged. Furthermore, the original local area containing the horizontal progress bar in the game can have its scaling type set to constrained scaling, so that its width stretches with the physical width of the container, but its height remains unchanged.

[0076] Step S220: Create an off-screen canvas adapted to a preset logical resolution, and redirect the original display content of the application to the off-screen canvas; the area on the off-screen canvas corresponds to the original local area.

[0077] It should be noted that the above-mentioned off-screen canvas creation steps are key steps to achieve non-intrusive adaptation. The purpose is to isolate the rendering process of the original application from the final display by creating an off-screen canvas. The application continues to draw at its preset resolution as usual, and all output is captured by the off-screen canvas. This ensures that a frame of "source image" unaffected by external devices is obtained, providing a foundation for subsequent smart screen adaptation.

[0078] Specifically, the adaptation layer creates an off-screen Canvas whose width and height are strictly set to be exactly the same as the preset logical resolution described in step S210. The off-screen canvas is created as shown in the following code example:

[0079] offscreenCanvas = document.createElement('canvas');

[0080] offscreenCanvas.width = layoutConfig.logicalSize.width;

[0081] offscreenCanvas.height = layoutConfig.logicalSize.height;

[0082] The code above generates an off-screen canvas with dimensions [logicalSize.width, logicalSize.height]. This canvas is a special use of the HTML Canvas element; it exists in the document object model's memory but is not inserted into the visible Document Object Model (DOM) tree. Therefore, the user cannot directly see it; its function is purely as a backup, intermediate drawing buffer. All drawing instructions from the original application will be relocated to this off-screen Canvas, which has the same logical resolution. For example, the rendering target of the original application can be replaced with the off-screen Canvas by overriding `document.getElementById` or by passing `offscreenCanvas` at application startup.

[0083] More specifically, redirecting the application's original display content to an off-screen canvas can also be achieved in the following ways:

[0084] Intercept the application's request to retrieve canvas elements and return the off-screen canvas as the response result to the application. The original display content is redirected to the off-screen canvas based on the response result. Alternatively, when initializing the application, pass the off-screen canvas as a rendering parameter to the application. The original display content is redirected to the off-screen canvas based on the rendering parameter.

[0085] The first path uses a system-level interception mechanism to intercept the application's request for canvas elements from its runtime environment and return a pre-created off-screen canvas as a valid response. This allows the application to automatically output all subsequent drawing commands to the off-screen canvas without being aware of the redirection. The second path directly passes the off-screen canvas as a necessary parameter in the application's rendering process (e.g., a Canvas element required for context initialization) during application initialization, guiding the application to actively set the off-screen canvas as its rendering target at startup. The core of both paths is to cleverly guide the application to draw the original display content onto the off-screen canvas as an intermediate carrier through the standard interaction interface between the external system and the application, without modifying the application's source code, thus providing a clean data source for subsequent intelligent adaptation. These two non-intrusive implementation schemes—interception-based redirection and parameter-based redirection—can cover Canvas applications with different technical backgrounds, ensuring that various legacy systems can seamlessly integrate with the adaptation framework.

[0086] The result is that a high-fidelity "source image" that is completely consistent with the original design and has not been scaled or distorted is generated and saved frame by frame on the off-screen canvas. At this point, each abstract "original local region" defined in the configuration file in step S210 has found a pixel-level precise corresponding real image block on this off-screen canvas, providing a unique and authoritative data source for the regional pixel operations in step S240.

[0087] Through the above steps, the application's multi-screen adaptation display can achieve functional expansion without modifying the application's original core source code, thus realizing a non-intrusive resolution adaptation method.

[0088] Step S230: Create a visible canvas whose size matches the physical size of the target screen.

[0089] This involves creating a user-visible Canvas and inserting it into the browser's viewport, serving as the sole graphical interface the user will ultimately see. Unlike the fixed size of the off-screen canvas, this visible canvas is dynamically sized. Its width and height properties are set in real-time to match the physical size of the target screen, typically by filling a parent container (such as the entire browser window) and dynamically updating its size by detecting the window's resize event. Below is one code example of initializing a visible canvas:

[0090] visibleCanvas = document.getElementById('visible-canvas');

[0091] visibleCtx = visibleCanvas.getContext('2d');

[0092] Technically, the canvas's size is measured in physical pixels, and its size determines the actual resolution of the frame buffer. It aims to fully utilize the available display space, avoiding any form of whitespace. This canvas is unrelated to the original application's logical resolution; it serves only one goal: to occupy the entire available physical display area. It is the final destination and display medium for all the complexly calculated and transformed graphic content in step S240, transforming "eliminating black borders and full-screen adaptation" from a goal into an achievable technical state. This provides a precise target coordinate system and spatial boundaries for subsequent hybrid rendering.

[0093] Step S240: Based on the scaling type and physical size, calculate the target display layout of multiple original local regions on the visible canvas; for each original local region, read the pixel data of the original display content from the corresponding area of ​​the off-screen canvas, and render the pixel data onto the visible canvas according to the target display layout.

[0094] In this step, a hybrid layout engine algorithm is run based on the physical dimensions of the currently visible canvas and the predefined scaling types for each region in the layout description file. The core task of this engine is to calculate a "target display layout," that is, to solve for the position and size of each original local region in the target space of the visible canvas. For example, for regions marked as "rigid," the algorithm strives to find a scaling factor and position that maintains their original aspect ratio; while for "flexible" regions, their target size is typically calculated by filling the remaining screen space after the rigid region layout, allowing them to scale independently in both width and height. The process of calculating the layout when the size changes can be implemented using the following code example:

[0095] Next, a rendering sub-step is performed for each frame of the application image (i.e., the original display content mentioned above). For example, the `drawImage` method in the Canvas 2D API (Canvas Application Programming Interface) can be used. This method is used here as an image cropping and transformation tool: its first parameter is the entire off-screen canvas (as the source), and then the pixel data of that area on the off-screen canvas is "cropped" by precisely specifying the source rectangular area (corresponding to the coordinates and size of that area on the off-screen canvas); finally, the cropped image block is drawn onto the visible canvas by specifying the target rectangular area (corresponding to the position and size of that area on the visible canvas obtained in the calculation sub-step). In this process, rigid areas are scaled proportionally because their target rectangles maintain the aspect ratio; flexible areas are filled and stretched because their target rectangles may stretch.

[0096] Ultimately, all these independently and differentiated image blocks are pieced together on the visible canvas into a completely new, seamless image that ensures that key content is not distorted and can perfectly fill any screen, thus technically achieving the dual goals of visual fidelity and maximizing space utilization.

[0097] To better understand this solution, a code example is provided here for the initialization, canvas creation, and rendering startup processes described above:

[0098] For HyperText Markup Language (HTML) initialization, HTML only requires a single visible Canvas for the final display:

[0099]

[0100] <canvas id="visible-canvas">< / canvas>

[0101]

[0102] JavaScript initialization:

[0103] / / Global variables

[0104] let layoutConfig;

[0105] let offscreenCanvas;

[0106] let visibleCanvas, visibleCtx;

[0107] let currentMapping = {}; / / Store the source and destination rectangles for each region;

[0108] async function initialize() {

[0109] / / 1. Load layout configuration:

[0110] layoutConfig = await (await fetch('layout.json')).json();

[0111] / / 2. Create an off-screen Canvas, which is where the original application draws:

[0112] offscreenCanvas = document.createElement('canvas');

[0113] offscreenCanvas.width = layoutConfig.logicalSize.width;

[0114] offscreenCanvas.height = layoutConfig.logicalSize.height;

[0115] / / Key step: Replace the original application's rendering target with an offscreen Canvas; this is typically achieved by overriding `document.getElementById` or by passing in an offscreenCanvas when the application starts.

[0116] / / game.init(offscreenCanvas);

[0117] / / 3. Initialize the visible Canvas:

[0118] visibleCanvas = document.getElementById('visible-canvas');

[0119] visibleCtx = visibleCanvas.getContext('2d');

[0120] / / 4. System Initialization:

[0121] window.addEventListener('resize', handleResize);

[0122] handleResize(); / / This line of code triggers layout calculation by actively calling handleResize;

[0123] setupInputInterceptor(); / / Start the input calibration system;

[0124] requestAnimationFrame(renderLoop); / / Start the rendering loop;

[0125] }

[0126] Through steps S210 to S240, the scaling characteristics of each region in the image are predefined using an external layout description file. This serves as the basis for driving the entire rendering process: while preserving the complete output of the original application (achieved through off-screen canvas redirection), a hybrid target display layout is intelligently calculated based on the scaling type of each original local region defined in the layout description file and the physical size of the target screen, thereby performing differentiated rendering for different types of regions. This mechanism ensures that every physical pixel on the screen can be effectively utilized, completely eliminating the black border problem caused by traditional proportional scaling. It enables the simultaneous maximization of screen space utilization and lossless display of core visual content without modifying the application source code, effectively solving the problem of low screen display space utilization. Furthermore, it breaks the technical limitations of traditional single proportional scaling schemes, which can only balance visual fidelity or pursue full-screen display. It overcomes the contradiction between low screen space utilization and visual content fidelity faced by traditional Canvas applications when adapting to cross-device applications.

[0127] In some embodiments, the above-described calculation of the target display layout of multiple original local regions on the visible canvas based on the scaling type and physical size may further include the following steps:

[0128] Based on the comparison between the physical size and the logical size of multiple original local regions, a baseline scaling factor is calculated. For rigid local regions with a rigid scaling type, scaling calculation is performed based on the baseline scaling factor, and the target display layout corresponding to the rigid local region on the visible canvas is determined. For flexible local regions with a flexible scaling type, the remaining display space on the visible canvas, determined by scaling the rigid local region, is filled based on the baseline scaling factor and the physical size, and the target display layout corresponding to the flexible local region is determined.

[0129] Specifically, by comparing the physical dimensions of the target screen with the overall logical span of all rigid local areas, a unified baseline scaling factor is calculated. This factor ensures that all rigid areas can be scaled proportionally and as a whole onto the current screen. Subsequently, the system rigorously applies this baseline scaling factor to each local area marked as rigid for proportional scaling calculations, thereby accurately determining the position and size of these areas on the visible canvas. This ensures that core content such as user interface (UI) buttons, icons, and data charts will not be deformed on any screen. Finally, the system allocates all remaining physical space on the screen not occupied by rigid areas to local areas marked as flexible, and calculates the target display layout of these flexible areas through stretching and filling. Through this method, black borders caused by differences in screen aspect ratio are completely eliminated, ultimately achieving 100% utilization of screen display space while perfectly maintaining the visual fidelity of core content.

[0130] In some embodiments, calculating the baseline scaling factor based on the comparison results between the physical size and the logical size of multiple original local regions may further include the following steps:

[0131] The baseline scaling factor is calculated based on the comparison between the physical dimensions and the logical dimensions of any rigid local region within the original local regions. Specifically, one region is arbitrarily selected from all the original local regions marked as rigid (e.g., the most important core content area, the center area, or a representative UI component). The physical dimensions of the target screen are then directly compared with the logical dimensions of this selected region. Specifically, the ratio of the physical width to the logical width of the region is calculated to obtain the horizontal scaling ratio, and the ratio of the physical height to the logical height of the region is calculated to obtain the vertical scaling ratio. Finally, the smaller of these two ratios is selected as the unified baseline scaling factor for the entire system. This simplified calculation of the baseline scaling factor based on the center area of ​​a rigid region is illustrated in the following code example:

[0132] const centerZone = layoutConfig.zones.center; / / This line of code retrieves all information about the center zone (including its coordinates, size, and type) from the layout configuration.

[0133] const scaleX = physicalWidth / centerZone.rect[2]; / / Horizontal scaling ratio = physical width of the target screen / logical width of the center area;

[0134] const scaleY = physicalHeight / centerZone.rect[3]; / / Vertical scaling ratio = physical height of the target screen / logical height of the center area;

[0135] const scale = Math.min(scaleX, scaleY); / / Base scaling factor const scale takes the minimum value between the horizontal scaling ratio and the vertical scaling ratio;

[0136] The above method uses the constraint of a local area to represent the constraint of the overall layout. Although it is a simplified solution, it can quickly and effectively ensure that the specific rigid area remains intact and undistorted on any screen, and use this as a benchmark to affect the adaptation effect of the entire screen.

[0137] Alternatively, based on the logical dimensions of each rigid local region in multiple original local regions, calculate the total logical spans occupied by the rigid local regions in the logical space along multiple different logical directions; for each of the multiple different logical directions, calculate the ratio between the physical dimensions and the corresponding total logical spans; and determine the minimum value among the multiple ratios as the baseline scaling factor.

[0138] It should be understood that the second strategy described above employs a global optimization strategy for calculating a baseline scaling factor. First, it iterates through all original local regions marked as rigid, analyzing the position and size of each region in a preset logical resolution coordinate system to calculate a minimum virtual bounding box that can completely enclose all these rigid regions. The total span of this bounding box along multiple different logical directions (e.g., standard width and height directions, or more complex custom axes) is then extracted. Subsequently, for each direction, the physical size of the target screen in that direction is divided by the calculated total logical span in that direction, yielding a series of direction-specific ratio values. Finally, by taking the minimum of these ratio values ​​as a unified baseline scaling factor, this mathematically ensures that all rigid regions, as a whole, will not be clipped due to exceeding the screen boundary in any direction, thus allowing them to be presented completely and without distortion within the target screen during subsequent adaptation.

[0139] Through the above embodiments, an optimal balance between adaptation robustness and computational efficiency is achieved by providing two strategies for calculating the baseline scaling factor: the first, a simplified strategy based on a single rigid region, ensures the complete display of core content with minimal computational overhead, suitable for scenarios with compact layouts or clear hierarchy; the second, a global strategy based on the total span of all rigid regions, fundamentally guarantees, through mathematical minimum proportional constraints, that all rigid regions as a whole can be completely and without pruning adapted to any screen, completely eliminating the risk of incomplete content display due to the remote location of individual secondary rigid elements. Both strategies together ensure that the system can select the most suitable solution to achieve perfect fidelity of rigid content under different complexity and reliability requirements, laying a reliable foundation for subsequent hybrid rendering.

[0140] In some embodiments, the above-mentioned determination of the remaining display space on the visible canvas, based on the reference scaling factor and physical size, after scaling of the rigid area, and determination of the target display layout corresponding to the flexible local area, may also include the following steps.

[0141] Based on the physical dimensions occupied by each rigid local region after scaling, calculated using a baseline scaling factor, the remaining unoccupied physical space on the visible canvas is calculated; the remaining physical space is then allocated to one or more flexible local regions to determine the target display layout corresponding to the flexible local regions.

[0142] In this embodiment, coordinate transformation calculations are first performed on each local region marked as rigid, strictly based on the unified reference scaling factor previously determined through a global optimization strategy. This calculation process is essentially a deterministic geometric mapping: the position coordinates and size parameters of each rigid region in the original logical coordinate system are converted into the specific position and occupied size in the target physical screen coordinate system through matrix transformation principles. Specifically, for any rigid region, its target physical width is equal to the product of the original logical width and the reference scaling factor, the target physical height is equal to the product of the original logical height and the reference scaling factor, and its target physical coordinates are determined by multiplying the original logical coordinates and the scaling factor plus a possible offset. After all rigid regions have completed this transformation, they form a series of layout elements on the physical screen that maintain the original proportions but vary in size. Although these elements are visually faithful, due to the inherent difference between the screen's physical aspect ratio and the original logical aspect ratio, they will inevitably leave unoccupied irregular areas in the screen space.

[0143] Subsequently, the system enters the remaining space quantization stage. At this point, the rendering engine treats the entire visible canvas as a two-dimensional rectangular space container, and uses efficient geometric algorithms to calculate the difference between this container space and the space occupied by all rigid regions. Specifically, a complete rectangular model of the screen space is first established, and then the physical rectangular area corresponding to each rigid region is subtracted sequentially. This calculation process also needs to consider the possible overlap between regions (although regions do not overlap in a typical 3x3 grid layout, the algorithm itself has the ability to handle more complex layouts). Finally, a polygon clipping algorithm or a rectangle segmentation algorithm is used to accurately calculate the set of all remaining space regions not occupied by rigid elements. Mathematically, these remaining spaces may be represented as one or more consecutive rectangular regions, whose total area is the difference between the total physical area of ​​the screen and the sum of the physical areas of all rigid regions.

[0144] Finally, the intelligent space allocation stage is executed. The core task of this stage is to allocate the remaining space calculated in the previous step to each flexible area according to preset layout rules. Taking a typical nine-grid layout model as an example, the allocation follows clear topological rules: the top flexible area receives the remaining horizontal strip space at the top of the screen, the bottom flexible area receives the horizontal strip space at the bottom of the screen, the left and right flexible areas receive the vertical strip spaces on the left and right sides of the screen respectively, and the central flexible area receives the remaining central area after all rigid areas have been laid out. From a technical implementation perspective, this is achieved by establishing a complete space allocation priority system: that is, firstly, the boundary frame of the screen is determined based on the layout of the rigid areas, and then the remaining space is divided according to predefined regional relationships. Each flexible area is allocated a corresponding remaining space partition according to its relative position in the original layout, and its precise position and size in physical space are determined by a linear interpolation algorithm.

[0145] Of particular note is that determining the size of the flexible areas is a passive process: their physical dimensions are not calculated through active scaling, but rather serve as "filler" after the rigid areas have been laid out, their size entirely determined by the size and shape of the remaining space. This constraint-based layout method ensures 100% utilization of screen space while maintaining logical consistency in the layout. The computational complexity of the entire process is linear, ensuring the performance requirements of real-time rendering and enabling the system to quickly recalculate the layout when the screen size dynamically changes, providing users with a smooth, adaptive visual experience.

[0146] More specifically, one concrete implementation of hybrid rendering using the above method can be seen in the following code example:

[0147] function handleResize() {

[0148] const container = visibleCanvas.parentElement;

[0149] const physicalWidth = container.clientWidth;

[0150] const physicalHeight = container.clientHeight;

[0151] visibleCanvas.width = physicalWidth;

[0152] visibleCanvas.height = physicalHeight;

[0153] / / 1. Traverse all regions and find the logical boundaries of all "rigid" regions:

[0154] let minX = Infinity, minY = Infinity, maxX = -Infinity, maxY = -Infinity;

[0155] let hasRigidZone = false;

[0156] for (const zoneName in layoutConfig.zones) {

[0157] const zone = layoutConfig.zones[zoneName];

[0158] if (zone.type === 'rigid') {

[0159] hasRigidZone = true;

[0160] const [x, y, w, h] = zone.rect;

[0161] minX = Math.min(minX, x);

[0162] minY = Math.min(minY, y);

[0163] maxX = Math.max(maxX, x + w);

[0164] maxY = Math.max(maxY, y + h);

[0165] }

[0166] }

[0167] / / 2. Calculate the "total rigid span" formed by all rigid regions; if there are no rigid regions, degenerate into stretching the entire canvas (or using logicalSize):

[0168] const totalLogicalRigidWidth = hasRigidZone ? (maxX - minX) :layoutConfig.logicalSize.width;

[0169] const totalLogicalRigidHeight = hasRigidZone ? (maxY - minY) :layoutConfig.logicalSize.height;

[0170] / / 3. Calculate the core scaling factor based on the "total rigid span"; this ensures that all rigid regions can be scaled proportionally as a whole and fit completely into the screen:

[0171] const scaleX = physicalWidth / totalLogicalRigidWidth;

[0172] const scaleY = physicalHeight / totalLogicalRigidHeight;

[0173] const scale = Math.min(scaleX, scaleY);

[0174] / / The following is a detailed calculation of the target rectangle in a nine-grid layout; the core of this solution is that the size of the rigid region is determined by `scale`, while the size of the flexible region is stretched to fill the remaining space:

[0175] const zones = layoutConfig.zones;

[0176] / / 4. Calculate the dimensions of the target (physical) mesh; the height and width of rigid columns / rows are determined by the scale:

[0177] const destColWidth_Left = zones.topLeft.rect[2] scale;

[0178] const destColWidth_Right = zones.topRight.rect[2] scale;

[0179] const destRowHeight_Top = zones.topLeft.rect[3] scale;

[0180] const destRowHeight_Bottom = zones.bottomLeft.rect[3] scale;

[0181] / / The height and width of the flexible columns / rows fill the remaining space; the stretching ratio of the flexible area is determined as follows: (Total physical dimensions - Rigid dimensions):

[0182] const destColWidth_Center = physicalWidth - destColWidth_Left -destColWidth_Right;

[0183] const destRowHeight_Center = physicalHeight - destRowHeight_Top -destRowHeight_Bottom;

[0184] / / 5. Calculate the coordinates of the target (physical) mesh:

[0185] const destX1 = destColWidth_Left;

[0186] const destX2 = destColWidth_Left + destColWidth_Center;

[0187] const destY1 = destRowHeight_Top;

[0188] const destY2 = destRowHeight_Top + destRowHeight_Center;

[0189] / / 6. Cache the source rectangle (src) and destination rectangle (dest) for all 9 regions:

[0190] currentMapping = {

[0191] 'topLeft': { src: zones.topLeft.rect, dest: [0, 0,destColWidth_Left, destRowHeight_Top]},

[0192] 'top': { src: zones.top.rect, dest: [destX1, 0, destColWidth_Center, destRowHeight_Top]},

[0193] 'topRight': { src: zones.topRight.rect, dest: [destX2, 0,destColWidth_Right, destRowHeight_Top]},

[0194] 'left': { src: zones.left.rect, dest: [0, destY1,destColWidth_Left, destRowHeight_Center]},

[0195] 'center': { src: zones.center.rect, dest: [destX1, destY1,destColWidth_Center, destRowHeight_Center]},

[0196] 'right': { src: zones.right.rect, dest: [destX2, destY1,destColWidth_Right, destRowHeight_Center]},

[0197] 'bottomLeft': { src: zones.bottomLeft.rect, dest: [0, destY2,destColWidth_Left, destRowHeight_Bottom]},

[0198] 'bottom': { src: zones.bottom.rect, dest: [destX1, destY2,destColWidth_Center, destRowHeight_Bottom]},

[0199] 'bottomRight': { src: zones.bottomRight.rect, dest: [destX2,destY2, destColWidth_Right, destRowHeight_Bottom]}

[0200] };

[0201] }

[0202] In some embodiments, the above-mentioned scaling type also includes a constrained scaling type; then the screen display control method further includes the following steps:

[0203] For constrained local regions with a scaling type of constrained scaling, the preset layout constraint dimension corresponding to the constraint scaling is determined, and scaling calculation is performed based on the physical size and the preset layout constraint dimension to determine the target display layout corresponding to the constrained local region on the visible canvas; for flexible local regions, based on the baseline scaling factor and the physical size, the remaining display space filled on the visible canvas is determined by the scaling of the rigid local region and the scaling of the constrained local region, and the target display layout corresponding to the flexible local region is determined.

[0204] Among them, constrained local areas refer to specific screen areas whose scaling behavior during screen adaptation does not follow the strict proportional scaling of rigid areas, nor the free stretching of flexible areas, but is limited by one or more preset layout constraint dimensions. The preset layout constraint dimensions are a set of rule parameters pre-configured for constrained local areas to quantitatively control their scaling behavior. They limit the transformation of the area through explicit mathematical or logical conditions. For example, the text area of ​​a news reading app can be defined as a constrained local area, and a preset layout constraint dimension of maximum width (maxWidth: 800px) can be configured for it. This ensures that the area will not expand infinitely on large-screen landscape devices, affecting the reading experience, but will instead limit its maximum display width to 800 physical pixels and display it in the center of the screen. At the same time, it can still shrink and adapt normally on small-screen portrait devices. Similarly, a product display card can be configured with an aspect ratio range (aspectRange: [1.2, 1.8]) constraint dimension to ensure that the area can adapt to different screens without compromising visual aesthetics due to excessive deformation (e.g., appearing bulky with an aspect ratio less than 1.2, or appearing elongated with an aspect ratio greater than 1.8). These constraint dimensions are implemented through external configuration, allowing developers to precisely control the adaptation strategy of specific areas in a declarative manner, achieving a fine balance between rigid conformation and flexible filling.

[0205] The scaling process for constrained and flexible local regions is explained in detail below:

[0206] First, the layout description file is parsed to obtain a set of predefined layout constraint dimension parameters for specific constrained local areas. These parameters are stored in a structured data format, and typical embodiments include, but are not limited to, key technical parameters such as: aspect ratio allowable range (e.g., aspectRange: [0.75, 1.25]), primary scaling direction identifier (e.g., primaryAxis: "x"), size constraints (e.g., maxWidth: 500, minHeight: 100), and elastic weight coefficient (e.g., flexGrow: 2). Then, the initial layout calculation phase begins. Based on the current screen physical size and the original logical attributes of the area, the system generates an initial target rectangle using a differentiated algorithm: for primary direction constraint types, the system calculates the size along the specified primary axis using a flexible stretching algorithm, while applying a rigid proportional scaling rule in the cross axis direction; for aspect ratio constraint types, a completely flexible initial rectangle is first generated as the calculation base.

[0207] When applying constraints, a multi-level conditional judgment architecture can achieve precise layout control: when an aspect ratio exceeding the limit is detected, the system recalculates the rectangle size based on the nearest boundary value and maintains visual balance through a translation transformation algorithm; when a size constraint is detected, a clamping algorithm is used to limit the out-of-bounds size to within a preset threshold; for flexible weight constraints, a weighted allocation algorithm is used to prioritize space allocation. In particular, during the constraint region processing, the system dynamically maintains an updated remaining space mapping table, which reflects the screen space occupancy after all rigid and constraint regions are laid out in real time.

[0208] Meanwhile, the scaling calculation process for flexible local areas employs a relatively simplified logic: based on the baseline scaling factor obtained from the aforementioned calculations, the system first accurately calculates the physical dimensions occupied by each rigid region after scaling. Then, it subtracts the occupied space of all rigid and constrained regions from the total physical area of ​​the container using a geometric difference algorithm, thereby quantifying the final usable remaining physical space. This remaining space is then allocated to each flexible region according to preset layout topology rules (such as the edge priority principle in a nine-grid model).

[0209] Through the above embodiments, by introducing constraint scaling types and establishing a collaborative layout mechanism of rigid-constraint-flexible regions, screen space utilization, visual fidelity, and layout flexibility are balanced. This establishes a broad and adjustable optimization range between full fidelity and full filling, enabling the system to maintain absolute fidelity of core content (rigidity) while allowing secondary content to adapt within a controllable range (constraint), and finally fill all remaining space with irrelevant content (flexibility). This simultaneously approaches the theoretical maximum value of fidelity and utilization, achieving perfect filling and maximizing space utilization for any screen physical size while absolutely ensuring the visual correctness of core content.

[0210] In some embodiments, after rendering pixel data onto a visible canvas for display, the screen display control method may further include the following steps:

[0211] In response to user input on the visible canvas, obtain the physical coordinates corresponding to the user input; determine the original local area hit by the physical coordinates according to the target display layout; map the physical coordinates to logical coordinates at a preset logical resolution based on the size ratio between the hit original local area and the target display layout; and provide the logical coordinates to the application.

[0212] First, let's explain the specific implementation process of rendering with code examples:

[0213] function renderLoop() {

[0214] / / Assuming the application has already drawn its content onto the offscreenCanvas

[0215] / / game.updateAndRender();

[0216] visibleCtx.clearRect(0, 0, visibleCanvas.width, visibleCanvas.height); / / Clear the screen;

[0217] for (const zoneName in currentMapping) { / / Iterate through and draw all zones;

[0218] const mapping = currentMapping[zoneName];

[0219] const [sx, sy, sWidth, sHeight] = mapping.src; / / Destructuring the source region;

[0220] const [dx, dy, dWidth, dHeight] = mapping.dest; / / Deconstruct the target region;

[0221] if (sWidth > 0 && sHeight > 0 && dWidth > 0 && dHeight > 0) {

[0222] visibleCtx.drawImage(offscreenCanvas, sx, sy, sWidth, sHeight, dx, dy, dWidth, dHeight); / / Security check;

[0223] }

[0224] }

[0225] requestAnimationFrame(renderLoop); / / Triggers in a loop;

[0226] }

[0227] After the visible canvas is rendered, when a user triggers an input operation (such as clicking or touching) on ​​the visible canvas, the browser's event object can obtain the original physical coordinates (in physical pixels) of the operation in the visible canvas coordinate system. Then, the target display layout data calculated in the current frame is traversed, and a geometric collision detection algorithm is used to compare the physical coordinates with the boundary of the target rectangle in each area, thus accurately determining which original local area (such as the top button area or the center game area) the input point hits. After determining the hit area, the system performs an inverse coordinate transformation calculation based on the size ratio between the source rectangle (the logical range on the off-screen canvas) and the target rectangle (the physical range on the visible canvas), accurately mapping the physical coordinates back to the logical coordinates in the preset logical resolution coordinate system recognized by the original application. Finally, through event simulation or function calls, the mapped logical coordinates are passed to the application's event handling system, enabling the application to respond correctly to interactions as if running at the original resolution, thus achieving visual adaptation while maintaining the accuracy of the interactive behavior.

[0228] Through the above embodiments, by tracking the target display layout output by the rendering engine in real time, the physical input coordinates of the user on the deformed interface are dynamically restored to the original application's inherent logical coordinate system. This process not only overcomes the coordinate system distortion caused by the proportional scaling of rigid areas and the stretching deformation of flexible areas, but also, through the region-aware intelligent mapping algorithm, enables the user's clicks, touches, and other operations on any adapted interface element (such as stretched background buttons or scaled rigid icons) to be losslessly converted into the logical coordinate input expected by the original application. This allows the application to obtain an interactive response experience that is completely consistent with the original design resolution at any screen size and ratio without being aware of the complexity of external adaptation. Ultimately, it achieves the adaptation and unification of visual presentation and interaction logic in cross-device adaptation.

[0229] In some embodiments, the above-mentioned mapping of physical coordinates to logical coordinates at a preset logical resolution based on the size ratio between the hit original local region and the target display layout may further include the following steps:

[0230] Obtain the source rectangle of the hit original local area, and the target rectangle in the target display layout corresponding to the hit original local area; calculate the physical coordinates relative to the target rectangle; perform linear interpolation on the relative physical coordinates based on the size ratio of the source rectangle and the target rectangle to obtain the logical coordinates relative to the source rectangle; determine the logical coordinates mapped to the preset logical resolution based on the relative logical coordinates and the position of the source rectangle.

[0231] Specifically, based on the hit region identifier, the source rectangle parameters of the region on the off-screen canvas are obtained from the layout configuration, and the corresponding target rectangle parameters are extracted from the rendering layout data of the current frame. Next, the relative position of the input point within the target region is calculated by subtracting the starting coordinates of the target rectangle from its absolute physical coordinates. Then, using the size ratio of the source and target rectangles in the width and height directions as scaling factors, linear interpolation is performed on the relative physical coordinates: X-direction logical coordinate offset = relative physical X-coordinate × (source rectangle width / target rectangle width), and the same applies to the Y-direction, thus restoring the relative position within the original logical region. Finally, this relative logical coordinate is added to the starting coordinates of the source rectangle to obtain the absolute logical coordinates in the preset logical resolution global coordinate system, completing a seamless mapping from the physical interaction space to the original application logical space.

[0232] Below is a code example of one way to implement input correction for region-aware systems:

[0233] function setupInputInterceptor() {

[0234] visibleCanvas.addEventListener('mousedown', function(event) {

[0235] const rect = visibleCanvas.getBoundingClientRect();

[0236] const physicalX = event.clientX - rect.left;

[0237] const physicalY = event.clientY - rect.top;

[0238] / / Traverse all areas and find the area that was clicked:

[0239] for (const zoneName in currentMapping) {

[0240] const mapping = currentMapping[zoneName];

[0241] const [dx, dy, dWidth, dHeight] = mapping.dest;

[0242] if (physicalX >= dx && physicalX <= dx + dWidth &&

[0243] physicalY >= dy && physicalY <= dy + dHeight) {

[0244] const [sx, sy, sWidth, sHeight] = mapping.src;

[0245] const zoneInfo = layoutConfig.zones[zoneName];

[0246] Let logicalX, logicalY;

[0247] / / Calculate relative physical coordinates (coordinates within the region):

[0248] const relativePx = physicalX - dx;

[0249] const relativePy = physicalY - dy;

[0250] / / Perform different inverse transformations based on the region type:

[0251] logicalX = sx + relativePx (sWidth / dWidth);

[0252] logicalY = sy + relativePy (sHeight / dHeight);

[0253] / / Pass the calibrated logical coordinates to the application;

[0254] return; / / Stop once found

[0255] }

[0256] }

[0257] });

[0258] }

[0259] In the code above, a unified linear interpolation formula is used for inverse coordinate transformation. The underlying principle is as follows: For flexible regions, due to non-uniform stretching along the X and Y axes, independent inter-axis ratios (X-axis: sWidth / dWidth, Y-axis: sHeight / dHeight) must be used for coordinate restoration via linear interpolation. For theoretically rigid regions, although their width and height scaling ratios are the same, causing the unified formula to naturally degenerate into an equivalent proportional calculation, the key issue is that in actual layouts, regions marked as rigid (such as the center area of ​​a nine-grid layout) may have unequal actual scaling ratios due to layout compression, resulting in a hybrid state of "nominal rigidity, actual flexibility." To ensure algorithm robustness, traditional conditional branching must be abandoned, and a unified linear interpolation formula must always be used. This ensures correct handling of purely flexible regions while also being compatible with compressed rigid regions, maintaining the calculation accuracy of purely rigid regions. The type field in the layout description has been transformed into a metadata support for semantic annotation and future functional expansion.

[0260] The above embodiments provide a coordinate inverse transformation technology, which can accurately map physical coordinates back to their original, unified logical coordinate system on a composite surface composed of multiple regions with different scaling ratios. This ensures that user interaction operations on any scaled area (whether it is a stretched flexible background or a proportionally scaled rigid button) can be losslessly converted into the logical coordinate input expected by the original application.

[0261] The present application will now be described and illustrated through specific embodiments. Figure 3 This is a schematic diagram of an adapted system architecture according to an embodiment of this application, such as... Figure 3 As shown, the system architecture includes:

[0262] A configuration parser is used to parse the input external layout description file, extract configuration information, and provide it for use by subsequent modules.

[0263] The hybrid rendering engine combines the original application content with external layout configurations to generate the final visual image. It consists of two sub-processes: providing an image source by passing in the original Canvas application (as an off-screen image source); and reassembling and drawing the image by recombinating the image content according to the layout configuration and drawing it onto the visible Canvas. The user then sees and interacts with the final rendered image.

[0264] The region-aware input calibration module handles user physical input (such as touch and mouse clicks) and calibrates the input coordinates based on the physical coordinate system context provided by the visible Canvas, combined with the layout and scaling of the currently displayed content, to match the logical coordinates of the original application. The input calibration module outputs the calibrated logical coordinates and passes them to the original application's event handlers to ensure accurate interaction.

[0265] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0266] This embodiment also provides a screen display control device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the terms "module," "unit," "subunit," etc., can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0267] Figure 4 This is a structural block diagram of a screen display control device according to an embodiment of this application, such as... Figure 4 As shown, the device includes:

[0268] The acquisition module 10 is used to acquire the layout description file of the application; the layout description file defines multiple original local regions of the application at a preset logical resolution, and the scaling type of each original local region.

[0269] The first creation module 20 is used to create an off-screen canvas adapted to a preset logical resolution and redirect the original display content of the application to the off-screen canvas; the area on the off-screen canvas corresponds to the original local area.

[0270] The second creation module 30 is used to create a visible canvas whose size matches the physical size of the target screen;

[0271] The display module 40 is used to calculate the target display layout of multiple original local areas on the visible canvas based on the scaling type and physical size; for each original local area, it reads the pixel data of the original display content from the corresponding area of ​​the off-screen canvas, and renders the pixel data onto the visible canvas for display according to the target display layout.

[0272] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination. Specific examples in this embodiment can be found in the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0273] This embodiment also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0274] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0275] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0276] S1, obtain the application's layout description file; the layout description file defines multiple original local regions of the application at a preset logical resolution, and the scaling type of each original local region.

[0277] S2 creates an off-screen canvas adapted to a preset logical resolution and redirects the application's original display content to the off-screen canvas; the area on the off-screen canvas corresponds to the original local area.

[0278] S3 creates a visible canvas whose size matches the physical size of the target screen.

[0279] S4, based on the scaling type and physical size, calculates the target display layout of multiple original local regions on the visible canvas; for each original local region, it reads the pixel data of the original display content from the corresponding area of ​​the off-screen canvas, and renders the pixel data onto the visible canvas according to the target display layout.

[0280] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0281] Furthermore, in conjunction with the screen display control methods in the above embodiments, this application embodiment can provide a storage medium for implementation. The storage medium stores a computer program; when the computer program is executed by a processor, it implements any of the screen display control methods in the above embodiments.

[0282] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0283] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0284] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0285] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A screen display control method characterized by comprising: The method is applied to an HTML5 Canvas application developed based on a fixed logical resolution, and comprises the following steps: obtaining an application-independent layout description file, wherein the layout description file defines a plurality of original local areas of the application at a preset logical resolution and a stretching type of each of the original local areas, the plurality of original local areas are determined based on a picture semantic function attribute of the application, and the stretching type of the original local area is determined based on a preset visual fidelity index of the application; creating an off-screen canvas adapted to the preset logical resolution, and redirecting original display content of the application to the off-screen canvas, wherein the areas on the off-screen canvas correspond to the original local areas; creating a visible canvas, wherein the size of the visible canvas matches a physical size of a target screen; calculating a target display layout of the plurality of original local areas on the visible canvas based on the stretching type and the physical size, and for each of the original local areas, using a drawImage API to read pixel data of the original display content from a corresponding area of the off-screen canvas and rendering the pixel data to the visible canvas according to the target display layout for display; the step of redirecting the original display content of the application to the off-screen canvas comprises: intercepting a request of the application for obtaining a canvas element, and returning the off-screen canvas as a response result of the request to the application, and the original display content is redirected to the off-screen canvas based on the response result; or when the application is initialized, the off-screen canvas is transmitted into the application as a rendering parameter, and the original display content is redirected to the off-screen canvas based on the rendering parameter; the step of calculating the target display layout of the plurality of original local areas on the visible canvas based on the stretching type and the physical size comprises: calculating a reference scaling factor based on a comparison result between the physical size and a logical size of the plurality of original local areas; for a rigid local area with a rigid stretching type, performing scaling calculation based on the reference scaling factor, and determining a corresponding target display layout of the rigid local area on the visible canvas; 2. The screen display control method according to claim 1, characterized by, for a flexible local area with a flexible stretching type, filling a remaining display space on the visible canvas determined after scaling of the rigid local area based on the reference scaling factor and the physical size, and determining a corresponding target display layout of the flexible local area. the step of calculating the reference scaling factor based on a comparison result between the physical size and a logical size of the plurality of original local areas comprises: calculating the reference scaling factor based on a comparison result between the physical size and a logical size of any rigid local area in the original local areas; or calculating the reference scaling factor based on a comparison result between the physical size and a logical size of any flexible local area in the original local areas. calculating, based on logical sizes of the rigid local regions in the plurality of the original local regions, a plurality of total logical spans along a plurality of different logical directions in a logical space occupied by the rigid local regions together; calculating, for the plurality of different logical directions, a ratio between the physical size and the corresponding total logical span; determining a minimum value of the plurality of ratios as the reference scaling factor.

3. The screen display control method according to claim 1, characterized by, filling, based on the reference scaling factor and the physical size, a remaining display space on the visible canvas determined after scaling of the rigid regions, and determining a target display layout corresponding to the flexible local region, comprises: calculating, according to a physical size of each of the rigid local regions calculated based on the reference scaling factor, a remaining physical space on the visible canvas that is not occupied; allocating the remaining physical space to one or more of the flexible local regions to determine the target display layout corresponding to the flexible local region.

4. The screen display control method according to claim 1, characterized by, The stretch type further includes a constraint scaling type; the method further comprises: for a constraint local region of the stretch type being constraint scaling, determining a preset layout constraint dimension corresponding to the constraint scaling, and performing scaling calculation based on the physical size and the preset layout constraint dimension to determine a target display layout on the visible canvas corresponding to the constraint local region; for the flexible local region, filling, based on the reference scaling factor and the physical size, a remaining display space on the visible canvas determined after scaling of the rigid local regions and scaling of the constraint local regions, and determining a target display layout corresponding to the flexible local region.

5. The screen display control method according to claim 1, characterized by, After rendering the pixel data to the visible canvas for display, the method further comprises: in response to a user input operation on the visible canvas, obtaining a physical coordinate corresponding to the user input operation; determining, according to the target display layout, an original local region hit by the physical coordinate; mapping the physical coordinate to a logical coordinate under the preset logical resolution based on a size ratio between the hit original local region and the target display layout; providing the logical coordinate to the application program.

6. The screen display control method according to claim 5, characterized by, The mapping of the physical coordinate to the logical coordinate under the preset logical resolution based on the size ratio between the hit original local region and the target display layout comprises: obtaining a source rectangle of the hit original local region and a target rectangle in a target display layout corresponding to the hit original local region; calculating a relative physical coordinate of the physical coordinate with respect to the target rectangle; performing linear interpolation calculation on the relative physical coordinate based on a size ratio of the source rectangle and the target rectangle to obtain a relative logical coordinate of the logical coordinate with respect to the source rectangle; determining a logical coordinate under the preset logical resolution based on a position of the relative logical coordinate and the source rectangle.

7. A storage medium, characterized by The storage medium stores a computer program, and the computer program is configured to execute the screen display control method in any one of claims 1 to 6 when running.

Images