Image rendering method and electronic device

Abstract

The present application discloses an image rendering method, comprising: generating thumbnails of multiple sizes for an original image at a first position; adjusting the resolution of an intermediate image based on the moving speed of the drag marker of the original image during the process of moving a drag marker of the original image from the first position to a second position; selecting a thumbnail of corresponding size based on the resolution, and rendering the thumbnail into an intermediate image of the moving process, so as to solve the problem of frame drops and freezes that are prone to occur during the dragging of the image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image rendering method and electronic device. Background Art

[0002] Image dragging is a common interaction in computer operating interfaces. If the image being dragged is a high-resolution image, either only a fixed label or a fixed thumbnail will be displayed during the dragging process. When the image reaches the end point of the drag, it will take a long time to re-render, and the dragging process is prone to frame drops and freezes.

[0003] How to avoid the long time spent on re-rendering after the image is dragged, and the proneness of frame drops and freezes during the dragging process, has become a technical problem that needs to be solved urgently. Summary of the Invention

[0004] The purpose of this application is to provide an image rendering method and electronic device to solve the problem of frame drops and freezes that easily occur during the dragging of images.

[0005] In a first aspect, the present application provides an image rendering method, comprising:

[0006] Generating thumbnails of multiple sizes for the original image at the first position;

[0007] During the process of moving the drag marker of the original image from the first position to the second position, adjusting the resolution of the intermediate image based on the moving speed of the drag marker of the original image;

[0008] Based on the resolution, a thumbnail of a corresponding size is selected, and the thumbnail is rendered as an intermediate image of the moving process.

[0009] Optionally, generating thumbnails of multiple sizes for the original image at the first position includes:

[0010] When the original image is a two-dimensional image, a convolution interpolation algorithm is used to obtain a plurality of downsampled images of different sizes corresponding to the original image as thumbnails of the plurality of sizes;

[0011] When the original image is a three-dimensional image, obtaining multiple downsampled images of different sizes corresponding to the original image as thumbnails of multiple sizes by at least one of octree compression, KD-Tree compression, 3D Gaussian splattering, deep learning compression, and Poisson disk sampling;

[0012] Rendering the thumbnail into an intermediate image of the moving process includes:

[0013] The thumbnail is enlarged according to the size of the original image, and the enlarged thumbnail is rendered as an intermediate image of the moving process.

[0014] Optionally, adjusting the resolution of the intermediate image based on the moving speed of the drag marker of the original image includes:

[0015] According to the target movement speed collected by the control sensor, the movement speed of the drag marker of the original image is calculated, and the resolution of the intermediate image is adjusted based on the movement speed of the drag marker of the original image, wherein the faster the movement speed of the drag marker, the smaller the resolution of the intermediate image.

[0016] Optionally, the control sensor includes at least one of a mouse, a stylus, a control panel, a joystick, a trackball, a touch screen, an eye tracker, a motion sensing ring, a radar, a depth sensor, and an image sensor;

[0017] The target to which the target movement speed collected by the control sensor belongs includes: at least one of: a mouse, a stylus, a hand, a joystick, a trackball, an eyeball, and a line of sight;

[0018] The sampling frequency of the control sensor for collecting the target movement speed is synchronized with the refresh frequency of the screen displaying the rendered image, and the screen includes a display and / or a projection device.

[0019] Optionally, calculating the moving speed of the drag marker of the original image according to the target moving speed collected by the control sensor, adjusting the resolution of the intermediate image based on the moving speed of the drag marker of the original image; selecting a thumbnail of a corresponding size based on the resolution, and rendering the intermediate image of the moving process includes:

[0020] Based on the first target movement speed at the current moment collected by the control sensor, a first screen coordinate of the corresponding cursor on the screen is obtained, the first screen coordinate is converted into a first canvas logical coordinate through a dual coordinate system mapping relationship, and the movement speed of the drag marker is calculated based on the difference of the first canvas logical coordinate at different times;

[0021] determining a resolution of the intermediate image at a current moment based on a moving speed of the drag marker;

[0022] Rendering the intermediate image at the first canvas logical coordinates according to the resolution of the intermediate image.

[0023] Optionally, the first position and the second position are located in different layers, wherein the first position is located in a picture layer in an image annotation interactive page, and the second position is located in a annotation layer in the image annotation interactive page.

[0024] After the drag mark of the original image enters the annotation layer, no coordinate transformation is performed on the initial annotation in the intermediate image.

[0025] Optionally, after the drag marker of the original image moves into the annotation layer, the method further includes:

[0026] An initial annotation is added to the intermediate image, where a conversion coefficient of the initial annotation is determined based on a resolution of the intermediate image and a Document Object Model (DOM) element size.

[0027] Optionally, after the drag marker of the original image enters the annotation layer, the method further includes:

[0028] Based on a transitional progressive restoration strategy, rendering the intermediate image in N stages of resolution until the resolution of the intermediate image reaches the resolution of the original image, where N is a positive integer greater than 2, and the value of N is inversely proportional to the movement speed;

[0029] The original image is rendered, and an initial annotation is added to the original image, where a conversion coefficient of the initial annotation is determined based on a resolution of the original image and a Document Object Model (DOM) element size.

[0030] Optionally, adding an initial annotation to the intermediate image includes:

[0031] A dual Canvas rendering pipeline is used to render the intermediate image and the initial annotation respectively.

[0032] Optionally, rendering the original image and adding an initial annotation to the original image includes:

[0033] A dual Canvas rendering pipeline is used to render the original image and the initial annotation respectively.

[0034] In a second aspect, an embodiment of the present application provides an electronic device, comprising: at least one memory and at least one processor,

[0035] The at least one memory stores executable code, and the at least one processor is configured to execute the executable code in the at least one memory to implement any one of the above methods.

[0036] In a third aspect, an embodiment of the present application further provides an image rendering device, the device comprising:

[0037] Thumbnail generation module: generates thumbnails of multiple sizes for the original image at the first position;

[0038] A speed detection module: detecting a moving speed of the drag mark of the original image during a process of moving the drag mark of the original image from a first position to a second position;

[0039] A resolution scheduling module: during a process of moving the drag marker of the original image from a first position to a second position, adjusting the resolution of the intermediate image based on a moving speed of the drag marker of the original image;

[0040] Image rendering module: used for selecting thumbnails of corresponding sizes based on the resolution, and rendering the thumbnails into intermediate images of the moving process.

[0041] The thumbnail generation module is further configured to: when the original image is a two-dimensional image, obtain, by a convolution interpolation algorithm, a plurality of downsampled images of different sizes corresponding to the original image as thumbnails of the plurality of sizes;

[0042] When the original image is a three-dimensional image, obtaining multiple downsampled images of different sizes corresponding to the original image as thumbnails of multiple sizes by at least one of octree compression, KD-Tree compression, 3D Gaussian splattering, deep learning compression, and Poisson disk sampling;

[0043] The rendering module is further configured to: enlarge the thumbnail according to the size of the original image, and render the enlarged thumbnail into an intermediate image of the moving process.

[0044] The above-mentioned image rendering device also includes a control sensor, and the above-mentioned resolution scheduling module is also used to: calculate the moving speed of the drag marker of the original image according to the target moving speed collected by the control sensor, and adjust the resolution of the intermediate image based on the moving speed of the drag marker of the original image, wherein the faster the moving speed of the drag marker, the smaller the resolution of the intermediate image.

[0045] Optionally, the image rendering device further includes a display and / or a projection device, and the control sensor includes at least one of a mouse, a stylus, a control panel, a joystick, a trackball, a touch screen, an eye tracker, a motion sensing ring, a radar, a depth sensor, and an image sensor;

[0046] The target to which the target movement speed collected by the control sensor belongs includes at least one of: a mouse, a stylus, a hand, a joystick, a trackball, an eyeball, and a line of sight;

[0047] The sampling frequency of the sensor used to collect the target movement speed is controlled to be synchronized with the refresh frequency of the screen displaying the rendered image, wherein the screen includes a display and / or a projection device.

[0048] Optionally, the resolution scheduling module is further configured to:

[0049] Based on the first target movement speed at the current moment collected by the control sensor, a first screen coordinate of the corresponding cursor on the screen is obtained, the first screen coordinate is converted into a first canvas logical coordinate through a dual coordinate system mapping relationship, and the movement speed of the drag marker is calculated based on the difference of the first canvas logical coordinate at different times; and the resolution of the intermediate image at the current moment is determined based on the movement speed of the drag marker;

[0050] The image rendering module is further configured to render the intermediate image on the first canvas logical coordinates according to the resolution of the intermediate image.

[0051] Optionally, the first position and the second position are located in different layers, wherein the first position is located in a picture layer in an image annotation interactive page, and the second position is located in a annotation layer in the image annotation interactive page.

[0052] The image rendering module is configured not to perform coordinate transformation on the initial annotation in the intermediate image after the dragged marker of the original image enters the annotation layer.

[0053] The above image rendering module is also used to:

[0054] An initial annotation is added to the intermediate image, where a conversion coefficient of the initial annotation is determined based on a resolution of the intermediate image and a Document Object Model (DOM) element size.

[0055] Optionally, the resolution scheduling module is further configured to:

[0056] Based on a transitional progressive restoration strategy, rendering the intermediate image in N stages of resolution until the resolution of the intermediate image reaches the resolution of the original image, where N is a positive integer greater than 2, and the value of N is inversely proportional to the movement speed;

[0057] The image rendering module is further configured to render the original image and add an initial annotation to the original image, wherein a conversion coefficient of the initial annotation is determined based on a resolution of the original image and a Document Object Model (DOM) element size.

[0058] The above image rendering module is also used to:

[0059] A dual Canvas rendering pipeline is used to render the intermediate image and the initial annotation respectively.

[0060] The above image rendering module is also used to:

[0061] A dual Canvas rendering pipeline is used to render the original image and the initial annotation respectively.

[0062] The embodiment of the present application selects the corresponding resolution based on the moving position of the image, renders the intermediate image displayed during the movement, and improves the visual effect of the image during the movement. Moreover, since thumbnails of multiple sizes are generated, this dynamic resolution adjustment based on the interaction speed can improve the display effect, avoid frame drops and freezes during the dragging of the image, and reduce the rendering time of the intermediate image. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A schematic diagram showing the original image during movement.

[0064] Figure 2 A schematic diagram of an image rendering method provided in an embodiment of the present application;

[0065] Figure 3 A schematic diagram of an algorithm for calculating movement speed provided in an embodiment of the present application;

[0066] Figure 4 A schematic diagram of rendering a thumbnail into an intermediate image of a moving process provided by an embodiment of the present application;

[0067] Figure 5 This is a schematic diagram of the picture layer, annotation layer, and the image after entering the annotation layer in an embodiment of the present application.

[0068] Figure 6 Schematic diagram of the original image and the intermediate image formed by the 1 / 2 thumbnail in the embodiment of the present application

[0069] Figure 7 Schematic diagram of an image rendering device in an embodiment of the present application. DETAILED DESCRIPTION

[0070] The present application will be described in detail below in conjunction with the specific embodiments shown in the accompanying drawings, but these embodiments do not limit the present application. Structural, methodological, or functional changes made by ordinary technicians in this field based on these embodiments are included in the scope of protection of the present application.

[0071] High-resolution images are usually replaced by labels during the dragging process, and are not displayed until they are moved to the target position, which has a poor visual effect. Figure 1 As shown, if the original image is always displayed during movement, the graphics card and processor will be burdened, which may cause lag. Figure 1 The resolution of the original image in the video is 3840*2160. If the same resolution is always used during the movement, it will put a heavy burden on the graphics card and processor, and the screen will easily drop frames or freeze.

[0072] The process of generating an AI model typically includes data preparation, data annotation, model training, model verification, and model deployment. During the data annotation process, images are dragged. If the images being annotated are high-resolution, re-rendering during the dragging process can be lengthy, leading to frame drops and lags, which can affect the display quality.

[0073] Reference Figure 2 As shown, an embodiment of the present application provides an image rendering method, comprising:

[0074] Step 201: Generate thumbnails of multiple sizes for the original image at the first position;

[0075] Step 202: During the process of moving the drag marker of the original image from the first position to the second position, adjusting the resolution of the intermediate image based on the moving speed of the drag marker of the original image;

[0076] Step 203: Based on the resolution, select a thumbnail of a corresponding size, and render the thumbnail into an intermediate image of the movement process.

[0077] In step 201 , a convolution interpolation algorithm may be used to obtain multiple downsampled images of different sizes corresponding to the original image as thumbnails of multiple sizes.

[0078] For example, if the original image is a 3D image, then multiple downsampled images of different sizes corresponding to the original image can be obtained as thumbnails of multiple sizes through at least one of octree compression, KD-Tree compression, 3D Gaussian splattering, deep learning compression, and Poisson disk sampling.

[0079] In step 202, the moving speed of the drag marker of the original image can be achieved in the following way: according to the target moving speed collected by the control sensor, the moving speed of the drag marker of the original image is calculated, and the resolution of the corresponding intermediate image is adjusted based on the moving speed of the drag marker of the original image, wherein the faster the moving speed of the drag marker is, the smaller the resolution of the corresponding intermediate image is.

[0080] The control sensor may include, but is not limited to, at least one of a mouse, a stylus, a control panel, a joystick, a trackball, a touch screen, an eye tracker, a motion sensing ring, a radar, a depth sensor, and an image sensor;

[0081] The target corresponding to the target movement speed collected by the control sensor may be, but is not limited to, at least one of a mouse, a stylus, a hand, a joystick, a trackball, an eyeball, and a line of sight.

[0082] For example, when the control sensor is a mouse, the user interacts with the image by moving the mouse. After obtaining the mouse movement speed, speed-aware rendering can be used to determine the image rendering decision based on the mouse movement speed. The core of this decision is the relationship between speed and resolution, which is associated through a negative correlation function.

[0083] Optionally, the sampling frequency of the control sensor for collecting the target movement speed is synchronized with the refresh frequency of the screen displaying the rendered image, and the screen includes a display and / or a projection device.

[0084] In the embodiments of this application, the sampling frequency of JavaScript mouse events depends primarily on the browser's refresh rate and the hardware configuration. The browser will try to synchronize with screen refreshes to avoid wasting computing resources. Therefore, the sampling frequency of JavaScript mouse events can be 60 Hz, 100 Hz, 120 Hz, 144 Hz, and so on, without limitation. Figure 3 As shown, assuming that the sampling frequency for judging the mouse movement speed is 100Hz, that is, the coordinate position of the drag mark corresponding to the mouse is sampled every 10 milliseconds, the coordinate position of point A is (x a ,y a ) at point B is (x b ,y b ), the moving speed v of the drag marker can be calculated according to the following formula:

[0085]

[0086]

[0087]

[0088] in, and Represents the difference between the two coordinate positions on the x-axis and y-axis, (x a ,y a ) represents the coordinate position of the drag marker at point A, (x b ,y b ) represents the coordinate position of the drag marker at point B.

[0089] For example, if the control sensor is a joystick, the user interacts with the image by moving the joystick. The change in the joystick's tilt angle can be used to determine the movement speed of the corresponding drag marker. In this case, the resolution is still negatively correlated with the movement speed of the drag marker, and therefore negatively correlated with the speed of the joystick angle change (user interaction speed).

[0090] For example, a multi-resolution image cache can be generated first, and the resolution of the rendered intermediate image can be switched according to the speed threshold (for example, switching in segments of 5px / ms and 15px / ms). In this way, the mechanism of changing the image rendering resolution in real time according to the user interaction speed can reduce the image display resolution when the user moves the image quickly, thereby reducing the burden on the graphics card and processor and improving the interactive perception experience.

[0091] Exemplarily, generating thumbnails of multiple sizes for the original image at the first position in step 201 may include:

[0092] Through the convolution interpolation algorithm, multiple downsampled images of different sizes corresponding to the original image are obtained as thumbnails of multiple sizes, for example, 1 / 2 thumbnail, 1 / 4 thumbnail and 1 / 8 thumbnail.

[0093] Referring to Table 1, the embodiment of the present application can adopt a three-stage resolution decision function. The moving speed of the drag mark corresponding to the first stage is between 0-5px / ms, and the resolution of the corresponding intermediate image is Figure 1 The total number of pixels is the same as the original. Figure 1 When the drag marker moves at a speed between 5 and 15 px / ms, the intermediate image uses a 1 / 2 thumbnail with a resolution of 1920×1080 and only 25% of the original image’s total pixels. When the drag marker moves at a speed greater than 15 px / ms, the intermediate image uses a 1 / 4 thumbnail with a resolution of 960×540 and 6.25% of the original image’s total pixels.

[0094]

[0095] Table 1

[0096] like Figure 4 As shown, Figure 4 The left side is the original image, the middle is the 1 / 2 thumbnail, and the right side is the intermediate image. When rendering the thumbnail as the intermediate image during the movement, you can enlarge the thumbnail according to the size of the original image and render the enlarged thumbnail as the intermediate image during the movement. When the drag marker moves at a low speed, render the intermediate image with a high resolution. As the speed increases, switch to using a lower resolution thumbnail and enlarge the width and height of the thumbnail to the original. Figure 1 The same size is used (that is, rendering a lower resolution image in the same size area). In this way, the size of the intermediate image remains unchanged, but the resolution will change, thus avoiding the image from being suddenly large or small. This can take into account both the display capabilities of the display and processor, and the user's visual experience.

[0097] In an optional embodiment of the present application, the coordinate system where the control sensor is located and the coordinate system where the screen where the image is displayed are located can be two different coordinate systems, and the movement speed conversion between the two coordinate systems can be achieved by, but is not limited to, the following formula.

[0098] There may be different layers of canvas on the screen. If the first position and the second position are located in different layers, the conversion from the coordinate system of the control sensor to the coordinate system of the canvas on the screen can be achieved in the following manner.

[0099] Exemplarily, the control sensor collects a first target movement speed at a current moment, obtains a first screen coordinate of a corresponding cursor on the screen, converts the first screen coordinate into a first canvas logical coordinate through a dual coordinate system mapping relationship, and calculates the movement speed of the drag marker based on the difference of the first canvas logical coordinate at different times;

[0100] determining a resolution of the intermediate image at a current moment based on a moving speed of the drag marker;

[0101] Rendering the intermediate image at the first canvas logical coordinates according to the resolution of the intermediate image.

[0102] In an embodiment of the present application, the cursor can be a cursor corresponding to a control sensor such as a mouse, stylus, hand, joystick, trackball, etc., and the drag mark can be a drag mark corresponding to the intermediate image or a drag mark converted from the cursor corresponding to the control sensor, which is not limited here.

[0103] like Figure 5 As shown, assume that the first position and the second position are located in different layers, with the first position being located in the image layer of the image annotation interactive page, and the second position being located in the annotation layer of the same page. When the intermediate image enters the annotation layer, the image and annotation are combined to form an intermediate image with the image annotation. After the drag marker is moved to the end position, the intermediate image slowly returns to the original image.

[0104] In this embodiment of the present application, an initial annotation (an automatically added annotation) can also be added to the intermediate image. The conversion factor of the initial annotation is determined based on the resolution of the intermediate image and the Document Object Model (DOM) element size. This ensures that the relative position of the annotation and the original image remains unchanged. DOM (Document Object Model) element size can include the width, height, margins, and padding of the element (annotation). For example, the DOM element size can be obtained by obtaining the element's CSS (Cascading Style Sheets) style attributes.

[0105] For example, in embodiments of the present application, when the original image is a two-dimensional image, algorithms such as logical coordinate mapping can be used to maintain the relative positions of the original image, intermediate image, and annotation layer. When the original image is a three-dimensional image, dynamic coordinate transformation (image editing / maps) or viewport-world coordinate separation (3D / GIS systems) can be used to maintain the relative positions of the original image, intermediate image, and annotation layer.

[0106] After dragging the marker into the annotation layer, the method further includes:

[0107] Based on a transitional progressive restoration strategy, rendering the intermediate image in N stages of resolution until the resolution of the intermediate image reaches the resolution of the original image (i.e., the intermediate image is restored to the original image), where N is a positive integer greater than 2, and the value of N is inversely proportional to the movement speed;

[0108] The original image is rendered on the annotation layer, and initial annotations are added to the original image. The conversion coefficient of the initial annotations is determined based on the resolution of the original image and the Document Object Model (DOM) element size.

[0109] For example, intermediate images can be rendered in three, four, or five stages of resolution grading. The number of stages can be adjusted based on the speed of the drag marker. For example, faster movement results in fewer stages, which reduces the amount of computation per unit time and avoids image lag.

[0110] After entering the annotation layer, the intermediate image and the initial annotation can be rendered separately using a dual Canvas rendering pipeline. After the intermediate image is restored to the original image, the original image is rendered in the annotation layer, and the dual Canvas rendering pipeline can be used to render the original image and the initial annotation separately.

[0111] When rendering the intermediate image, a dual Canvas rendering pipeline may be used to render the intermediate image and the initial annotation separately.

[0112] Since dual Canvas rendering pipelines are used to render images and annotations separately, one pipeline renders images and one pipeline renders annotations, which can improve rendering efficiency and does not interfere with each other, thereby improving display effects.

[0113] like Figure 6 As shown in the figure, after the drag marker of the original image enters the annotation layer, the coordinates of the annotation in the intermediate image do not need to be transformed. This is because the intermediate image is rendered by enlarging the thumbnail to the original image size. The center position of the annotation before and after this operation remains unchanged relative to the original image. Therefore, in the annotation layer, the coordinates of the initial annotation do not need to be transformed.

[0114] Due to the generation of multi-level thumbnails and speed-aware decisions, the rendering resolution is dynamically selected based on the mouse movement speed (such as 100Hz sampling) (high resolution for low speed and low resolution for high speed). In addition, through logical coordinate mapping, the coordinates of the annotation layer remain unchanged, and only the image layer resolution is adjusted, thereby ensuring the relative consistency of the annotation position.

[0115] See also Figure 7 , an embodiment of the present application further provides an image rendering device, the device comprising:

[0116] Thumbnail generation module 701: generates thumbnails of multiple sizes for the original image at the first position;

[0117] The speed detection module 702 detects the moving speed of the drag marker of the original image during the process of moving the drag marker of the original image from the first position to the second position.

[0118] A resolution scheduling module 703 is configured to adjust the resolution of the intermediate image based on the moving speed of the drag marker of the original image during the process of moving the drag marker of the original image from the first position to the second position.

[0119] Image rendering module 704: configured to select a thumbnail of a corresponding size based on the resolution, and render the thumbnail into an intermediate image of the movement process.

[0120] The thumbnail generation module is further configured to: when the original image is a two-dimensional image, obtain, by a convolution interpolation algorithm, a plurality of downsampled images of different sizes corresponding to the original image as thumbnails of the plurality of sizes;

[0121] When the original image is a three-dimensional image, obtaining multiple downsampled images of different sizes corresponding to the original image as thumbnails of multiple sizes by at least one of octree compression, KD-Tree compression, 3D Gaussian splattering, deep learning compression, and Poisson disk sampling;

[0122] The rendering module is further configured to: enlarge the thumbnail according to the size of the original image, and render the enlarged thumbnail into an intermediate image of the moving process.

[0123] The image rendering device in the embodiment of the present application also includes a control sensor, and the above-mentioned resolution scheduling module is also used to: calculate the moving speed of the drag marker of the original image based on the target moving speed collected by the control sensor, and adjust the resolution of the intermediate image based on the moving speed of the drag marker of the original image, wherein the faster the moving speed of the drag marker, the smaller the resolution of the intermediate image.

[0124] Optionally, the image rendering device in the embodiment of the present application further includes a display and / or a projection device, and the control sensor includes at least one of a mouse, a stylus, a control panel, a joystick, a trackball, a touch screen, an eye tracker, a motion sensing ring, a radar, a depth sensor, and an image sensor;

[0125] The target to which the target movement speed collected by the control sensor belongs includes at least one of: a mouse, a stylus, a hand, a joystick, a trackball, an eyeball, and a line of sight;

[0126] The sampling frequency of the sensor used to collect the target movement speed is controlled to be synchronized with the refresh frequency of the screen displaying the rendered image, wherein the screen includes a display and / or a projection device.

[0127] Optionally, the resolution scheduling module is further configured to:

[0128] Based on the first target movement speed at the current moment collected by the control sensor, a first screen coordinate of the corresponding cursor on the screen is obtained, the first screen coordinate is converted into a first canvas logical coordinate through a dual coordinate system mapping relationship, and the movement speed of the drag marker is calculated based on the difference of the first canvas logical coordinate at different times; and the resolution of the intermediate image at the current moment is determined based on the movement speed of the drag marker;

[0129] The image rendering module is further configured to render the intermediate image on the first canvas logical coordinates according to the resolution of the intermediate image.

[0130] Optionally, the first position and the second position are located in different layers, wherein the first position is located in a picture layer in an image annotation interactive page, and the second position is located in a annotation layer in the image annotation interactive page.

[0131] The image rendering module is configured not to perform coordinate transformation on the initial annotation in the intermediate image after the dragged marker of the original image enters the annotation layer.

[0132] The above image rendering module is also used to:

[0133] An initial annotation is added to the intermediate image, where a conversion coefficient of the initial annotation is determined based on a resolution of the intermediate image and a Document Object Model (DOM) element size.

[0134] Optionally, the resolution scheduling module is further configured to:

[0135] Based on a transitional progressive restoration strategy, rendering the intermediate image in N stages of resolution until the resolution of the intermediate image reaches the resolution of the original image, where N is a positive integer greater than 2, and the value of N is inversely proportional to the movement speed;

[0136] The image rendering module is further configured to render the original image and add an initial annotation to the original image, wherein a conversion coefficient of the initial annotation is determined based on a resolution of the original image and a Document Object Model (DOM) element size.

[0137] The above image rendering module is also used to:

[0138] A dual Canvas rendering pipeline is used to render the intermediate image and the initial annotation respectively.

[0139] The above image rendering module is also used to:

[0140] A dual Canvas rendering pipeline is used to render the original image and the initial annotation respectively.

[0141] The embodiment of the present application selects the corresponding resolution based on the moving position of the image, renders the intermediate image displayed during the movement, and improves the visual effect of the image during the movement. Moreover, since thumbnails of multiple sizes are generated, this dynamic resolution adjustment based on the interaction speed can improve the display effect, avoid frame drops and freezes during the dragging of the image, and reduce the rendering time of the intermediate image.

[0142] Based on the same inventive concept, an embodiment of the present application also provides an electronic device, comprising: at least one memory and at least one processor, the at least one memory storing executable code, and the at least one processor being used to execute the executable code in the at least one memory to implement the above-mentioned image rendering method.

[0143] Exemplarily, the electronic device may further include a display for displaying at least one of the original image, the intermediate image, and the annotation.

[0144] The electronic device may further include a control sensor. The control sensor may include at least one of a mouse, a stylus, a control panel, a joystick, a trackball, a touch screen, an eye tracker, a motion sensing ring, a radar, a depth sensor, and an image sensor. The target to which the target movement speed collected by the control sensor belongs includes at least one of a mouse, a stylus, a hand, a joystick, a trackball, an eyeball, and a line of sight.

[0145] The sampling frequency of the control sensor for collecting the target movement speed is synchronized with the refresh frequency of the screen displaying the rendered image, and the screen includes a display and / or a projection device.

[0146] The processor may, when the original image is a two-dimensional image, obtain a plurality of downsampled images of different sizes corresponding to the original image as thumbnails of the plurality of sizes by using a convolution interpolation algorithm; when the original image is a three-dimensional image, obtain a plurality of downsampled images of different sizes corresponding to the original image as thumbnails of the plurality of sizes by using at least one of octree compression, KD-Tree compression, 3D Gaussian splattering, deep learning compression, and Poisson disk sampling;

[0147] The processor may enlarge the thumbnail according to the size of the original image, and render the enlarged thumbnail into an intermediate image of the moving process.

[0148] The processor can calculate the moving speed of the drag marker of the original image according to the target moving speed collected by the control sensor, and adjust the resolution of the intermediate image based on the moving speed of the drag marker of the original image, wherein the faster the moving speed of the drag marker, the smaller the resolution of the intermediate image.

[0149] The processor may acquire a first screen coordinate of a corresponding cursor on the screen based on a first target movement speed at a current moment collected by the control sensor, and convert the first screen coordinate into a first canvas logical coordinate through a dual coordinate system mapping relationship;

[0150] determining a resolution of the intermediate image at a current moment based on the first target moving speed;

[0151] Rendering the intermediate image at the first canvas logical coordinates according to the resolution of the intermediate image.

[0152] The processor may not perform coordinate transformation on the initial annotation in the intermediate image after the dragged marker of the original image enters the annotation layer.

[0153] The processor may add an initial annotation to the intermediate image after the drag marker of the original image enters the annotation layer, wherein a conversion coefficient of the initial annotation is determined based on a resolution of the intermediate image and a Document Object Model (DOM) element size.

[0154] The processor may further render the intermediate image in N stages of resolution after the drag marker of the original image enters the annotation layer based on a transitional progressive recovery strategy until the resolution of the intermediate image reaches the resolution of the original image, where N is a positive integer greater than 2 and the value of N is inversely proportional to the movement speed;

[0155] The original image is rendered, and an initial annotation is added to the original image, where a conversion coefficient of the initial annotation is determined based on a resolution of the original image and a Document Object Model (DOM) element size.

[0156] The processor may also use a dual Canvas rendering pipeline to render the intermediate image and the initial annotation respectively.

[0157] The processor may also use dual Canvas rendering pipelines to render the original image and the initial annotation respectively.

[0158] The above processor may be a GPU or a CPU.

[0159] The memory may be a random access memory, a read-only memory, a non-volatile memory, a programmable ROM, an erasable PROM, an electrically erasable memory, a flash memory, an optical memory, a register, and the like. The processor may be a general-purpose processor, which may be a processor that performs specific steps and / or operations by reading and executing a computer program stored in the memory, and the general-purpose processor may use data stored in the memory in the process of performing the steps and / or operations. The general-purpose processor may be a central processing unit, an ASIC, an FPGA, and the like. During implementation, each step of the above method may be completed by an integrated logic circuit of hardware in the processor or by instructions in the form of software. The method disclosed in conjunction with the embodiments of the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0160] Exemplarily, the input device includes but is not limited to at least one of a keyboard, a touch panel, a voice input device, and an image sensor.

[0161] In the above embodiments, all or part of the embodiments can be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or solid-state drive (SSD).

[0162] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0163] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The above is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application are included in the scope of protection of this application.

Claims

1. An image rendering method, characterized in that: include: Generating thumbnails of multiple sizes for the original image at the first position; During the process of moving the drag marker of the original image from the first position to the second position, adjusting the resolution of the intermediate image based on the moving speed of the drag marker of the original image; Based on the resolution, selecting a thumbnail of a corresponding size, and rendering the thumbnail into an intermediate image of the movement process; The first position and the second position are located in different layers, wherein the first position is located in the picture layer in the image annotation interaction page, and the second position is located in the annotation layer in the image annotation interaction page. After the drag mark of the original image enters the annotation layer, no coordinate transformation is performed on the initial annotation in the intermediate image.

2. The method according to claim 1, wherein Generating thumbnails of multiple sizes for the original image at the first position includes: When the original image is a two-dimensional image, a convolution interpolation algorithm is used to obtain a plurality of downsampled images of different sizes corresponding to the original image as thumbnails of the plurality of sizes; When the original image is a three-dimensional image, obtaining multiple downsampled images of different sizes corresponding to the original image as thumbnails of multiple sizes by at least one of octree compression, KD-Tree compression, 3D Gaussian splattering, deep learning compression, and Poisson disk sampling; Rendering the thumbnail into an intermediate image of the moving process includes: The thumbnail is enlarged according to the size of the original image, and the enlarged thumbnail is rendered as an intermediate image of the moving process.

3. The method according to claim 1, wherein Adjusting the resolution of the intermediate image based on the moving speed of the drag marker of the original image includes: According to the target movement speed collected by the control sensor, the movement speed of the drag marker of the original image is calculated, and the resolution of the intermediate image is adjusted based on the movement speed of the drag marker of the original image, wherein the faster the movement speed of the drag marker, the smaller the resolution of the intermediate image.

4. The method according to claim 3, wherein The control sensor includes at least one of a mouse, a stylus, a control panel, a joystick, a trackball, a touch screen, an eye tracker, a motion sensing ring, a radar, a depth sensor, and an image sensor; The target to which the target movement speed collected by the control sensor belongs includes: at least one of: a mouse, a stylus, a hand, a joystick, a trackball, an eyeball, and a line of sight; The sampling frequency of the control sensor for collecting the target movement speed is synchronized with the refresh frequency of the screen displaying the rendered image, and the screen includes a display and / or a projection device.

5. The method according to claim 3, wherein calculating a moving speed of the drag marker of the original image according to a target moving speed acquired by a control sensor, and adjusting a resolution of the intermediate image based on the moving speed of the drag marker of the original image; Based on the resolution, a thumbnail of a corresponding size is selected, and an intermediate image of the movement process is rendered, comprising: Based on the first target movement speed at the current moment collected by the control sensor, a first screen coordinate of the corresponding cursor on the screen is obtained, the first screen coordinate is converted into a first canvas logical coordinate through a dual coordinate system mapping relationship, and the movement speed of the drag marker is calculated based on the difference of the first canvas logical coordinate at different times; determining a resolution of the intermediate image at a current moment based on a moving speed of the drag marker; Rendering the intermediate image at the first canvas logical coordinates according to the resolution of the intermediate image.

6. The method according to claim 1, wherein After the drag marker of the original image moves into the annotation layer, the method further includes: An initial annotation is added to the intermediate image, where a conversion coefficient of the initial annotation is determined based on a resolution of the intermediate image and a Document Object Model (DOM) element size.

7. The method according to claim 1, wherein After the drag mark of the original image enters the annotation layer, the method further includes: Based on a transitional progressive restoration strategy, rendering the intermediate image in N stages of resolution until the resolution of the intermediate image reaches the resolution of the original image, where N is a positive integer greater than 2, and the value of N is inversely proportional to the movement speed; The original image is rendered, and an initial annotation is added to the original image, where a conversion coefficient of the initial annotation is determined based on a resolution of the original image and a Document Object Model (DOM) element size.

8. The method according to claim 6, wherein Adding an initial annotation to the intermediate image includes: A dual Canvas rendering pipeline is used to render the intermediate image and the initial annotation respectively.

9. The method according to claim 7, wherein The rendering of the original image and adding an initial annotation to the original image includes: A dual Canvas rendering pipeline is used to render the original image and the initial annotation respectively.

10. An electronic device, characterized in that: include: at least one memory and at least one processor, The at least one memory stores an executable code, and the at least one processor is configured to execute the executable code in the at least one memory to implement the method according to any one of claims 1 to 9.

Images