Method, apparatus, device, medium and program product for image rendering

Abstract

The present disclosure provides an image rendering method, which can be applied to the field of image processing technology. The image rendering method includes: obtaining a model file, idle time, and remaining memory space, where the model file includes a plurality of mesh data, the idle time includes the time when processing resources are not occupied in each frame, and the remaining memory space includes the data generated during related processing; setting batch information, where the batch information includes a preset number of mesh data as a batch; and when both the idle time is greater than a preset time threshold and the remaining memory space is greater than a preset memory threshold, performing related processing on the preset number of mesh data based on the batch to obtain a rendered image. The present disclosure also provides an image rendering device, device, storage medium, and program product.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, specifically including the field of image processing, and more specifically to a method, apparatus, device, medium, and program product for image rendering. Background Art

[0002] Due to the huge advantages of 3D visualization in terms of experience and intuitiveness, users' pursuit of 3D visualization is increasing both in quantity and quality.

[0003] However, existing 3D visualization has many problems such as lag, unsmooth rendering, and preemption of threads by non-visual items.

[0004] Therefore, how to perform image rendering is an urgent problem for those skilled in the art. Summary of the Invention

[0005] In view of the above problems, the present disclosure provides a method, apparatus, device, medium, and program product for image rendering that improves the user experience.

[0006] According to a first aspect of the present disclosure, a method for image rendering is provided, including: obtaining a model file, idle time, and remaining memory space, where the model file includes a plurality of mesh data, the idle time includes the time when processing resources are not occupied in each frame, and the remaining memory space includes the data generated during related processing; setting batch information, where the batch information includes a preset number of mesh data as one batch; and when both the idle time is greater than a preset time threshold and the remaining memory space is greater than a preset memory threshold, performing related processing on the preset number of mesh data based on the batch to obtain a rendered image.

[0007] According to an embodiment of the present disclosure, the related processing includes a first process, a second process, and a third process. Among them, the first process includes converting the mesh data into task data, the second process includes converting the task data into a memory object, and the third process includes rendering the memory object to obtain a rendered image.

[0008] According to an embodiment of the present disclosure, the performing related processing on the preset number of mesh data based on the batch includes: for the mesh data in the same batch, when both the idle time is greater than a preset time threshold and the remaining memory space is greater than a preset memory threshold, performing the first process on the preset number of mesh data to obtain a preset number of task data; retrieving the preset number of task data one by one; performing the second process on the preset number of task data to obtain a preset number of memory objects; and performing the third process on the preset number of memory objects to obtain a preset number of rendered images.

[0009] According to an embodiment of the present disclosure, when the remaining memory space reaches a preset memory threshold, performing a first process on a preset number of grid data to obtain a preset number of task data, including: after each piece of task data obtained through the first process, caching the task data into the remaining memory space one by one.

[0010] According to an embodiment of the present disclosure, setting batch information, the batch information including that a preset number of grid data is one batch, further including: setting the priority processing level of each grid data, and setting the grid data with a higher priority processing level in the batch to be preferentially processed.

[0011] According to an embodiment of the present disclosure, the grid data includes: a first parameter and a second parameter, wherein the first parameter includes the position parameters between each memory object, and the second parameter includes the texture mapping parameters of the memory object.

[0012] According to an embodiment of the present disclosure, setting the processing priority level of each grid data, and setting the grid data with a higher processing priority level in the batch to be preferentially processed, includes: classifying based on the first parameter and the second parameter of each grid quantity to obtain a classification result; and setting the priority processing level based on each classification result.

[0013] A second aspect of the present disclosure provides an image rendering apparatus, including: an acquisition module, a setting module, and a processing module, wherein the acquisition module is configured to acquire a model file, idle time, and remaining memory space, the model file includes a plurality of grid data, the idle time includes the time when the processing resources are not occupied in each frame, and the remaining memory space includes the data generated during the relevant processing; the setting module is configured to set batch information, the batch information including that a preset number of grid data is one batch; and the processing module is configured to, when both the idle time is greater than a preset time threshold and the remaining memory space is greater than a preset memory threshold, perform relevant processing on a preset number of the grid data based on the batch to obtain a rendered image.

[0014] A third aspect of the present disclosure provides an electronic device, including: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are caused to execute the above-mentioned image rendering method.

[0015] A fourth aspect of the present disclosure further provides a computer-readable storage medium, on which executable instructions are stored, and when the instructions are executed by a processor, the processor is caused to execute the above-mentioned image rendering method.

[0016] The fifth aspect of the present disclosure also provides a computer program product, including a computer program which, when executed by a processor, implements the above-mentioned method for image rendering. Description of the Drawings

[0017] Through the following description of the embodiments of the present disclosure with reference to the drawings, the content of the present disclosure and other objects, features and advantages will become clearer. In the drawings:

[0018] Figure 1 Schematically shows an application scenario diagram of a method, apparatus, device, medium and program product for image rendering according to an embodiment of the present disclosure.

[0019] Figure 2 Schematically shows a flowchart of a method for image rendering according to an embodiment of the present disclosure.

[0020] Figure 3 Schematically shows a flowchart of related processing for image rendering according to an embodiment of the present disclosure.

[0021] Figure 4 Schematically shows a flowchart of a method for a priority processing level according to an embodiment of the present disclosure.

[0022] Figure 5 Schematically shows a schematic diagram of a method for the entire process of image rendering according to an embodiment of the present disclosure.

[0023] Figure 6 Schematically shows a block diagram of the structure of an apparatus for image rendering according to an embodiment of the present disclosure.

[0024] Figure 7 Schematically shows a block diagram of an electronic device suitable for implementing the method for image rendering according to an embodiment of the present disclosure. Detailed Description of the Embodiments

[0025] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the following detailed description, for the sake of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, obviously, one or more embodiments can also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present disclosure.

[0026] The terms used herein are merely for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprising", "including", etc. used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0027] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0028] In cases where expressions similar to "at least one of A, B, and C, etc." are used, generally, it should be interpreted according to the meaning commonly understood by those of ordinary skill in the art (for example, "a system having at least one of A, B, and C" should include, but not be limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0029] Before describing the technical solutions of the present disclosure, the technical fields in the art are described as follows:

[0030] 3D rendering engine: It refers to a graphics library that can utilize graphics hardware to obtain high-performance graphics rendering capabilities. A typical one is threejs, which is a simple and easy-to-use 3D graphics library that encapsulates WebGL. The rendering engine renders the model into a 3D effect.

[0031] Mesh: In this computer, an arc is composed of a finite number of points connected by line segments. A three-dimensional model, a solid figure, is composed of a finite number of meshes. Generally speaking, the more the number of Meshes, the higher the accuracy. Such a model is called a Mesh model; the Mesh data format is mainly composed of Geometry and Material.

[0032] Geometry: It stores the points and the relationships between points used in the Mesh model, and can describe cubes, planes, spheres, etc. Complex shapes can also be made and exported into model files through professional software.

[0033] Material: It is used to describe the parameters other than Geometry in the model, including color, texture, reflectivity, luminosity, etc.

[0034] Memory tool: A built-in function used by the browser to obtain the memory usage situation.

[0035] RequestIdleCallback: A built-in function on the browser side. The first parameter of this function is a callback function that will be called during the browser's idle period. This enables developers to perform background or low-priority work on the main event loop without affecting latency-critical events such as animations and input responses. Functions generally execute in the order of first-in, first-out. However, if the callback function specifies an execution timeout (the second parameter of requestIdleCallback, in milliseconds), then the execution order may be disrupted in order to execute the function before the timeout.

[0036] Action: After the 3D model is rendered on the browser, the operations performed by the user on the model or a part of the model, such as translation and flipping.

[0037] setInterval: A built-in timer function on the browser side. The first parameter is the callback, and the second parameter is delay (indicating how long after which to execute, in milliseconds). The theoretical interval between each call of the callback is delay milliseconds.

[0038] setTimeout: A built-in timer function on the browser side. The first parameter is the callback, and the second parameter is delay (indicating how long after which to execute, in milliseconds). The callback will be executed after delay milliseconds, so as to give priority to executing the processing logic with higher priority. The difference from setInterval is that it only executes once at most.

[0039] requestAnimationFrame: A built-in function on the browser side that requests the browser to call the specified callback function before the next rendering. If you want to continue updating the next frame of the animation before the browser's next redraw, you can call requestAnimationFrame again in the callback function.

[0040] The prior art related to the present disclosure is described as follows:

[0041] Prior art (1): By using the built-in setInterval method of the browser, the rendering logic of different components of the model is executed at fixed time intervals. This method is a relatively conventional technique that can spread complex rendering over more time to avoid the phenomenon of lag. After the rendering is completed, the timer can be cancelled. setInterval is often used to handle scenarios of repetitive behaviors such as carousels, polling, and animations.

[0042] The prior art (2) uses the setTimeout method built into the browser to defer the rendering of the model until a specified number of milliseconds have passed. In the callback function, the model can be rendered in portions, that is, only a part is rendered each time the callback is executed. After the current part is completed, another setTimeout is set to enter the next round. This method is logically similar to the setInterval scheme in 1.2. setTimeout is often used to handle scenarios such as delayed operations, throttling, and debouncing.

[0043] The prior art (3), requestAnimationFrame is often used to execute animations because the callback method of this method can ensure that the browser is executed once in each frame, thus ensuring the smoothness of the animation, and thus is superior to setInterval in the animation scenario. Using this method, it can be ensured that a part is rendered in each frame until all are rendered. When the page is hidden or minimized, the callback will be paused, thus saving CPU resources and battery life.

[0044] For the prior art (1), setInterval is a macro task and will be incorporated into the js event loop. If there is no more idle time in the current browser rendering frame, setInterval will be postponed, that is, the timer may not be timed and may be delayed beyond expectation. Effects such as animations may appear unsmooth. In addition, when the page is hidden or minimized, the callback will still be executed.

[0045] For the prior art (2), setTimeout is a macro task and will be incorporated into the js event loop. If there is no more idle time in the current browser rendering frame, setTimeout will be postponed for execution. In addition, when the page is hidden or minimized, the callback will still be executed.

[0046] For the prior art (3), the callback is executed in each frame, which may cause the page to freeze or the user to be unable to perform other click actions.

[0047] In summary, the problem that the rendering operation preempts the thread in the prior art, resulting in a stuck rendering process and a poor user experience, has not been solved yet.

[0048] It should be noted that at the technical level, WebGL and browsers are also constantly providing support. Three.js, which is encapsulated based on WebGL, stands out for its simplicity, ease of use, open source, cross-platform nature, support for various formats of models, and pure front-end implementation, and is widely used in fields such as operation and maintenance, gaming, VR, and architecture. However, three.js itself can support models of any data volume, and at the API level, it also provides methods such as decimation, normal mapping, level of detail, mesh cloning, and geometry parameters to optimize rendering performance, but it does not provide a performance optimization solution at the overall public relations level. When encountering a large model, the browser page often freezes, resulting in a poor user experience.

[0049] Embodiments of the present disclosure provide a method for image rendering, which includes obtaining a model file, idle time, and remaining memory space. The model file includes a plurality of mesh data. The idle time includes the time when the processing resources are not occupied in each frame. The remaining memory space includes the data generated during the relevant processing. Set batch information, where the batch information includes a preset number of mesh data as a batch. And when both the idle time is greater than a preset time threshold and the remaining memory space is greater than a preset memory threshold, perform relevant processing on the preset number of mesh data based on the batch to obtain a rendered image.

[0050] In the embodiments of the present disclosure, by performing the rendering method during idle time, the relevant processing performed for rendering does not preempt the thread resources in each frame, giving priority to ensuring the occupation of processing resources by other operations of the user, so that the relevant processing of rendering hinders other non-image rendering operations of the user as little as possible, ensuring the smoothness of use.

[0051] Figure 1 A schematic application scenario diagram of the image rendering method according to an embodiment of the present disclosure is shown.

[0052] As Figure 1 shown, the application scenario 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0053] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (for example only).

[0054] Terminal devices 101, 102, and 103 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop computers, desktop computers, and so on.

[0055] Server 105 can be a server that provides various services, such as a background management server that supports the websites browsed by users using terminal devices 101, 102, and 103 (for example only). The background management server can analyze and process data such as received user requests, and feedback the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0056] It should be noted that the method for image rendering provided by the embodiments of the present disclosure can generally be executed by server 105. Correspondingly, the device for image rendering provided by the embodiments of the present disclosure can generally be set in server 105. The method for image rendering provided by the embodiments of the present disclosure can also be executed by a server or a server cluster different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Correspondingly, the device for image rendering provided by the embodiments of the present disclosure can also be set in a server or a server cluster different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105.

[0057] It should be understood that Figure 1 the numbers of terminal devices, networks, and servers in

[0058] are merely illustrative. According to the implementation requirements, there can be any number of terminal devices, networks, and servers. Figure 1 Based on the scenario described below Figures 2 to 5 the method for image rendering of the disclosed embodiments will be described in detail through

[0059] Figure 2 Schematically shows a flowchart of the method for image rendering according to an embodiment of the present disclosure.

[0060] As Figure 2 shown, the method for image rendering of this embodiment includes operation S210 to operation S230, and this method for image rendering can be executed by server 105.

[0061] In operation S210, a model file, idle time and remaining memory space are obtained, wherein the model file includes a plurality of grid data, the idle time includes the time when processing resources are not occupied in each frame, and the remaining memory space includes data generated during storage of related processing.

[0062] According to an embodiment of the present disclosure, the model file includes a json format, or other formats that can be converted into a json format by a specific loader. The model file is obtained by direct reception.

[0063] According to an embodiment of the present disclosure, the grid data includes a first parameter and a second parameter, wherein the first parameter includes a position parameter between each memory object, and the second parameter includes a mapping parameter of the memory object.

[0064] It should be noted that the embodiments of the present disclosure can be applied to the application scenario of browser rendering 3D images, and a model file is composed of grid data. For the browser, the grid data in the received model file cannot be rendered directly, and it needs to be converted into a memory object before it can be rendered. In addition, the grid data comes from a wide range of sources, and the format of network data in different software is not unified. In this case, it is necessary to convert the format of the network data through a specific loader, convert it into json format or other formats that can be recognized by the browser, and then operate it.

[0065] For example, the idle time can be obtained by using a requestIdleCallback function built into the browser, and the requestIdleCallback function is periodically executed to retrieve the idle time in each frame. For another example, the remaining memory space can be obtained by using a memory tool built into the browser.

[0066] According to an embodiment of the present disclosure, the method for acquiring the idle time includes: periodically executing a callback function, wherein the callback function includes returning the remaining time after other processing logics in each frame have completed executing related processing.

[0067] According to an embodiment of the present disclosure, the processing resources include thread processing resources.

[0068] It should be noted that most monitors on the market have a fixed refresh rate. A certain time interval needs to be maintained between two refreshes to enable users to have a normal visual experience. Thus, this time interval is the frame in the present disclosure. At the same time, too fast a refresh rate (i.e., too short an interval time) cannot be perceived by the naked eye, resulting in meaningless overhead for the hardware. Therefore, it is more appropriate to take 16 ms for one frame in the present disclosure. Furthermore, the browser will utilize this 16-ms time interval to optimize the rendering of the page. However, for the browser, during the 16 ms of one frame, other user processing logics will also be executed. Especially in a single-threaded processing environment, when other processing logics preempt this thread with the processing logic of image rendering, it will lead to chaos in the processing logic, and further result in a poor experience for users when using the browser to view image rendering. Thus, the embodiments of the present disclosure perform image rendering processing using the remaining idle time in one frame while giving priority to ensuring the processing resources of other processing logics. Of course, the value of one frame is only illustrative, and other values can also be used in actual applications.

[0069] In operation S220, batch information is set, and the batch information includes grid data of a preset processing quantity as one batch.

[0070] According to an embodiment of the present disclosure, the setting of the batch information, where the batch information includes grid data of a preset processing quantity as one batch, further includes: setting the priority processing level of each grid data, and setting the grid data with a higher priority processing level in the batch to be preferentially processed.

[0071] By preferentially setting the grid data with a higher processing level in the batch to be preferentially processed, the smoothness of the user experience is ensured visually.

[0072] In operation S230, when both the idle time is greater than a preset time threshold and the remaining memory space is greater than a preset memory threshold are satisfied, relevant processing is performed on the grid data of the preset processing quantity based on the batch to obtain a rendered image.

[0073] According to an embodiment of the present disclosure, when the idle time being greater than the preset time threshold is not satisfied and the remaining memory space being greater than the preset memory threshold is satisfied, the relevant processing in this frame is paused, and the idle time in subsequent frames is continuously obtained until relevant processing is performed on the grid data of the preset processing quantity based on the batch when both the idle time is greater than the preset time threshold and the remaining memory space is greater than the preset memory threshold are satisfied. Or, the preset time threshold is appropriately reduced based on the actual situation to make the preset time threshold less than the idle time.

[0074] According to an embodiment of the present disclosure, when the idle time is greater than a preset time threshold and the remaining memory space does not satisfy being greater than a preset memory threshold, the related processing in this frame is paused, and the remaining memory space in subsequent frames is continuously acquired until the idle time is greater than the preset time threshold and the remaining memory space is greater than the preset memory threshold, and then related processing is performed on the grid data of the preset processing quantity based on batches. Alternatively, the value of the preset processing quantity in the batch information is appropriately reduced, and at the same time, the preset memory threshold is appropriately reduced.

[0075] According to an embodiment of the present disclosure, the related processing includes a first processing, a second processing, and a third processing. Among them, the first processing includes converting the grid data into task data, the second processing includes converting the task data into a memory object, and the third processing includes rendering the memory object to obtain a rendered image.

[0076] It should be noted that there will be time overhead for the first processing, the second processing, and the third processing. That is to say, if within a frame, the idle time is less than the time overhead required for any one of the first processing, the second processing, and the third processing, and the idle time in this frame is simply not sufficient to support any time overhead, then there will be no relevant operations in this frame. Therefore, the preset time threshold should be set to be at least greater than the maximum time overhead among the first processing, the second processing, and the third processing, because it may be the first processing, the second processing, or the third processing that is executed during the idle time. Through actual operations, it can be known that the time overhead of the third processing is the largest, and thus the preset time threshold should be at least greater than the time overhead of the third processing. Of course, the actual situation is more complex, and the preset time threshold needs to be set according to the actual situation, at least in combination with the scale of the model file and the remaining memory space.

[0077] It should also be noted that during the process of performing related processing, data that needs to be stored will be generated, such as task data. And because task data is stored in batches, the preset memory threshold should be at least greater than the total data volume of the task data in one batch. That is to say, the preset processing quantity and the preset memory threshold are positively correlated. When setting the preset memory threshold, the impact of the set preset processing quantity on the preset memory threshold should be fully considered.

[0078] According to an embodiment of the present disclosure, by means of the method of performing rendering during idle time, the related processing performed for rendering does not preempt the processing resources in each frame, giving priority to ensuring the occupation of thread resources by other operations of the user, so that the related processing of rendering hinders other non-image-rendering operations of the user as little as possible, ensuring the smoothness of the user experience.

[0079] Figure 3 A flowchart schematically showing related processing of image rendering according to an embodiment of the present disclosure is shown.

[0080] As Figure 3 described, the related processing method of this embodiment includes operations S310 to S340.

[0081] In operation S310, for the grid data of the same batch, when both the idle time is greater than a preset time threshold and the remaining memory space is greater than a preset memory threshold, a first process is performed on a preset number of grid data to obtain a preset number of task data.

[0082] According to an embodiment of the present disclosure, after each task data obtained by the first process, the task data is cached into the remaining memory space one by one.

[0083] In operation S320, a preset number of task data is retrieved one by one.

[0084] It should be noted that the task data obtained by the first process is first stored in the remaining memory space. When the current frame ends and not all of the network data of the preset number of this batch has completed the first process, for the task data obtained by the completed first process, it has been saved and will not be lost. When starting the second process batch, it is retrieved one by one. Among them, the way of retrieving one by one can be first in first out or first in last out.

[0085] In operation S330, a second process is performed on a preset number of task data to obtain a preset number of memory objects.

[0086] According to an embodiment of the present disclosure, the memory object includes a memory object in Mesh format.

[0087] In operation S340, a third process is performed on a preset number of memory objects to obtain a preset number of rendered images.

[0088] For example, the task data can be tasks, and the memory object can be a Mesh object. Among them, tasks are converted from the grid data of the model file. Each task is a description of a certain part of the model corresponding to the model file, mainly including shape, width and height, color, etc., and then is converted into a Mesh memory object (geometry and material). Mesh objects can be generated in batches or in the reading order in memory; the timing of generating Mesh objects depends on the event sharding mechanism. Among them, geometry and material are converted from the first parameter and the second parameter.

[0089] For another example, based on RequestIdleCallback and Memory tools, when there is enough remaining memory space and idle time, tasks are cyclically extracted to generate Mesh objects; when the memory or time is insufficient, the execution logic enters a waiting state.

[0090] In an embodiment of the present disclosure, in one batch, after obtaining a preset number of preset tasks by first processing a preset number of mesh data, the task data is stored in the memory, and then retrieved and second processed. After the second processing, the obtained memory object is third processed to obtain a rendered image, and so on until all the mesh data of the entire model file is processed. Among them, when storing the task data obtained after the first processing in the remaining memory space, the task data is saved in a frame where a relevant process cannot be executed, so that the loss of task data does not occur, and the repeated execution of the first processing on the same mesh data is avoided, making full use of the occupied thread resources. Furthermore, the rendering efficiency is greatly improved.

[0091] Figure 4 A flowchart of a method for prioritizing processing levels according to an embodiment of the present disclosure is schematically shown.

[0092] Such as Figure 4 As shown, the relevant processing method of this embodiment includes operations S410 to S420.

[0093] In operation S410, classification is performed based on the first parameter and the second parameter for each number of meshes to obtain a classification result.

[0094] In operation S420, the priority processing level is set based on each classification result.

[0095] For example, when the scene of the model file is a 3D image of a computer room, based on the viewing and usage habits of the user, the first obtained image after relevant processing should be the arrangement of the computer cases in the computer room, or the building information in the computer room, such as pipes, wiring, etc. Therefore, such data is pre-rendered. Therefore, it is necessary to set the priority processing level of such data to high, and such mesh data is classified by the first parameter and the second parameter and marked as a category with a high priority processing level. The data with a low priority processing level can be data such as the internal structure of the computer case. Such mesh data is classified by the first parameter and the second parameter and marked as a category with a low priority processing level. After processing the mesh data with a higher priority processing level, such data is processed. Of course, the high or low of the above priority processing levels only represents a relative concept, and there are not only these two setting methods. It is not limited here.

[0096] In the embodiments of the present disclosure, by setting the priority processing level after classification based on the first parameter and the second parameter, important and more intuitive data is processed first, ensuring the user's sensory experience.

[0097] Figure 5 The figure schematically shows a schematic diagram of a full process of image rendering according to an embodiment of the present disclosure.

[0098] As Figure 5 shown, in practical applications, generally, the preset processing quantity is set to infinity or 1, and then two different schemes occur.

[0099] For the scheme with the preset processing quantity being infinity, that is to say, there is only one batch. In this batch, after all the mesh data is all processed for the first time and converted into task data tasks, then for the second processing, all the task data tasks are converted into memory objects Mesh, and finally, all the memory objects Mesh are processed for the third time to obtain the overall rendered image.

[0100] For the scheme with the preset processing quantity being 1, that is to say, only one mesh data is processed in each batch. After the first processing, 1 task data task is obtained. Then, this 1 task data task is processed for the second time to obtain a memory object Mesh, and then the memory object Mesh is processed for the third time to obtain a rendered image. Then, the above operations are repeated until the overall rendered image is obtained.

[0101] Of course, whether it is the scheme with the preset processing quantity being infinity or the scheme with the preset processing quantity being 1, it is necessary to store the task data task / tasks after the first processing into the remaining space of the memory.

[0102] It is worth mentioning that for the scheme with the preset processing quantity being set to 1, that is, only one task data task is stored in the remaining space of the memory each time. Thus, when the memory space is tight, it is better to choose to use this scheme. When actually testing the rendering speed, for the scheme with the preset processing quantity being infinity, the time overhead required is smaller, which means that the time overhead of the scheme for processing a large number of mesh data is smaller and the rendering speed is faster. Therefore, when there is a large amount of free memory space, it is preferred to choose the scheme with the preset processing quantity being infinity. Among them, it can be speculated that the reason why the scheme with the preset processing quantity being infinity has a faster processing speed may be that the browser kernel has optimized the batch processing of mesh data.

[0103] Based on the method of image rendering, the present disclosure also provides an image rendering device. The following will be combined with Figure 6 to describe this device in detail.

[0104] Figure 6 A structural block diagram of an image rendering device according to an embodiment of the present disclosure is schematically shown.

[0105] As Figure 6 shown, the image rendering device 600 of this embodiment includes an acquisition module 610, a setting module 620, and a processing module 630.

[0106] The acquisition module 610 is used to acquire model data, idle time, and remaining memory space. The model data includes a plurality of mesh data. The idle time includes the time when the processing resources are not occupied in each frame. The remaining memory space includes the data generated during the relevant processing. In one embodiment, the acquisition module 610 can be used to perform the operation S210 described above, which will not be elaborated here.

[0107] The setting module 620 is used to set batch information, and the batch information includes a preset number of mesh data as a batch. In one embodiment, the setting module 620 can be used to perform the operation S220 described above, which will not be elaborated here.

[0108] The module 630 is used to perform relevant processing on the preset number of mesh data based on batches when both the idle time is greater than a preset time threshold and the remaining memory space is greater than a preset memory threshold, so as to obtain a rendered image. In one embodiment, the module 630 can be used to perform the operation S230 described above, which will not be elaborated here.

[0109] According to an embodiment of the present disclosure, any of the acquisition module 610, the setting module 620, and the processing module 630 can be combined and implemented in one module, or any one of them can be split into multiple modules. Or, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 610, the setting module 620, and the processing module 630 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on a substrate, a system in a package, an application specific integrated circuit (ASIC), or can be implemented by any other reasonable way of integrating or packaging circuits, etc., in hardware or firmware, or implemented in any one of the three implementation manners of software, hardware, and firmware, or in any appropriate combination of several of them. Or, at least one of the acquisition module 610, the setting module 620, and the processing module 630 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding functions can be executed.

[0110] Figure 7 A block diagram of an electronic device suitable for implementing an image rendering method according to an embodiment of the present disclosure is schematically shown.

[0111] As Figure 7 shown, an electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage section 708 into a random access memory (RAM) 703. The processor 701 can include, for example, a general microprocessor (e.g., CPU), an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (e.g., an application specific integrated circuit (ASIC)), etc. The processor 701 can also include on-board memory for caching purposes. The processor 701 can include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0112] In the RAM 703, various programs and data required for the operation of the electronic device 700 are stored. The processor 701, ROM 702, and RAM 703 are connected to each other via a bus 704. The processor 701 performs various operations of the method flow according to an embodiment of the present disclosure by executing the programs in the ROM 702 and / or RAM 703. It should be noted that the program can also be stored in one or more memories other than the ROM 702 and RAM 703. The processor 701 can also perform various operations of the method flow according to an embodiment of the present disclosure by executing the programs stored in the one or more memories.

[0113] According to an embodiment of the present disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, and the input / output (I / O) interface 705 is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed so that a computer program read from it can be installed into the storage section 708 as needed.

[0114] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the embodiments; or may exist alone without being assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when the one or more programs are executed, the methods according to the embodiments of the present disclosure are implemented.

[0115] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 702 and / or RAM 703 described above and / or one or more memories other than the ROM 702 and RAM 703.

[0116] An embodiment of the present disclosure further includes a computer program product, which includes a computer program that contains program code for executing the method shown in the flowchart. When the computer program product runs in a computer system, the program code is used to cause the computer system to implement the item recommendation method provided by the embodiments of the present disclosure.

[0117] When the computer program is executed by the processor 701, the functions defined in the system / apparatus of the embodiments of the present disclosure are executed. According to an embodiment of the present disclosure, the systems, apparatuses, modules, units, etc. described above may be implemented by computer program modules.

[0118] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices and magnetic storage devices. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and be downloaded and installed through the communication part 709, and / or be installed from the removable medium 711. The program code included in the computer program can be transmitted by any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0119] In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the functions defined in the system of the embodiments of the present disclosure are executed. According to the embodiments of the present disclosure, the systems, devices, apparatuses, modules, units, etc. described above can be implemented by computer program modules.

[0120] According to the embodiments of the present disclosure, the program code for executing the computer program provided by the embodiments of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The programming languages include, but are not limited to, such as Java, C++, Python, the "C" language, or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., by connecting through the Internet using an Internet service provider).

[0121] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0122] Those skilled in the art can understand that the features recited in the various embodiments and / or claims of the present disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly recited in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features recited in the various embodiments and / or claims of the present disclosure can be combined and combined in various ways. All such combinations and / or combinations fall within the scope of the present disclosure.

[0123] The embodiments of the present disclosure have been described above. However, these embodiments are merely for illustrative purposes and not for limiting the scope of the present disclosure. Although the embodiments have been described separately above, this does not mean that the measures in each embodiment cannot be used advantageously in combination. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art can make various substitutions and modifications, and these substitutions and modifications should all fall within the scope of the present disclosure.

Claims

1. A method for image rendering, characterized in that, Including: Obtain a model file, idle time, and remaining memory space, where the model file includes multiple mesh data, and the idle time includes the time when processing resources are not occupied in each frame; Set batch information, where the batch information includes a preset number of mesh data as one batch; And When both the idle time is greater than a preset time threshold and the remaining memory space is greater than a preset memory threshold, perform relevant processing on a preset number of the mesh data based on the batch to obtain a rendered image; Wherein, the relevant processing includes a first processing, a second processing, and a third processing. Among them, the first processing includes converting the mesh data into task data, the second processing includes converting the task data into a memory object, and the third processing includes rendering the memory object to obtain a rendered image; Wherein, the performing relevant processing on a preset number of the mesh data based on the batch includes: for the mesh data of the same batch, when both the idle time is greater than a preset time threshold and the remaining memory space is greater than a preset memory threshold, perform the first processing on a preset number of the mesh data to obtain a preset number of task data; retrieve a preset number of task data one by one; perform the second processing on a preset number of the task data to obtain a preset number of memory objects; and perform the third processing on a preset number of the memory objects to obtain a preset number of rendered images.

2. The method according to claim 1, characterized in that The performing the first processing on a preset number of the mesh data to obtain a preset number of task data when both the idle time is greater than a preset time threshold and the remaining memory space is greater than a preset memory threshold includes: after the first processing, cache the task data to the remaining memory space one by one.

3. The method according to claim 1, wherein The method further includes: setting the processing priority level of each mesh data, and setting the mesh data into corresponding batches according to the processing priority level.

4. The method according to claim 3, characterized in that, The mesh data includes: a first parameter and a second parameter, where the first parameter includes the position parameter between each memory object, and the second parameter includes the texture mapping parameter of the memory object.

5. The method according to claim 4, wherein The setting the processing priority level of each mesh data and setting the mesh data into corresponding batches according to the processing priority level includes: Classify based on the first parameter and the second parameter of each mesh data to obtain a classification result; and Set the processing priority level based on each classification result.

6. An image rendering device, characterized in that, Including: An acquisition module, a setting module, and a processing module, Wherein, The acquisition module is used to obtain a model file, idle time, and remaining memory space, where the model file includes multiple mesh data, and the idle time includes the time when processing resources are not occupied in each frame; The setting module is used to set batch information, where the batch information includes a preset number of mesh data as one batch; and The processing module is used to when both the idle time is greater than a preset time threshold and the remaining memory space When it is greater than a preset memory threshold, perform relevant processing on the grid data of a preset number of processes based on batches to obtain a rendered image; wherein, the relevant processing includes a first process, a second process, and a third process. The first process includes converting the grid data into task data, the second process includes converting the task data into a memory object, and the third process includes rendering the memory object to obtain a rendered image; wherein, the performing relevant processing on the grid data of a preset number of processes based on batches includes: for the grid data of the same batch, when both the idle time is greater than a preset time threshold and the remaining memory space is greater than a preset memory threshold, perform the first process on the grid data of a preset number of processes to obtain the task data of a preset number of processes; retrieve the task data of a preset number of processes one by one; perform the second process on the task data of a preset number of processes to obtain the memory objects of a preset number of processes; and perform the third process on the memory objects of a preset number of processes to obtain the rendered images of a preset number of processes.

7. An electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors are caused to execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, having executable instructions stored thereon, which when executed by a processor cause the processor to execute the method according to any one of claims 1 to 5.

9. A computer program product, comprising a computer program, which when executed by a processor implements the method according to any one of claims 1 to 5.

Images