Digital twin image rendering method, device and system and medium

Abstract

The invention provides a digital twin image rendering method, device and system and a medium. The method comprises the following steps: creating a corresponding bounding box for each model object in a target three-dimensional scene; determining a unique color, except the configured background color, corresponding to each model object, and generating bounding box information corresponding to each model object, so as to render a bounding box of each model object under the current view angle in a frame buffer and obtain corresponding pixel information, rendering the bounding box of the plurality of model objects under the current view angle in the frame buffer area, and obtaining corresponding pixel information; according to the pixel information and the unique color corresponding to each model object, determining a model object needing to be rendered under the current view angle; the model objects needing to be rendered comprise model objects which are not shielded or not completely shielded; and rendering the model object to be rendered. According to the method, the rendering performance is improved, and the accuracy of the rendering result is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and more specifically, to a method, device, system and medium for rendering an image of a digital twin. Background Art

[0002] Digital twin technology is widely used in the field of warehousing and logistics, such as real-time monitoring of warehouses, or simulating warehousing and distribution plans through virtual models to optimize operational processes. These three-dimensional scenes are large and complex. When rendering and displaying them, most of the objects are non-transparent, and there are a large number of objects blocked behind these non-transparent objects. There are not many objects that can be seen from different perspectives. Therefore, only unobstructed objects can be rendered to improve rendering efficiency. Then, in the model drawing of large-scale three-dimensional scenes, it is first necessary to judge or query the object occlusion for each batch and each frame of the image, and then render the visible objects after the objects are occluded and culled from the current perspective.

[0003] However, the existing occlusion culling solutions have the problem of too high CPU-side calculation time in the process of occlusion culling in large scenes, while the GPU query-based solution has a large number of API access interactions between the CPU and GPU in large scenes, resulting in performance waste. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a method, device, system and medium for image rendering of a digital twin, which greatly improves the rendering performance by using a bounding box instead of a model object for occlusion rendering, utilizes the conversion of random colors and IDs to form a match between the occlusion result and the scene model, thereby improving the accuracy of the rendering result, and optimizes the occlusion calculation process by combining GPU and CPU processing to avoid the interaction time-consuming and calculation time-consuming problems of occlusion query on the GPU side and occlusion calculation on the CPU side.

[0005] In a first aspect, a method for rendering an image of a digital twin is provided, and the method may include:

[0006] For multiple model objects in the target three-dimensional scene, create a corresponding bounding box for each model object;

[0007] Determine a unique color corresponding to each model object except for the configured background color, and generate bounding box information corresponding to each model object; the bounding box information corresponding to each model object at least includes triangle facet data and color information of the bounding box corresponding to the model object;

[0008] Based on the bounding box information of each bounding box, the bounding boxes of the multiple model objects at the current viewing angle are rendered in the frame buffer, and corresponding pixel information is obtained;

[0009] Determine the model objects to be rendered under the current viewing angle according to the pixel information corresponding to each bounding box after rendering and the unique color corresponding to each model object; the model objects to be rendered include all model objects that are not blocked or not completely blocked;

[0010] Render the model object to be rendered.

[0011] In some possible implementations, for each of the multiple model objects in the target three-dimensional scene, creating a corresponding bounding box includes:

[0012] Acquire three-dimensional model data of multiple model objects in a target three-dimensional scene;

[0013] A bounding box of each model object is created according to the three-dimensional model data of each model object and a bounding box generation algorithm.

[0014] In some possible implementations, determining a unique color corresponding to each model object other than a configured background color includes:

[0015] Assigning unique model codes to multiple model objects of the target three-dimensional scene;

[0016] According to the model code of each model object, the unique color of the bounding box corresponding to each model object is determined; the model code has a unique corresponding color and is different from the background color of the target three-dimensional scene.

[0017] In some possible implementations, the pixel information includes a pixel value;

[0018] The step of determining the model object to be rendered at the current viewing angle according to the pixel information corresponding to each bounding box after rendering and the unique color corresponding to each model object includes:

[0019] Detect pixel values ​​in the cross-sectional image corresponding to the target three-dimensional scene at the current viewing angle;

[0020] If the detected pixel value is a pixel value of a non-background color, then according to the detected pixel value and the unique color corresponding to each model object, the model object corresponding to the detected pixel value is determined and marking information is generated accordingly to mark the model object as requiring rendering;

[0021] After the pixel value detection in the cross-sectional image is completed, the tag information of all model objects is traversed to determine the model objects to be rendered at the current viewing angle.

[0022] In some possible implementations, if the detected pixel value is a pixel value of a non-background color, determining the model object corresponding to the detected pixel value according to the detected pixel value and the unique color corresponding to each model object and generating marking information accordingly to mark the model object to be rendered, including:

[0023] If the detected pixel value is a pixel value of a non-background color, a bounding box of a pixel block corresponding to each bounding box is generated, and a ratio of a diagonal line of the bounding box of the pixel block corresponding to each bounding box to a diagonal line of a texture rectangle corresponding to the frame buffer is determined;

[0024] If the determined ratio is not less than the first ratio threshold, the model object corresponding to the detected pixel value is determined according to the detected pixel value and the unique color corresponding to each model object, and marking information is generated accordingly to mark the model object for rendering.

[0025] In some possible implementations, if the detected pixel value is a pixel value of a non-background color, determining the model object corresponding to the detected pixel value according to the detected pixel value and the unique color corresponding to each model object and generating marking information accordingly to mark the model object to be rendered includes:

[0026] If the detected pixel value is a pixel value of a non-background color, then based on a preset convex hull algorithm, the polygonal convex hull contour and the corresponding polygonal area of ​​the pixel block of the bounding box corresponding to the pixel value are obtained;

[0027] Determine the ratio of the area of ​​the polygon to the area of ​​the texture rectangle corresponding to the frame buffer;

[0028] If the determined ratio is not less than the second ratio threshold, the model object corresponding to the detected pixel value is determined according to the detected pixel value and the unique color corresponding to each model object, and marking information is generated accordingly to mark the model object for rendering.

[0029] In a second aspect, a digital twin image rendering device is provided, which may include:

[0030] A creation unit, used for creating a corresponding bounding box for each of the multiple model objects in the target three-dimensional scene;

[0031] A determination unit, used for determining a unique color corresponding to each model object except for a configured background color;

[0032] A generating unit, configured to generate bounding box information corresponding to each model object; the bounding box information corresponding to each model object at least includes triangular facet data and color information of the bounding box corresponding to the model object;

[0033] A rendering unit, configured to render the bounding boxes of the plurality of model objects at the current viewing angle in a frame buffer based on the bounding box information of each bounding box, and obtain corresponding pixel information;

[0034] The determining unit is used to determine the model objects to be rendered under the current viewing angle according to the pixel information corresponding to each bounding box after rendering and the unique color corresponding to each model object; the model objects to be rendered include all model objects that are not blocked or not completely blocked;

[0035] The rendering unit is also used for the model object to be rendered.

[0036] In some possible implementations, a unit is created, specifically for:

[0037] Acquire three-dimensional model data of multiple model objects in a target three-dimensional scene;

[0038] A bounding box of each model object is created according to the three-dimensional model data of each model object and a bounding box generation algorithm.

[0039] In a third aspect, a digital twin image rendering system is provided, characterized in that the image rendering system comprises: a central processing unit CPU and a graphics processing unit GPU; wherein,

[0040] The CPU creates a corresponding bounding box for each of the multiple model objects in the target three-dimensional scene; and determines a unique color corresponding to each model object except for the configured background color, and generates a bounding box rendering command, wherein the bounding box rendering command includes bounding box information corresponding to each model object, and sends the bounding box rendering command to the GPU; the bounding box information corresponding to each model object includes at least triangle facet data and color information of the bounding box corresponding to the model object;

[0041] Based on the bounding box rendering command, the GPU renders the bounding boxes of the multiple model objects at the current viewing angle in the frame buffer and obtains corresponding pixel information;

[0042] The CPU obtains pixel information corresponding to each bounding box after rendering from the frame buffer, and determines the model object to be rendered under the current viewing angle according to the pixel information corresponding to each bounding box after rendering and the unique color corresponding to each model object, and generates and sends a rendering command of the model object to be rendered to the GPU; the model object display information includes all unobstructed model objects;

[0043] The GPU receives a rendering command for the model object to be rendered, and renders the model object to be rendered.

[0044] In the digital twin-based image rendering method provided by the present application, for multiple model objects in the target three-dimensional scene, a corresponding bounding box is created for each model object to segment and enclose the multiple model objects respectively; the unique color corresponding to each model object except the configured background color is determined, and the bounding box information corresponding to each model object is generated; the bounding box information corresponding to each model object at least includes the triangle facet data and color information of the bounding box corresponding to the model object; based on the bounding box information of each bounding box, the bounding boxes of multiple model objects under the current viewing angle are rendered in the frame buffer, and the corresponding pixel information is obtained; according to the pixel information corresponding to each bounding box after rendering and the unique color corresponding to each model object, the model object to be rendered under the current viewing angle is determined; the model object to be rendered includes all model objects that are not blocked or not completely blocked; the model object to be rendered is rendered. This method uses bounding boxes instead of model objects for occlusion rendering to greatly improve the rendering performance, and uses the conversion of random colors and IDs to form a match between the occlusion result and the scene model, thereby improving the accuracy of the rendering result. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A schematic diagram of a flow chart of a method for rendering an image of a digital twin provided in an embodiment of the present application;

[0047] Figure 2 A schematic diagram of the structure of a digital twin image rendering device provided in an embodiment of the present application;

[0048] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application. Unless otherwise defined, the technical terms or scientific terms used in this application should be the common meanings understood by people with ordinary skills in the field to which the present invention belongs. The "first", "second" and similar words used in this application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Including" or "including" and other similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connected", "coupled" or "connected" and other similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0050] The digital twin image rendering method provided in the embodiment of the present application can be applied to the digital twin image rendering system composed of a central processing unit (CPU) and a graphics processing unit (GPU). In the scene organization, a corresponding bounding box is generated for each model object in the target three-dimensional scene, and the bounding box is used instead of the model object in the rendering occlusion process, and the bounding box is assigned an RGB color that matches the 32-bit integer model encoding ID. A one-to-one correspondence is achieved using 32-bit integers and RGB color values, and the rendering is finally submitted through the bounding box rendering process, the frame buffer frameBuf processing process, and the model command indicator CMD generation process. Specifically:

[0051] The CPU creates a corresponding bounding box for each of the multiple model objects in the target three-dimensional scene; and determines a unique color corresponding to each model object except for the configured background color, and generates a bounding box rendering command, the bounding box rendering command including bounding box information corresponding to each model object, and sends it to the GPU; the bounding box information corresponding to each model object includes at least the triangle facet data and color information of the bounding box corresponding to the model object;

[0052] Based on the bounding box rendering command, the GPU renders the bounding boxes of multiple model objects at the current viewing angle in the frame buffer and obtains the corresponding pixel information;

[0053] After the GPU finishes rendering, the CPU obtains the pixel information corresponding to each bounding box after rendering from the frame buffer, and determines the model object to be rendered under the current viewing angle according to the pixel information corresponding to each bounding box after rendering and the unique color corresponding to each model object, and generates and sends the rendering command of the model object to be rendered to the GPU; the model objects to be rendered include all model objects that are not blocked or not completely blocked;

[0054] The GPU receives a rendering command for a model object to be rendered, and renders the model object to be rendered.

[0055] In this way, the occlusion calculation process is optimized by combining GPU and CPU processing, avoiding the time-consuming interaction and calculation problems of GPU-side occlusion query and CPU-side occlusion calculation, and improving the speed of occlusion calculation.

[0056] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.

[0057] Figure 1 A flowchart of a method for rendering an image of a digital twin provided in an embodiment of the present application is shown below. Figure 1 As shown, the method may include:

[0058] Step S110: for multiple model objects in the target three-dimensional scene, create a corresponding bounding box for each model object.

[0059] In a specific implementation, three-dimensional model data of multiple model objects in a target three-dimensional scene are obtained; and a bounding box of each model object is created according to the three-dimensional model data of each model object and a bounding box generation algorithm.

[0060] The bounding box generation algorithm may adopt the existing bounding box generation algorithm, and the present application does not limit it here. The created bounding box may be a bounding sphere (Sphere), an axis-aligned bounding box (Axis-Aligned

[0061] One or more of convex hull bounding boxes (AABB), oriented bounding boxes (OBB), fixed direction bounding boxes (FDH), or convex hull bounding boxes. For example, if the number of sides of the bounding box of the model object is greater than a preset value, a bounding sphere, AABB, or OBB can be created for the model object; if the number of sides of the bounding box of the model object is not greater than a preset value, a FDH or convex hull bounding box can be created for the model object. It is understandable that the same bounding box can also be created for all model objects, and this application does not limit this.

[0062] It should be noted that in the above bounding boxes, when the occlusion accuracy requirement is not very high and performance is required, the AABB bounding box is preferred. The AABB bounding box has the least number of triangles, and the algorithm is simple, which puts very little pressure on the graphics card rendering. If you want to consider both performance and occlusion accuracy, you can choose the FDH and convex hull bounding box strategies to minimize the false occlusion problems caused by AABB, OBB, and Sphere spherical bounding boxes.

[0063] In some embodiments, when AABB bounding boxes are created for all model objects and high occlusion accuracy is required, a triangular mesh simplification algorithm can be used to process the bounding boxes to obtain processed bounding boxes to optimize the false occlusion problem caused by the AABB bounding boxes. Surface mesh simplification is a process of reducing the number of faces used in the surface mesh while maintaining the overall shape, volume and boundary as much as possible, using a method called edge collapse to simplify any oriented manifold surface.

[0064] Step S120: determine a unique color corresponding to each model object except the configured background color, and generate bounding box information corresponding to each model object.

[0065] The bounding box information corresponding to each model object at least includes the triangle face data and color information of the bounding box corresponding to the model object.

[0066] In a specific implementation, the background color of the frame buffer frameBuffer is preconfigured, such as white.

[0067] Assigning unique model coding IDs to multiple model objects of the target three-dimensional scene. For example, the coding ID may be a 32-bit integer.

[0068] According to the model code of each model object, the unique color of the bounding box corresponding to each model object is determined, that is, the model code ID of each model object has a unique corresponding color, and it is different from the background color of the target three-dimensional scene. In other words, the color of each bounding box is different and non-repeating, and the assigned color does not include the configured background color.

[0069] After that, an indirect drawing CMD command stream CommandBuf of the model bounding box is generated, and the bounding box information after the assigned color is sent to the GPU.

[0070] Step S130: Based on the bounding box information of each bounding box, the bounding boxes of the multiple model objects at the current viewing angle are rendered in the frame buffer and corresponding pixel information is obtained.

[0071] The pixel information may include the pixel position of the bounding box, the pixel value and the number of pixels in the bounding box, etc. The specific information may be determined according to actual needs and is not limited here.

[0072] Specifically, based on the drawing of glDrawElementsInstanced, bounding boxes of different colors will be rendered in the frame buffer frameBuffer, and the color result after the occlusion relationship will be generated, that is, the pixel information of the rendered bounding box, and it will be sent to the CPU.

[0073] Step S140: Determine the model object to be rendered at the current viewing angle according to the pixel information corresponding to each bounding box after rendering and the unique color corresponding to each model object.

[0074] The model objects to be rendered include all model objects that are not blocked or not completely blocked.

[0075] In a specific implementation, the pixel values ​​in the cross-sectional image corresponding to the target three-dimensional scene at the current viewing angle are detected. The cross-sectional image may include the pixel blocks formed by the bounding box corresponding to each model object at the current viewing angle, that is, the pixel blocks corresponding to the projection view at the current viewing angle; if the detected pixel value is a pixel value of a non-background color, the model object corresponding to the detected pixel value is determined based on the detected pixel value and the unique color corresponding to each model object, and the marking information is generated accordingly to mark the model object to be rendered; after the pixel value detection in the cross-sectional image is completed, the marking information of all model objects is traversed to determine the model object to be rendered at the current viewing angle.

[0076] In some embodiments, the generated rendering color buffer is processed in parallel to retrieve pixel blocks: memory comparison is performed according to the memory blocks of each row of pixels. If the entire row is white, the pixel data of the current row is discarded, that is, the next step of pixel matching is not performed. After memory comparison, pixel matching is performed to filter the color in a row of data, and match the corresponding model objects in the original scene data according to the color, so as to determine which models are occluded and which models are not occluded. The unoccluded models are generated with indirect drawing commands commandBuffer and submitted to the graphics card for actual model rendering.

[0077] In some embodiments, if the detected pixel value is a pixel value of a non-background color, then according to the detected pixel value and the unique color corresponding to each model object, the model object corresponding to the detected pixel value is determined and marking information is generated accordingly to mark the model object as requiring rendering, which may specifically include:

[0078] If the detected pixel value is a pixel value of a non-background color, a bounding box of a pixel block corresponding to each bounding box is generated, and a ratio of a diagonal line of the bounding box of the pixel block corresponding to each bounding box to a diagonal line of a texture rectangle corresponding to the frame buffer is determined;

[0079] If the determined ratio is not less than the first ratio threshold, the model object corresponding to the detected pixel value is determined according to the detected pixel value and the unique color corresponding to each model object, and marking information is generated accordingly to mark the model object for rendering.

[0080] If the determined ratio is less than the first ratio threshold, pixel matching is not performed on the pixel block corresponding to the bounding box of the model object, that is, the step of determining the model object corresponding to the detected pixel value is not performed.

[0081] It should be noted that the method for determining the first ratio threshold may include:

[0082] Method 1: The first ratio threshold can be determined by a heuristic algorithm, namely a genetic algorithm. Specifically:

[0083] Initialize the population: Select an appropriate population size N, for example, N = 50, and assume that the threshold value range is [0.1, 0.9]. Generate N random thresholds as the initial population. For example, if N = 50, generate 50 random numbers in the interval [0.1, 0.9] as the initial population.

[0084] Define the fitness function: Assuming that response time is selected as the main indicator, the fitness function can be defined as: Where x is the threshold and ResponseTime(x) represents the response time under the given threshold x.

[0085] Roulette selection: According to the fitness value of each individual, the parent individuals are selected through roulette to participate in the subsequent genetic operation. The individuals with higher fitness values ​​have a greater probability of being selected.

[0086] Crossover operator: Single-point crossover: For the selected parent individuals, a crossover point is randomly selected and some genes after the point are exchanged to generate new offspring individuals. For example, if the selected thresholds are 0.3 and 0.6 and the crossover point is 0.4, the new offspring individuals may be a combination of 0.3 and 0.6, such as 0.3 and 0.45, or 0.45 and 0.6.

[0087] Mutation operator: Random mutation: Randomly mutate the genes of the offspring individuals with a certain probability pm (for example, pm = 0.1). The mutation can be a random increase or decrease of a small value. For example, if the probability of mutation is 10%, then for each offspring individual, there is a 10% probability of mutation.

[0088] Calculate the fitness of the new population: Use the fitness function to evaluate each individual in the new population.

[0089] Replacement rule: retain the top m individuals (e.g. m=5) with the highest fitness to the next generation population to ensure that excellent individuals are not lost. After that, the new population replaces the old population.

[0090] Iteration number: Set the maximum number of iterations G (for example, G=100), and stop the algorithm when this number is reached.

[0091] Fitness convergence: If the average fitness of the population does not increase significantly within several consecutive generations (such as 10 generations), the algorithm is considered to have converged and can be stopped.

[0092] Best individual: Output the individual with the highest fitness in the current population as the optimal solution.

[0093] Method 2: Dynamic adjustment: This threshold can be adjusted dynamically by monitoring system performance. For example, when the system load is high, the threshold can be increased to reduce the number of pixel matches, thereby improving rendering efficiency.

[0094] In other embodiments, before determining the model object corresponding to the detected pixel value, the polygonal convex hull contour and the corresponding polygonal area of ​​the pixel block corresponding to the bounding box of any model object may be obtained based on a preset convex hull algorithm;

[0095] Get the ratio of the polygon area to the texture rectangle area corresponding to the frame buffer;

[0096] If the determined ratio is not less than the second ratio threshold, the model object corresponding to the detected pixel value is determined based on the detected pixel value and the unique color corresponding to each model object; if the determined ratio is less than the second ratio threshold, pixel matching is not performed on the pixel block corresponding to the bounding box of the model object, that is, the step of determining the model object corresponding to the detected pixel value is not performed.

[0097] It should be noted that the method for determining the second ratio threshold may include:

[0098] Method 1: The second ratio threshold may be determined by a heuristic algorithm, namely a genetic algorithm, which is similar to the first ratio threshold mentioned above and will not be elaborated in detail in this application.

[0099] Method 2: Dynamic adjustment: This threshold can be adjusted dynamically by monitoring system performance. For example, when the system load is high, the threshold can be increased to reduce the number of pixel matches, thereby improving rendering efficiency.

[0100] Step S150: Render the model object to be rendered.

[0101] This method combines the computing performance of the GPU and the logic processing of the CPU to improve the speed of occlusion calculation. It makes full use of the new feature of OpenGL graphics card indirect rendering technology. In the first process rendering, it does not need to perform lighting calculation and only retains the color value, which improves the efficiency of occlusion rendering results.

[0102] Corresponding to the above method, the embodiment of the present application also provides an image rendering device of a digital twin, such as Figure 2 As shown, the device comprises:

[0103] A creating unit 210, for creating a corresponding bounding box for each of the multiple model objects in the target three-dimensional scene;

[0104] A determination unit 220, configured to determine a unique color corresponding to each model object except for a configured background color;

[0105] A generating unit 230 is used to generate bounding box information corresponding to each model object; the bounding box information corresponding to each model object at least includes triangle facet data and color information of the bounding box corresponding to the model object;

[0106] A rendering unit 240, configured to render the bounding boxes of the plurality of model objects at the current viewing angle in a frame buffer based on the bounding box information of each bounding box, and obtain corresponding pixel information;

[0107] The determination unit 220 is used to determine the model objects to be rendered under the current viewing angle according to the pixel information corresponding to each bounding box after rendering and the unique color corresponding to each model object; the model objects to be rendered include all model objects that are not blocked or not completely blocked;

[0108] The rendering unit 240 is also used for the model object to be rendered.

[0109] The functions of each functional unit of the digital twin image rendering device provided in the above embodiments of the present application can be realized through the above method steps. Therefore, the specific working process and beneficial effects of each unit in the digital twin image rendering device provided in the embodiments of the present application will not be repeated here.

[0110] The present application also provides an electronic device, such as Figure 3 As shown, it includes a processor 310 , a communication interface 320 , a memory 330 and a communication bus 340 , wherein the processor 310 , the communication interface 320 , and the memory 330 communicate with each other via the communication bus 340 .

[0111] Memory 330, for storing computer programs;

[0112] The processor 310 is used to execute the program stored in the memory 330 to implement the following steps:

[0113] For multiple model objects in the target three-dimensional scene, create a corresponding bounding box for each model object;

[0114] Determine a unique color corresponding to each model object except for the configured background color, and generate bounding box information corresponding to each model object; the bounding box information corresponding to each model object at least includes triangle facet data and color information of the bounding box corresponding to the model object;

[0115] Based on the bounding box information of each bounding box, the bounding boxes of the multiple model objects at the current viewing angle are rendered in the frame buffer, and corresponding pixel information is obtained;

[0116] Determine the model objects to be rendered under the current viewing angle according to the pixel information corresponding to each bounding box after rendering and the unique color corresponding to each model object; the model objects to be rendered include all model objects that are not blocked or not completely blocked;

[0117] Render the model object to be rendered.

[0118] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0119] The communication interface is used for communication between the above electronic device and other devices.

[0120] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0121] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0122] The implementation methods and beneficial effects of the components of the electronic device in the above embodiments to solve the problems can be seen in Figure 1 The various steps in the illustrated embodiment are implemented, therefore, the specific working process and beneficial effects of the electronic device provided by the embodiment of the present application are not repeated here.

[0123] In another embodiment provided in the present application, a computer-readable storage medium is also provided, which stores instructions. When the computer-readable storage medium is run on a computer, the computer executes the image rendering method of the digital twin described in any of the above embodiments.

[0124] In another embodiment provided in the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute the image rendering method of the digital twin described in any of the above embodiments.

[0125] Those skilled in the art will appreciate that the embodiments in the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt a complete hardware embodiment, a complete software embodiment, or a form of an embodiment combining software and hardware. Moreover, the present application may adopt a form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0127] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0129] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0130] Obviously, those skilled in the art can make various changes and modifications to the embodiments in the present application without departing from the spirit and scope of the embodiments in the present application. Thus, if these modifications and variations of the embodiments in the present application fall within the scope of the claims and their equivalents in the embodiments of the present application, the embodiments of the present application are also intended to include these modifications and variations.

Claims

1. A digital twin image rendering method, characterized in that: The method comprises: For multiple model objects in the target three-dimensional scene, create a corresponding bounding box for each model object; Determine a unique color corresponding to each model object except for the configured background color, and generate bounding box information corresponding to each model object; the bounding box information corresponding to each model object at least includes triangle facet data and color information of the bounding box corresponding to the model object; Based on the bounding box information of each bounding box, the bounding boxes of the multiple model objects at the current viewing angle are rendered in the frame buffer and corresponding pixel information is obtained; Determine the model objects to be rendered under the current viewing angle according to the pixel information corresponding to each bounding box after rendering and the unique color corresponding to each model object; the model objects to be rendered include all model objects that are not blocked or not completely blocked; Render the model object to be rendered.

2. The method according to claim 1, characterized in that The step of creating a corresponding bounding box for each of the multiple model objects in the target three-dimensional scene comprises: Acquire three-dimensional model data of multiple model objects in a target three-dimensional scene; A bounding box of each model object is created according to the three-dimensional model data of each model object and a bounding box generation algorithm.

3. The method according to claim 1, characterized in that The determining of a unique color corresponding to each model object except for a configured background color includes: Assigning unique model codes to multiple model objects of the target three-dimensional scene; According to the model code of each model object, the unique color of the bounding box corresponding to each model object is determined; the model code has a unique corresponding color and is different from the background color of the target three-dimensional scene.

4. The method according to claim 3, characterized in that The pixel information includes pixel values; The step of determining the model object to be rendered at the current viewing angle according to the pixel information corresponding to each bounding box after rendering and the unique color corresponding to each model object includes: Detect pixel values ​​in the cross-sectional image corresponding to the target three-dimensional scene at the current viewing angle; If the detected pixel value is a pixel value of a non-background color, then according to the detected pixel value and the unique color corresponding to each model object, the model object corresponding to the detected pixel value is determined and marking information is generated accordingly to mark the model object as requiring rendering; After the pixel value detection in the cross-sectional image is completed, the tag information of all model objects is traversed to determine the model objects to be rendered at the current viewing angle.

5. The method according to claim 4, characterized in that If the detected pixel value is a pixel value of a non-background color, the model object corresponding to the detected pixel value is determined according to the detected pixel value and the unique color corresponding to each model object, and marking information is generated accordingly to mark the model object as requiring rendering, including: If the detected pixel value is a pixel value of a non-background color, a bounding box of a pixel block corresponding to each bounding box is generated, and a ratio of a diagonal line of the bounding box of the pixel block corresponding to each bounding box to a diagonal line of a texture rectangle corresponding to the frame buffer is determined; If the determined ratio is not less than the first ratio threshold, the model object corresponding to the detected pixel value is determined according to the detected pixel value and the unique color corresponding to each model object, and marking information is generated accordingly to mark the model object for rendering.

6. The method according to claim 4, characterized in that If the detected pixel value is a pixel value of a non-background color, then according to the detected pixel value and the unique color corresponding to each model object, the model object corresponding to the detected pixel value is determined and marking information is generated accordingly to mark that the model object needs to be rendered, including: If the detected pixel value is a pixel value of a non-background color, then based on a preset convex hull algorithm, the polygonal convex hull contour and the corresponding polygonal area of ​​the pixel block of the bounding box corresponding to the pixel value are obtained; Determine the ratio of the area of ​​the polygon to the area of ​​the texture rectangle corresponding to the frame buffer; If the determined ratio is not less than the second ratio threshold, the model object corresponding to the detected pixel value is determined according to the detected pixel value and the unique color corresponding to each model object, and marking information is generated accordingly to mark the model object for rendering.

7. A digital twin image rendering device, characterized in that: The device comprises: A creation unit, used for creating a corresponding bounding box for each of the multiple model objects in the target three-dimensional scene; A determination unit, used for determining a unique color corresponding to each model object except for a configured background color; A generating unit, configured to generate bounding box information corresponding to each model object; the bounding box information corresponding to each model object at least includes triangular facet data and color information of the bounding box corresponding to the model object; A rendering unit, configured to render the bounding boxes of the plurality of model objects at the current viewing angle in a frame buffer based on the bounding box information of each bounding box, and obtain corresponding pixel information; The determining unit is used to determine the model objects to be rendered under the current viewing angle according to the pixel information corresponding to each bounding box after rendering and the unique color corresponding to each model object; the model objects to be rendered include all model objects that are not blocked or not completely blocked; The rendering unit is also used for the model object to be rendered.

8. The device according to claim 7, characterized in that Create units specifically for: Acquire three-dimensional model data of multiple model objects in a target three-dimensional scene; A bounding box of each model object is created according to the three-dimensional model data of each model object and a bounding box generation algorithm.

9. A digital twin image rendering system, characterized in that: The image rendering system includes: a central processing unit (CPU) and a graphics processing unit (GPU); wherein, The CPU creates a corresponding bounding box for each of the multiple model objects in the target three-dimensional scene; and determines a unique color corresponding to each model object except for the configured background color, and generates a bounding box rendering command, wherein the bounding box rendering command includes bounding box information corresponding to each model object, and sends the bounding box rendering command to the GPU; the bounding box information corresponding to each model object includes at least triangle facet data and color information of the bounding box corresponding to the model object; Based on the bounding box rendering command, the GPU renders the bounding boxes of the multiple model objects at the current viewing angle in the frame buffer and obtains corresponding pixel information; The CPU obtains pixel information corresponding to each bounding box after rendering from the frame buffer, and determines the model object to be rendered under the current viewing angle according to the pixel information corresponding to each bounding box after rendering and the unique color corresponding to each model object, and generates and sends a rendering command for the model object to be rendered to the GPU; the model objects to be rendered include all model objects that are not blocked or not completely blocked; The GPU receives a rendering command for the model object to be rendered, and renders the model object to be rendered.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images