Graphic processing engine development method and system based on virtual simulation

Abstract

The invention discloses a graphic processing engine development method and system based on virtual simulation, and relates to the technical field of computer graphics and virtual reality, and the method comprises the steps: collecting three-dimensional point cloud data, carrying out the convolution operation through a convolutional neural network algorithm, extracting a three-dimensional space feature matrix, and constructing a graphic recognition model. Inputting original data of a to-be-processed graph into the graph recognition model, outputting a graph parameter set, matching a virtual scene model in a preset scene library, and adjusting parameters of the model; establishing a physical coordinate system according to the adjusted model, converting a to-be-processed graph into grid units through a grid division algorithm, and calculating displacement data of the grid units; and reading displacement data, generating a graphic processing instruction sequence and sending the graphic processing instruction sequence to a graphic processor, and outputting an imaging result by superposing texture information and a ray tracing effect on the grid units. According to the method, a complete processing chain from data acquisition to rendering output is established, and the automation level and accuracy of graphic processing are improved.

Description

Technical Field

[0001] The present invention relates to the technical fields of computer graphics and virtual reality, and particularly to a method and system for developing a graphics processing engine based on virtual simulation. Background Art

[0002] Traditional graphics processing engines mainly use geometric modeling-based methods to construct 3D scenes and complete image generation through the rasterization rendering pipeline. In recent years, with the rise of deep learning technology, 3D feature extraction and scene understanding methods based on neural networks have gradually shown superiority. At the same time, the introduction of physical simulation technology enables virtual scenes to have more realistic dynamic effects, and the development of real-time ray tracing technology provides a new solution for high-quality rendering. Currently, mainstream graphics processing engines in the industry, such as Unity3D and Unreal Engine, have begun to integrate technologies such as deep learning, physical simulation, and ray tracing, but there is still much room for optimization in system architecture and processing flow.

[0003] However, existing graphics processing engines still face many challenges when dealing with complex scenes: First, traditional feature extraction methods are difficult to accurately capture the deep semantic information in 3D space, resulting in insufficient accuracy of scene understanding and reconstruction; Second, during the virtual scene matching process, due to the lack of an effective parameter optimization mechanism, problems such as model deformation and detail loss often occur; Third, the physical simulation calculation efficiency is low and it is difficult to meet the requirements of real-time interaction; Finally, texture mapping and lighting calculation in the rendering process often use simplified models, resulting in insufficient image realism. These technical bottlenecks severely restrict the further development of virtual simulation applications.

[0004] In view of the above problems, the present invention proposes a method and system for developing a graphics processing engine based on virtual simulation. This method improves the 3D feature extraction ability through deep learning technology, uses improved scene matching and parameter optimization algorithms to improve the model reconstruction quality, introduces an efficient physical simulation calculation method, and combines advanced rendering technology to enhance the image realism, thereby effectively solving the technical problems of existing graphics processing engines in aspects such as feature extraction accuracy, scene reconstruction quality, physical simulation efficiency, and rendering effect. Summary of the Invention

[0005] In view of the problems of insufficient 3D feature extraction accuracy, poor virtual scene matching quality, low physical simulation calculation efficiency, and insufficient rendering realism of existing graphics processing engines, the present invention is proposed.

[0006] Therefore, the problem to be solved by the present invention is how to improve the modeling accuracy and rendering realism of virtual scenes through deep learning and physical simulation technologies.

[0007] To solve the above technical problems, the present invention provides the following technical solutions:

[0008] In a first aspect, an embodiment of the present invention provides a method for developing a graphics processing engine based on virtual simulation, which includes collecting three-dimensional point cloud data output by a depth camera of a training scene, performing convolution operations on the three-dimensional point cloud data through a convolutional neural network algorithm, and extracting a three-dimensional spatial feature matrix; constructing a graphics recognition model according to the three-dimensional spatial feature matrix, and inputting the original data of the graphics to be processed into the graphics recognition model to output a graphics parameter set; based on the graphics parameter set, matching a virtual scene model in a preset scene library, and adjusting the parameters of the virtual scene model; establishing a physical coordinate system according to the adjusted virtual scene model, converting the graphics to be processed into grid cells through a grid division algorithm, and calculating the displacement data of the grid cells based on physical kinematic formulas; reading the displacement data, generating and sending a graphics processing instruction sequence to a graphics processor, and outputting an imaging result by superimposing texture information and ray tracing effects on the grid cells.

[0009] As a preferred solution of the method for developing a graphics processing engine based on virtual simulation according to the present invention, wherein: the method for generating the imaging result is to sort the displacement data by timestamp using a time-series data stream processing method and enter a graphics processing buffer, where the displacement data in the graphics processing buffer is mapped to a graphics rendering pipeline after interpolation smoothing processing to generate a graphics processing instruction sequence; based on the graphics processing instruction sequence, assign texture coordinates to the grid cells according to the physical coordinate system, and use an anisotropic filtering method to improve the texture sampling quality; at the same time, enhance the surface details through normal maps and displacement maps, and apply a preset material template to the grid cells to perform bidirectional reflectance distribution function calculations to simulate the light scattering characteristics of the material surface; use a hierarchical bounding box structure to accelerate the ray-scene intersection operation, calculate the combined effects of direct illumination, indirect illumination, and global illumination, and add ambient occlusion and soft shadow effects to achieve the effect of ray tracing; based on the effect of ray tracing, output the rendering result to a display buffer through tone mapping and anti-aliasing processing of a post-processing pipeline to generate an imaging result.

[0010] As a preferred solution of the method for developing a graphics processing engine based on virtual simulation according to the present invention, wherein: the method for forming the displacement data is as follows: based on the right-handed Cartesian coordinate system, a physical coordinate system is established according to the adjusted virtual scene model; an adaptive quadrilateral mesh division algorithm is used to perform mesh division on the graph to be processed, and the mesh density is dynamically adjusted according to the local curvature and boundary characteristics of the graph to generate a set of mesh elements; based on the set of mesh elements, with Newton's second law as the basis of the physical kinematic formula and combined with Lagrangian mechanics equations, forces are applied to the mesh elements; at the same time, the Runge-Kutta integration algorithm is used to solve the differential equations to calculate the velocity vector and displacement vector of the mesh elements within the discrete time step; according to the displacement vector, the spatial position of the mesh elements in the physical coordinate system is updated to form displacement data.

[0011] As a preferred solution of the method for developing a graphics processing engine based on virtual simulation according to the present invention, wherein: the corresponding virtual scene model is matched in the preset scene library according to the set of graph parameters, and the parameters of the virtual scene model are adjusted, including: constructing a hierarchical scene index structure, and performing feature vector normalization processing on the set of graph parameters to generate a normalized feature vector; based on the fast retrieval algorithm of the KD tree, the set of graph parameters is mapped to the category space of the scene library, and a feature vector index is constructed within the category space; the similarity between the normalized feature vector and the feature vector index is calculated to determine the matched virtual scene model; the geometric parameters in the set of graph parameters are extracted, and geometric transformation is performed on the virtual scene model according to the principle of affine transformation, where the geometric transformation includes size scaling, rotation and translation, and projection mapping; the transformed virtual scene model is adjusted through the bilinear interpolation algorithm, and at the same time, the material parameters, lighting parameters and shadow parameters of the virtual scene model are optimized and configured.

[0012] As a preferred solution of the method for developing a graphics processing engine based on virtual simulation according to the present invention, wherein: the specific formula for the similarity is as follows:

[0013]

[0014] where S is the similarity score, V1 is the normalized feature vector to be matched, V2 is the feature vector index in the scene library, v 1i and v 2i are the i-th components in V1 and V2 respectively, σ is the bandwidth parameter of the Gaussian kernel function, λ is the attenuation coefficient, w j is the weight coefficient of the j-th feature, f 1j and f 2j are the j-th global features of V1 and V2 respectively, n is the dimension of the feature vector, and m is the number of global features.

[0015] As a preferred solution of the method for developing a graphics processing engine based on virtual simulation according to the present invention, wherein: the method for obtaining the graphics parameter set is to establish a graphics recognition model based on a three-dimensional space feature matrix through a neural network architecture, wherein the neural network architecture includes an encoder and a decoder; the encoder includes a fully connected layer, a normalization layer, a ReLU activation function, and a bidirectional long short-term memory network unit; the decoder includes a parallel branch structure; the parallel branch structure includes a first branch structure and a second branch structure; the first branch structure includes a transposed convolution layer; the second branch structure includes an attention mechanism; use the fully connected layer to perform dimensionality reduction mapping on the three-dimensional space feature matrix, and input the dimensionality-reduced feature vector into the bidirectional long short-term memory network unit for temporal feature encoding; use the first branch structure to use the transposed convolution layer to perform spatial restoration on the temporal feature encoding, and at the same time use the second branch structure to use the attention mechanism to extract local key features; perform feature fusion on the spatially restored temporal feature encoding and the local key features through a connection operation in the channel dimension; input the original data of the to-be-processed graphics into the graphics recognition model, and extract the geometric parameters, position parameters, and orientation parameters of the graphics through the sequential processing of the encoder and the decoder to form a graphics parameter set.

[0016] As a preferred solution of the method for developing a graphics processing engine based on virtual simulation according to the present invention, wherein: the depth camera includes an original point cloud data stream that outputs spatial coordinate information and reflection intensity values of the X-axis, Y-axis, and Z-axis; the method for extracting the three-dimensional space feature matrix is to perform spatial noise reduction and density equalization preprocessing on the original point cloud data stream to generate a point cloud data set; divide the point cloud data set into three-dimensional voxel grids of equal size, and perform statistical encoding on the point cloud features of the three-dimensional voxel grids to generate local feature descriptors; based on a convolutional neural network algorithm, use multi-scale three-dimensional convolutional kernels to perform layer-by-layer feature extraction on the local feature descriptors; adaptively weight and combine the feature maps extracted at different scales through a cross-layer feature fusion module to output a three-dimensional feature matrix including multi-scale spatial information, wherein the row vector of the three-dimensional feature matrix represents the number of sampling points, and the column vector represents the feature dimension.

[0017] In a second aspect, an embodiment of the present invention provides a graphics processing engine development system based on virtual simulation, which includes: a data acquisition module for acquiring three-dimensional point cloud data output by a depth camera of a training scene, and performing convolution operations on the three-dimensional point cloud data through a convolutional neural network algorithm to extract a three-dimensional space feature matrix; a feature extraction module for constructing a graphics recognition model based on the three-dimensional space feature matrix, inputting the original data of the graphics to be processed into the graphics recognition model, and outputting a set of graphics parameters; a parameter adjustment module for matching a virtual scene model in a preset scene library based on the set of graphics parameters and adjusting the parameters of the virtual scene model; a grid processing module for establishing a physical coordinate system according to the adjusted virtual scene model, converting the graphics to be processed into grid cells through a grid division algorithm, and calculating the displacement data of the grid cells based on physical kinematic formulas; a graphics imaging module for reading the displacement data, generating and sending a graphics processing instruction sequence to a graphics processor, and outputting an imaging result by superimposing texture information and ray tracing effects on the grid cells.

[0018] In a third aspect, an embodiment of the present invention provides a computer device, including a memory and a processor, where the memory stores a computer program, and: when the computer program instructions are executed by the processor, the steps of the method for developing a graphics processing engine based on virtual simulation as described in the first aspect of the present invention are implemented.

[0019] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and: when the computer program instructions are executed by the processor, the steps of the method for developing a graphics processing engine based on virtual simulation as described in the first aspect of the present invention are implemented.

[0020] The beneficial effects of the present invention are as follows: By collecting three-dimensional point cloud data through a depth camera and combining it with a convolutional neural network for feature extraction, the accuracy of spatial feature expression is improved; by constructing a graphics recognition model to achieve dimensionality reduction and parameterization of features, the computational complexity is reduced; by using a scene library matching and parameter dynamic adjustment mechanism, the accuracy of virtual scene reconstruction is ensured; by combining a grid division algorithm and physical kinematic calculations, the computational efficiency of physical simulation is improved; by using an optimized rendering technique for texture overlay and ray tracing, the image realism is enhanced. Description of the Drawings

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings based on these drawings without creative efforts. Among them:

[0022] Figure 1 Flowchart of the method for developing a graphics processing engine based on virtual simulation in Embodiment 1. Detailed implementation manners

[0023] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will describe the detailed implementation manners of the present invention in conjunction with the accompanying drawings of the specification.

[0024] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0025] Secondly, the so-called "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation manner of the present invention. The "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that excludes other embodiments.

[0026] Embodiment 1

[0027] Referring to Figure 1 , which is the first embodiment of the present invention. This embodiment provides a method for developing a graphics processing engine based on virtual simulation, including

[0028] S1: Collect the three-dimensional point cloud data output by the depth camera of the training scene, and perform convolution operations on the three-dimensional point cloud data through a convolutional neural network algorithm to extract a three-dimensional spatial feature matrix.

[0029] Specifically, the depth camera includes an original point cloud data stream that outputs the spatial coordinate information and reflection intensity values of the X-axis, Y-axis, and Z-axis; perform spatial noise reduction and density equalization preprocessing on the original point cloud data stream to generate a point cloud data set; divide the point cloud data set into three-dimensional voxel grids of equal size, and perform statistical coding on the point cloud features of the three-dimensional voxel grids to generate local feature descriptors.

[0030] Furthermore, based on the convolutional neural network algorithm, use multi-scale three-dimensional convolutional kernels to perform layer-by-layer feature extraction on the local feature descriptors; adaptively weight and combine the feature maps extracted at different scales through a cross-layer feature fusion module to output a three-dimensional feature matrix including multi-scale spatial information, where the row vectors of the three-dimensional feature matrix represent the number of sampling points, and the column vectors represent the feature dimensions.

[0031] It should be noted that the multi-scale three-dimensional convolution kernel includes a first-layer convolution kernel, a second-layer convolution kernel, and a third-layer convolution. Among them, the size of the first-layer convolution kernel is 3×3×3, which is used to capture local geometric structure features; the size of the second-layer convolution kernel is 5×5×5, which is used to extract medium-scale spatial relationship features; the size of the third-layer convolution kernel is 7×7×7, which is used to obtain large-scale global context features.

[0032] S2: Construct a graphic recognition model based on the three-dimensional space feature matrix, and input the original data of the graphic to be processed into the graphic recognition model to output a set of graphic parameters.

[0033] Specifically, based on the three-dimensional space feature matrix, a graphic recognition model is established through a neural network architecture, where the neural network architecture includes an encoder and a decoder.

[0034] It should be noted that the encoder includes a fully connected layer, a normalization layer, a ReLU activation function, and a bidirectional long short-term memory network unit; the decoder includes a parallel branch structure; the parallel branch structure includes a first branch structure and a second branch structure; the first branch structure includes a transposed convolution layer; the second branch structure includes an attention mechanism.

[0035] Furthermore, the fully connected layer is used to perform dimensionality reduction mapping on the three-dimensional space feature matrix, and the dimension-reduced feature vector is input into the bidirectional long short-term memory network unit for temporal feature encoding; the output dimensions of each fully connected layer are 1024, 512, and 256 respectively.

[0036] Even further, the first branch structure is adopted to use the transposed convolution layer to perform spatial restoration on the temporal feature encoding, and at the same time, the second branch structure is adopted to use the attention mechanism to extract local key features; the spatially restored temporal feature encoding and local key features are fused through a concatenation operation in the channel dimension; the original data of the graphic to be processed is input into the graphic recognition model, and through the sequential processing of the encoder and decoder, the geometric parameters, position parameters, and orientation parameters of the graphic are extracted to form a set of graphic parameters.

[0037] S3: Based on the set of graphic parameters, match a virtual scene model in a preset scene library and adjust the parameters of the virtual scene model.

[0038] Specifically, a hierarchical scene index structure is constructed, and the set of graphic parameters is processed through feature vector normalization to generate a normalized feature vector; based on the fast retrieval algorithm of the KD tree, the set of graphic parameters is mapped to the category space of the scene library, and a feature vector index is constructed within the category space; the similarity between the normalized feature vector and the feature vector index is calculated to determine the matching virtual scene model.

[0039] Furthermore, the specific formula for similarity is as follows:

[0040]

[0041] Among them, S is the similarity score, V1 is the normalized feature vector to be matched, V2 is the feature vector index in the scenario library, v 1i and v 2i are the i-th components in V1 and V2 respectively, σ is the bandwidth parameter of the Gaussian kernel function, λ is the attenuation coefficient, w j is the weight coefficient of the j-th feature, f 1j and f 2j are the j-th global features of V1 and V2 respectively, n is the dimension of the feature vector, and m is the number of global features.

[0042] It should be noted that when the similarity score is greater than the preset threshold, the corresponding virtual scene model is marked as a candidate matching model; if the similarity score of a certain candidate matching model is higher than the scores of other candidate models by a preset difference, this model is directly determined as the final matching model; if the difference in similarity scores of multiple candidate matching models is less than the preset difference, the similarity scores are weighted and re-sorted according to the usage frequency weights of the candidate matching models, and the model with the highest weighted score is selected as the final matching model; when the similarity score is less than or equal to the preset threshold, an empty matching result is returned and a prompt message indicating that there is no suitable matching model is sent to the user. The preset threshold is the interval boundary value obtained by performing K-means clustering analysis based on the similarity distribution of the historical matching dataset.

[0043] Furthermore, geometric parameters in the graphic parameter set are extracted, and geometric transformation is performed on the virtual scene model according to the principle of affine transformation; the transformed virtual scene model is adjusted through the bilinear interpolation algorithm, and at the same time, the material parameters, lighting parameters, and shadow parameters of the virtual scene model are optimally configured.

[0044] It should be noted that the geometric transformation includes size scaling, rotation and translation, and projection mapping.

[0045] S4: Establish a physical coordinate system based on the adjusted virtual scene model, convert the graphic to be processed into grid cells through the grid division algorithm, and calculate the displacement data of the grid cells based on the physical kinematic formula.

[0046] Specifically, based on the right-handed Cartesian coordinate system, a physical coordinate system is established according to the adjusted virtual scene model.

[0047] It should be noted that the origin is set to coincide with the center of the virtual scene in the physical coordinate system, the X-axis, Y-axis, and Z-axis respectively correspond to the length, height, and depth directions of the scene, and the coordinate unit is scaled to the actual physical scale.

[0048] Furthermore, an adaptive quadrilateral mesh generation algorithm is used to perform mesh dissection on the to-be-processed graph, and the mesh density is dynamically adjusted according to the local curvature and boundary features of the graph to generate a set of mesh elements, where each mesh element contains node coordinates, element quality, and boundary constraint conditions.

[0049] Even further, based on the set of mesh elements, taking Newton's second law as the basis of the physical kinematic formula and combining with Lagrangian mechanics equations, forces are applied to the mesh elements; at the same time, the Runge-Kutta integration algorithm is used to solve the differential equation system to calculate the velocity vector and displacement vector of the mesh elements within the discrete time step; the spatial positions of the mesh elements in the physical coordinate system are updated according to the displacement vector to form displacement data.

[0050] S5: Read the displacement data, generate and send a sequence of graphics processing instructions to the graphics processor, and output an imaging result by superimposing texture information and ray tracing effects on the mesh elements.

[0051] Specifically, a time-series data stream processing method is used to sort the displacement data by timestamp and enter it into the graphics processing buffer, where the displacement data in the graphics processing buffer is mapped to the graphics rendering pipeline through interpolation and smoothing processing to generate a sequence of graphics processing instructions.

[0052] It should be noted that the sequence of graphics processing instructions includes vertex shading instructions, fragment shading instructions, and post-processing special effect instructions, where the vertex shading instructions control the spatial transformation and geometric deformation of the mesh elements; the fragment shading instructions are responsible for texture mapping and material rendering; the post-processing special effect instructions are used for ray tracing and environmental special effects.

[0053] Furthermore, based on the sequence of graphics processing instructions, texture coordinates are assigned to the mesh elements according to the physical coordinate system, and an anisotropic filtering method is used to improve the texture sampling quality; at the same time, the surface details are enhanced through normal mapping and displacement mapping, and a preset material template is applied to the mesh elements to perform bidirectional reflectance distribution function calculation to simulate the light scattering characteristics of the material surface.

[0054] It should be noted that the preset material template defines the diffuse reflection coefficient, specular reflection coefficient, and bump mapping parameters.

[0055] Even further, a hierarchical bounding box structure is used to accelerate the ray-scene intersection operation, calculate the combined effects of direct illumination, indirect illumination, and global illumination, and add ambient occlusion and soft shadow effects to achieve the effect of ray tracing; based on the effect of ray tracing, through tone mapping and anti-aliasing processing of the post-processing pipeline, the rendering result is output to the display buffer to generate an imaging result.

[0056] Furthermore, this embodiment also provides a graphics processing engine development system based on virtual simulation, including: a data acquisition module, configured to acquire three-dimensional point cloud data output by a depth camera of a training scenario, and perform convolution operations on the three-dimensional point cloud data through a convolutional neural network algorithm to extract a three-dimensional space feature matrix; a feature extraction module, configured to construct a graphics recognition model based on the three-dimensional space feature matrix, input the original data of a to-be-processed graphic into the graphics recognition model, and output a graphic parameter set; a parameter adjustment module, configured to match a virtual scene model in a preset scene library based on the graphic parameter set, and adjust the parameters of the virtual scene model; a meshing processing module, configured to establish a physical coordinate system according to the adjusted virtual scene model, convert the to-be-processed graphic into grid cells through a grid division algorithm, and calculate displacement data of the grid cells based on physical kinematic formulas; a graphic imaging module, configured to read the displacement data, generate and send a graphic processing instruction sequence to a graphics processor, and output an imaging result by superimposing texture information and ray tracing effects on the grid cells.

[0057] This embodiment also provides a computer device applicable to the case of a graphics processing engine development method based on virtual simulation, including a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the graphics processing engine development method based on virtual simulation as proposed in the above embodiment.

[0058] This computer device can be a terminal. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a carrier network, NFC (Near Field Communication), or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0059] In summary, the present invention collects three-dimensional point cloud data through a depth camera and combines it with a convolutional neural network for feature extraction, improving the accuracy of spatial feature expression; realizes dimensionality reduction and parameterization processing of features by constructing a graphic recognition model, reducing the computational complexity; ensures the accuracy of virtual scene reconstruction by using a scene library matching and parameter dynamic adjustment mechanism; improves the computational efficiency of physical simulation by combining a mesh generation algorithm and physical kinematics calculation; and enhances the image realism by using an optimized rendering technology for texture overlay and ray tracing.

[0060] Embodiment 2

[0061] Referring to Table 1, this is the second embodiment of the present invention. This embodiment provides a method for developing a graphics processing engine based on virtual simulation. To verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0062] Specifically, an industrial part design scenario is selected as the test object, and a depth camera Intel RealSense D455 is used to collect three-dimensional point cloud data of 6 mechanical part models with different complexities; the sampling frequency of the depth camera is set to 30 Hz, the spatial resolution is 1280×720 pixels, and the depth accuracy is ±0.5 mm. During the collection process, the part to be measured is placed on a calibration platform, and the depth camera is driven by a robotic arm to perform a 360-degree circumferential scan. The collection time for each part is 60 seconds, and approximately 1800 frames of original point cloud data are obtained.

[0063] Furthermore, the collected point cloud data is denoised by voxel filtering, and the voxel size is set to 0.5 mm×0.5 mm×0.5 mm; an improved PointNet++ network architecture is used for feature extraction. The network includes 4 layers of point cloud convolutional layers, and the convolutional kernel sizes are 16, 32, 64, and 128 respectively, with a stride of 1. The activation function uses LeakyReLU; after feature extraction, a three-dimensional spatial feature matrix with a dimension of 256 is obtained; based on the extracted three-dimensional spatial feature matrix, a deep neural network including an encoder and a decoder is constructed as a graphic recognition model; the encoder uses 3 fully connected layers (the number of nodes is 512, 256, and 128 respectively), and the decoder uses a parallel branch structure. The first branch uses transposed convolution for feature restoration, and the second branch extracts key feature points through an attention mechanism. The outputs of the two branches are fused through a 1×1 convolutional layer, and finally a graphic parameter set including geometric parameters, position parameters, and orientation parameters is output.

[0064] Specifically, model matching is performed in a preset scenario library of 10,000 industrial parts. An improved KD-tree algorithm is used to establish an index structure, and the similarity threshold of the feature vector is set to 0.85. Parameter adjustment is performed on the matched virtual scene model, including geometric transformation (scaling factor 0.8 - 1.2, rotation angle ±15°) and material optimization (diffuse reflection coefficient 0.2 - 0.8, roughness 0.1 - 0.9); in the physical simulation stage, an adaptive quadrilateral mesh generation algorithm is used, with the mesh size range of 0.1 mm - 2 mm and the curvature adaptive threshold of 0.05. The fourth-order Runge-Kutta method is used to solve the motion equation, and the time step is set to 0.001 s. In the rendering stage, a physically based material system is used, the ray tracing sample count is set to 1024, the recursion depth is 4, and at the same time, ambient occlusion and soft shadow effects are enabled.

[0065] Furthermore, as shown in Table 1, the point cloud acquisition accuracy of the present invention reaches 0.48 mm, which is 49.5% higher than 0.95 mm of the traditional method, improving the acquisition quality of three-dimensional data; the feature extraction time is reduced from 195 ms of the traditional method to 58 ms, improving the processing efficiency by 70.3%. This benefits from the improved convolutional neural network algorithm and the optimized parallel computing architecture. The feature dimension compression rate reaches 85.5%, which is 89.2% higher than 45.2% of the traditional method, proving that the present invention has obvious advantages in data dimensionality reduction and feature expression; the model matching accuracy of the present invention reaches 95.8%, which is 16.1% higher than 82.5% of the traditional method. At the same time, the scene reconstruction error is reduced from 1.85 mm to 0.62 mm, an improvement of 66.5%. This shows that the present invention has made remarkable progress in virtual scene model matching and parameter optimization, providing more reliable basic data for subsequent physical simulation and rendering processing. In terms of physical simulation, the present invention achieves a simulation frame rate of 105 fps, which is 133.3% higher than 45 fps of the traditional method. The mesh generation accuracy is improved to 0.15 mm, which is 66.7% higher than 0.45 mm of the traditional method. These improvements benefit from the adaptive mesh generation algorithm and the efficient physical kinematics calculation framework, significantly improving the accuracy and real-time performance of the simulation process.

[0066] Table 1. Comparison Table between the Method of the Present Invention and the Traditional Method

[0067]

[0068]

[0069] Furthermore, the rendering quality score reaches 9.2 points (out of 10), a 24.3% increase compared to 7.4 points of the traditional method. The accuracy of material restoration is increased from 75.8% to 92.5%, an increase of 22.0%. The ray tracing sampling efficiency reaches 960 samples / ms, a 152.6% increase compared to 380 samples / ms of the traditional method. These indicators fully demonstrate the advantages of the present invention in terms of graphics rendering effect and processing efficiency. The system response time is reduced from 125 ms of the traditional method to 32 ms, an increase of 74.4%. At the same time, the memory occupancy is reduced from 1250 MB to 485 MB, saving 61.2% of system resources. This shows that the present invention not only improves the processing effect but also optimizes the resource utilization efficiency, with better practicality.

[0070] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A method for developing a graphics processing engine based on virtual simulation, characterized in that: include, Collect the 3D point cloud data output by the depth camera of the training scene, and perform convolution operation on the 3D point cloud data through the convolutional neural network algorithm to extract the 3D spatial feature matrix; Constructing a graphic recognition model according to the three-dimensional spatial feature matrix, inputting the original data of the graphic to be processed into the graphic recognition model, and outputting a graphic parameter set; Based on the graphic parameter set, matching a virtual scene model in a preset scene library, and adjusting parameters of the virtual scene model; A physical coordinate system is established according to the adjusted virtual scene model, the graphics to be processed are converted into grid units through a grid division algorithm, and the displacement data of the grid units are calculated based on a physical kinematics formula; The displacement data is read, a graphics processing instruction sequence is generated and sent to a graphics processor, and an imaging result is output by superimposing texture information and ray tracing effects on the grid units.

2. The method for developing a graphics processing engine based on virtual simulation according to claim 1, characterized in that: The method for generating the imaging result is: The displacement data is sorted by time stamp using a time series data stream processing method and enters a graphics processing buffer, wherein the displacement data in the graphics processing buffer is mapped to a graphics rendering pipeline after interpolation and smoothing processing to generate a graphics processing instruction sequence; Based on the graphics processing instruction sequence, texture coordinates are allocated to the grid cells according to the physical coordinate system, and an anisotropic filtering method is used to improve the texture sampling quality; At the same time, the surface details are enhanced through normal maps and displacement maps, and the preset material template is applied to the grid unit to calculate the bidirectional reflectance distribution function to simulate the light scattering characteristics of the material surface; A hierarchical bounding box structure is used to accelerate the light-scene intersection operation, calculate the composite effect of direct lighting, indirect lighting and global lighting, and add ambient occlusion and soft shadow effects to achieve the effect of ray tracing; Based on the effect of ray tracing, the rendering results are output to the display buffer through tone mapping and anti-aliasing processing of the post-processing pipeline to generate imaging results.

3. The method for developing a graphics processing engine based on virtual simulation according to claim 2, characterized in that: The method for forming the displacement data is: Based on the right-handed Cartesian coordinate system, a physical coordinate system is established according to the adjusted virtual scene model; Adopting adaptive quadrilateral meshing algorithm to mesh the graphics to be processed, and dynamically adjusting the mesh density according to the local curvature and boundary characteristics of the graphics to generate a set of mesh units; Based on the set of grid cells, using Newton's second law as the basis of the physical kinematics formula and combining the Lagrangian mechanics equation, a force is applied to the grid cells; At the same time, the differential equations are solved by the Runge-Kutta integration algorithm to calculate the velocity vector and displacement vector of the grid unit in the discrete time step; The spatial position of the grid unit in the physical coordinate system is updated according to the displacement vector to form displacement data.

4. The method for developing a graphics processing engine based on virtual simulation according to claim 3, characterized in that: Matching a corresponding virtual scene model in a preset scene library according to the graphic parameter set, and adjusting parameters of the virtual scene model, including: Construct a hierarchical scene index structure, and process the graphics parameter set through feature vector normalization to generate a standardized feature vector; Based on the KD tree fast retrieval algorithm, the graphic parameter set is mapped to the category space of the scene library, and a feature vector index is constructed in the category space; Calculating the similarity between the standardized feature vector and the feature vector index to determine a matching virtual scene model; Extracting geometric parameters from the graphic parameter set, and performing geometric transformation on the virtual scene model according to the affine transformation principle, wherein the geometric transformation includes size scaling, rotation translation and projection mapping; The transformed virtual scene model is adjusted by a bilinear interpolation algorithm, and the material parameters, lighting parameters and shadow parameters of the virtual scene model are optimized.

5. The method for developing a graphics processing engine based on virtual simulation according to claim 4, characterized in that: The specific formula of the similarity is as follows: Among them, S is the similarity score, V1 is the standardized feature vector to be matched, V2 is the feature vector index in the scene library, and v 1i and v 2i are the i-th components in V1 and V2 respectively, σ is the bandwidth parameter of the Gaussian kernel function, λ is the attenuation coefficient, and w j is the weight coefficient of the jth feature, f 1j and f 2j are the j-th global features of V1 and V2 respectively, n is the feature vector dimension, and m is the number of global features.

6. The method for developing a graphics processing engine based on virtual simulation according to claim 4, characterized in that: The method for obtaining the graphics parameter set is: Based on the three-dimensional spatial feature matrix, a graphic recognition model is established through a neural network architecture, wherein the neural network architecture includes an encoder and a decoder; the encoder includes a fully connected layer, a normalization layer, a ReLU activation function, and a bidirectional long short-term memory network unit; the decoder includes a parallel branch structure; the parallel branch structure includes a first branch structure and a second branch structure; the first branch structure includes a deconvolution layer; the second branch structure includes an attention mechanism; Use the fully connected layer to reduce the dimension of the three-dimensional spatial feature matrix, and input the reduced-dimensional feature vector into the bidirectional long short-term memory network unit for temporal feature encoding; The first branch structure is used to use the deconvolution layer to perform spatial restoration on the temporal feature encoding, and the second branch structure is used to use the attention mechanism to extract local key features; The temporal feature encoding after spatial restoration and the local key features are connected in the channel dimension and then feature fused; The original data of the graphics to be processed is input into the graphics recognition model, and the geometric parameters, position parameters and direction parameters of the graphics are extracted through the sequential processing of the encoder and the decoder to form a graphics parameter set.

7. The method for developing a graphics processing engine based on virtual simulation according to claim 1, characterized in that: The depth camera includes an original point cloud data stream that outputs spatial coordinate information of the X-axis, Y-axis and Z-axis and a reflection intensity value; The method for extracting the three-dimensional space feature matrix is: Performing spatial noise reduction and density equalization preprocessing on the original point cloud data stream to generate a point cloud data set; Dividing the point cloud data set into three-dimensional voxel grids of equal size, and performing statistical encoding on point cloud features of the three-dimensional voxel grids to generate local feature descriptors; Based on the convolutional neural network algorithm, a multi-scale three-dimensional convolution kernel is used to extract features of the local feature descriptor layer by layer; The feature maps extracted at different scales are adaptively weighted and combined through a cross-layer feature fusion module to output a three-dimensional feature matrix including multi-scale spatial information, wherein the row vector of the three-dimensional feature matrix represents the number of sampling points and the column vector represents the feature dimension.

8. A graphics processing engine development system based on virtual simulation, based on the graphics processing engine development method based on virtual simulation according to any one of claims 1 to 7, characterized in that: Also includes, The data acquisition module is used to collect the 3D point cloud data output by the depth camera of the training scene, and perform convolution operation on the 3D point cloud data through the convolutional neural network algorithm to extract the 3D spatial feature matrix; A feature extraction module, used to construct a graphic recognition model according to the three-dimensional space feature matrix, input the original data of the graphic to be processed into the graphic recognition model, and output a graphic parameter set; A parameter adjustment module, used to match a virtual scene model in a preset scene library based on the graphic parameter set, and adjust the parameters of the virtual scene model; A grid processing module is used to establish a physical coordinate system according to the adjusted virtual scene model, convert the graphics to be processed into grid units through a grid division algorithm, and calculate the displacement data of the grid units based on physical kinematics formulas; The graphics imaging module is used to read the displacement data, generate and send a graphics processing instruction sequence to the graphics processor, and output an imaging result by superimposing texture information and ray tracing effects on the grid units.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for developing a graphics processing engine based on virtual simulation according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for developing a graphics processing engine based on virtual simulation according to any one of claims 1 to 7 are implemented.

Images