VR-based textile culture heritage three-dimensional reconstruction method and system

Abstract

The invention relates to the technical field of three-dimensional image reconstruction, in particular to a VR-based three-dimensional reconstruction method and system for textile culture heritage, and the method comprises the steps: obtaining a multi-view image collection sequence of the textile culture heritage, carrying out the geometric topological feature extraction of the multi-view image collection sequence, generating a three-dimensional point cloud geometric model, and carrying out the reconstruction of the three-dimensional point cloud geometric model. Performing multi-resolution texture feature mapping processing on the three-dimensional point cloud geometric model to generate an initial three-dimensional model fusing geometric topological features and multi-resolution texture features; performing dynamic geometric optimization processing on the initial three-dimensional model based on preset VR display parameters to generate a target three-dimensional model after surface continuity correction; and performing dynamic matching processing on the target three-dimensional model and a preset VR interaction algorithm to generate a VR visual model with a multi-dimensional interaction attribute, and transmitting the VR visual model to a VR display terminal to perform three-dimensional space rendering, so that the three-dimensional reconstruction and interaction intelligence degree of the textile culture heritage can be improved in combination with a VR technology.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional image reconstruction, and more specifically, to a method and system for three-dimensional reconstruction of textile cultural heritage based on VR. Background Art

[0002] With the continuous development of virtual reality (VR) and 3D reconstruction technologies, they are finding widespread application in numerous fields. VR provides users with an immersive experience, while 3D reconstruction technology allows real-world objects to be presented as digital 3D models. These technologies are also gradually emerging in the preservation and display of cultural heritage, providing new avenues for its transmission.

[0003] However, when existing VR and 3D reconstruction technologies are applied to the field of cultural heritage, there are many deficiencies: (1) Previous 3D reconstructions often only focus on the geometric shape of objects, ignoring the accurate restoration of complex texture features, resulting in the reconstruction model not being realistic enough; (2) In the VR display process, the surface continuity of the model is poor, and the visual effect is not natural and smooth enough; (3) The existing interaction method is single, and users can only perform simple observations and cannot deeply participate in the interaction with cultural heritage. In summary, how to combine VR technology to improve the 3D reconstruction and interactive intelligence of textile cultural heritage is a technical problem that needs to be overcome. Summary of the Invention

[0004] In view of this, the present invention provides a VR-based three-dimensional reconstruction method and system for textile cultural heritage.

[0005] An embodiment of the present invention provides a VR-based three-dimensional reconstruction method for textile cultural heritage, which is applied to a textile cultural heritage three-dimensional reconstruction system. The method includes: acquiring a multi-perspective image acquisition sequence of textile cultural heritage, performing geometric topological feature extraction processing on the multi-perspective image acquisition sequence to generate a three-dimensional point cloud geometric model, and performing multi-resolution texture feature mapping processing on the three-dimensional point cloud geometric model to generate an initial three-dimensional model that integrates geometric topological features and multi-resolution texture features; performing dynamic geometric optimization processing on the initial three-dimensional model based on preset VR display parameters to generate a target three-dimensional model after surface continuity correction; dynamically matching the target three-dimensional model with a preset VR interaction algorithm to generate a VR visualization model with multi-dimensional interactive properties, and transmitting the VR visualization model to a VR display terminal for three-dimensional space rendering.

[0006] The present invention also provides a three-dimensional reconstruction system for textile cultural heritage, comprising: a memory for storing program instructions and data; a processor for coupling with the memory and executing instructions in the memory to implement the above method.

[0007] The present invention also provides a computer storage medium comprising instructions, which implement the above method when executed on a processor.

[0008] The embodiment of the present invention obtains a multi-view image acquisition sequence of textile cultural heritage, extracts geometric topological features and maps multi-resolution texture features to generate an initial three-dimensional model that integrates multiple features. This model can fully and meticulously restore the original appearance and complex textures of the textile cultural heritage. Dynamic geometric optimization of the initial three-dimensional model based on preset VR display parameters can improve the surface continuity of the target three-dimensional model and present a natural and smooth visual effect. The target three-dimensional model is dynamically matched with the VR interaction algorithm, and the generated VR visualization model has multi-dimensional interactive properties, thereby enabling user interaction with the virtual model of textile cultural heritage. This design can be combined with VR technology to improve the three-dimensional reconstruction and interactive intelligence of textile cultural heritage. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0010] Figure 1 A schematic flow chart of the steps of a VR-based three-dimensional reconstruction method for textile cultural heritage provided in an embodiment of the present invention.

[0011] Figure 2 This is a structural block diagram of a textile cultural heritage 3D reconstruction system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0012] The technical solutions of the present invention will be described below in conjunction with the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation methods described in the following exemplary embodiments do not represent all implementation methods consistent with the present invention. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present invention. It should be noted that the terms "first", "second", etc. in the specification of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0013] See also Figure 1 , Figure 1 1 is a flow chart of a method for three-dimensional reconstruction of textile cultural heritage based on VR provided by an embodiment of the present invention. The method is applied to a three-dimensional reconstruction system of textile cultural heritage and may further include steps 101 to 103.

[0014] Step 101: Acquire a multi-view image acquisition sequence of textile cultural heritage, perform geometric topological feature extraction processing on the multi-view image acquisition sequence to generate a three-dimensional point cloud geometric model, and perform multi-resolution texture feature mapping processing on the three-dimensional point cloud geometric model to generate an initial three-dimensional model that integrates geometric topological features and multi-resolution texture features.

[0015] In this embodiment, a textile cultural heritage with unique patterns and complex weaving structures is used as an example. To obtain a comprehensive and accurate multi-view image acquisition sequence, the system can use multiple high-definition cameras to capture the textile cultural heritage at different angles and distances. During the shooting process, various lighting conditions are set, such as strong light from the side and weak light from the top, to obtain original texture images and geometric structure images under different lighting conditions. This image data is stored in a corresponding data storage device, forming a multi-view image acquisition sequence.

[0016] Next, the multi-view image acquisition sequence is processed for geometric topological feature extraction. Using common image processing algorithms, geometric information such as the lines and contours of the textile cultural heritage in the image is analyzed to extract its geometric topological features. For example, the topological structure of the woven texture is determined by analyzing the line orientation and intersections. Using these features, a 3D point cloud geometric model is generated. This model represents the 3D shape of the textile cultural heritage as a collection of points, with each point containing spatial coordinate information.

[0017] The generated 3D point cloud geometric model is then subjected to multi-resolution texture feature mapping. Texture images of varying resolutions are selected from the multi-view image acquisition sequence and mapped onto the 3D point cloud geometric model according to the geometric model's topological structure. For example, for detail-rich portions of textile cultural heritage, high-resolution texture images are used to create exquisite patterns. For relatively smooth portions, lower-resolution texture images are used to minimize data volume while maintaining visual quality. Ultimately, an initial 3D model is generated that integrates geometric topological features with multi-resolution texture features.

[0018] In some examples, the multi-view image acquisition sequence includes original texture images and geometric structure images acquired under different lighting conditions. Based on this, step 101 includes:

[0019] Step 1011: Divide the multi-view image acquisition sequence into a synchronous acquisition image set and an asynchronous acquisition image set, wherein the synchronous acquisition image set includes original texture images and geometric structure images of different view angles at the same timestamp.

[0020] In this embodiment, a multi-view image acquisition sequence stored in a data storage device is analyzed. Images captured at the same time from different viewpoints are grouped into a synchronously captured image set based on their timestamp information. For example, at one moment, original texture images and geometric structure images captured simultaneously from three different viewpoints—the front, 45 degrees to the left, and 30 degrees to the right—are combined to form a synchronously captured image set. Images captured at other different times are grouped into asynchronously captured image sets. This division allows for more targeted processing of different image types, improving processing efficiency and accuracy.

[0021] Step 1012: Perform stereo vision matching processing on the synchronously collected image set to extract depth difference features and edge continuity features of the synchronously collected image set.

[0022] In this embodiment, a stereo vision matching algorithm is applied to images of different perspectives at the same timestamp in a synchronously captured image set. The algorithm calculates depth information by comparing the pixel position differences of the same object in images of different perspectives. For example, the position of a pattern element on a textile cultural heritage will be different in images of different perspectives. By analyzing this position difference and combining the parameter information of the camera, the depth difference characteristics of the pattern element in three-dimensional space are calculated. At the same time, the edge information in the image is analyzed, and the edge contours of the textile cultural heritage are identified by edge detection algorithms, such as the Canny algorithm. Then, the continuity of these edges in images of different perspectives is tracked, and the edge continuity features are extracted. These depth difference features and edge continuity features are crucial for the subsequent construction of an accurate three-dimensional model.

[0023] Step 1013: construct an initial three-dimensional point cloud space according to the depth difference feature, and perform topological alignment processing on the edge continuity feature and the initial three-dimensional point cloud space to generate a three-dimensional point cloud geometric model with spatial topological constraints.

[0024] In this embodiment, the initial three-dimensional point cloud space is constructed based on the extracted depth difference features. Based on the depth information, the corresponding points in the images of different perspectives are mapped into the three-dimensional space to form an initial spatial structure composed of points. For example, the corresponding points of the pattern elements whose depth differences have been calculated before are placed in the three-dimensional space according to their depth and spatial position relationship. Then, the edge continuity feature is topologically aligned with the initial three-dimensional point cloud space. By analyzing the direction and connection relationship of the edge in the three-dimensional space, the position and connection method of the point cloud are adjusted so that the point cloud can accurately reflect the topological structure of the textile cultural heritage. For example, for the continuous weaving lines in the textile cultural heritage, it is ensured that the corresponding points in the three-dimensional point cloud space can also be connected and arranged according to the continuity of the lines. After the above processing, a three-dimensional point cloud geometric model with spatial topological constraints is generated. The model not only contains the position information in the three-dimensional space, but also reflects the topological structure characteristics of the textile cultural heritage.

[0025] Step 1014: Perform illumination consistency compensation processing on the asynchronously acquired image set to generate a compensated image sequence after illumination equalization, and perform dynamic texture coordinate mapping processing on the compensated image sequence and the three-dimensional point cloud geometric model to generate the initial three-dimensional model that integrates the geometric topological features and the multi-resolution texture features.

[0026] In this embodiment, for asynchronously captured image sets, due to varying lighting conditions during acquisition, the images exhibit illumination variations. First, illumination consistency compensation is performed. Using an illumination analysis algorithm, information such as illumination intensity and color distribution is calculated for each image. For example, for a dimly lit image, the area and degree of insufficient illumination are analyzed. The image is then adjusted based on a pre-defined illumination compensation model, which can be based on histogram equalization or other illumination correction algorithms, to achieve a more uniform illumination effect. After this processing, a sequence of compensated, illumination-equalized images is generated.

[0027] Next, the compensated image sequence is dynamically mapped to the generated 3D point cloud geometry model using texture coordinates. By analyzing the surface structure and topological relationships of the 3D point cloud geometry model, corresponding texture coordinates are assigned to each point cloud. Then, based on the texture features and positional information in the compensated image sequence, the texture is accurately mapped to the 3D point cloud geometry model. For example, for a raised portion of a textile cultural heritage, its corresponding point cloud region is determined in the 3D point cloud geometry model. Then, a suitable texture image is selected from the compensated image sequence, and the texture is mapped to the region according to the texture coordinates of the point cloud. Ultimately, an initial 3D model is generated that integrates geometric topological features with multi-resolution texture features.

[0028] Step 102: Dynamically optimize the geometry of the initial three-dimensional model based on preset VR display parameters to generate a target three-dimensional model after surface continuity correction.

[0029] In this embodiment, VR display parameters, including field of view and display resolution, are preset, taking into account the characteristics and requirements of VR displays. Based on these preset VR display parameters, dynamic geometric optimization is performed on the generated initial 3D model. This aims to enhance the visual quality of the model in a VR environment, particularly by resolving potential surface discontinuities and generating a target 3D model with corrected surface continuity.

[0030] In an optional embodiment, step 102 includes:

[0031] Step 1021: Obtain the field of view angle parameters and display resolution parameters of the VR display terminal, and divide the polygonal facet density distribution intervals of the initial three-dimensional model according to the field of view angle parameters.

[0032] In this embodiment, the field of view angle parameters and display resolution parameters are first obtained from the device information of the VR display terminal. For example, the obtained field of view angle is α degrees, and the display resolution is I×J pixels. According to the field of view angle parameters, the initial three-dimensional model is analyzed. Since the human eye has different perceptions of model details within different field of view ranges, it is necessary to divide the polygonal patch density distribution intervals of the initial three-dimensional model according to the field of view angle. For example, the field of view is divided into a central area and an edge area. The central area is the area that the human eye focuses on and has high requirements for details, so a higher density of polygonal patches is divided in this area; while the edge area is relatively less paid attention to by the human eye and has low requirements for details, so a lower density of polygonal patches can be divided. Through the above division, the patch resources of the model can be reasonably allocated while ensuring the visual effect, thereby improving the rendering efficiency.

[0033] Step 1022: performing surface curvature analysis on the facets within the polygonal facet density distribution range to extract the geometric mutation features of the first curvature region and the geometric smooth features of the second curvature region.

[0034] In this embodiment, surface curvature analysis is performed on the faces within the divided polygonal face density distribution range. The surface curvature of each face is analyzed by a preset curvature calculation algorithm. For the three-dimensional model of textile cultural heritage, some parts may have obvious turning points or protrusions. The surface curvature of these parts is large and is defined as the first curvature area; while some parts are relatively smooth and have a small surface curvature, which is defined as the second curvature area. For example, the edge of the woven pattern in the textile cultural heritage may belong to the first curvature area, with geometric mutation features, such as sudden angle changes; while the large area of flat woven area belongs to the second curvature area, with geometric flat features. Through this analysis, the characteristics of different areas of the model surface can be accurately identified, providing a basis for subsequent processing.

[0035] Step 1023: Adaptively triangulate the first curvature region according to the display resolution parameter and the geometric mutation feature to generate a subdivided facet set, and merge the facets of the second curvature region according to the display resolution parameter and the geometric smooth feature to generate a simplified facet set.

[0036] In this embodiment, the first curvature region is adaptively triangulated in combination with the display resolution parameter and the previously extracted geometric mutation features. The display resolution parameter determines the minimum level of detail that can be clearly displayed. For the parts of the first curvature region with obvious geometric mutations, in order to accurately present these details on the VR display terminal, the facets of the area are further divided into smaller triangular facets according to the display resolution. For example, when the display resolution is high, a more refined triangulation will be performed on the mutation part of the edge of the textile cultural heritage pattern to generate a set of subdivided facets.

[0037] At the same time, meshes are merged in the second curvature region based on the display resolution parameters and the geometrically flat features. Because the geometry of the second curvature region is relatively flat, it does not require too many meshes to represent it at a certain display resolution. Therefore, adjacent, similarly shaped meshes are merged to form larger meshes, generating a simplified mesh set. This reduces the model's data size without affecting the visual quality of the model in this region.

[0038] Step 1024: Perform spatial continuity correction processing on the subdivided facet set and the simplified facet set to eliminate geometric seam errors between adjacent facets and generate a target three-dimensional model after surface continuity correction.

[0039] In this embodiment, after obtaining the subdivided patch set and the simplified patch set, they need to be subjected to spatial continuity correction processing. Since the subdivision and merging operations may cause geometric seam errors between adjacent patches in the previous processing process, affecting the surface continuity of the model. Therefore, the position and normal direction of adjacent patches are adjusted through a preset spatial correction algorithm. For example, for the patches at the junction of the subdivided patch set and the simplified patch set, their position differences and normal direction differences in three-dimensional space are analyzed. Through operations such as translation and rotation, the boundaries of adjacent patches can be accurately aligned and the normal directions are kept consistent, thereby eliminating geometric seam errors. After the above processing, a target three-dimensional model with surface continuity correction is generated, which can present a smoother and more natural visual effect in a VR display environment.

[0040] Step 103: Dynamically match the target three-dimensional model with a preset VR interaction algorithm to generate a VR visualization model with multi-dimensional interactive properties, and transmit the VR visualization model to a VR display terminal for three-dimensional space rendering.

[0041] In this embodiment, a set of VR interaction algorithms is preset, which is designed to realize various interactive functions between users and three-dimensional models of textile cultural heritage. The target three-dimensional model after the generated surface continuity correction is dynamically matched with the preset VR interaction algorithm. Through the matching mechanism in the algorithm, the geometric topological features and texture features of the target three-dimensional model are analyzed, and compared and adapted with the interaction rules preset in the algorithm. For example, the algorithm stipulates that the user interacts with specific areas of the model through gesture operations. Then, in the matching process, it is determined which areas of the model meet these interaction conditions. After dynamic matching processing, a VR visualization model with multi-dimensional interactive properties is generated. This model not only contains the three-dimensional information of the textile cultural heritage, but also has the ability to interact with the user in various ways.

[0042] The generated VR visualization model is then transmitted to a VR display terminal for 3D rendering. The model data is sent to the VR display terminal via a network transmission protocol. After receiving the data, the VR display terminal uses its own graphics processing capabilities to render the model in 3D, presenting it to the user in a three-dimensional form.

[0043] In an optional embodiment, dynamically matching the target three-dimensional model with a preset VR interaction algorithm to generate a VR visualization model with multi-dimensional interactive properties includes:

[0044] Step 1031: Obtain the user interaction behavior type configured in the VR interaction algorithm.

[0045] In this embodiment, user interaction behavior types are obtained from a preset VR interaction algorithm configuration file. These interaction behavior types are pre-defined to meet the needs of different users in exploring textile cultural heritage. For example, the configuration file defines how users can interact with the model through head rotation, hand movements, and other methods. Head rotation can be used to change the viewing angle, while hand movements can be used to touch and zoom the model. By obtaining these user interaction behavior types, a clear direction is provided for subsequent interaction processing.

[0046] Step 1032: performing interaction-sensitive area recognition processing on the geometric topological features of the target three-dimensional model to generate a set of dynamic response areas corresponding to the user interaction behavior type.

[0047] In this embodiment, a preset recognition algorithm is used to identify interactive sensitive areas based on the geometric topological features of the target three-dimensional model. First, the geometric structure of the model is analyzed to determine which parts may be areas where users are likely to pay attention to and interact. For example, for a three-dimensional model of a textile cultural heritage, its unique patterns, edge parts, etc. may be interactive sensitive areas. Then, combined with the acquired user interaction behavior type, the dynamic response area corresponding to each interaction behavior is further determined. For example, if the user interaction behavior type includes touch operation, then the area with a protruding surface and obvious texture in the model may be identified as a dynamic response area for touch interaction. Through this recognition process, a set of dynamic response areas corresponding to the user interaction behavior type is generated.

[0048] Preferably, step 1032 includes:

[0049] Step 10321: extracting the texture complexity index and geometric visibility index of each facet in the target three-dimensional model, and generating a facet texture weight distribution map according to the texture complexity index.

[0050] In this embodiment, each facet in the target three-dimensional model is analyzed in detail. The texture complexity index of each facet is calculated by a texture analysis algorithm. For example, for a facet in a textile cultural heritage model, if it contains complex weaving textures, multiple color changes, etc., then its texture complexity index is relatively high; conversely, if the facet texture is relatively simple, with only a single color or a small number of lines, then the texture complexity index is low. At the same time, the geometric visibility index of each facet is calculated by a geometric analysis algorithm, taking into account factors such as the position of the facet in the overall structure of the model and whether it is blocked by other parts. For example, the geometric visibility index of a facet located on the surface of the model and not blocked by other parts is high, while the geometric visibility index of a facet located inside the model or blocked by a large number of other faces is low.

[0051] Based on the calculated texture complexity index, a patch texture weight distribution map is generated. In this distribution map, patches with high texture complexity are assigned higher weights, while patches with low texture complexity are assigned lower weights. This method intuitively demonstrates the texture importance of different patches in the model.

[0052] Step 10322: The geometric visibility index is overlaid and analyzed with the preset user gaze hotspot area to generate a patch visibility heat distribution map.

[0053] In this embodiment, user gaze hotspot areas are preset, which are derived based on the analysis of user observation habits and areas of interest in textile cultural heritage. The geometric visibility index of each facet calculated previously is superimposed on the preset user gaze hotspot area for analysis. For example, for a facet located in the user gaze hotspot area and having a high geometric visibility, its heat value in the superimposed analysis will increase accordingly; while for a facet far from the user gaze hotspot area and having a low geometric visibility, its heat value will decrease. Through this superimposed analysis process, a facet visibility heat distribution map is generated, which reflects the heat situation of each facet under the user's attention perspective.

[0054] Step 10323: Perform dual-channel feature fusion processing on the patch texture weight distribution map and the patch visibility heat distribution map to generate an interactive sensitive area probability map.

[0055] In this embodiment, dual-channel feature fusion processing is performed on the generated patch texture weight distribution map and patch visibility heat distribution map. A feature fusion algorithm is used to comprehensively analyze the information in the two distribution maps. For example, for a patch with a high weight in the texture weight distribution map and a high heat in the visibility heat distribution map, after the fusion process, its probability value in the interaction-sensitive area probability map will increase accordingly; while for patches with low performance in both distribution maps, their probability values will decrease. Through this fusion process, an interaction-sensitive area probability map is generated, which integrates multiple aspects of information such as texture and visibility, and more accurately reflects the possibility of each patch in the model becoming an interaction-sensitive area.

[0056] Step 10324: performing binary segmentation processing on the interaction-sensitive area probability map according to a preset probability threshold, and extracting a set of facets whose probability values exceed the probability threshold as the dynamic response area set.

[0057] In this embodiment, a probability threshold is preset, which is determined based on actual application requirements and expectations for interaction effects. The generated interaction-sensitive area probability map is subjected to binary segmentation processing, and the probability value of each facet in the probability map is compared with the preset probability threshold. Facets with probability values exceeding the probability threshold are extracted and formed into a set, which is the dynamic response area set. For example, if the preset probability threshold is 0.6, then the faces with probability values greater than 0.6 in the probability map are included in the dynamic response area set, and these faces will serve as the main areas corresponding to user interaction behaviors.

[0058] Step 1033: Calculate the user viewing angle direction in real time based on the spatial positioning data of the VR display terminal, and perform spatial matching processing on the user viewing angle direction and the dynamic response area set to generate the interactive response priority under the current viewing angle.

[0059] In this embodiment, the VR display terminal is equipped with a spatial positioning sensor that can acquire spatial positioning data of the user's viewing angle in real time. By analyzing and processing this data, the user's current viewing angle is calculated. For example, the sensor acquires information such as the rotation angle and position of the user's head. Through coordinate conversion and calculation, the user's viewing angle vector in three-dimensional space is determined.

[0060] The calculated user viewing direction is then spatially matched with the dynamic response area set. The spatial relationship between the center point of each patch in the dynamic response area set and the user viewing direction is analyzed. For example, parameters such as the angle and distance between each patch center point and the user viewing direction vector are calculated. Based on these parameters, the interaction response priority of each patch under the current viewing angle is determined. For example, patches with a smaller angle and a closer distance to the user viewing direction have a higher interaction response priority; conversely, patches with a larger angle and a farther distance have a lower interaction response priority. Through this spatial matching process, the interaction response priority under the current viewing angle is generated.

[0061] Preferably, step 1033 includes:

[0062] Step 10331: Acquire the spatial coordinate transformation matrix of the user viewing direction in real time, and calculate the spatial angle deviation value between each patch center point in the dynamic response area set and the user viewing direction in combination with the spatial coordinate transformation matrix.

[0063] In this embodiment, the spatial positioning system built into the VR display terminal continuously collects data related to the user's viewing angle in real time. This data is processed to generate a spatial coordinate transformation matrix, which contains the conversion relationship between the user's viewing angle's position and orientation in three-dimensional space. For example, if the user's head rotates or moves in the VR environment, the spatial positioning system will promptly capture these changes and convert them into a spatial coordinate transformation matrix.

[0064] For each facet in the dynamic response area set, determine the coordinates of its center point. Then, use the obtained spatial coordinate transformation matrix to convert the coordinates of the facet center point to the same coordinate system as the user's perspective. Through vector operations, calculate the angle between the facet center point and the user's perspective direction vector. This angle is the spatial angle deviation value. For example, the coordinates of the facet center point are represented as vector F1, and the user's perspective direction vector is represented as vector F2. The cosine value of the angle between the two vectors is calculated using the vector dot product formula, and then the specific value of the angle, that is, the spatial angle deviation value, is obtained through the inverse trigonometric function. In this way, the spatial angle difference between each facet and the user's perspective direction can be accurately measured.

[0065] Step 10332: Generate a viewing angle deviation weight for each facet based on the spatial angle deviation value, and perform weighted fusion processing on the viewing angle deviation weight and the texture complexity index of the corresponding facet to generate a facet interaction response weight.

[0066] In this embodiment, a perspective deviation weight is generated for each patch based on the calculated spatial angle deviation value. Generally speaking, the smaller the spatial angle deviation value, the closer the patch is to the user's viewing direction, and the higher its perspective deviation weight; conversely, the larger the spatial angle deviation value, the lower the perspective deviation weight. For example, a mapping relationship can be set such that when the spatial angle deviation value is between 0 and 30 degrees, the perspective deviation weight is 0.8; when it is between 30 and 60 degrees, the perspective deviation weight is 0.5; and when it is above 60 degrees, the perspective deviation weight is 0.2.

[0067] Next, the view deviation weight is weighted and fused with the texture complexity index of the corresponding patch. For example, the weight of the texture complexity index is w1, the weight of the view deviation weight is w2, and wl + w2 = 1. For example, for a patch with a high texture complexity index (set to 0.7) and a view deviation weight of 0.6, set w1 = 0.6 and w2 = 0.4, then the interaction response weight of the patch is 0.7 × 0.6 + 0.6 × 0.4. Through this weighted fusion method, the texture characteristics of the patch and its relationship with the user's perspective are comprehensively considered to generate a more reasonable patch interaction response weight.

[0068] Step 10333: Sort the facets in the dynamic response area set in descending order of the facet interaction response weights to generate the interaction response priority; wherein the interaction response priority is used as a basis for dynamically allocating rendering computing resources of the VR visualization model.

[0069] In this embodiment, all facets in the dynamic response region set are sorted according to their calculated facet interaction response weights. These facets are arranged from high to low, with faces with higher weights placed first and faces with lower weights placed last. For example, after sorting, a facet sequence is obtained, where the first facet has the highest interaction response weight and the last facet has the lowest interaction response weight. This sorted sequence is the interaction response priority.

[0070] Interaction response priority plays a crucial role in the rendering process of VR visualization models. Because VR display terminals have limited rendering computing resources, to ensure a better user experience, these resources are dynamically allocated based on interaction response priority. For patches with high interaction response priority, more computing resources are allocated for fine rendering, resulting in greater detail and better visuals. For patches with low interaction response priority, relatively fewer computing resources are allocated, conserving computing resources while ensuring overall visual quality.

[0071] Step 1034: Dynamically adjust the texture rendering order and geometric detail level of the target three-dimensional model based on the interaction response priority to generate a VR visualization model with multi-dimensional interaction properties.

[0072] In this embodiment, the texture rendering order and geometric level of detail of the target 3D model are dynamically adjusted based on the generated interaction response priority. Texture rendering is prioritized for patches with high interaction response priority, and higher-resolution texture maps are used to display rich details. For example, for parts of a textile cultural heritage model that are close to the user's perspective and have complex textures, high-resolution texture maps are prioritized, making these parts appear clearer and more vivid in the VR environment.

[0073] At the same time, for these high-priority patches, their geometric level of detail is increased. For example, the number of polygon subdivisions is increased to make the model surface smoother and present a more realistic geometry. For patches with a low priority for interactive response, texture rendering is deferred, and lower-resolution texture maps are used to reduce the amount of computation. At the geometric level of detail, the geometric structure is simplified and the number of polygons is reduced. For example, for background parts of the model that are less likely to attract user attention, simplified geometry and low-resolution textures are used.

[0074] Through this dynamic adjustment, combined with user interaction behavior and the current perspective, a VR visualization model with multi-dimensional interactive properties is generated. This model can not only respond in real time according to user operations and perspective changes, but also reasonably allocate resources in different interaction scenarios to provide a good visual experience.

[0075] Preferably, step 1034 includes:

[0076] Step 10341: Assign target resolution texture maps and multi-level geometric subdivision patches to the first target priority patch, and use a real-time lighting baking strategy to generate dynamic light and shadow output results for the first target priority patch; assign compressed texture maps and simplified geometric patches to the second target priority patch, and use a deferred rendering strategy to determine the texture mapping results for the second target priority patch; wherein, the priority of the first target priority patch is higher than that of the second target priority patch.

[0077] In this embodiment, based on the interaction response priority, facets are divided into first-target priority facets and second-target priority facets, with first-target priority facets having a higher priority. For first-target priority facets, a target-resolution texture map is assigned. This resolution is set based on the VR display terminal's capabilities and the required detail, ensuring that the model's texture details are fully displayed. For example, a high-resolution texture map is assigned to the key patterned portion of a textile cultural heritage model, ensuring that every detail of the pattern is clearly presented.

[0078] At the same time, multi-level geometric subdivision patches are provided for the first-priority patches, increasing the number of polygons to improve the smoothness and detail of the model surface. For example, multi-level subdivision is applied to raised areas of the model to make their surfaces smoother and more realistic. A real-time lighting baking strategy is used to calculate lighting effects in real time during the rendering process, generating dynamic lighting and shadow output. This allows the lighting and shadow effects of the first-priority patches to update in real time as the user's perspective changes and the ambient light changes, enhancing the realism of the model.

[0079] For the second-priority facets, given their lower priority and to conserve computational resources, compressed texture maps are assigned. These texture maps are compressed, resulting in a smaller data size, but still sufficient to meet basic visual requirements. For example, compressed texture maps are used for relatively unimportant background areas of the model. Simultaneously, simplified geometry is employed to reduce the number of polygons and thus the complexity of the model. A deferred rendering strategy is employed, whereby texture mapping is performed on the second-priority facets after processing of other high-priority facets is completed, and the resulting texture mapping result is determined.

[0080] Step 10342: Based on the dynamic light and shadow output results and the texture mapping results, combined with the real-time changes in the user interaction behavior type, dynamically switch the rendering modes of the first target priority patch and the second target priority patch to obtain the patch output results under different rendering modes.

[0081] In this embodiment, changes in the type of user interaction behavior are monitored in real time. For example, when the user's attention shifts from one part of the model to another, the type of user interaction behavior changes. Based on this change, the rendering mode of the first target priority patch and the texture mapping results of the second target priority patch are dynamically switched based on the dynamic lighting and shadow output results.

[0082] If the user's attention shifts to the area that originally belonged to the second target priority patch, and the interactive response priority of this area increases, then the patch in this area is switched from the rendering mode of the second target priority patch to the rendering mode of the first target priority patch. That is, from using compressed texture maps and simplified geometric patches, to using target resolution texture maps and multi-level geometric subdivision patches, and adopting a real-time lighting baking strategy. Conversely, if the user's attention to the area where the first target priority patch was originally located decreases and its interactive response priority decreases, then the patch in this area is switched to the rendering mode of the second target priority patch. By dynamically switching rendering modes, rendering resources can be reasonably allocated according to the real-time needs of users, and patch output results under different rendering modes can be obtained to provide a better interactive experience.

[0083] Step 10343: Synchronize the output results of the meshes in different rendering modes in the frame buffer to generate a VR visualization model with multi-dimensional interactive properties.

[0084] In this embodiment, the patch output results for different rendering modes are generated at different time points. To ensure that these results can be displayed smoothly and synchronously on the VR display terminal, frame buffer synchronization is required. First, the patch output results generated by all different rendering modes are collected into the frame buffer. The frame buffer is an area dedicated to storing image frame data.

[0085] The data in the frame buffer is then synchronized. Timestamps and synchronization signals are used to ensure temporal consistency across all mesh outputs. For example, each mesh output is timestamped and the data is sorted and aligned based on the timestamp, ensuring that meshes generated at the same time are correctly combined. Furthermore, mesh outputs from different rendering modes are spatially aligned to ensure their accurate placement in 3D space. For example, adjacent meshes from different rendering modes are checked for alignment to avoid gaps or overlap.

[0086] After frame buffer synchronization processing, the processed results are combined into a complete image frame to generate a VR visualization model with multi-dimensional interactive properties. This model can be presented to users in a smooth and accurate manner on the VR display terminal, providing a rich interactive experience.

[0087] In an optional embodiment, transmitting the VR visualization model to a VR display terminal for three-dimensional space rendering includes:

[0088] Step 1035: Obtain hardware performance parameters of the VR display terminal, and dynamically adjust the rendering frame rate and texture compression rate of the VR visualization model according to the hardware performance parameters.

[0089] In this embodiment, hardware performance parameters are first obtained from the VR display terminal. These parameters include information such as the performance indicators of the graphics processor, memory size, and bandwidth. For example, the model of the graphics processor determines the graphics complexity and frame rate limit it can handle, while the memory size affects the amount of data that can be stored and processed simultaneously.

[0090] Based on these hardware performance parameters, the rendering frame rate and texture compression rate of the VR visualization model are dynamically adjusted. If the graphics processor has strong performance, sufficient memory, and high bandwidth, the rendering frame rate is appropriately increased to provide a smoother visual experience, while the texture compression rate is reduced to make the textures clearer. For example, the rendering frame rate can be increased from 60 frames per second to 90 frames per second, and the texture compression rate can be reduced from 80% to 60%. Conversely, if the hardware performance is limited, the rendering frame rate is reduced and the texture compression rate is increased to ensure that the model can be rendered normally on the terminal. For example, the rendering frame rate can be adjusted to 45 frames per second and the texture compression rate can be increased to 90%. Through this dynamic adjustment, users can be provided with relatively good visual effects on VR display terminals with different hardware performance.

[0091] Step 1036: Divide the adjusted VR visualization model into multiple rendering task blocks, and assign an independent graphics computing thread to each rendering task block.

[0092] In this embodiment, the VR visualization model, after adjusting the rendering frame rate and texture compression rate, is segmented. Based on the model's geometric structure and texture distribution, it is divided into multiple relatively independent rendering task blocks. For example, for a textile cultural heritage model, the model can be divided into different regions, such as the pattern, edge, and background, into different rendering task blocks.

[0093] Assign a separate graphics computing thread to each rendering task block. A graphics computing thread is an execution unit within the VR display terminal's graphics processor, capable of processing graphics computing tasks in parallel. By assigning a separate thread to each task block, each task block can perform rendering calculations simultaneously, improving rendering efficiency. For example, there are four rendering task blocks: task block MA, task block MB, task block MC, and task block MD. Threads Th1, Th2, Th3, and Th4 are assigned to each of them, respectively. These threads can run simultaneously within the graphics processor, each processing its corresponding task block.

[0094] Step 1037: The rendering task blocks are transmitted in parallel to the graphics processor of the VR display terminal through an asynchronous transmission channel for distributed rendering calculation.

[0095] In this embodiment, the asynchronous transmission channel provided by the VR display terminal is used to transmit the divided rendering task blocks to the graphics processor in parallel. The asynchronous transmission channel allows data to be transmitted without blocking the main thread, improving transmission efficiency. Each rendering task block is sent to the graphics processor simultaneously via an independent transmission path.

[0096] For example, rendering task blocks MA, MB, MC, and MD simultaneously transmit data to the GPU via different transmission channels. After receiving these task blocks, the GPU distributes them to different computing cores for distributed rendering. This distributed rendering method fully utilizes the multi-core performance of the GPU, enabling rapid processing of large numbers of graphics computing tasks and accelerating rendering speed.

[0097] Step 1038: Perform time synchronization and spatial alignment processing on the image frames generated by the distributed rendering calculation to generate a continuous three-dimensional space rendering picture.

[0098] In this embodiment, distributed rendering computation generates multiple image frames, which require time synchronization and spatial alignment to produce a continuous 3D rendering image. First, time synchronization is performed to add a timestamp to each rendering task block. As the graphics processor generates the image frames, the generation time of each frame is recorded.

[0099] The GPU monitors the frame generation rate based on timestamps. If a task block is found to be generating frames too quickly or too slowly, the synchronization tolerance of the timestamp marker is dynamically adjusted according to pre-set rules. For example, if a task block generates frames faster than the others, the synchronization tolerance of its timestamp marker is appropriately increased to bring it closer to the frame generation time of the other task blocks. For image frames that exceed the synchronization tolerance, interpolation compensation is performed, and intermediate frames are generated through an algorithm to ensure the continuity of the frame sequence.

[0100] Next, spatial alignment is performed to extract the depth buffer data from the spatial position markers. The depth buffer data records the depth information of each pixel in three-dimensional space. Edge smoothing is performed on the depth buffers of adjacent image frames, analyzing the depth differences between object edges in adjacent frames. An algorithm is used to adjust the pixel depth values to eliminate spatial stitching gaps.

[0101] Finally, the processed image frames are input into the display buffer of the VR display terminal in timestamp order, generating a 3D rendering image without delay or spatial dislocation. Users can see a smooth and accurate rendering of the 3D model of the textile cultural heritage on the VR display terminal.

[0102] Preferably, step 1038 includes:

[0103] Step 10381: Add a timestamp and a spatial position mark to each rendering task block, and monitor the frame generation rate of the graphics processor.

[0104] In this embodiment, before transmitting a rendering task block to the GPU, a timestamp and spatial position marker are added to each task block. The timestamp records the time when the task block processing begins, while the spatial position marker contains the location information of the task block in the 3D model. For example, for the rendering task block MA, the timestamp t1 and spatial position coordinates (x1, y1, z1) are added to indicate that the task block processing begins at time t1, corresponding to the position (x1, y1, z1) in the 3D model.

[0105] The VR display terminal's system monitoring function also monitors the GPU's frame rate in real time. While processing rendering blocks, the GPU continuously generates image frames. The monitoring function counts the number of frames generated per unit time. For example, the number of frames generated may be counted every one second and recorded as the current frame rate. By monitoring the frame rate, the GPU's operating status can be monitored in real time, providing a basis for subsequent synchronization and adjustments.

[0106] Step 10382: Dynamically adjust the synchronization tolerance value of the timestamp mark according to the frame generation rate, and perform interpolation compensation processing on the image frames that exceed the synchronization tolerance value.

[0107] In this embodiment, the synchronization tolerance value of the timestamp is dynamically adjusted based on the monitored frame generation rate of the graphics processor. If the frame generation rate is fast, indicating that the frame generation speed of each task block is relatively uniform, the synchronization tolerance value can be appropriately reduced to ensure accurate synchronization of the image frames. If the frame generation rate is slow and the frame generation speeds of different task blocks vary significantly, the synchronization tolerance value can be appropriately increased to avoid synchronization issues caused by speed differences.

[0108] For example, when the frame generation rate stabilizes at 80 frames per second, the synchronization tolerance is set to 0.01 seconds. When the frame generation rate drops to 50 frames per second and the frame generation times of some task blocks fluctuate significantly, the synchronization tolerance is increased to 0.03 seconds. For image frames that exceed the synchronization tolerance, interpolation compensation is performed. For example, if the image frames generated by one task block exceed the synchronization tolerance value compared to the frames of other task blocks, linear interpolation or more complex algorithms are used to generate intermediate transition frames by analyzing the content and motion trends of the preceding and following frames, making the entire frame sequence smoother and more continuous.

[0109] Step 10383: Extract the depth buffer data in the spatial position mark, and perform edge smoothing on the depth buffers of adjacent image frames to eliminate spatial splicing gaps.

[0110] In this embodiment, depth buffer data is extracted from the spatial position marker of each rendering task block. The depth buffer data records the three-dimensional spatial depth information corresponding to each pixel in the image. For adjacent image frames, their depth buffer data is analyzed, especially the depth differences at the edges of objects.

[0111] For example, in two adjacent frames, the edge of an object has a depth value of d1 in the first frame and a depth value of d2 in the second frame. If d1 and d2 differ significantly, this may cause spatial stitching gaps. An edge smoothing algorithm adjusts the depth values of edge pixels in the depth buffer. For example, a weighted average method is used to adjust the depth values of edge pixels to a value between d1 and d2. This makes the edges of objects in adjacent frames appear more spatially continuous, eliminating spatial stitching gaps.

[0112] Step 10384: Input the processed image frames into the display buffer of the VR display terminal in the order of timestamps to generate a three-dimensional space rendering picture without delay and spatial dislocation.

[0113] In this embodiment, after timing synchronization and spatial alignment, the processed image frames are sequentially input into the display buffer of the VR display terminal in timestamp order. The display buffer is responsible for storing and displaying the image frames, and the user sees the contents of the display buffer on the VR display terminal screen.

[0114] Image frames are input in timestamp order, ensuring temporal continuity and avoiding delays. Furthermore, thanks to prior spatial alignment, the frames are spatially aligned, ensuring no misalignment. This allows users to view a continuous, smooth, and spatially accurate 3D rendering on the VR display, as if they were truly immersed in the virtual environment of textile cultural heritage.

[0115] In an independent embodiment, after transmitting the VR visualization model to the VR display terminal for three-dimensional space rendering, the method further includes any one of the following embodiments:

[0116] Embodiment 1: Real-time collection of user eye tracking data, identification of gaze point coordinates in the user eye tracking data; determination of frequently watched areas in the target three-dimensional model based on the gaze point coordinates, and extraction of geometric detail levels and texture resolution parameters of the frequently watched areas; dynamic improvement of polygonal face density of the frequently watched areas based on the geometric detail levels, simultaneous loading of lossless compressed texture maps based on the texture resolution parameters, to obtain processed frequently watched areas; gradual transition processing of the processed frequently watched areas with the remaining areas, to generate a three-dimensional rendering image with visual continuity fusion.

[0117] In this embodiment, the VR display terminal is equipped with a high-precision eye-tracking device that can collect real-time user eye tracking data. For example, using technologies such as infrared cameras, the device accurately captures the user's eye movement trajectory and gaze direction. The collected data is recorded in a time series format, with each time point corresponding to the user's eye position on the screen.

[0118] From the collected eye tracking data, a pre-set algorithm identifies the coordinates of the user's gaze point. These coordinates represent the actual location of the user's eyes in the VR scene. For example, at a certain moment, the user's gaze point coordinates are (x1, y1), which represents the user's gaze position in the VR display at that moment.

[0119] Based on the identified gaze point coordinates, they are mapped onto the target 3D model to identify frequently attended areas within the model. By analyzing the distribution of gaze point coordinates over time, the user's frequently observed areas are identified. For example, if the majority of gaze points within a minute are concentrated on a certain patterned area on a textile cultural heritage model, that area is identified as a frequently attended area.

[0120] For the frequently observed areas, the geometric level of detail and texture resolution parameters are extracted. The geometric level of detail can be determined by analyzing factors such as the number of polygons and the complexity of the patches in the area; the texture resolution parameter directly reflects the clarity of the texture image used in the area. For example, the geometric level of detail in a frequently observed area may be manifested as a high polygon density to reveal fine shapes, while the texture resolution parameter may be a high resolution to make the pattern clearly visible.

[0121] Based on the extracted geometric detail level, the polygon density of frequently visited areas is dynamically increased, which means increasing the number of polygons in the area to make the model surface smoother and more detailed. For example, the original polygon density of the area was n per square centimeter, and based on the requirements of the geometric detail level improvement, it was increased to 1.5n per square centimeter. At the same time, according to the texture resolution parameters, lossless compressed texture maps are loaded. Lossless compressed texture maps can reduce the amount of data and increase the loading speed without losing texture quality. For example, for frequently visited areas that originally used low-resolution textures, they are replaced with high-resolution lossless compressed texture maps to make the textures clearer and more vivid.

[0122] After obtaining the processed frequently watched area, a gradual transition is performed between it and the remaining area. By gradually changing the polygon density and texture clarity at the boundary between the frequently watched area and the remaining area, a smooth visual transition is achieved. For example, at the boundary, the polygon density gradually decreases from high density in the frequently watched area to low density in the remaining area, and the texture resolution also gradually transitions from high resolution to low resolution. This gradual transition process generates a 3D rendering with visual continuity and fusion. When users observe the model, they will not perceive a clear visual gap between the frequently watched area and the remaining area, thus improving the overall visual experience.

[0123] Example 2: Real-time monitoring of the light intensity and color temperature parameters of the environment in which the VR display terminal is located; real-time adjustment of the dynamic light and shadow effects in the VR visualization model according to the light intensity and the color temperature parameters to generate an environment-adaptive light map; extraction of the color difference value between the environment-adaptive light map and the original texture of the target three-dimensional model, and pixel-by-pixel color compensation processing to obtain compensated color data; matching and calibrating the compensated color data with the display color gamut of the VR display terminal to output a three-dimensional rendering picture with lighting consistency.

[0124] In this embodiment, light sensors are placed around the VR display terminal to monitor the ambient light intensity and color temperature parameters in real time. The light sensor can accurately measure the intensity of the ambient light, for example, by quantifying it in lux (1x); it can also accurately detect the color temperature of the ambient light, in Kelvin (K). For example, in an indoor environment during the day, the light intensity may be measured as 5001x and the color temperature as 5500K; while in an indoor environment at night, the light intensity may drop to 1001x and the color temperature may be 3000K.

[0125] Based on the acquired light intensity and color temperature parameters, the dynamic light and shadow effects in the VR visualization model are adjusted in real time. The system has a built-in light and shadow effect adjustment model, which adjusts the light reflection, shadow, and other effects on the model surface according to changes in light intensity and color temperature. For example, when the light intensity is high, the shadow area on the model surface is reduced, and the intensity of the reflected light is enhanced to make the model appear brighter; when the color temperature is low, the color tone of the model is adjusted to make it warmer to match the color characteristics of the ambient light. Through these adjustments, an environment-adaptive light map is generated, which can reflect the light and shadow effects that the model should present under the current ambient lighting conditions.

[0126] Next, the color difference between the environment-adapted lightmap and the original texture of the target 3D model is extracted. Using a color analysis algorithm, the color information of the lightmap and the original texture is compared pixel by pixel. For example, for each pixel, the numerical difference in the red, green, and blue color channels is analyzed. If the pixel value of the lightmap on the red channel is r1 and the pixel value of the original texture is r2, then the difference value Δr = r1 - r2 is calculated; similarly, the difference values Δg and Δb for the green and blue channels are calculated. After obtaining the color difference value for each pixel, pixel-by-pixel color compensation is performed. For example, for each pixel in the original texture, the corresponding difference value is added to its value on the three color channels, that is, the new red channel value is r2 + Δr, the green channel value is g2 + Δg, and the blue channel value is b2 + Δb, thereby obtaining the compensated color data.

[0127] Finally, the compensated color data is matched and calibrated with the display color gamut of the VR display terminal. Different VR display terminals have different display color gamut ranges. In order to ensure that the compensated colors can be accurately displayed on the terminal, color gamut matching calibration is required. Through the color gamut conversion algorithm, the compensated color data is mapped to the display color gamut range of the VR display terminal. For example, if one of the color values exceeds the upper limit of the display color gamut, it is adjusted to the upper limit value; if it is below the lower limit, it is adjusted to the lower limit value. After the above matching calibration, a three-dimensional rendering picture with lighting consistency is output, so that the model can present natural and realistic colors and light and shadow effects under different ambient lighting conditions, enhancing the user's sense of immersion.

[0128] Example 3: Acquire the user's displacement vector and viewing angle change rate in three-dimensional space in real time; predict the user's field of view focus area at the next moment based on the displacement vector and the viewing angle change rate; improve the density of geometric subdivision surfaces and the texture resolution level within the field of view focus area based on preset timing advance weights, and simultaneously perform progressive detail attenuation processing on non-focus areas and perform visual transition buffering to generate a focus-driven adaptive three-dimensional rendering picture.

[0129] In this embodiment, the VR display terminal uses built-in sensors to obtain the user's displacement vector and viewing angle change rate in three-dimensional space in real time. The displacement vector is obtained through devices such as accelerometers and gyroscopes, and can accurately measure the user's movement distance and direction in the X, Y, and Z directions. For example, if the user moves forward 1 meter in the VR environment, the displacement vector may be expressed as (0, 0, 1) (setting the Z axis to the forward direction). The viewing angle change rate is measured by devices such as gyroscopes to measure the speed at which the user's head rotates, such as the angle of rotation per second.

[0130] Based on the acquired displacement vector and viewing angle change rate, a prediction algorithm is used to predict the user's next focal area. This algorithm combines the user's current position, viewing angle direction, and movement and rotation speed to analyze the areas the user is likely to focus on. For example, if the user is currently moving rapidly in a certain direction and their viewing angle is also rotating in that direction, the algorithm predicts that the area within a certain range in front of that direction will be the focal area of the user's field of view at the next moment.

[0131] According to the preset timing advance weight, the density of geometric subdivision patches and the level of texture resolution in the focal area of the field of view are improved. The preset timing advance weight is a parameter set according to actual application needs and user experience, which is used to determine the amount of resources prepared in advance. For example, when the weight value is higher, the details of the focal area will be improved earlier and more fully. For the focal area of the field of view, the number of polygon subdivisions is increased, the density of geometric subdivision patches is improved, and the model surface is smoother and more detailed; at the same time, higher resolution texture maps are loaded to improve the texture resolution level. For example, an area with 100 polygons per square centimeter is increased to 200 polygons, and the texture resolution is increased from 512×512 pixels to 1024×1024 pixels.

[0132] Synchronously perform progressive detail attenuation processing on non-focus areas and perform visual transition buffering. For non-focus areas, gradually reduce the number of polygons, lower the level of geometric detail, and at the same time reduce the texture resolution. For example, reduce the polygon density in non-focus areas from 100 to 50 per square centimeter, and reduce the texture resolution from 512×512 pixels to 256×256 pixels. In order to avoid visual mutations, perform visual transition buffering. At the junction of the focus area and the non-focus area, a gradient algorithm is used to make the change of details gradually transition. For example, within a certain range, the number of polygons and texture resolution gradually decrease so that the user does not feel an obvious visual jump.

[0133] Through the above processing, a focus-driven adaptive 3D rendering image is generated. This image can dynamically adjust the details and performance of the model according to the user's real-time actions and potential areas of attention, improve rendering efficiency, and provide users with a smoother, more natural and attention-distribution-compliant visual experience, enhancing the user's immersion and interactivity in the VR environment.

[0134] It should be noted that, in the process of implementing the above technical solutions, those skilled in the art can optimize the matching of rendering parameters based on dynamic load balancing technology and GPU performance prediction model, such as integrating DLSS or FidelityFX framework to establish a frame rate-resolution-texture compression rate regression model, and combining OpenGL / Vulkan asynchronous computing pipeline to realize resource pre-allocation of high-priority patches.

[0135] For field of view focus prediction, the gaze heat map of the Tobii eye tracker and the LSTM displacement prediction algorithm can be integrated, and the sensor data can be processed in real time through Unity's Job System to construct a motion intention probability distribution map. In the lighting compensation link, Agisoft Metashape's photometric stereo vision algorithm is introduced to separate surface reflectivity and lighting components, combined with OpenCV's Retinex multi-scale enhancement to achieve cross-device color temperature alignment.

[0136] Distributed rendering synchronization can draw on the principles of NVIDIA VRSS technology, use a time warp algorithm to compensate for asynchronous frame motion deviations, and maintain buffer consistency through the Vulkan multi-GPU synchronization extension; texture compression can implement a block differentiation strategy based on the ASTC encoder, retaining the RGBA uncompressed format for highly sensitive areas, and enabling ASTC 8x8 efficient compression in low-priority areas.

[0137] Geometry optimization integrates tessellation shaders with the Quadric Error Metrics simplification algorithm, leveraging Houdini's retopology tools to ensure UV continuity. Eye tracking applications employ Varjo's bionic foveated rendering technology, dynamically adjusting the display panel's local refresh rate via FPGA, and integrating with the Nanite system to achieve instanced rendering of billions of polygons. Color calibration utilizes an X-Rite colorimeter to generate a 3DLUT, which is then compensated for ΔE2000 differences using the DaVinci color grading engine. Ultimately, end-to-end color matching is achieved using the HDMI 2.1 dynamic HDR protocol.

[0138] Those skilled in the art also know that the above-mentioned improvement scheme can be based on the modular integration of commercial engine plug-ins, focusing on foveated rendering and ASTC compression on the mobile side, and enhancing ray tracing and DLSS super sampling on the PC side, and collaboratively improving the visual fidelity and real-time interactive performance of the cultural heritage model through layered technology.

[0139] In summary, the embodiment of the present invention obtains a multi-view image acquisition sequence of textile cultural heritage, extracts geometric topological features and maps multi-resolution texture features to generate an initial three-dimensional model that integrates multiple features. This can comprehensively and meticulously restore the original appearance and complex texture of the textile cultural heritage. Dynamic geometric optimization of the initial three-dimensional model based on preset VR display parameters can improve the surface continuity of the target three-dimensional model and present a natural and smooth visual effect. The target three-dimensional model is dynamically matched with the VR interaction algorithm, and the generated VR visualization model has multi-dimensional interactive properties, thereby enabling the user to interact with the virtual model of the textile cultural heritage. Such a design can be combined with VR technology to improve the three-dimensional reconstruction and interactive intelligence of textile cultural heritage.

[0140] Further, Figure 2 The structural block diagram of the textile cultural heritage 3D reconstruction system 300 is shown, which includes: a memory 310 for storing program instructions and data; a processor 320 for coupling with the memory 310 and executing the instructions in the memory 310 to implement the above method.

[0141] Furthermore, a computer storage medium is provided, comprising instructions, which implement the above method when executed on a processor.

[0142] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A VR-based three-dimensional reconstruction method for textile cultural heritage, characterized in that: include: Acquiring a multi-view image acquisition sequence of the textile cultural heritage, performing geometric topological feature extraction processing on the multi-view image acquisition sequence to generate a three-dimensional point cloud geometric model, and performing multi-resolution texture feature mapping processing on the three-dimensional point cloud geometric model to generate an initial three-dimensional model that integrates the geometric topological features and the multi-resolution texture features; Performing dynamic geometric optimization processing on the initial three-dimensional model based on preset VR display parameters to generate a target three-dimensional model after surface continuity correction; The target three-dimensional model is dynamically matched with a preset VR interaction algorithm to generate a VR visualization model with multi-dimensional interactive properties, and the VR visualization model is transmitted to a VR display terminal for three-dimensional space rendering.

2. The method according to claim 1, wherein The multi-view image acquisition sequence includes original texture images and geometric structure images acquired under different lighting conditions, the multi-view image acquisition sequence is subjected to geometric topological feature extraction processing to generate a three-dimensional point cloud geometric model, and the three-dimensional point cloud geometric model is subjected to multi-resolution texture feature mapping processing to generate an initial three-dimensional model that fuses geometric topological features and multi-resolution texture features, including: Dividing the multi-view image acquisition sequence into a synchronous acquisition image set and an asynchronous acquisition image set, wherein the synchronous acquisition image set includes original texture images and geometric structure images from different viewpoints at the same timestamp; Performing stereo vision matching processing on the synchronously collected image set to extract depth difference features and edge continuity features of the synchronously collected image set; Constructing an initial three-dimensional point cloud space according to the depth difference feature, and performing topological alignment processing on the edge continuity feature and the initial three-dimensional point cloud space to generate a three-dimensional point cloud geometric model with spatial topological constraints; The asynchronously acquired image set is subjected to illumination consistency compensation processing to generate a compensated image sequence after illumination equalization, and the compensated image sequence is subjected to dynamic texture coordinate mapping processing with the three-dimensional point cloud geometric model to generate the initial three-dimensional model that integrates the geometric topological features and the multi-resolution texture features.

3. The method according to claim 2, wherein The dynamic geometric optimization processing of the initial three-dimensional model based on the preset VR display parameters to generate a target three-dimensional model after surface continuity correction includes: Obtaining a field of view angle parameter and a display resolution parameter of the VR display terminal, and dividing the polygonal facet density distribution intervals of the initial three-dimensional model according to the field of view angle parameter; Performing surface curvature analysis on the polygonal patch density distribution interval to extract geometric abrupt changes in the first curvature region and geometric gentle changes in the second curvature region; performing adaptive triangulation processing on the first curvature region according to the display resolution parameter and the geometric mutation feature to generate a subdivided facet set, and performing face merging processing on the second curvature region according to the display resolution parameter and the geometric smooth feature to generate a simplified facet set; The subdivided facet set and the simplified facet set are subjected to spatial continuity correction processing to eliminate geometric seam errors between adjacent facets and generate a target three-dimensional model after surface continuity correction.

4. The method according to claim 3, wherein The dynamically matching processing of the target three-dimensional model with a preset VR interaction algorithm to generate a VR visualization model with multi-dimensional interactive properties includes: Obtaining the user interaction behavior type configured in the VR interaction algorithm; Performing interaction-sensitive area recognition processing on the geometric topological features of the target three-dimensional model to generate a set of dynamic response areas corresponding to the user interaction behavior type; Calculating the user's viewing angle direction in real time based on the spatial positioning data of the VR display terminal, and performing spatial matching processing on the user's viewing angle direction and the dynamic response area set to generate an interactive response priority under the current viewing angle; The texture rendering order and geometric detail level of the target three-dimensional model are dynamically adjusted based on the interaction response priority to generate a VR visualization model with multi-dimensional interaction properties.

5. The method according to claim 4, wherein The performing interaction-sensitive area identification processing on the geometric topological features of the target three-dimensional model to generate a set of dynamic response areas corresponding to the user interaction behavior type includes: Extracting a texture complexity index and a geometric visibility index of each facet in the target three-dimensional model, and generating a facet texture weight distribution map according to the texture complexity index; Overlaying and analyzing the geometric visibility index with the preset user gaze hotspot area to generate a patch visibility heat distribution map; Performing dual-channel feature fusion processing on the patch texture weight distribution map and the patch visibility heat distribution map to generate an interactive sensitive area probability map; The interaction-sensitive area probability map is subjected to binary segmentation processing according to a preset probability threshold, and a facet set whose probability value exceeds the probability threshold is extracted as the dynamic response area set.

6. The method according to claim 5, wherein The performing spatial matching processing on the user viewing angle direction and the dynamic response area set to generate the interaction response priority under the current viewing angle includes: Acquire the spatial coordinate transformation matrix of the user's viewing angle in real time, and calculate the spatial angle deviation value between each patch center point in the dynamic response area set and the user's viewing angle in combination with the spatial coordinate transformation matrix; Generating a viewing angle deviation weight for each facet according to the spatial angle deviation value, and performing weighted fusion processing on the viewing angle deviation weight and the texture complexity index of the corresponding facet to generate a facet interaction response weight; The facets in the dynamic response area set are sorted in descending order according to the facet interaction response weights to generate the interaction response priority; wherein the interaction response priority is used as a basis for dynamically allocating rendering computing resources of the VR visualization model.

7. The method according to claim 4, wherein The method dynamically adjusting the texture rendering order and geometric detail level of the target three-dimensional model based on the interaction response priority to generate a VR visualization model with multi-dimensional interactive attributes includes: Allocate a target resolution texture map and a multi-level geometric subdivision patch to a first target priority patch, and use a real-time lighting baking strategy to generate a dynamic light and shadow output result for the first target priority patch; allocate a compressed texture map and a simplified geometric patch to a second target priority patch, and use a deferred rendering strategy to determine the texture mapping result for the second target priority patch; wherein the first target priority patch has a higher priority than the second target priority patch; According to the dynamic light and shadow output result and the texture mapping result, combined with the real-time change of the user interaction behavior type, dynamically switch the rendering mode of the first target priority patch and the second target priority patch to obtain patch output results under different rendering modes; The output results of the patches under different rendering modes are processed synchronously in the frame buffer to generate a VR visualization model with multi-dimensional interactive properties.

8. The method according to claim 1, wherein The step of transmitting the VR visualization model to a VR display terminal for three-dimensional space rendering includes: Obtaining hardware performance parameters of the VR display terminal, and dynamically adjusting the rendering frame rate and texture compression rate of the VR visualization model according to the hardware performance parameters; Divide the adjusted VR visualization model into multiple rendering task blocks and assign an independent graphics computing thread to each rendering task block; Transmitting the rendering task blocks in parallel to the graphics processor of the VR display terminal through an asynchronous transmission channel for distributed rendering calculation; Perform temporal synchronization and spatial alignment on the image frames generated by distributed rendering calculations to generate continuous 3D rendering images: The step of performing time synchronization and spatial alignment processing on the image frames generated by the distributed rendering calculation to generate a continuous three-dimensional space rendering image includes: Adding a timestamp and a spatial position mark to each rendering task block, and monitoring the frame generation rate of the graphics processor; Dynamically adjusting the synchronization tolerance value of the timestamp mark according to the frame generation rate, and performing interpolation compensation processing on the image frames exceeding the synchronization tolerance value; Extracting the depth buffer data in the spatial position mark, and performing edge smoothing on the depth buffers of adjacent image frames to eliminate spatial splicing gaps; The processed image frames are input into the display buffer of the VR display terminal in the order of timestamps to generate a three-dimensional space rendering picture without delay and spatial dislocation.

9. A three-dimensional reconstruction system for textile cultural heritage, characterized in that: include: Memory, used to store program instructions and data; A processor, coupled to a memory, and configured to execute instructions in the memory to implement the method according to any one of claims 1 to 8.

10. A computer storage medium, characterized in that The method comprises instructions which, when executed on a processor, implement the method according to any one of claims 1 to 8.

Images