Model reconstruction method, computer readable storage medium and electronic device

Abstract

The embodiment of the invention provides a model reconstruction method, a computer readable storage medium and an electronic device, and relates to the technical field of image processing and vehicles. The method comprises the steps that color correction processing is carried out on an image frame data set, a first frame data set is obtained, and the image frame data set is used for representing a multi-view image of a target object; performing illumination decomposition processing on the first frame data set to obtain a second frame data set; a texture image set is determined based on a semantic tag graph of the second frame data set, the semantic tag graph is used for distinguishing different areas of an initial model of the target object, and the texture image set comprises texture images corresponding to the different areas; and mapping the texture image set to the initial model to obtain a target model. The technical problems that a three-dimensional modeling method in the prior art is low in efficiency and high in cost, and the map content is disordered and fragmented are solved.

Description

Technical Field

[0001] This application relates to the fields of image processing and vehicle technology, and more specifically, to a model reconstruction method, a computer-readable storage medium, and an electronic device. Background Technology

[0002] In the field of intelligent connected vehicles, 3D model reconstruction technology is gradually becoming a key to improving the simulation effects of virtual reality, augmented reality, and autonomous driving systems. Traditional 3D modeling methods, such as manual modeling, 3D scanning, and image or video-based modeling, while each with its advantages, also exhibit limitations such as low efficiency, high cost, or inconsistent model quality. Especially in texture mapping, traditional methods often only achieve passive repair of texture coordinates (UV) maps, lacking active design capabilities. This results in messy and fragmented texture content, severely affecting the secondary use value and visual effects of the model.

[0003] There is currently no good solution to the above problems. Summary of the Invention

[0004] This application provides a model reconstruction method, a computer-readable storage medium, and an electronic device to at least solve the technical problems of low efficiency, high cost, and disorganized and fragmented texture content in related 3D modeling methods.

[0005] According to one aspect of the embodiments of this application, a model reconstruction method is provided, comprising: performing color correction processing on an image frame data set to obtain a first frame data set, wherein the image frame data set is used to represent multi-view images of a target object; performing illumination decomposition processing on the first frame data set to obtain a second frame data set; determining a texture image set based on a semantic label map of the second frame data set, wherein the semantic label map is used to distinguish different regions of an initial model of the target object, and the texture image set includes texture images corresponding to different regions; and mapping the texture image set to the initial model to obtain a target model.

[0006] Furthermore, the method also includes: acquiring a video file; performing frame extraction on the video file to obtain an initial image frame data set; evaluating the exposure quality of the initial image frame data set to obtain an evaluation result, wherein the evaluation result is used to reflect the exposure quality of the initial image frame data set; and updating the initial image frame data set based on the evaluation result to obtain an image frame data set.

[0007] Further, color correction processing is performed on the image frame data set to obtain the first frame data set, which includes: fusing the pixel values ​​in the image frame data set to generate a reference frame; and performing histogram matching on the image frame data set based on the color temperature information of the reference frame to obtain the first frame data set.

[0008] Further, the illumination decomposition process of the first frame data set to obtain the second frame data set includes: decomposing the first frame data set to obtain an illumination component set and a reflection component set; performing pose weighting processing on the illumination component set based on camera pose information to obtain a target illumination component set; filtering the reflection component set to obtain a target reflection component set; and obtaining the second frame data set based on the target illumination component set and the target reflection component set.

[0009] Further, based on the target illumination component set and the target reflection component set, the second frame data set is obtained by: determining the third frame data set based on the target illumination component set and the target reflection component set; determining the viewpoint contribution weight of the third frame data set, wherein the viewpoint contribution weight is used to represent the contribution weight of different viewpoints to texture generation; and performing texture weighted fusion processing on the third frame data set based on the viewpoint contribution weight to obtain the second frame data set.

[0010] Further, determining the texture image set based on the semantic label map of the second frame data set includes: performing semantic segmentation processing on the second frame data set to obtain a semantic label map; determining the processing priority of the target object based on the semantic label map, wherein the processing priority is used to represent the texture enhancement processing priority corresponding to different parts of the target object; and performing texture enhancement processing on the second frame data set based on the processing priority to obtain the texture image set.

[0011] Furthermore, mapping the texture image set to the initial model to obtain the target model includes: determining texture coordinate layout information based on the semantic label graph, wherein the texture coordinate layout information includes texture coordinates and vertex coordinates; using a non-rigid iterative nearest-neighbor algorithm to match the initial model with the texture coordinate layout information to obtain an intermediate model; and mapping the texture image set to the intermediate model to obtain the target model.

[0012] Furthermore, the method also includes: creating an initial model of the target object based on a set of image frame data.

[0013] According to another aspect of the embodiments of this application, a model reconstruction apparatus is also provided, comprising: a correction module, configured to perform color correction processing on an image frame data set to obtain a first frame data set, wherein the image frame data set is used to represent a multi-view image of a target object; a decomposition module, configured to perform illumination decomposition processing on the first frame data set to obtain a second frame data set; a determination module, configured to determine a texture image set based on a semantic label map of the second frame data set, wherein the semantic label map is used to distinguish different regions of the initial model of the target object, and the texture image set includes texture images corresponding to different regions; and a mapping module, configured to map the texture image set to the initial model to obtain a target model.

[0014] According to another aspect of the embodiments of this application, a vehicle is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0015] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0016] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the methods in various embodiments of this application.

[0017] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0018] In this embodiment, a first frame data set is obtained by performing color correction processing on the image frame data set, wherein the image frame data set is used to represent multi-view images of the target object; a second frame data set is obtained by performing illumination decomposition processing on the first frame data set; a texture image set is determined based on the semantic label map of the second frame data set, wherein the semantic label map is used to distinguish different regions of the initial model of the target object, and the texture image set includes texture images corresponding to different regions; the texture image set is mapped to the initial model to obtain the target model. Thus, precise color and illumination control is achieved, thereby realizing a high-quality, low-fragmentation texture mapping effect, and solving the technical problems of low efficiency, high cost, and disorganized, fragmented texture content in related 3D modeling methods. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0020] Figure 1 This is a flowchart of a model reconstruction method according to one embodiment of this application;

[0021] Figure 2 This is a flowchart of a full-link collaborative optimization method for reconstructing a lighting-geometry-semantic model according to one embodiment of this application;

[0022] Figure 3 This is a structural block diagram of a model reconstruction apparatus according to one embodiment of this application. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] Currently, 3D modeling can be summarized as follows: creating a model mesh, adding bones, adjusting skinning weights, and designing model animations. Several implementation methods exist:

[0026] Method 1: 3D software modeling: The above process is carried out manually using 3D design software, constructing the model through basic geometric shapes (such as cubes and spheres) and geometric operations (such as translation, rotation, stretching, and Boolean operations). This method starts almost from scratch and is extremely time-consuming and labor-intensive.

[0027] Method 2: 3D scanning tool modeling: Use a 3D scanner to capture the 3D coordinates and color information of the object's surface to generate a digital model. Then, use algorithms to drive the virtual digital character image. This method requires certain costs and technical support.

[0028] Method 3: Image or video-based modeling. This method involves capturing photos or videos of an object from multiple angles and using computer vision technology to reconstruct its 3D geometric structure. It is low-cost and highly automated. Currently, it supports importing single or multiple images, automatically cutting out the subject, generating a mesh model, and even supporting binding skeletons on the page before outputting the model.

[0029] In model reconstruction of UV maps, traditional methods can only guarantee an absolute one-to-one relationship between the current UV and the current model. However, the generated results suffer from serious problems such as messy and fragmented texture content, which is not conducive to the secondary use of UV maps. Traditional tools cannot automatically and reasonably divide UV islands and automatically perform UV mapping according to different parts of the model.

[0030] According to an embodiment of this application, a method embodiment for model reconstruction is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0031] This embodiment provides a model reconstruction method. Figure 1 This is a flowchart of a model reconstruction method according to one embodiment of this application, such as... Figure 1 As shown, the process includes the following steps:

[0032] Step S11: Perform color correction processing on the image frame data set to obtain the first frame data set, wherein the image frame data set is used to represent the multi-view image of the target object.

[0033] In this embodiment of the application, the image frame data set can be a series of image data in continuous shooting or video, and each frame image is a capture of a single perspective or instantaneous state of the target object. The image frame data set can also be multiple images captured by the camera device.

[0034] Color correction processing refers to adjusting the colors of an image using algorithms to ensure color consistency in images taken from different perspectives or at different times, thereby providing a unified color basis for 3D model reconstruction.

[0035] The first frame data set consists of a set of images with consistent colors obtained after color correction, which will be used for subsequent processing.

[0036] For example, this application can use dynamic framing technology to extract keyframes from videos or continuous images, ensuring that images from each viewpoint are effectively utilized. The quality screening stage removes image frames with abnormal exposure (too bright or too dark) by calculating histograms and analyzing exposure conditions. Through this process, high-quality, properly exposed images are retained for reconstruction, improving the color accuracy and consistency of the reconstructed model.

[0037] This ensures the quality of the input data, provides a good starting point for subsequent lighting and texture processing, and reduces model errors caused by uneven lighting.

[0038] Step S12: Perform illumination decomposition processing on the first frame data set to obtain the second frame data set.

[0039] In this embodiment of the application, the illumination decomposition process refers to decomposing the input image into illumination components (such as ambient light and light from the direction of the light source) and reflection components (the true color and texture of the object's surface) so as to better control and optimize the illumination effect in subsequent processing.

[0040] The second frame data set is an image data set containing illumination and reflection components after illumination decomposition processing, providing clearer texture information for subsequent processing.

[0041] For example, this application can process the first frame dataset using the trajectory-aware Retinex (Trajectory-aware Retinex with Integrated Noise Expungement, TRINEX) algorithm to separate the illumination and reflection information in the image. Illumination decomposition processing helps eliminate the impact of illumination changes under different viewpoints on model reconstruction, ensuring the clarity and accuracy of the model's surface texture.

[0042] This improves the lighting consistency and texture quality of the reconstructed model, enabling the generation of visually better models even in complex lighting environments.

[0043] Step S13: Determine the texture image set based on the semantic label map of the second frame data set, wherein the semantic label map is used to distinguish different regions of the initial model of the target object, and the texture image set includes texture images corresponding to different regions.

[0044] In this embodiment of the application, semantic labeling refers to using deep learning techniques, such as the Mask2Former model, to segment an image and assign semantic labels (such as face, clothing, background) to each pixel in order to distinguish different parts of the target object.

[0045] A texture image set refers to a set of texture images of a specific region formed by extracting texture information from each region of the model based on the semantic label map, so as to carry out targeted texture optimization and mapping in the future.

[0046] For example, this application can generate a semantic label map based on the image information in the second frame data set using semantic segmentation technology. By analyzing the label map, different regions of the initial model are determined, that is, UV islands are divided according to different parts of the object (such as a face or clothing), and the best texture image is selected for each region to form a texture image set.

[0047] UV islands, in the context of texture mapping for 3D models, refer to the shapes formed by flattening a continuous area of ​​the model's surface into two-dimensional space. UV mapping is a crucial technique in 3D modeling and animation, used to apply 2D texture images onto the surface of 3D models to enhance detail and realism. During UV mapping, the model's surface is divided into different blocks, which appear like islands in two-dimensional space, hence the name UV islands.

[0048] This enables an active design strategy in model reconstruction, allowing for refined texture processing of different regions and significantly improving the model's visual fidelity and texture coherence. Through semantic understanding, the reconstructed model is more realistic, the texture is no longer fragmented, and the reusability and aesthetics of UV mapping are improved.

[0049] Step S14: Map the texture image set to the initial model to obtain the target model.

[0050] In this embodiment, the initial model refers to the textureless mesh model generated in the early stage of 3D reconstruction. The target model refers to the 3D model with complete texture mapping, which is the final output of model reconstruction.

[0051] For example, this application can utilize the aforementioned generated set of texture images, combined with optimized UV mapping, to precisely fit the texture images onto corresponding areas of the initial model, generating a target model with high-quality texture and detail. This process considers texture optimization parameters for different regions, ensuring that the model achieves optimal visual effects in all detailed areas.

[0052] As a result, the final generated model not only highly replicates the target object in terms of geometric structure, but also achieves precise matching in terms of texture details, enhancing the model's realism and application value. Through the coordinated optimization of the above series of steps, the upper limit of model reconstruction has been raised, realizing a shift from "passive repair" to "active design."

[0053] In summary, the model reconstruction method presented in this application synergistically improves the efficiency and quality of 3D model reconstruction through color correction, illumination decomposition, semantic tag map generation, and texture image mapping. It not only solves the problems of low efficiency, high cost, and poor texture mapping quality in traditional solutions, but also achieves the goal of high-quality reconstruction at a low technical cost. The method in this application not only improves the automation level of model reconstruction but also significantly enhances the detail reproduction and visual effects of the model, providing a more efficient and refined model reconstruction solution for fields such as virtual reality, augmented reality, and simulation design.

[0054] Furthermore, this application standardizes the colors of multi-view images through color correction, ensuring the consistency of input data. Next, it employs illumination decomposition technology to separate the illumination and reflection components, mitigating the impact of illumination variations on model quality. Then, by generating semantic label maps, it intelligently distinguishes model regions and proactively designs texture image sets, overcoming the challenges of messy and fragmented textures. Finally, the optimized textures are mapped onto the model, achieving efficient and low-cost high-quality model reconstruction. This results in more coherent texture maps with richer details, significantly improving the efficiency of 3D modeling and the visual effect of the model. It solves the technical problems of low efficiency, high cost, and messy, fragmented texture content in related 3D modeling methods.

[0055] The above steps of this application involve color correction processing of the image frame data set to obtain a first frame data set, which represents a multi-view image of the target object; performing illumination decomposition processing on the first frame data set to obtain a second frame data set; determining a texture image set based on the semantic label map of the second frame data set, where the semantic label map is used to distinguish different regions of the initial model of the target object, and the texture image set includes texture images corresponding to different regions; and mapping the texture image set to the initial model to obtain the target model. This achieves precise color and illumination control, resulting in high-quality, low-fragmentation texture mapping technology, thereby solving the technical problems of low efficiency, high cost, and disorganized, fragmented texture content in related 3D modeling methods.

[0056] Optionally, the method may further include the following execution steps:

[0057] Step S101: Obtain the video file.

[0058] Step S102: Perform frame extraction on the video file to obtain an initial set of image frame data.

[0059] Step S103: Perform an exposure quality assessment on the initial image frame data set to obtain an assessment result, wherein the assessment result is used to reflect the exposure quality of the initial image frame data set.

[0060] Step S104: Update the initial image frame data set based on the evaluation results to obtain the image frame data set.

[0061] In this embodiment of the application, when acquiring the image frame data set, a video file is acquired first. The video file refers to a file that records a series of continuous images, which can be captured images of the target object from different angles or time periods.

[0062] As can be seen, the system receives a video file as input, which contains multi-view image data of the target object. The purpose is to acquire static image frames from a dynamic scene to facilitate subsequent 3D model reconstruction. Therefore, acquiring image data from a video source, compared to a traditional single still image, provides more perspectives and details, increasing the amount of information and possibilities for model reconstruction.

[0063] Then, the video file is processed by frame extraction to obtain an initial set of image frame data.

[0064] Frame extraction refers to selecting specific frames from a video as the image source for reconstruction. These frames are usually selected based on a certain strategy (such as time intervals, keyframes, etc.). The initial image frame data set is the collection of all image frames obtained after frame extraction.

[0065] As can be seen, video processing tools (such as the Fast Forward Video Audio Multiplexing and Demultiplexing Program, FFmpeg) are used to extract a series of still image frames from the video file, forming an initial image frame data set. This initial image frame data set will be used for subsequent processing to generate a 3D model. Therefore, frame extraction ensures that the reconstruction process can utilize key information from the video, while reducing the demand for computing resources and improving processing speed and efficiency.

[0066] Next, the exposure quality of the initial image frame data set was evaluated, and the evaluation results were obtained.

[0067] Exposure quality assessment is a process that analyzes features such as the image's histogram, brightness, and contrast to determine whether the image's exposure is suitable for 3D modeling. The assessment results reflect the exposure quality of the initial image frame dataset; that is, the assessment results reflect the exposure quality and suitability of the image frames.

[0068] As can be seen, the system automatically analyzes each extracted image frame, evaluating its exposure level, including whether it is overexposed or underexposed, thereby determining which frames are suitable sources for model reconstruction. Thus, through exposure quality assessment, images with appropriate exposure can be selected, avoiding reconstruction errors and model quality issues caused by lighting problems.

[0069] Finally, the initial image frame data set is updated based on the evaluation results to obtain the image frame data set.

[0070] Updating the initial image frame dataset means removing substandard images from the initial set based on the exposure quality assessment results. The image frame dataset is the filtered dataset containing only images that meet the exposure quality requirements.

[0071] As can be seen, based on the aforementioned evaluation results, the system removes image frames with abnormal exposure (such as overexposure or underexposure), leaving high-quality images suitable for model reconstruction, thus obtaining an image frame data set. This further improves the quality of the input data, ensuring that only the optimal images are used for reconstruction, thereby improving the accuracy and efficiency of 3D model reconstruction.

[0072] Optionally, in step S11, performing color correction processing on the image frame data set to obtain the first frame data set may include the following steps:

[0073] Step S111: The pixel values ​​in the image frame data set are fused to generate a reference frame.

[0074] Step S112: Histogram matching is performed on the image frame data set based on the color temperature information of the reference frame to obtain the first frame data set.

[0075] In this embodiment of the application, when color correction processing is performed on the image frame data set to obtain the first frame data set, the pixel values ​​in the image frame data set can be fused first to generate a reference frame.

[0076] Pixel value fusion refers to statistically processing the pixel values ​​at the same location in an image frame dataset, such as taking the median or average, to generate an image frame that integrates information from all viewpoints, i.e., a reference frame. The reference frame is the standard image frame obtained through fusion, representing the common characteristics of all frame data, and is used as a benchmark for subsequent color correction.

[0077] For example, by processing each frame of image data in the input image frame dataset and selecting the median pixel value at each location, a reference frame that integrates all viewpoint information is generated. This effectively reduces random noise and provides a stable and balanced image as a reference for subsequent operations.

[0078] Then, histogram matching is performed on the image frame data set based on the color temperature information of the reference frame to obtain the first frame data set.

[0079] Color temperature information refers to an index that describes the temperature of the light source in an image. Color temperature information determines the color tendency of an image, such as being cool (blue) or warm (yellow).

[0080] Histogram matching is an image processing technique that achieves color consistency by matching the grayscale distribution of one image with that of another.

[0081] For example, using the aforementioned generated reference frame as a benchmark, based on the gray-scale world assumption, the color temperature difference between the reference frame and the original image is calculated, and histogram matching technology is used to adjust the color of the original image to align it with the color characteristics of the reference frame, thereby generating the first frame data set.

[0082] This ensures color consistency between images taken from different perspectives or under different lighting conditions, eliminates color deviation caused by changes in lighting, provides color-balanced input for 3D model reconstruction, and enhances the visual realism of the model.

[0083] Optionally, in step S12, performing illumination decomposition processing on the first frame data set to obtain the second frame data set may include the following steps:

[0084] Step S121: Decompose the first frame data set to obtain the illumination component set and the reflection component set.

[0085] Step S122: Perform pose weighting processing on the illumination component set based on the camera pose information to obtain the target illumination component set.

[0086] Step S123: Filter the set of reflection components to obtain the set of target reflection components.

[0087] Step S124: Based on the target illumination component set and the target reflection component set, obtain the second frame data set.

[0088] In this embodiment of the application, when performing illumination decomposition processing on the first frame data set to obtain the second frame data set, the first frame data set can be decomposed to obtain an illumination component set and a reflection component set.

[0089] The illumination component set refers to the set of information representing the intensity and direction of light in the environment, separated from the first frame data set. The reflection component set refers to the set of information representing the material color and texture of the object's surface, separated from the first frame data set.

[0090] For example, the TRINEX algorithm is used to perform illumination decomposition on the first frame of color-corrected data, that is, to decompose each image into its illumination component (the influence of ambient light) and reflection component (the color and texture of the object itself). These two components constitute the basis of illumination changes and object surface information.

[0091] Therefore, the decomposition process eliminates the impact of lighting variations on subsequent model reconstruction and texture mapping, ensuring consistent texture representation even for images captured under different lighting conditions, thus improving the accuracy and stability of model reconstruction.

[0092] Then, the illumination component set is subjected to pose weighting processing based on the camera pose information to obtain the target illumination component set.

[0093] Among them, camera pose information refers to the position and rotation angle of the camera, which is used to describe the relative position and orientation of the camera with respect to the object.

[0094] The target illumination component set is a smoother and more consistent illumination component set obtained after pose weighting.

[0095] For example, camera pose information is introduced into the set of illumination components. By assigning weights, the illumination components of adjacent viewpoints can transition more smoothly, reducing abrupt changes in illumination caused by changes in viewpoint. Thus, pose-weighted processing ensures the consistency of illumination information across different viewpoints, avoiding illumination breaks or unnatural phenomena that may occur in model reconstruction, and enhancing the realism and visual effect of the model.

[0096] Next, the set of reflection components is filtered to obtain the target set of reflection components. For example, applying nonlocal mean filtering to the set of reflection components can effectively reduce noise in the image while keeping important edges and details unaffected, ensuring the sharpness and accuracy of the reflection components.

[0097] Therefore, filtering improves the purity of the reflection component, reduces artifacts in the model reconstruction process, makes the texture more delicate, and enhances the visual quality of the model.

[0098] Finally, based on the target illumination component set and the target reflection component set, a second frame data set is obtained. For example, based on the optimized target illumination component set and target reflection component set, a new, high-quality image set is generated through a reverse merging process, serving as improved input for subsequent 3D model reconstruction.

[0099] This ensures that the lighting and color information of the input image are in optimal condition during the data preparation stage before model reconstruction, providing a solid foundation for subsequent model reconstruction and texture mapping, and improving the overall quality and efficiency of model reconstruction.

[0100] Optionally, in step S124, obtaining the second frame data set based on the target illumination component set and the target reflection component set may include the following execution steps:

[0101] Step S1241: Determine the third frame data set based on the target illumination component set and the target reflection component set.

[0102] Step S1242: Determine the view contribution weight of the third frame data set, wherein the view contribution weight is used to represent the contribution weight of different viewpoints to texture generation.

[0103] Step S1243: Perform texture weighted fusion processing on the third frame data set based on the view contribution weight to obtain the second frame data set.

[0104] In this embodiment of the application, when the second frame data set is obtained based on the target illumination component set and the target reflection component set, the third frame data set can be determined based on the target illumination component set and the target reflection component set.

[0105] For example, the target illumination component set and target reflection component set obtained above are merged to recover the information of the original image. At this point, the dataset has already undergone illumination correction, making it more suitable for subsequent texture synthesis and mapping processing. Thus, the image set after illumination and reflection decomposition can effectively eliminate the influence of light changes during merging, ensuring stronger color consistency of images from different viewpoints and providing higher quality input for subsequent texture generation.

[0106] Then, the viewpoint contribution weights of the third frame dataset are determined. These viewpoint contribution weights represent the contribution of different viewpoints to texture generation; they are numerical values ​​used to measure the importance of each viewpoint image in texture generation, taking into account factors such as viewpoint angle, exposure level, and the accuracy of feature point matching.

[0107] As can be seen, by calculating the viewpoint contribution weight of each view image, we can determine which images are more representative in texture synthesis. The viewpoint contribution weight reflects the quality of a single image and its relative contribution to the texture of the reconstructed model. Therefore, determining the viewpoint contribution weight helps to utilize the information of each image more scientifically and rationally during multi-view texture fusion, avoiding model texture distortion caused by the limitations of a single viewpoint, and improving the accuracy and effect of texture synthesis.

[0108] Finally, the third frame dataset is subjected to texture-weighted fusion processing based on the viewpoint contribution weights to obtain the second frame dataset. Texture-weighted fusion processing refers to the process of weighting the images in the third frame dataset according to their viewpoint contribution weights to generate the final texture image.

[0109] As can be seen, based on the aforementioned determined viewpoint contribution weights, the images from each viewpoint in the third frame dataset are weighted and fused to generate a second frame dataset with richer texture details, which is used for the final texture mapping of the model. Therefore, weight-based fusion processing can maximize the use of information from high-quality viewpoint images and reduce the influence of low-quality viewpoint images, resulting in more coherent and higher-resolution texture images, significantly improving the model's visual effect and the integrity of the texture mapping.

[0110] Optionally, in step S13, determining the texture image set based on the semantic label map of the second frame data set may include the following execution steps:

[0111] Step S131: Perform semantic segmentation on the second frame data set to obtain a semantic label map.

[0112] Step S132: Determine the processing priority of the target object based on the semantic label map, wherein the processing priority is used to represent the texture enhancement processing priority corresponding to different parts of the target object.

[0113] Step S133: Perform texture enhancement processing on the second frame data set based on processing priority to obtain a texture image set.

[0114] In this embodiment of the application, when determining the texture image set based on the semantic label map of the second frame data set, semantic segmentation processing can be performed on the second frame data set to obtain the semantic label map. Semantic segmentation is an image analysis technique that uses a deep learning model to assign a category label to each pixel in the image, thereby distinguishing different semantic regions (such as faces, clothing, backgrounds, etc.) in the image.

[0115] For example, semantic segmentation algorithms, such as Mask2Former, are used to process each image in the second frame dataset to generate a detailed semantic label map. This step can accurately distinguish the various components of the target object, providing the necessary regional information for subsequent texture enhancement and mapping. Thus, through semantic segmentation, different regions of the model can be intelligently distinguished, setting the basis for texture processing in each region, avoiding the one-size-fits-all approach to the entire model found in traditional methods, thereby enhancing the precision and fidelity of model reconstruction.

[0116] Then, the processing priority of the target object is determined based on the semantic label map. The processing priority is used to represent the priority of texture enhancement processing for different parts of the target object. That is, the processing priority refers to the order of importance assigned to different parts of the target object during the texture enhancement process, so as to ensure that the texture quality of key areas is optimized first.

[0117] As can be seen, based on the semantic label map, the system automatically identifies and determines which regions should be prioritized for texture enhancement, such as high-detail areas like faces. This determination can be based on factors such as the complexity of the region and its importance to the visual effect. Therefore, setting processing priorities ensures efficient resource allocation, prioritizing the optimization of visually impactful regions, such as faces and clothing, thereby maximizing the model's visual effect and user experience with limited computational resources.

[0118] Finally, based on processing priority, the second frame dataset is subjected to texture enhancement processing to obtain a texture image set. Texture enhancement processing refers to strengthening the texture details in the image to make its mapping on the 3D model clearer and more natural, including operations such as sharpening, noise reduction, and contrast enhancement.

[0119] As can be seen, based on a defined processing priority, targeted texture enhancement processing is performed on the images in the second frame dataset. Texture enhancement processing includes, but is not limited to, applying additional filters and enhancement algorithms to high-priority regions to ensure that these regions have rich details and clear boundaries during texture mapping.

[0120] Therefore, by enhancing textures, the problems of messy and fragmented texture content in traditional solutions are solved. This allows the generated texture image set to maintain details while avoiding edge blurring and information loss, thus improving the texture quality and visual effect of the final model.

[0121] Optionally, in step S14, mapping the texture image set to the initial model to obtain the target model may include the following execution steps:

[0122] Step S141: Determine texture coordinate layout information based on semantic tag graph, wherein the texture coordinate layout information includes texture coordinates and vertex coordinates.

[0123] Step S142: The initial model is matched with the texture coordinate layout information using a non-rigid iterative nearest-point algorithm to obtain an intermediate model.

[0124] Step S143: Map the texture image set to the intermediate model to obtain the target model.

[0125] In this embodiment, when mapping a set of texture images to an initial model to obtain a target model, texture coordinate layout information can be determined based on a semantic label map. The texture coordinate layout information represents the location on the surface of the 3D model where each pixel in the texture image set is mapped. The texture coordinate layout information includes not only the texture coordinates themselves but also the vertex coordinates of the model, guiding how the texture is correctly applied to the model surface. The texture coordinate layout information includes both texture coordinates and vertex coordinates.

[0126] As can be seen, by using semantic tag maps to assign corresponding UV coordinates to each part of the 3D model, each semantic region (such as a face or clothing) has corresponding texture coordinates. This ensures that even with a complex model, it is possible to accurately determine where to apply which texture, avoiding arbitrariness and chaos in texture mapping. Therefore, by combining semantic tags with texture coordinate layout information, textures can be intelligently assigned to different parts of the model, improving the accuracy and consistency of texture mapping. The semantic-based design of this application allows for finer control, ensuring the consistency between texture maps and actual body parts, enhancing the realism and usability of the final model.

[0127] Then, a non-rigid iterative closest point (ICP) algorithm is used to match the initial model with the texture coordinate layout information to obtain an intermediate model. The non-rigid iterative closest point (ICP) algorithm is a point cloud registration algorithm suitable for handling the elastic deformation of non-rigid objects. Non-rigid ICP iteratively optimizes the model's position, orientation, and shape to ensure that the vertices on the model match the vertices in the texture coordinate layout information as closely as possible, thereby minimizing the distance between them.

[0128] The intermediate model is the version of the model where the vertex coordinates and texture coordinates are initially matched after adjustments using the non-rigid ICP algorithm.

[0129] As can be seen, the non-rigid ICP algorithm is used to adjust the shape and position of the initial model to better match the texture coordinate layout information, especially considering the possible non-rigid deformation of the model, such as the bending of human limbs. This application ensures that the distance between the model vertices and the UV template vertices is minimized through elastic deformation, while maintaining the smoothness of the deformation process and preventing the distortion of the model.

[0130] Therefore, the application of the non-rigid ICP algorithm greatly improves the accuracy of texture mapping, especially in the case of complex and non-rigid deformations. The non-rigid ICP algorithm ensures the reasonable distribution of textures on the model, reduces unnatural transitions at UV seams, and improves the visual effect and quality of model reconstruction.

[0131] Finally, the texture image set is mapped onto the intermediate model to obtain the target model. It can be seen that the optimized texture image set is applied to the intermediate model, completing the texture mapping process. Here, each pixel in the texture image set is accurately assigned to its corresponding position on the model surface, forming a target model with fine texture.

[0132] Therefore, the final generated target model not only possesses geometric accuracy but also boasts meticulously designed and optimized textures, presenting a highly realistic and visually appealing appearance. This precise texture mapping significantly improves the final rendering effect of the model, making it more suitable for various application scenarios, such as game development and virtual reality, while also simplifying the designer's subsequent editing work.

[0133] Optionally, the method may further include the following execution steps:

[0134] An initial model of the target object is created based on a set of image frame data.

[0135] In this embodiment, 3D reconstruction techniques such as structured light, stereo vision, photometric stereo, or deep learning can be used to analyze and reconstruct the geometry of the target object from an image frame dataset, creating a preliminary initial model without detailed texture and color information. This provides a geometric skeleton, laying the foundation for adding texture and color details in subsequent steps. The creation of the initial model reduces the direct need for high-quality texture mapping, making the processing flow more modular.

[0136] For example, based on the initial model, the model can be further refined, such as by adding more geometric details, like subtle bumps and smoothing, to more accurately reflect the true appearance of the target object. The model can also be further optimized, such as by improving the computational resource efficiency and visual effects of the initial model, including topology optimization and reducing the number of triangles, to ensure that the model is both refined and efficient; this is not limited to specific examples.

[0137] In summary, this application proposes a collaborative optimization scheme for the entire model reconstruction chain of illumination-geometry-semantics, which mainly includes two modules: TRINEX multi-view image preprocessing and semantic-aware UV extension pipeline (SemUVex) semantic UV generation and mapping.

[0138] The specific implementation steps are as follows:

[0139] S1, Dynamic Frame Segmentation and Quality Filtering: Use FFmpeg to extract frames from the input video data and extract video metadata; batch process all frames for exposure evaluation, quantify the quality of each frame through histogram analysis, and delete frames with abnormal exposure.

[0140] S2, Cross-frame exposure and color alignment: The median method is used to fuse multiple input images to generate a reference frame to reduce noise; the color temperature of the reference frame is transferred to other frames (based on the gray-world assumption) and histogram matching ensures color consistency between frames.

[0141] S3, TRINEX core processing:

[0142]

[0143] in, For light component, For the reflection component, For dynamic Gaussian kernel ( (Adapts to scene brightness) This represents the original image or frame at time t.

[0144] Smoothness of illumination components is constrained by camera pose:

[0145]

[0146]

[0147] in, Let i represent the corrected illumination component image of the i-th frame. Let be the camera rotation matrix for the j-th frame. Let be the camera rotation matrix for the i-th frame. To prevent small constants from being divided by zero, This represents the weight of the j-th frame. This represents the original image or frame of the j-th frame. Let represent the neighborhood set of the i-th frame image.

[0148] Noise suppression:

[0149] Apply nonlocal mean filtering to the reflection component R to preserve edges:

[0150]

[0151] in, Let C(x) represent the denoising result obtained after nonlocal mean filtering of the reflection component R at position x, where C(x) is the normalization coefficient. It represents the domain of an image, which is the set of all pixel locations in the image. The similarity weights between locations x and y are calculated using a function based on their distance; closer locations x and y receive a higher weight, and vice versa. The parameter h controls the rate of weight decay, determining the filter's effective range. R(y) represents the value of the original reflectance component image at location y.

[0152] S4, Multi-view Texture Blending: Multi-view texture weighted blending reduces the impact of uneven projection weight distribution from different viewpoints during automatic texture baking.

[0153] S5, Semantic Segmentation + Region-Based Texture Optimization: Mask2Former single-frame image segmentation generates semantic label maps. In each frame, the generated object's surface is divided into multiple UV islands based on its semantic labels. UV space priority is determined, and high-frequency detail enhancement is performed on details such as faces.

[0154] S6, Pre-generated UV-mapped object mesh (mesh) white model: Non-rigid ICP bridges the pre-generated UV and white model through elastic deformation, minimizing the distance between the white model vertices and the UV template vertices, while constraining deformation smoothness:

[0155]

[0156] in, Let T be the optimal non-rigid transformation function to be determined, where T is the non-rigid transformation function and N is the number of all vertices. For sliding term weights (to prevent excessive distortion). For white model vertices, This represents the i-th vertex in the white model. For UV template vertices, This represents the i-th vertex in the UV template. Let T represent the gradient of T, that is, the rate of change of T in space.

[0157] Furthermore, Figure 2 This is a flowchart of a full-link collaborative optimization method for light-geometry-semantic model reconstruction according to one embodiment of this application, such as... Figure 2 As shown, for the input multi-angle images / videos, dynamic frame segmentation and quality screening are first performed.

[0158] Dynamic framing: When using FFmpeg to extract frames, it's crucial to ensure consistent exposure across the video encoding. If the video is in RAW or Log format, each frame independently records sensor data, resulting in complete exposure information for each frame after extraction, suitable for post-processing compensation. Most videos (such as H.264 / HEVC) are compressed, potentially leading to inter-frame compression and metadata loss. Therefore, when using FFmpeg commands to extract frames, it's essential to preserve as much dynamic metadata as possible from the video.

[0159] Quality screening: Define a Python function `evaluate_exposure` to calculate the exposure score of all frames and select the most neutrally exposed image from the group as a reference. Delete frames with abnormal exposure (e.g., overexposed / underexposed by more than 20%).

[0160] Exposure score calculation:

[0161]

[0162] in, is the count of pixel value i in the grayscale histogram, and N is the total number of pixels in the image.

[0163] Then, cross-frame exposure and color alignment are performed.

[0164] Among them, cross-frame exposure: color difference caused by different shooting time / angle in multi-view images is synthesized by median method with adaptive reference frame. For each pixel position (x,y), the median of the red, green and blue (RGB) values ​​at that position in all images is taken to generate a local reference.

[0165]

[0166] in, This represents the pixel value of the composited reference frame at coordinates (x, y). This represents the pixel value at the same coordinates (x, y) for each image in a video frame sequence or image set. `mendian` is a statistical concept representing the median; here, the `mendian` function calculates the median of the RGB pixel values ​​for all input images at the specified coordinates (x, y).

[0167] The color temperature of the reference frame is transferred to other frames (based on the grayscale world). Histogram matching ensures color consistency between frames.

[0168] White balance gain calculation:

[0169]

[0170] in, The white balance gain for channel c. The average value of channel c in the reference frame. This is the mean value of channel c of the frame to be corrected.

[0171] The lookup table (LUT) for histogram matching is represented as follows:

[0172]

[0173]

[0174] in, This represents the mapping function in the lookup table, where CDF is the cumulative distribution function. Let v be the value of the cumulative distribution function of the source image at brightness v. Let be the value of the cumulative distribution function of the template image at brightness u.

[0175] Then, TRINEX core processing is performed.

[0176] The TRINEX algorithm is as follows:

[0177]

[0178] in, For light component, For the reflection component, For dynamic Gaussian kernel ( (Adapts to scene brightness) This represents the original image or frame at time t.

[0179] Pose-weighted Retinex, by fusing camera pose inverse constraints on the smoothness of illumination components:

[0180]

[0181]

[0182] in, Let i represent the corrected illumination component image of the i-th frame. Let be the camera rotation matrix for the j-th frame. Let be the camera rotation matrix for the i-th frame. To prevent small constants from being divided by zero, This represents the weight of the j-th frame. This represents the original image or frame of the j-th frame. Let represent the neighborhood set of the i-th frame image.

[0183] Noise suppression:

[0184] Apply nonlocal mean filtering to the reflection component R to preserve edges:

[0185]

[0186] in, Let C(x) represent the denoising result obtained after nonlocal mean filtering of the reflection component R at position x, where C(x) is the normalization coefficient. It represents the domain of an image, which is the set of all pixel locations in the image. The similarity weights between locations x and y are calculated using a function based on their distance; closer locations x and y receive a higher weight, and vice versa. The parameter h controls the rate of weight decay, determining the filter's effective range. R(y) represents the value of the original reflectance component image at location y.

[0187] Then, multi-view texture fusion is performed.

[0188] When generating texture maps from multi-view images, issues such as significant color differences of the same object under different viewpoints, breaks or misalignments at UV stitching, and smoothing of high-frequency information during averaging can occur due to differences in viewpoints, lighting variations, and occlusion. The following steps can be used to optimize this process:

[0189] (1) Calculate the contribution weight of each viewpoint .

[0190]

[0191] in, As the weight of the angle of view, ( (The angle between the current viewpoint and the surface normal). For consistent exposure weighting, ( This represents the average brightness of the frame. (This represents the global average brightness). The confidence score for the feature points is the proportion of feature points matched with inliers from SfM.

[0192] (2) Color consistency detection.

[0193] Color values ​​projected onto the same 3D point from different viewpoints Remove outliers:

[0194]

[0195] in, The threshold is dynamically adjusted to represent the color standard deviation.

[0196] (3) Texture weighted blending.

[0197] Achieve pixel-level blending:

[0198]

[0199] Where T(x, y) represents the color value of the fused image at pixel position (x, y), and valid represents all viewpoints that are considered valid and do not contain outliers during the color consistency detection process. This is a high-frequency detail layer extracted using the Laplacian operator (preserving texture sharpness). This is the detail enhancement factor (default 0.3).

[0200] (4) Poisson fusion.

[0201] Solve the Poisson equation at the UV seam boundary to minimize the gradient difference:

[0202]

[0203] Where Ω represents the seam area. and Let T and I represent the gradients of image T and image I, respectively. This represents the boundary of the seam area, that is, the boundary region where image T and image I overlap or connect. Indicates the color of the target boundary.

[0204] Next, semantic segmentation and regional texture optimization are performed.

[0205] Through semantic understanding, different object regions (such as faces / clothing) are segmented and differentiated texture enhancement strategies are applied to improve the visual fidelity of the reconstructed model. Texture maps partitioned by semantic labels, along with region optimization parameters, are used for subsequent UV pre-generation.

[0206] 2D segmentation + 3D label fusion: A pre-trained Mask2Former model is used to segment a single frame image. The 2D segmentation results are then back-projected onto a sparse point cloud, and the semantic label for each 3D point is determined through voting.

[0207]

[0208] in, The final semantic label representing the 3D point p. Let be the projection function of the camera at the k-th viewpoint, be the indicator function, and c represent the category, such as face, clothing, background, etc.

[0209] Verify cross-view consistency: Check whether the semantic labels of the same three-dimensional (3D) point are consistent across multiple views, and remove abnormal labels (such as misidentifying an arm as background from a certain view).

[0210] Region-specific optimization: For each semantic region Apply the corresponding filter :

[0211]

[0212] in, This represents the value at pixel position (x, y) in the optimized image. This represents the filtering operation performed on the value at pixel position (x, y) of image I.

[0213] The final processing results are shown in Table 1 below.

[0214] Table 1

[0215]

[0216] Finally, a white model of a UV-mapped object (Mesh) is pre-generated.

[0217] Pre-generated UV template: Create a UV layout in Blender that matches the semantic tags (e.g., centering facial UV islands), and export it as a .obj file containing vertex coordinates. and UV coordinates .

[0218] Export the .obj file from RealityScan (a 3D reconstruction software), with vertices as follows: However, the UV did not unfold.

[0219] Non-rigid ICP bridges the gap between pre-generated UVs and white models through elastic deformation, minimizing the distance between the vertices of the white model and the UV template, while simultaneously constraining deformation smoothness.

[0220]

[0221] in, Let T be the optimal non-rigid transformation function to be determined, where T is the non-rigid transformation function and N is the number of all vertices. For sliding term weights (to prevent excessive distortion). For white model vertices, This represents the i-th vertex in the white model. For UV template vertices, This represents the i-th vertex in the UV template. Let T represent the gradient of T, that is, the rate of change of T in space.

[0222] Through the above steps, the complete model is finally output.

[0223] Therefore, the importance of this application in achieving high-quality reproduction of 3D models of real-world objects with low manpower and technical costs, using output from ordinary cameras and mobile phones, is self-evident. This application can convert videos or photos into models while ensuring a high degree of fidelity in essential details such as meshes, colors, and textures, enabling rapid application in fields such as design and simulation.

[0224] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0225] According to an embodiment of this application, a model reconstruction apparatus is provided. It should be noted that the apparatus can be used to perform the above-described model reconstruction method.

[0226] Figure 3 This is a structural block diagram of a model reconstruction apparatus according to one embodiment of this application, such as... Figure 3 As shown, a model reconstruction device 300 is used as an example. The device includes: a correction module 301, used to perform color correction processing on an image frame data set to obtain a first frame data set, wherein the image frame data set is used to represent multi-view images of the target object; a decomposition module 302, used to perform illumination decomposition processing on the first frame data set to obtain a second frame data set; a determination module 303, used to determine a texture image set based on the semantic label map of the second frame data set, wherein the semantic label map is used to distinguish different regions of the initial model of the target object, and the texture image set includes texture images corresponding to different regions; and a mapping module 304, used to map the texture image set to the initial model to obtain the target model.

[0227] Embodiments of this application also provide a vehicle, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods described in various embodiments of this application when it runs.

[0228] Embodiments of this application also provide a computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0229] Embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0230] Embodiments of this application also provide an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the methods in various embodiments of this application.

[0231] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0232] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0233] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0234] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0235] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0236] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A model reconstruction method, characterized in that, The method includes: Color correction processing is performed on the image frame data set to obtain the first frame data set, wherein the image frame data set is used to represent the multi-view image of the target object; The first frame data set is subjected to illumination decomposition processing to obtain the second frame data set; A texture image set is determined based on the semantic label map of the second frame data set, wherein the semantic label map is used to distinguish different regions of the initial model of the target object, and the texture image set includes texture images corresponding to the different regions; The texture image set is mapped to the initial model to obtain the target model.

2. The method according to claim 1, characterized in that, The method further includes: Get the video file; The video file is subjected to frame extraction to obtain an initial set of image frame data; An exposure quality assessment is performed on the initial image frame data set to obtain an assessment result, wherein the assessment result is used to reflect the exposure quality of the initial image frame data set; The initial image frame data set is updated based on the evaluation results to obtain the image frame data set.

3. The method according to claim 1, characterized in that, The process of performing color correction on the image frame data set to obtain the first frame data set includes: The pixel values ​​in the image frame data set are fused to generate a reference frame; Histogram matching is performed on the image frame data set based on the color temperature information of the reference frame to obtain the first frame data set.

4. The method according to claim 1, characterized in that, The step of performing illumination decomposition processing on the first frame data set to obtain the second frame data set includes: The first frame data set is decomposed to obtain a set of illumination components and a set of reflection components; Based on the camera pose information, the set of illumination components is subjected to pose weighting processing to obtain the target set of illumination components. The set of reflection components is filtered to obtain the set of target reflection components; The second frame data set is obtained based on the target illumination component set and the target reflection component set.

5. The method according to claim 4, characterized in that, The process of obtaining the second frame data set based on the target illumination component set and the target reflection component set includes: The third frame data set is determined based on the target illumination component set and the target reflection component set; Determine the view contribution weight of the third frame data set, wherein the view contribution weight is used to represent the contribution weight of different viewpoints to texture generation; Based on the viewpoint contribution weight, the third frame data set is subjected to texture weighted fusion processing to obtain the second frame data set.

6. The method according to claim 1, characterized in that, The determination of the texture image set based on the semantic label map of the second frame data set includes: The second frame data set is subjected to semantic segmentation processing to obtain the semantic tag map; The processing priority of the target object is determined based on the semantic tag map, wherein the processing priority is used to represent the texture enhancement processing priority corresponding to different parts of the target object; Based on the processing priority, the second frame data set is subjected to texture enhancement processing to obtain the texture image set.

7. The method according to claim 1, characterized in that, The step of mapping the texture image set to the initial model to obtain the target model includes: Texture coordinate layout information is determined based on the semantic tag graph, wherein the texture coordinate layout information includes texture coordinates and vertex coordinates; A non-rigid iterative nearest-point algorithm is used to match the initial model with the texture coordinate layout information to obtain an intermediate model; The texture image set is mapped to the intermediate model to obtain the target model.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: The initial model of the target object is created based on the image frame data set.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the model reconstruction method according to any one of claims 1 to 8 when run on a computer or processor.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the model reconstruction method as described in any one of claims 1 to 8.

Images