Ship outfitting piece three-dimensional reconstruction method based on single RGB image

Abstract

The invention relates to the field of ship design and manufacturing, in particular to a ship outfitting piece three-dimensional reconstruction method based on a single RGB image. The method comprises the following steps: acquiring a multi-view data set of an outfitting piece, and performing geometry and texture optimization on an input single RGB image through a diffusion model to generate a multi-view image consistent with the features of the outfitting piece; performing feature extraction on the multi-view image by using a convolutional network to generate a geometric feature sequence and a texture feature sequence; reconstructing a three-dimensional model of the fitting-out part through a geometric mapping network in combination with three-dimensional geometric prior; and carrying out real-time geometric verification and detail optimization on the generated three-dimensional model. According to the method, through combination of deep learning and the geometric mapping network, the three-dimensional model of the ship outfitting piece is rapidly reconstructed from a single RGB image with high precision, the modeling efficiency and precision are remarkably improved, manual intervention is reduced, the precision requirement of ship design and manufacturing is met, and the practical value is high.

Description

Technical Field

[0001] The present invention relates to the field of ship design and manufacturing, and in particular to a three-dimensional reconstruction method of ship outfitting based on a single RGB image. Background Art

[0002] Ship outfitting is a vital component in the shipbuilding process, including ladders, hatch covers, pipes, valves, etc. These outfitting parts usually have complex geometric shapes and unique structural features. Traditional 3D modeling methods rely on multi-view images or physical scanning equipment, which is cumbersome and costly. In addition, manual modeling is easily affected by human factors, resulting in low modeling efficiency and uncertainty in accuracy.

[0003] With the rapid development of computer vision and deep learning technologies, image-based 3D reconstruction methods have gradually been introduced into the field of ship design and manufacturing. However, existing 3D reconstruction methods still have several technical bottlenecks, especially in how to accurately restore the 3D geometric structure and texture details of outfitting parts from a single RGB image. Due to the complex shapes, material properties and different lighting conditions of ship outfitting parts, existing technologies often cannot meet the requirements of industrial applications in terms of both accuracy and efficiency.

[0004] Most current 3D reconstruction methods rely on multi-view images or multiple image data sources. Although they can improve reconstruction accuracy, they still require a large amount of image data support and complex post-processing. Moreover, how to quickly and accurately generate 3D models that meet design requirements in ship design and manufacturing is still a technical problem that needs to be solved urgently.

[0005] Therefore, how to achieve high-precision 3D reconstruction of ship outfitting parts based on a single RGB image through deep learning and geometric mapping network technology has become an important challenge in the field of ship design and manufacturing. This invention is aimed at this technical problem and proposes a novel solution to achieve rapid and high-precision generation of a 3D outfitting part model that meets the design requirements from a single RGB image, significantly improving the efficiency and accuracy of ship design and manufacturing. Summary of the invention

[0006] The present invention provides a three-dimensional reconstruction method for ship outfitting parts based on a single RGB image, so as to solve the problem of how to achieve high-precision three-dimensional reconstruction of ship outfitting parts based on a single RGB image through deep learning and geometric mapping network to meet the precise requirements of ship design and manufacturing.

[0007] In order to solve the above technical problems, the present invention provides a method for 3D reconstruction of ship outfitting based on a single RGB image, comprising:

[0008] Obtain a multi-view dataset of outfitting parts, and use a diffusion model to optimize the geometry and texture of a single input RGB image to generate a multi-view image that is consistent with the features of the outfitting parts;

[0009] Performing convolutional network feature extraction on the multi-view images to generate a geometric feature sequence and a texture feature sequence of the outfitting parts;

[0010] Based on the geometric feature sequence and the texture feature sequence, combined with the three-dimensional geometric prior, the three-dimensional model of the outfitting part is reconstructed through a geometric mapping network;

[0011] The three-dimensional model is subjected to real-time geometric verification and detail optimization, and a high-precision three-dimensional model that can be directly applied to ship design and manufacturing is generated based on industrial specifications.

[0012] Furthermore, before the step of obtaining the multi-view data set of outfitting parts, the method further includes:

[0013] Collect 3D model data related to outfitting parts and construct a multi-view dataset for training.

[0014] Furthermore, the diffusion model generates an image sequence that conforms to the geometric features and texture features of the outfitting parts by fine-tuning the input RGB image.

[0015] Furthermore, during the training of the diffusion model, the training data set includes RGB images and standard coordinate maps of six viewing angles, and denoising, illumination correction and texture detail extraction are performed on each image.

[0016] Furthermore, the convolutional network feature extraction step specifically includes:

[0017] Performing a partition operation on the multi-view image to extract geometric features and texture features of each region;

[0018] Based on the feature extraction results, a geometric feature sequence and a texture feature sequence of the outfitting parts are generated.

[0019] Furthermore, when reconstructing the three-dimensional model, the geometric mapping network fits the model by combining geometric prior information of the outfitting parts.

[0020] Furthermore, the real-time geometry verification and detail optimization steps specifically include:

[0021] Correcting the geometric structure of the three-dimensional model and repairing inconsistent areas;

[0022] The texture map is calibrated.

[0023] Furthermore, the step of generating the high-precision three-dimensional model includes:

[0024] The proportions and materials of the 3D model are optimized and adjusted according to industrial design specifications.

[0025] Furthermore, the format conversion step of the three-dimensional model includes:

[0026] Convert the generated 3D model into a standardized 3D data file format.

[0027] Furthermore, a three-dimensional reconstruction system for ship outfitting based on a single RGB image comprises:

[0028] Data acquisition module, which acquires image data and feature information related to 3D reconstruction of ship outfitting parts from different sources;

[0029] A data preprocessing and feature extraction module preprocesses the RGB image and the standard coordinate map, and extracts the geometric feature sequence and texture feature sequence of the outfitting parts from the processed image;

[0030] The diffusion model training and fine-tuning module trains and fine-tunes the multi-view diffusion model based on the extracted feature data to generate an image sequence that conforms to the geometric and texture features of the outfitting parts, and uses the trained diffusion model to generate multi-view images of the outfitting parts;

[0031] A geometric mapping network module generates a three-dimensional model of the outfitting part based on the geometric feature sequence and the texture feature sequence combined with three-dimensional geometric priors;

[0032] Model optimization and detail enhancement module, which performs geometric repair, detail enhancement, and material and texture optimization on the generated 3D model;

[0033] Real-time verification and adaptation module, which verifies the generated 3D model in real time and adapts it according to the requirements of ship design and manufacturing;

[0034] The rendering and format conversion module performs high-precision rendering of the optimized 3D model and converts the rendered 3D model into a universal 3D data format;

[0035] The user interaction and visualization module provides users with an intuitive operation interface, supports interactive operations and result display, helps users view model details through a graphical interface and provide real-time feedback on optimization results, and supports multi-scenario display and model adjustment.

[0036] The key innovative features of the present invention include:

[0037] (1) Generate a high-precision 3D model based on a single RGB image. Traditional 3D reconstruction methods rely on multi-view images or complex sensor equipment. The innovation of the present invention is to generate a high-precision 3D outfit model based on a single RGB image by optimizing geometric and texture features through a diffusion model, thus solving the problem that a single image cannot directly generate a high-quality 3D model.

[0038] (2) Combination of deep learning and geometric mapping network. This invention innovatively combines the diffusion model in deep learning with the geometric mapping network, which can extract geometric features and texture features from RGB images, and combine them with three-dimensional geometric prior information to accurately reconstruct the three-dimensional structure of the outfitting parts, solving the problem of insufficient accuracy of traditional methods in modeling complex structures.

[0039] (3) Automated feature extraction and model optimization. The use of convolutional neural networks to extract features from images, as well as subsequent geometric verification and detail optimization, greatly improves the degree of automation, reduces manual intervention, and avoids the inefficiency and error problems caused by traditional manual modeling.

[0040] The following are its main beneficial effects:

[0041] (1) Improve modeling accuracy and efficiency. The present invention uses a diffusion model to optimize the geometry and texture of a single RGB image, generates a multi-view image that is consistent with the features of the outfitting parts, extracts geometric and texture features through a convolutional network, and finally reconstructs a high-precision three-dimensional model through a geometric mapping network. Compared with traditional methods that rely on multi-view images, the processing of a single RGB image greatly reduces the complexity and time cost of image acquisition while maintaining a high-precision modeling effect.

[0042] (2) Reduce manual intervention and improve automation. The present invention automatically extracts geometric and texture features through a deep learning model, reduces the tedious steps of manual modeling and image annotation, greatly improves the automation of 3D modeling, avoids errors that may occur in manual modeling, and improves the consistency and reliability of modeling.

[0043] (3) Optimize the ship design and manufacturing process. Through real-time verification and detail optimization modules, the three-dimensional model is adapted and adjusted in combination with industrial specifications. The present invention can directly generate a high-precision three-dimensional model that meets the ship design and manufacturing standards, greatly shortening the cycle from design to production and improving the design and production efficiency of ship outfitting parts. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic diagram of a process of a method for three-dimensional reconstruction of ship outfitting based on a single RGB image provided in an embodiment of the present application;

[0045] Figure 2A structural block diagram of a method for three-dimensional reconstruction of ship outfitting based on a single RGB image provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0047] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0048] Example 1: Reference Figure 1 , is a schematic flow chart of a method for 3D reconstruction of ship outfitting based on a single RGB image provided by an embodiment of the present invention. The process may at least include steps S100-S400:

[0049] S100, based on the multi-view dataset of outfitting parts, uses the diffusion model to optimize the geometry and texture of the input single RGB image to generate multi-view images consistent with the features of the outfitting parts.

[0050] S200, performing convolutional network feature extraction on the multi-view images to generate a geometric feature sequence and a texture feature sequence of the outfitting parts.

[0051] S300, based on the geometric feature sequence and the texture feature sequence, combined with the three-dimensional geometric prior, reconstruct the three-dimensional model of the outfitting part through a geometric mapping network.

[0052] S400, performing real-time geometric verification and detail optimization on the three-dimensional model, and generating a high-precision three-dimensional model based on industrial specifications that can be directly applied to ship design and manufacturing.

[0053] Step S100 at least includes steps S110-S130:

[0054] S110: Acquire RGB images and standard coordinate diagram data in the outfitting parts multi-view data set, pre-process the data, and generate an optimized training data set.

[0055] Specifically, a multi-view data set is obtained, including RGB images and standard coordinate diagram data of ship outfitting parts. The RGB images mainly cover six view angles: front, back, left, right, top, and bottom, and the standard coordinate diagram provides depth and feature distribution information corresponding to the geometric information of the outfitting parts.

[0056] First, obtain a multi-view dataset for outfitting training and parse the metadata of each image, including UID, outfitting name, classification, image type (RGB or standard coordinate map), and image perspective. Remove low-resolution or substandard images from the dataset to ensure the uniformity and integrity of the dataset image quality.

[0057] Furthermore, for RGB images, normalization processing is further performed, including brightness balance and color contrast adjustment, to eliminate the interference of illumination differences. For standard coordinate diagrams, the geometric features of outfitting parts are combined to improve the resolution through stretching and smoothing operations. Both types of image data are stored in a six-view arrangement to ensure the correspondence between features.

[0058] Furthermore, the generated training data is stored in the form of folders, and each outfitting data corresponds to a set of six-view images and its associated geometric information table. Furthermore, the preprocessed RGB images are matched with the standard coordinate diagram in sequence to form a unified data structure for subsequent diffusion model use.

[0059] S120: Based on the preprocessed data set, a diffusion model is used to fine-tune the input single RGB image to generate a diffusion model that conforms to the geometric features and texture features of the outfitting parts.

[0060] Specifically, based on the optimized training data set generated by S110, a multi-view diffusion model is used for training, so that the model can efficiently generate multi-view data of outfitting parts.

[0061] First, the general multi-view diffusion model is initialized and the RGB image and standard coordinate map training dataset generated by S110 are loaded. Furthermore, the initial weights of the model are adjusted by using the prior knowledge in the field of ship outfitting parts to make it initially adapt to the diversity of geometric features of outfitting parts.

[0062] Furthermore, based on the input single RGB image, the diffusion model generates a complete six-view image through the inverse process of gradually adding noise. In this process, the canonical coordinate map is used as an auxiliary input and participates in the fitting of geometric and texture features together with the RGB image. The model learns the mapping relationship between the RGB image and the canonical coordinate map through multiple iterations to capture the geometric structural characteristics of the outfitting parts.

[0063] Furthermore, after multiple rounds of training, the diffusion model is fine-tuned to further improve its accuracy and consistency in generating multi-view images of outfitting parts. The optimized model is able to generate multi-view image data that matches the geometry and texture features of real outfitting parts.

[0064] S130: Inputting the single RGB image to be processed into the diffusion model to generate a multi-view image sequence consistent with the geometric structure and texture of the outfitting part.

[0065] Specifically, based on the diffusion model trained by S120, a single RGB image is used as input to generate a six-view image sequence corresponding to the outfitting parts.

[0066] First, obtain a single RGB image of the outfitting part to be processed, and standardize it according to the preprocessing rules mentioned in S110, including brightness correction, noise filtering and contrast equalization. Ensure that the input image is consistent with the model training data distribution.

[0067] Furthermore, the preprocessed RGB image is input into the diffusion model, which extracts features from the input image layer by layer and combines the learned geometric and texture prior knowledge to generate a six-view image sequence of the outfitting parts. Each view image has accurate geometric information and detailed texture, which conforms to the characteristic pattern of the outfitting parts.

[0068] Furthermore, the geometric consistency and texture detail integrity of the generated six-view image sequence are verified. The geometric continuity between each view is ensured by matching with the standard coordinate map. After verification, the six-view image is output in sequence to provide input data for subsequent modules.

[0069] Connection Note: The optimized training dataset generated in S110 provides high-quality input for the diffusion model training in S120;

[0070] The diffusion model fine-tuned in S120 is used in S130 to generate multi-view images of outfitting parts;

[0071] The six-view image sequence generated by S130 is used as the convolution feature extraction input of S200 and is subsequently used for the analysis of geometric and texture features.

[0072] Through the close connection between the previous and next steps, the integrity of the outfitting parts 3D reconstruction process and the continuity of data flow are ensured.

[0073] Step S200 at least includes steps S210-S230:

[0074] S210: Performing partition feature extraction on the multi-view image to obtain preliminary geometric features and texture features of the outfitting parts.

[0075] Specifically, the multi-view image sequence generated in S130 is used as input to extract geometric features and texture features of the outfitting parts view by view.

[0076] First, the six-view image sequence generated by S130 (including RGB images of six viewing angles: front, back, left, right, top, and bottom) is input into the partition feature extraction module. The image is stored in the form of a two-dimensional matrix, and its pixel values ​​represent the texture details of the outfitting. Specifically, each image is partitioned into multiple small areas, and the features in each area represent the local geometry and texture information of the outfitting at that specific viewing angle.

[0077] Further, a convolution operation is applied to the partitioned regions to extract the geometric and texture features of each local region. The convolution kernel scans each region through a sliding window mechanism and outputs a feature vector to describe the local geometric details and texture distribution. Further, the extracted feature vector is encoded to form a preliminary set of geometric and texture features.

[0078] Furthermore, the local features of each view are aggregated to generate the overall geometric features and texture features of the view. The feature sets of each view are saved in a unified format for easy calling by subsequent modules. This step ensures the spatial continuity of feature extraction while retaining the overall structural information of the outfitting.

[0079] Connection Note: The set of geometric and texture features extracted in S210 will be used as input data of S220 for further aggregation analysis and sequence generation.

[0080] S220: Perform aggregation analysis on the preliminary features based on a convolutional network to generate a geometric feature sequence of the outfitting parts.

[0081] Specifically, the multi-view feature set extracted by S210 is further aggregated through a convolutional network to generate a geometric feature sequence of the outfitting parts.

[0082] First, the six-view feature set saved in S210 is loaded into the convolutional network, and the feature vector of each view is parsed to ensure that it contains the integrity of the geometric features of the outfitting parts. Specifically, the six-view feature set includes the geometric details and texture distribution of each area, which are represented in matrix form.

[0083] Based on the geometric symmetry of the outfitting parts and the correlation between views, the convolution aggregation operation is performed on the six-view feature set. The joint features across views are extracted through the convolution layer and the pooling layer to generate the global geometric feature sequence of the outfitting parts. This sequence represents the overall geometric shape of the outfitting parts and also has local detail information.

[0084] Furthermore, the aggregated global feature sequence is subjected to dimension reduction and optimization processing to ensure that the geometric feature sequence has low computational redundancy and maintains information integrity. Furthermore, the generated geometric feature sequence is formatted and output for use by the S230 module.

[0085] Connection Note: The geometric feature sequence generated by S220 will be directly used as the input of S230 for matching and normalizing texture features.

[0086] S230: performing matching calibration on the geometric feature sequence to generate a normalized texture feature sequence.

[0087] Specifically, the geometric feature sequence generated in S220 is used in combination with texture information for matching and calibration, and finally a normalized texture feature sequence is generated.

[0088] The geometric feature sequence generated in S220 is loaded and matched with the local texture features extracted in S210. Specifically, the correlation between the geometric features and the texture features is determined by calculating the Euclidean distance of the feature vectors to ensure the spatial correspondence between the two. Based on the global information of the geometric feature sequence, the spatial position and scale of the local texture features are calibrated to eliminate the deviation that may be introduced in the multi-view generation process. The calibrated texture features have a consistent spatial distribution and are accurately matched with the geometric features.

[0089] Furthermore, the calibrated texture features are normalized, including resolution unification and color adjustment, to generate a highly consistent and high-quality normalized texture feature sequence. Furthermore, the generated texture feature sequence is output in a standard format for direct call by subsequent modules.

[0090] Connection description: The normalized texture feature sequence generated by S230 will be directly used as the input data for the 3D reconstruction of the S300 module and will participate in the generation of the 3D model of the outfitting parts.

[0091] Connection summary: The preliminary feature set extracted in S210 provides input for the geometric feature aggregation in S220;

[0092] The geometric feature sequence generated by S220 provides a basis for the texture feature calibration of S230;

[0093] The normalized texture feature sequence output by S230 will be directly used in the subsequent 3D reconstruction process.

[0094] Through the gradual connection of the above modules, the integrity and continuity of the geometric and texture feature extraction of outfitting parts are ensured, providing high-quality data support for subsequent modules.

[0095] Step S300 at least includes steps S310-S330:

[0096] S310: Input the geometric feature sequence and the texture feature sequence into a three-dimensional geometric priori module to generate spatial geometric relationship parameters of the outfitting parts.

[0097] Specifically, the geometric feature sequence and the normalized texture feature sequence generated in S230 are loaded into the three-dimensional geometric prior module to derive the spatial geometric relationship parameters of the outfitting parts. The geometric feature sequence and texture feature sequence output by S230 are loaded to analyze the dimension and meaning of each feature vector. Specifically, the geometric feature sequence describes the global morphology of the outfitting parts, including vertex coordinates, boundary information, and local geometric changes; the texture feature sequence provides texture distribution information on the surface of the outfitting parts, including color gradients and detail textures.

[0098] Furthermore, the geometric prior model of ship outfitting is used to perform structural analysis on the input geometric feature sequence and texture feature sequence. The geometric prior model includes typical structural templates of outfitting parts (such as ladders, hatch covers, etc.), which provide references for generating the spatial geometric relationship of outfitting parts. Specifically, based on the prior information of a specific outfitting part category, its key point position, edge relationship and symmetry are derived.

[0099] Furthermore, the spatial geometric relationship parameters of the outfitting parts are generated based on the geometric prior model and the input features. The parameters include point cloud data, bounding box information, and geometric constraints to ensure that the subsequent 3D model generation process can maintain the geometric consistency and spatial rationality of the outfitting parts.

[0100] Connection description: The spatial geometric relationship parameters generated by S310 will be used as direct input to the S320 geometric mapping network to fit the preliminary three-dimensional model of the outfitting parts.

[0101] S320: Fitting spatial geometric relationship parameters based on a geometric mapping network to generate a preliminary three-dimensional model.

[0102] Specifically, the spatial geometric relationship parameters generated in S310 are input into the geometric mapping network, and a preliminary three-dimensional model of the outfitting part is generated through a fitting operation.

[0103] First, the geometric mapping network is initialized, and the spatial geometric relationship parameters generated in S310 are loaded as input. The structure of the geometric mapping network includes a feature mapping layer and a geometric construction layer, which are used to map the parameters to the three-dimensional space and generate preliminary geometric shapes.

[0104] The spatial geometric relationship parameters are mapped layer by layer to the three-dimensional space using the geometric mapping network. Specifically, the geometric feature sequence is used to define the global shape of the outfitting parts, and the texture feature sequence is used to refine the surface details. The geometric constraints of the input parameters are maintained during the mapping process to ensure that the generated preliminary three-dimensional model conforms to the actual structure of the outfitting parts.

[0105] Furthermore, after the mapping is completed, the geometric mapping network outputs a preliminary three-dimensional model of the outfitting part. The model is represented in the form of a point cloud or a mesh, including the geometric framework and preliminary texture details of the outfitting part, providing a basis for subsequent feature alignment and texture fusion.

[0106] Connection Note: The preliminary 3D model generated by S320 will be used as the input of S330 for feature alignment and texture fusion to further improve the accuracy and detail expression of the model.

[0107] S330: Perform feature alignment and texture fusion on the preliminary three-dimensional model to generate a high-precision three-dimensional model of the outfitting part.

[0108] Specifically, based on the preliminary 3D model generated by S320, the final high-precision outfitting 3D model is generated through feature alignment and texture fusion operations.

[0109] First, the geometric features of the preliminary 3D model are aligned with the normalized texture features generated in S230. Specifically, through point cloud matching and boundary alignment operations, the position and scale of the geometric features are adjusted to be consistent with the spatial distribution of the texture features.

[0110] Furthermore, based on the aligned geometric features, the texture features are mapped to the surface of the 3D model. The surface curvature and texture gradient distribution of the outfitting parts are considered during the texture mapping process to ensure that the fused texture details are consistent with the texture of the actual outfitting parts.

[0111] After alignment and fusion, a high-precision 3D model of the outfitting parts is generated. The model contains complete geometric shapes and detailed textures and is output in a standard 3D data format to support subsequent verification and industrial applications.

[0112] Connection Note: The high-precision 3D model generated by S330 will be directly used for real-time verification and industrial adaptation of S400 modules.

[0113] Connection summary: The spatial geometric relationship parameters generated by S310 provide input for the generation of the preliminary three-dimensional model in S320;

[0114] The preliminary 3D model output by S320 provides the basis for feature alignment and texture fusion in S330;

[0115] The high-precision 3D model generated by S330 serves as the direct input of the next module, supporting verification and application.

[0116] Through the gradual connection of the above modules, the integrity of the outfitting parts 3D reconstruction process and the efficient use of data are ensured, laying a solid foundation for subsequent industrial applications.

[0117] Step S400 at least includes steps S410-S430:

[0118] S410: Perform real-time geometric verification on the generated 3D model to check the consistency between its geometric structure and the input features.

[0119] Specifically, the high-precision three-dimensional model generated by S330 is used as input, its geometric structure is verified in real time, and the matching degree between the model and the previous input features (such as geometric feature sequence and texture feature sequence) is checked.

[0120] First, the high-precision 3D model outputted by S330 is obtained, and the normalized texture feature sequence generated in S230 and the geometric feature sequence generated in S320 are loaded as verification benchmark data. The 3D model is represented in the form of point cloud or mesh, and the feature data is stored in the form of structured feature vectors.

[0121] The geometric vertex information of the 3D model is compared with the geometric feature sequence generated by S320 through the geometric verification module. Specifically, the vertex distribution, boundary shape and structural symmetry of the model are verified to be consistent with the input features. If inconsistencies are found, the abnormal points are marked and their position information is recorded.

[0122] Furthermore, the texture distribution information of the three-dimensional model is checked with the normalized texture feature sequence in S230 to further verify the integrity of the texture mapping and the continuity of the distribution. By comparing the spatial coordinates and color gradient values ​​of each texture block, it is identified whether there is a texture break or distortion phenomenon.

[0123] Connection Note: The geometric consistency and texture mapping integrity verified in S410 provide key reference data for detail optimization in S420. The outlier location information and texture correction requirements will be directly used in the optimization step.

[0124] S420: Optimizing the details of the three-dimensional model according to industrial design specifications, including material adjustment and scaling.

[0125] Specifically, according to the verification data generated in S410 and the requirements of the industrial design specifications, the geometry and texture details of the three-dimensional model are optimized to meet the actual needs of ship design and manufacturing.

[0126] Based on the abnormal point location information recorded in S410, the geometric structure of the three-dimensional model is partially repaired. Specifically, the coordinates of the abnormal point and its neighborhood vertices are adjusted to ensure that the model surface is smooth and the structure is symmetrical. Furthermore, the boundary shape of the model is refined according to industrial specifications to improve its matching degree with the design requirements.

[0127] Regenerate and repair the broken or distorted areas found in the texture mapping. Specifically, use the normalized texture feature sequence to recalculate the texture mapping parameters, and perform local texture enhancement on the abnormal areas to ensure the continuity of texture distribution and richness of details.

[0128] According to the actual material characteristics and industrial design specifications of the ship's outfitting parts, the material properties of the 3D model are adjusted, including parameters such as reflectivity, glossiness and transparency. At the same time, the model is scaled according to the design specifications of the outfitting parts to ensure that its size meets the actual manufacturing requirements.

[0129] Connection description: The geometry repair and texture optimization results completed by S420 will be directly used as input to S430 to generate the final 3D data file.

[0130] S430: Perform final rendering and format conversion on the optimized three-dimensional model to generate a three-dimensional data file that meets the requirements of ship design.

[0131] Specifically, the three-dimensional model optimized in S420 is input into the rendering and format conversion module to generate a standard three-dimensional data file that can be used for ship design and manufacturing.

[0132] First, the optimized 3D model is loaded into the rendering module for full-view rendering. Specifically, the lighting parameters and background environment are set according to industrial specifications, the geometry and texture details of the model are rendered with high precision, and a concept map is generated that fully displays the model details.

[0133] Convert the rendered 3D model into a standardized data format (such as STEP, IGES or GLTF format) for use in different design and manufacturing software. Specifically, all the geometry, texture and material information of the model are retained during the format conversion process, and the data is compressed and optimized to reduce storage and transmission costs.

[0134] Furthermore, the standardized 3D data files and their corresponding verification reports and optimization records are packaged and stored, and then exported to the specified design platform. The generated files include geometry information tables, texture mapping tables, and material attribute tables, providing comprehensive reference data for ship designers and manufacturing engineers.

[0135] Connection Note: The standardized 3D data file output by S430 is the final result of the entire process and can be directly used in subsequent design, manufacturing and performance analysis.

[0136] Conclusion: The geometric consistency and texture distribution results verified in S410 provide key input for detail optimization in S420;

[0137] The optimized model completed by S420 provides high-quality basic data for rendering and format conversion of S430;

[0138] The 3D data file generated by S430, as the final output of the entire process, can be directly applied to ship design and manufacturing.

[0139] Through the gradual connection of the above modules, the integrity and data quality of the outfitting parts 3D reconstruction process are ensured, laying a solid foundation for subsequent industrial applications.

[0140] The key innovative features of the present invention include:

[0141] (1) Generate a high-precision 3D model based on a single RGB image. Traditional 3D reconstruction methods rely on multi-view images or complex sensor equipment. The innovation of the present invention is to generate a high-precision 3D outfit model based on a single RGB image by optimizing geometric and texture features through a diffusion model, thus solving the problem that a single image cannot directly generate a high-quality 3D model.

[0142] (2) Combination of deep learning and geometric mapping network. This invention innovatively combines the diffusion model in deep learning with the geometric mapping network, which can extract geometric features and texture features from RGB images, and combine them with three-dimensional geometric prior information to accurately reconstruct the three-dimensional structure of the outfitting parts, solving the problem of insufficient accuracy of traditional methods in modeling complex structures.

[0143] (3) Automated feature extraction and model optimization. The use of convolutional neural networks to extract features from images, as well as subsequent geometric verification and detail optimization, greatly improves the degree of automation, reduces manual intervention, and avoids the inefficiency and error problems caused by traditional manual modeling.

[0144] The following are its main beneficial effects:

[0145] (1) Improve modeling accuracy and efficiency. The present invention uses a diffusion model to optimize the geometry and texture of a single RGB image, generates a multi-view image that is consistent with the features of the outfitting parts, extracts geometric and texture features through a convolutional network, and finally reconstructs a high-precision three-dimensional model through a geometric mapping network. Compared with traditional methods that rely on multi-view images, the processing of a single RGB image greatly reduces the complexity and time cost of image acquisition while maintaining a high-precision modeling effect.

[0146] (2) Reduce manual intervention and improve automation. The present invention automatically extracts geometric and texture features through a deep learning model, reduces the tedious steps of manual modeling and image annotation, greatly improves the automation of 3D modeling, avoids errors that may occur in manual modeling, and improves the consistency and reliability of modeling.

[0147] (3) Optimize the ship design and manufacturing process. Through real-time verification and detail optimization modules, the three-dimensional model is adapted and adjusted in combination with industrial specifications. The present invention can directly generate a high-precision three-dimensional model that meets the ship design and manufacturing standards, greatly shortening the cycle from design to production and improving the design and production efficiency of ship outfitting parts.

[0148] Embodiment 2: Figure 2 FIG. 4 is a block diagram showing a method for 3D reconstruction of ship outfitting parts based on a single RGB image according to an embodiment of the present invention. Figure 2 As shown, the structure may include:

[0149] The data acquisition module 10 is used to obtain image data and feature information related to the three-dimensional reconstruction of ship outfitting parts from different sources. Specifically, it includes:

[0150] Get a single RGB image and the canonical coordinate map corresponding to the image.

[0151] According to the design requirements of ship outfitting parts, multi-view images are collected, covering data from six viewpoints: front, back, left, right, top and bottom.

[0152] Collect other environmental data (such as lighting conditions, image clarity, etc.) to provide support for subsequent feature extraction and reconstruction optimization.

[0153] The data preprocessing and feature extraction module 20 is responsible for preprocessing the collected RGB images and standard coordinate maps, including denoising, normalization, illumination correction, etc., to ensure the consistency and availability of the data. The specific functions are as follows:

[0154] Perform denoising on a single RGB image to eliminate noise and unnecessary interference in the image.

[0155] The canonical coordinate map is rectified to ensure that it is consistent with the geometric and texture features of the RGB image.

[0156] The geometric and texture features of the outfitting parts are extracted from the processed images, and preliminary geometric feature sequences and texture feature sequences are generated to provide input data for subsequent model generation.

[0157] The diffusion model training and fine-tuning module 30 trains and fine-tunes the multi-view diffusion model based on the extracted feature data. The specific functions are as follows:

[0158] Based on the multi-view dataset of outfitting parts, the diffusion model is trained by deep learning method to generate image sequences that conform to the geometric and texture characteristics of outfitting parts.

[0159] The trained diffusion model is used to generate multi-view images of the outfitting parts as the basic data for 3D reconstruction.

[0160] Fine-tune the model and optimize the generated results to ensure its adaptability and accuracy in specific outfitting categories.

[0161] The geometric mapping network module 40 is used to generate a three-dimensional model of the outfitting parts based on the extracted geometric feature sequence and texture feature sequence combined with the three-dimensional geometric prior. The specific functions are as follows:

[0162] The geometric and texture feature sequences obtained from the data preprocessing module are input into the geometric mapping network.

[0163] The input data is fitted based on the geometric mapping network to generate a preliminary three-dimensional model.

[0164] By combining geometric priors and deep learning algorithms, the accuracy and consistency of the generated model are improved to ensure that it conforms to the geometric structure of the actual outfitting parts.

[0165] The model optimization and detail enhancement module 50 is responsible for optimizing and enhancing the initially generated three-dimensional model to improve its accuracy and detail performance. The specific functions are as follows:

[0166] The preliminary 3D model is repaired and enhanced with details to ensure that its surface is smooth and has no obvious defects.

[0167] Adjust the material and detail texture of the model according to the material and actual application requirements of the ship's outfitting parts.

[0168] Calibrate texture maps and geometry to ensure your model is consistent and accurate across all viewing angles.

[0169] The real-time verification and adaptation module 60 is used to verify the generated three-dimensional model in real time and adapt it according to the requirements of ship design and manufacturing. The specific functions are as follows:

[0170] Perform real-time geometric verification on the generated 3D model to ensure that the model's geometric structure is consistent with the design requirements.

[0171] Based on the ship design specifications, adjust the model proportions and materials to ensure that they meet manufacturing requirements.

[0172] Verify the optimized 3D model to ensure its accurate application in the actual manufacturing process.

[0173] The rendering and format conversion module 70 is responsible for rendering the optimized 3D model and converting it into a standard format suitable for ship design and manufacturing. The specific functions are as follows:

[0174] Render the 3D model with high precision to show the final effect of the model, including details such as lighting, materials and textures.

[0175] Convert the rendered 3D model into common 3D data formats (such as STL, IGES, STEP, etc.) for use in ship design and manufacturing software.

[0176] Generate standardized data files and ensure their compatibility with ship design software to facilitate subsequent processing by engineers.

[0177] The user interaction and visualization module 80 is used to provide users with an intuitive operation interface and support interactive operations and result display. The specific functions are as follows:

[0178] Provides a graphical interface to display various perspectives and details of the 3D reconstructed model.

[0179] Allow users to adjust models and view details through the interface, and provide real-time feedback on optimization results.

[0180] It supports multi-scene display and interactive adjustment of models, helping designers better understand and operate 3D reconstruction models.

[0181] Beneficial effects of the embodiment: The three-dimensional reconstruction method of ship outfitting based on a single RGB image of the present invention can effectively realize the conversion from a single image to a high-precision three-dimensional model by combining deep learning technology, geometric mapping network, and geometric and texture feature extraction. Its beneficial effects include:

[0182] 1. High-precision reconstruction. By using comprehensive analysis of multi-view data, texture features and geometric features, a 3D model that is highly consistent with the actual outfitting parts can be generated, with accuracy meeting industrial design requirements.

[0183] 2. Reduce manual intervention. The present invention reduces the tedious steps of traditional manual modeling through automated deep learning training and fine-tuning, improves work efficiency and reduces human errors.

[0184] 3. Adapt to complex structures. The method of combining geometric mapping network with geometric prior can accurately reconstruct complex and asymmetric ship outfitting parts, which is suitable for the 3D reconstruction of various ship components.

[0185] 4. Support ship design and manufacturing. The generated 3D model complies with ship design and manufacturing specifications and can be directly used for subsequent design, verification and production, greatly improving the collaborative efficiency of design and production.

[0186] 5. Real-time and interactive. The system has real-time verification and interactive operation functions, which can quickly adapt to changes in design requirements, adjust and optimize 3D models in real time, and meet the needs of rapid design and production.

[0187] The present invention provides an efficient, accurate and operable solution for the design and manufacture of ship outfitting parts through innovative image processing and three-dimensional reconstruction technology.

[0188] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A method for 3D reconstruction of ship outfitting based on a single RGB image, characterized in that: The steps include: Obtain a multi-view dataset of outfitting parts, and use a diffusion model to optimize the geometry and texture of a single input RGB image to generate a multi-view image that is consistent with the features of the outfitting parts; Performing convolutional network feature extraction on the multi-view images to generate a geometric feature sequence and a texture feature sequence of the outfitting parts; Based on the geometric feature sequence and the texture feature sequence, combined with the three-dimensional geometric prior, the three-dimensional model of the outfitting part is reconstructed through a geometric mapping network; The three-dimensional model is subjected to real-time geometric verification and detail optimization, and a high-precision three-dimensional model that can be directly applied to ship design and manufacturing is generated based on industrial specifications.

2. The method for three-dimensional reconstruction of ship outfitting based on a single RGB image according to claim 1, characterized in that: Before the step of obtaining the multi-view data set of outfitting parts, the method further includes: Collect 3D model data related to outfitting parts and construct a multi-view dataset for training.

3. The method for three-dimensional reconstruction of ship outfitting based on a single RGB image according to claim 1, characterized in that: The diffusion model generates an image sequence that conforms to the geometric features and texture features of the outfitting parts by fine-tuning the input RGB image.

4. The method for three-dimensional reconstruction of ship outfitting based on a single RGB image according to claim 3, characterized in that: During the training process of the diffusion model, the training data set includes RGB images and standard coordinate maps of six viewing angles, and denoising, illumination correction and texture detail extraction are performed on each image.

5. The method for 3D reconstruction of ship outfitting based on a single RGB image according to claim 1, characterized in that: The convolutional network feature extraction step specifically includes: Performing a partition operation on the multi-view image to extract geometric features and texture features of each region; Based on the feature extraction results, a geometric feature sequence and a texture feature sequence of the outfitting parts are generated.

6. The method for 3D reconstruction of ship outfitting based on a single RGB image according to claim 1, characterized in that: When reconstructing the three-dimensional model, the geometric mapping network fits the model by combining the geometric prior information of the outfitting parts.

7. The method for three-dimensional reconstruction of ship outfitting based on a single RGB image according to claim 1, characterized in that: The real-time geometry verification and detail optimization steps specifically include: Correcting the geometric structure of the three-dimensional model and repairing inconsistent areas; The texture map is calibrated.

8. The method for 3D reconstruction of ship outfitting based on a single RGB image according to claim 1, characterized in that: The steps of generating the high-precision three-dimensional model include: The proportions and materials of the 3D model are optimized and adjusted according to industrial design specifications.

9. The method for 3D reconstruction of ship outfitting based on a single RGB image according to claim 1, characterized in that: The format conversion step of the three-dimensional model includes: Convert the generated 3D model into a standardized 3D data file format.

10. A three-dimensional reconstruction system for ship outfitting based on a single RGB image, characterized in that: include: Data acquisition module, which acquires image data and feature information related to 3D reconstruction of ship outfitting parts from different sources; A data preprocessing and feature extraction module preprocesses the RGB image and the standard coordinate map, and extracts the geometric feature sequence and texture feature sequence of the outfitting parts from the processed image; The diffusion model training and fine-tuning module trains and fine-tunes the multi-view diffusion model based on the extracted feature data to generate an image sequence that conforms to the geometric and texture features of the outfitting parts, and uses the trained diffusion model to generate multi-view images of the outfitting parts; A geometric mapping network module generates a three-dimensional model of the outfitting part based on the geometric feature sequence and the texture feature sequence combined with three-dimensional geometric priors; Model optimization and detail enhancement module, which performs geometric repair, detail enhancement, and material and texture optimization on the generated 3D model; Real-time verification and adaptation module, which verifies the generated 3D model in real time and adapts it according to the requirements of ship design and manufacturing; The rendering and format conversion module performs high-precision rendering of the optimized 3D model and converts the rendered 3D model into a universal 3D data format; The user interaction and visualization module provides users with an intuitive operation interface, supports interactive operations and result display, helps users view model details through a graphical interface and provide real-time feedback on optimization results, and supports multi-scenario display and model adjustment.

Images