A three-dimensional reconstruction method and system of nipple protection area based on super-resolution

Abstract

The present invention discloses a method and system for three-dimensional reconstruction of a nipple protection zone based on super-resolution, which relates to the field of image processing technology, including obtaining two-dimensional slice images of the nipple protection zone through medical imaging equipment; performing preliminary three-dimensional structural estimation on the two-dimensional slice images to obtain spatial geometric constraints; building a super-resolution network based on three-dimensional structure guidance for the two-dimensional images according to the geometric constraints; and generating a dense high-precision point cloud for three-dimensional reconstruction through multi-view stereo matching. The method described in the present invention integrates two-dimensional image enhancement with three-dimensional structural constraints through a three-dimensional structure-guided super-resolution method, thereby improving the accuracy of three-dimensional reconstruction of the nipple protection zone while also enhancing the detailed information of the image; adopting innovative structural consistency loss and depth constraint loss, it effectively improves the structural continuity and spatial consistency of the reconstructed model, solving the problem of three-dimensional structural distortion that may occur in traditional methods.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a super-resolution-based three-dimensional reconstruction method and system for a nipple protection area. Background Art

[0002] With the rapid development of medical imaging technology, 3D reconstruction has become an important adjunct to medical diagnosis, surgical planning, and postoperative evaluation, with growing application in breast surgery and plastic surgery. Currently, 3D reconstruction methods for medical images typically process traditional 2D medical images (such as MRI, CT, and ultrasound) or 2D photo sequences, then utilize computer vision techniques to construct 3D models. However, due to the delicate structures and subtle, complex anatomical features of medical images, conventional imaging equipment is limited by its resolution and scanning accuracy. This results in insufficient 2D image clarity and blurred tissue boundaries, compromising the accuracy and realism of 3D reconstruction. Therefore, to improve image clarity, some studies have attempted to enhance the resolution of 2D images using image super-resolution algorithms before performing 3D reconstruction. While these efforts have achieved some success, conventional super-resolution algorithms often neglect the guiding role of 3D spatial structural information, resulting in a lack of spatial consistency in the enhanced images and a tendency for structural distortion and loss of detail in the reconstructed 3D models.

[0003] Specifically, existing technologies generally fail to consider three-dimensional structural constraints when performing super-resolution enhancement on medical images, limiting themselves to pixel-level optimization of two-dimensional images. This makes it difficult to ensure spatial consistency after super-resolution. Furthermore, most existing three-dimensional reconstruction techniques employ dense reconstruction directly from two-dimensional images. However, due to a lack of effective prior guidance on three-dimensional structure, they are unable to fully leverage spatial geometric relationships to improve reconstruction accuracy. This results in insufficient refinement of the three-dimensional models of medical images, and unclear representation of local details (such as blood vessels, tissue edges, and skin texture). This can have adverse effects in medical diagnostic and surgical planning applications, making it difficult to meet the model accuracy and authenticity requirements of precision medicine. Summary of the Invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by the present invention is: how to solve the problems of insufficient image spatial consistency and low model accuracy in the process of three-dimensional reconstruction of medical images.

[0006] To solve the above technical problems, the present invention provides the following technical solution: a super-resolution-based three-dimensional reconstruction method for the nipple protection area, comprising:

[0007] Obtain two-dimensional slice images of the nipple protection area through medical imaging equipment;

[0008] Perform preliminary 3D structure estimation on 2D slice images to obtain spatial geometric constraints;

[0009] Build a super-resolution network based on 3D structure guidance for 2D images according to geometric constraints;

[0010] Dense and high-precision point clouds are generated through multi-view stereo matching for 3D reconstruction.

[0011] As a preferred embodiment of the super-resolution-based 3D reconstruction method for the nipple protection zone of the present invention, the 2D slice image comprises acquiring low-resolution images of the nipple protection zone from multiple angles and viewpoints to obtain a plurality of 2D image data sequences at different viewpoints;

[0012] The captured images are preprocessed, including denoising, blur removal and illumination normalization, preliminary calibration of camera internal and external parameters, and distortion correction; the processed low-resolution image sequence is output.

[0013] As a preferred embodiment of the super-resolution-based 3D reconstruction method for the nipple protection area of ​​the present invention, the preliminary 3D structure estimation includes extracting feature points of each viewpoint image using a pre-processed low-resolution image sequence and a structural motion recovery algorithm;

[0014] Obtain reliable matching points through feature matching method;

[0015] Based on reliable matching points, the intrinsic and extrinsic parameters of each camera are estimated, and a preliminary coarse-grained 3D sparse point cloud structure is obtained;

[0016] The initial coarse-grained 3D structure is used as spatial constraint information for super-resolution processing.

[0017] As a preferred solution of the super-resolution-based three-dimensional reconstruction method of the nipple protection area of ​​the present invention, the construction of a super-resolution network guided by three-dimensional structure includes:

[0018] Prepare network input, the input data is the preliminary coarse-grained 3D structure and preprocessed multi-view low-resolution 2D images;

[0019] Use shallow convolutional networks to extract basic image features from low-resolution image sequences and output basic two-dimensional feature maps ;Introduce preliminary 3D structural information into the network and fuse 3D structural features with 2D image features through the attention mechanism;

[0020] Apply spatial constraint information to the two-dimensional features and output a structure-guided fusion feature map ;

[0021] Fusion feature map It takes the image as input and uses the sub-pixel convolution method to perform efficient upsampling processing, outputting a two-dimensional high-quality image after super-resolution enhancement;

[0022] To train the super-resolution network, real high-resolution images of the nipple protection area are selected as the supervision data for the training set;

[0023] Define a hybrid loss function and use the Adam optimizer to iteratively optimize the super-resolution network parameters until convergence.

[0024] As a preferred solution of the super-resolution-based 3D reconstruction method for the nipple protection area of ​​the present invention, the hybrid loss function is defined to include content loss, perception loss, structural consistency loss, and depth constraint loss;

[0025] Specific definitions include,content loss is the measurement of the pixel-level difference of the image;

[0026] Perceptual loss is used to supervise the reconstruction of image semantic features;

[0027] The structural consistency loss is to constrain the continuity of pixel feature spaces at different perspectives through the consistency of corresponding points in three-dimensional space;

[0028] The depth-constrained loss provides additional constraints for network training by utilizing preliminary 3D structure depth information.

[0029] As a preferred embodiment of the super-resolution-based 3D reconstruction method for the nipple protection area of ​​the present invention, generating a dense high-precision point cloud includes performing multi-view stereo matching using camera parameters generated in the preliminary 3D structure estimation and high-resolution images enhanced by the super-resolution network;

[0030] The disparity estimation algorithm is used to calculate disparity and generate depth maps, and multi-view depth information is integrated to obtain a high-density 3D point cloud of the nipple protection area.

[0031] The obtained high-density point cloud is denoised, filtered and optimized to remove clutter and isolated points and improve the clarity and accuracy of the point cloud.

[0032] As a preferred embodiment of the super-resolution-based three-dimensional reconstruction method for the nipple protection zone of the present invention, the three-dimensional reconstruction includes meshing the optimized high-density point cloud using a Poisson reconstruction algorithm to generate a fine three-dimensional surface mesh model of the nipple protection zone;

[0033] Further optimization of the mesh model, including:

[0034] Mesh smoothing and removal of redundant vertices, repairing holes and irregular edges, and streamlining model structure;

[0035] For the high-resolution image obtained based on super-resolution enhancement, a multi-view texture mapping method is used to perform high-precision texture mapping optimization.

[0036] As a preferred solution of the super-resolution-based three-dimensional reconstruction system for the nipple protection zone of the present invention, it includes: a data acquisition module, a preliminary estimation module, a 3D guided super-resolution processing module, and a three-dimensional reconstruction module;

[0037] The data acquisition module includes a data acquisition unit and an image processing unit; it is used to collect multi-angle low-resolution two-dimensional images and depth scanning data around the nipple protection area and complete preliminary processing of the nipple protection area data;

[0038] The preliminary estimation module includes a feature extraction unit, a parameter estimation unit and an initial structure generation unit; it is used to estimate the preliminary three-dimensional structure and output coarse-grained three-dimensional spatial prior information;

[0039] The 3D guided super-resolution processing module is used to use the three-dimensional structural information as a guide to perform super-resolution processing on the two-dimensional image of the nipple protection area;

[0040] The three-dimensional reconstruction module includes a reconstruction unit and a texture mapping unit, which is used to realize high-precision three-dimensional reconstruction of the nipple protection area based on the super-resolution image and output a refined three-dimensional model.

[0041] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement a method for three-dimensional reconstruction of a nipple protection area based on super-resolution.

[0042] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a super-resolution-based three-dimensional reconstruction method for a nipple protection area.

[0043] The beneficial effects of the present invention are as follows: through a three-dimensional structure-guided super-resolution method, two-dimensional image enhancement and three-dimensional structural constraints are integrated, which not only improves the three-dimensional reconstruction accuracy of the nipple protection area, but also enhances the image detail information; by adopting innovative structural consistency loss and depth constraint loss, the structural continuity and spatial consistency of the reconstructed model are effectively improved, solving the three-dimensional structural distortion problem that may occur in traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 This is an overall flow chart of a super-resolution-based 3D reconstruction method for a nipple protection area provided in the first embodiment of the present invention. DETAILED DESCRIPTION

[0046] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the following detailed description of the specific embodiments of the present invention is given in conjunction with the accompanying drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in this field without creative work should fall within the scope of protection of the present invention.

[0047] Example 1, with reference to Figure 1 , as one embodiment of the present invention, provides a super-resolution-based three-dimensional reconstruction method for a nipple protection area, comprising:

[0048] S1: Obtain two-dimensional slice images of the nipple protection area through medical imaging equipment.

[0049] Furthermore, the two-dimensional slice image includes acquiring low-resolution images of the nipple protection area at multiple angles and viewpoints to obtain a plurality of two-dimensional image data sequences at different viewpoints;

[0050] Furthermore, the captured images are preprocessed, including denoising, blur removal and illumination normalization, preliminary calibration of camera internal and external parameters, and distortion correction; the processed low-resolution image sequence is output.

[0051] Furthermore, image denoising uses Gaussian filtering, median filtering and other image noise reduction methods to eliminate noise; illumination normalization is performed to balance the brightness and contrast differences between images from different perspectives.

[0052] Furthermore, the image correction algorithm is used to correct the distortion and make a preliminary estimate of the internal and external parameters of the camera; the preprocessed low-resolution image sequence is output to form a standardized data set.

[0053] It should be noted that the collected two-dimensional images are systematically preprocessed, including denoising using methods such as Gaussian filtering and non-local mean filtering to eliminate noise caused by equipment imaging or environmental factors; and image deblurring methods are used to remove motion blur and sampling interference, and histogram equalization or illumination normalization methods are used to standardize image brightness and contrast; the acquisition equipment is calibrated for internal and external parameters and image distortion is corrected based on known calibration plates or reference objects; finally, a processed standardized low-resolution two-dimensional image sequence is obtained, laying the foundation for the next step of preliminary three-dimensional structure estimation.

[0054] S2: Perform preliminary 3D structure estimation on 2D slice images to obtain spatial geometric constraints.

[0055] Furthermore, the preliminary three-dimensional structure estimation includes using a pre-processed low-resolution image sequence and a structural motion recovery algorithm to extract feature points of images from each viewing angle.

[0056] Extract the feature points of each image from the pre-processed low-resolution two-dimensional image sequence;

[0057] A feature matching algorithm is used to match feature points, and a random sampling consensus algorithm is used to eliminate mismatched points to ensure the reliability of feature point matching.

[0058] Based on reliable feature matching points, the structure-from-motion recovery algorithm is used to estimate the preliminary three-dimensional sparse point cloud structure of the nipple protection area and obtain the camera parameters at the same time.

[0059] The preliminary feature extraction formula is expressed as:

[0060] ;

[0061] Reliable matching points are obtained through feature matching method, where Represents the i-th low-resolution image, and the number of images is N. Indicates the SIFT feature extraction operation on the image. Indicates that from the image The feature point set extracted from the dataset contains information such as key point position, scale, direction and feature descriptor.

[0062] Based on reliable matching points, the intrinsic and extrinsic parameters of each camera view are estimated, and a preliminary coarse-grained three-dimensional sparse point cloud structure is obtained.

[0063] The obtained coarse-grained three-dimensional structure is used as the spatial constraint information of the next super-resolution network for use by the super-resolution network. The initial three-dimensional structure SfM construction formula is expressed as:

[0064] ;

[0065] in, Represents the preliminary three-dimensional sparse point cloud structure (SparsePointCloud), , Represents the camera internal and external parameters corresponding to the i-th image.

[0066] It should be noted that the Structure from Motion (SfM) algorithm, based on the initial internal and external parameter estimation method of feature matching relationship and camera calibration, gradually optimizes and calculates the camera pose and position information corresponding to each perspective image, and constructs the initial coarse-grained three-dimensional sparse point cloud structure of the nipple protection area; finally, the coarse-grained sparse three-dimensional structure information is output in the form of spatial structure prior, which is used to guide the next stage of two-dimensional super-resolution image enhancement based on deep learning network, further improving the three-dimensional reconstruction accuracy.

[0067] S3: Build a super-resolution network based on 3D structure guidance for 2D images according to geometric constraints.

[0068] Furthermore, the construction of a super-resolution network guided by three-dimensional structure includes preparing network input, where the input data is a preliminary coarse-grained three-dimensional structure and a preprocessed multi-view low-resolution two-dimensional image.

[0069] For input low-resolution images :

[0070] ;

[0071] in, represents the feature map extracted from the i-th low-resolution image; σ represents the nonlinear activation function; W and b represent the convolutional network parameters.

[0072] Preliminary three-dimensional structural information Projecting to a 2D view to obtain depth-guided features , specifically:

[0073] ;

[0074] Use shallow convolutional networks to extract basic image features from low-resolution image sequences and output basic two-dimensional feature maps The two-dimensional convolution operation Conv2D is used for local feature extraction, including convolution, activation, and feature normalization. The convolution kernel size is selected to be 3×3, with a stride of 1, to extract shallow basic features of the image.

[0075] The preliminary three-dimensional structure information is introduced into the network, and the three-dimensional structure features are fused with the two-dimensional image features through the attention mechanism.

[0076] Apply spatial constraint information to the two-dimensional features and output a structure-guided fusion feature map .

[0077] Fusion feature map It takes the image as input and uses the sub-pixel convolution method for efficient upsampling, outputting a two-dimensional high-quality image after super-resolution enhancement.

[0078] By fusing two-dimensional features and depth-guided features through the attention mechanism, the spatial attention weight formula is expressed as:

[0079] ;

[0080] Fusion feature representation:

[0081] ;

[0082] in, Represents the Sigmoid function; Represents convolution operation; [] represents feature concatenation; Represents element-wise multiplication.

[0083] First, feature channels are globally pooled, and channel attention weights are obtained using the FC layer. The fused features are then subjected to channel-weighted fusion to achieve 3D guidance. A spatial attention module is used to integrate spatial structures, and a 2D convolutional layer is used to obtain a spatial attention weight map. The correlation between 2D features and 3D structural information is calculated to highlight the feature weights of structural regions.

[0084] The formula for upsampling the feature map using sub-pixel convolution is expressed as:

[0085] ;

[0086] Get a high-resolution output image:

[0087] ;

[0088] Among them, pixelshuffle represents the sub-pixel rearrangement operation; Represents the fused feature map of the i-th frame perspective.

[0089] To train the super-resolution network, real high-resolution images of the nipple protection area are selected as supervision data for the training set.

[0090] Define a hybrid loss function and use the Adam optimizer to iteratively optimize the super-resolution network parameters until convergence.

[0091] The defined hybrid loss function includes content loss, perceptual loss, structural consistency loss and depth constraint loss.

[0092] The specific definition includes that content loss is a measure of the pixel-level difference in the image. The content loss formula is expressed as:

[0093] ;

[0094] in, Represents a real high-resolution image.

[0095] Perceptual loss is the supervised reconstruction of image semantic features. The perceptual loss formula is expressed as:

[0096] ;

[0097] in, Feature extraction function for training networks (such as VGG) for prediction.

[0098] The structural consistency loss is to constrain the spatial continuity of pixel features from different perspectives through the consistency of corresponding points in three-dimensional space. Based on the corresponding feature consistency constraint after projection into three-dimensional space, let P be the projection function of two-dimensional points to three-dimensional space, and the formula is expressed as:

[0099] ;

[0100] in, is the coordinate of the super-resolution result of the image projected into the three-dimensional space, express The corresponding three-dimensional space point from another perspective, is the projection function.

[0101] The depth constraint loss provides additional constraints for network training by utilizing preliminary 3D structure depth information. The depth information constraint formula based on the coarse-grained 3D structure is expressed as:

[0102] ;

[0103] in, represents the corresponding estimated depth map during super-resolution image reconstruction, Represents a coarse-grained depth prior map. represents the squared Euclidean distance (squared L2 norm), which represents the pixel-by-pixel difference between the estimated depth and the initial depth.

[0104] It should be noted that the surface reconstruction algorithm performs surface meshing on the point cloud to generate a fine three-dimensional surface mesh model of the nipple protection area; the generated mesh model is further refined and optimized, including mesh smoothing, redundant vertex deletion and mesh hole filling, to improve the continuity and smoothness of the three-dimensional model; finally, using the enhanced high-resolution two-dimensional image sequence, the fine texture information is mapped to the three-dimensional mesh surface through a multi-view texture mapping method to generate a fine three-dimensional reconstructed model of the nipple protection area with precise spatial structure and realistic texture, and the output is in a standard format for medical analysis or auxiliary diagnosis.

[0105] S4: Generate dense high-precision point clouds for 3D reconstruction through multi-view stereo matching.

[0106] Furthermore, generating a dense high-precision point cloud includes performing multi-view stereo matching using camera parameters generated in the preliminary three-dimensional structure estimation and high-resolution images enhanced by a super-resolution network;

[0107] The disparity estimation algorithm is used to calculate disparity and generate depth maps, and multi-view depth information is integrated to obtain a high-density 3D point cloud of the nipple protection area.

[0108] The obtained high-density point cloud is denoised, filtered and optimized to remove clutter and isolated points and improve the clarity and accuracy of the point cloud.

[0109] The three-dimensional reconstruction includes meshing the optimized high-density point cloud using a Poisson reconstruction algorithm to generate a fine three-dimensional surface mesh model of the nipple protection area;

[0110] Further optimization of the mesh model, including:

[0111] Mesh smoothing and removal of redundant vertices, repairing holes and irregular edges, and streamlining model structure;

[0112] For the high-resolution image obtained based on super-resolution enhancement, a multi-view texture mapping method is used to perform high-precision texture mapping optimization.

[0113] It should be noted that the depth constraint loss is introduced to calculate the depth supervision signal through the difference between the depth map predicted by the network and the initial coarse-grained depth structure, further enhancing the depth accuracy of the two-dimensional image super-resolution processing; finally, through iterative training of the Adam optimizer, the network weights are optimized with a hybrid loss function, so that the final output result of the network has high-precision spatial consistency, accurate depth contour and good perceptual quality, effectively improving the detail accuracy of the three-dimensional reconstruction results of the nipple protection area.

[0114] Example 2 is an embodiment of the present invention, which provides a super-resolution-based three-dimensional reconstruction method for the nipple protection area. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0115] First, the nipple protection area was systematically scanned using advanced medical imaging equipment, and low-resolution two-dimensional slice images were collected from different angles and viewpoints. In the experiment, a combination of high-resolution CT and MRI was used to ensure that the acquired images had a certain degree of spatial diversity and information redundancy during the acquisition process. After the acquired images were transmitted to the image processing terminal, they were first preprocessed. The preprocessing process included denoising the images using Gaussian filtering and non-local mean filtering algorithms, and normalizing the images using adaptive histogram equalization to eliminate brightness and contrast differences caused by different acquisition angles and equipment parameters. This step also includes preliminary calibration of the camera's intrinsic and extrinsic parameters using a preset calibration plate, thereby correcting image distortion and outputting a processed low-resolution image sequence.

[0116] Next, the preprocessed image sequence is subjected to a structure-from-motion (SfM) algorithm for preliminary 3D structure estimation. Specifically, the SIFT algorithm extracts key feature points from each viewpoint image, and the RANSAC method removes mismatched points. Based on the multi-view feature matching results, the intrinsic and extrinsic parameters of each camera are estimated to reconstruct a preliminary coarse-grained 3D sparse point cloud structure. This coarse-grained point cloud not only preserves the key spatial distribution characteristics of the nipple protection area but also provides spatial constraints for subsequent super-resolution processing.

[0117] After obtaining the preliminary 3D structure, it is fed into a 3D structure-guided super-resolution network along with the low-resolution image sequence. The network first uses a shallow convolutional network to extract the basic features of each low-resolution image, forming a basic 2D feature map. Simultaneously, the preliminary 3D structure is projected onto the 2D image plane to generate a corresponding depth-guided feature map. An attention mechanism module is designed to fuse the depth-guided features with the 2D image features. Using a spatial attention mechanism, different weights are assigned to features at different locations, effectively focusing on key structural regions within the image and outputting a structure-guided fused feature map.

[0118] Sub-pixel convolution is then used to upsample the fused feature map to generate a high-quality two-dimensional image after super-resolution enhancement. During the network training phase, real high-resolution images of the nipple protection area are used as supervision data. A hybrid loss function consisting of content loss, perceptual loss, structural consistency loss, and depth constraint loss is used, and the network parameters are iteratively optimized using the Adam optimizer until the loss converges.

[0119] After the super-resolution network training is complete, disparity calculation is performed using the network's output high-resolution image and the camera parameters obtained from the preliminary 3D structure estimation via multi-view stereo matching (MVS). The PatchMatch algorithm is used to generate depth maps corresponding to each viewpoint. The multi-view depth information is then fused to form a high-density 3D point cloud within the nipple protection zone. The generated point cloud data is then denoised using statistical filtering and radius filtering to remove isolated and noisy points, ensuring high accuracy and continuity. Finally, the point cloud data is meshed using the Poisson reconstruction algorithm to generate a detailed 3D surface mesh model of the nipple protection zone. After mesh reconstruction, the model is smoothed, redundant vertices are removed, and holes are repaired to ensure structural continuity and rich detail. Finally, the high-resolution image texture obtained through super-resolution enhancement is mapped onto the 3D model surface using a multi-view texture mapping method, resulting in a 3D reconstructed model of the nipple protection zone with realistic texture and precise structure.

[0120] The super-resolution-based 3D reconstruction method for the nipple protection zone demonstrated significant advantages across all key metrics. First, the image acquisition resolution remained between 150dpi and 170dpi for all test subjects, demonstrating the performance stability of the acquisition equipment. Regarding image preprocessing, the noise reduction rate for each test subject exceeded 85%, with a maximum of 89%, demonstrating that the preprocessing algorithm can effectively eliminate noise and blurring effects in the image, ensuring high-quality basic data for subsequent 3D reconstruction. This is particularly important for refined 3D reconstruction, as improvements in the quality of the original image are directly related to the accuracy of subsequent structural estimation and super-resolution reconstruction.

[0121] In terms of preliminary 3D reconstruction error, the error for each test object was less than 0.75 mm, with the lowest error being 0.65 mm. This demonstrates that by employing a structural motion recovery algorithm and feature matching strategy, this method can achieve high accuracy while maintaining speed when extracting preliminary 3D structures from low-resolution images, thus providing a reliable spatial constraint prior for the subsequent super-resolution network. The super-resolution improvement factor was 3.0 times for all test objects, indicating that after processing with the super-resolution network, the 2D image resolution was significantly improved, resulting in a significant improvement in the rendering of image details.

[0122] Point cloud density, an important parameter for measuring 3D reconstruction quality, showed high values ​​in all test subjects, ranging from 2.10 points / mm³ to 2.50 points / mm³, indicating that this method can generate high-density, high-precision 3D point clouds during multi-view stereo matching and parallax fusion. This high point cloud density facilitates subsequent mesh reconstruction and texture mapping, giving the final 3D model a higher level of detail restoration capabilities. At the same time, the texture mapping accuracy remained above 92% in all test subjects, reaching a maximum of 95%. This fully demonstrates that the high-resolution images generated based on super-resolution enhancement can accurately reflect the actual tissue texture and details during the texture mapping process, enhancing the realism and diagnostic value of the model.

[0123] Example 3 is an embodiment of the present invention, which provides a super-resolution-based three-dimensional reconstruction system for the nipple protection area, including a data acquisition module, a preliminary estimation module, a 3D-guided super-resolution processing module, and a three-dimensional reconstruction module.

[0124] The data acquisition module includes a data acquisition unit and an image processing unit; it is used to collect multi-angle low-resolution two-dimensional images and depth scanning data around the nipple protection area and complete preliminary processing of the nipple protection area data.

[0125] The preliminary estimation module includes a feature extraction unit, a parameter estimation unit and an initial structure generation unit; it is used to estimate the preliminary three-dimensional structure and output coarse-grained three-dimensional space prior information.

[0126] The 3D guided super-resolution processing module is used to perform super-resolution processing on the two-dimensional image of the nipple protection area using the three-dimensional structural information as a guide.

[0127] The three-dimensional reconstruction module includes a reconstruction unit and a texture mapping unit, which is used to realize high-precision three-dimensional reconstruction of the nipple protection area based on the super-resolution image and output a refined three-dimensional model.

[0128] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0129] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0130] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting, or processing it in another suitable manner as necessary, and then storing it in a computer memory.

[0131] It should be understood that various aspects of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one or a combination of the following technologies known in the art may be used: a discrete logic circuit having logic gates for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gates, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc. It should be noted that the above embodiments are intended only to illustrate the technical solutions of the present invention and are not intended to be limiting. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art will understand that modifications or equivalent substitutions may be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and such modifications are intended to be encompassed by the claims of the present invention.

[0132] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A super-resolution-based 3D reconstruction method for the nipple protection area, characterized in that: include: Obtain two-dimensional slice images of the nipple protection area through medical imaging equipment; Perform preliminary 3D structure estimation on 2D slice images to obtain spatial constraints; Build a super-resolution network based on 3D structure guidance for 2D images according to spatial constraints; Generate dense and high-precision point clouds for 3D reconstruction through multi-view stereo matching; The preliminary three-dimensional structure estimation includes extracting feature points of each view image using a pre-processed low-resolution image sequence and a structural motion recovery algorithm; Obtain reliable matching points through feature matching method; Based on reliable matching points, the intrinsic and extrinsic parameters of each camera are estimated, and a preliminary coarse-grained 3D sparse point cloud structure is obtained; The initial coarse-grained 3D structure is used as spatial constraint information for super-resolution processing; The construction of a super-resolution network based on three-dimensional structure guidance includes: Prepare network input, the input data is the preliminary coarse-grained 3D structure and preprocessed multi-view low-resolution 2D images; Use shallow convolutional networks to extract basic image features from low-resolution image sequences and output basic two-dimensional feature maps ; Introducing preliminary 3D structural information into the network, and fusing 3D structural features with 2D image features through the attention mechanism; Apply spatial constraint information to the two-dimensional features and output a structure-guided fusion feature map ; Fusion feature map As input, the sub-pixel convolution method is used for upsampling processing, and the output is a two-dimensional high-quality image after super-resolution enhancement; To train the super-resolution network, real high-resolution images of the nipple protection area are selected as the supervision data for the training set; Define a hybrid loss function and use the Adam optimizer to iteratively optimize the super-resolution network parameters until convergence.

2. The super-resolution-based 3D reconstruction method for the nipple protection area according to claim 1, wherein: The two-dimensional slice image includes collecting low-resolution images of the nipple protection area at multiple angles and viewpoints to obtain a plurality of two-dimensional image data sequences at different viewpoints; Preprocess the captured images, including denoising, blur removal, and illumination normalization, preliminarily calibrate the camera's internal and external parameters, and correct distortion; Outputs the processed low-resolution image sequence.

3. The super-resolution-based 3D reconstruction method for the nipple protection area according to claim 2, wherein: The defined hybrid loss function includes content loss, perception loss, structural consistency loss and depth constraint loss; Specific definition Including, content loss is to measure the pixel-level difference of images; Perceptual loss is used to supervise the reconstruction of image semantic features; The structural consistency loss is to constrain the continuity of pixel feature spaces at different perspectives through the consistency of corresponding points in three-dimensional space; The depth-constrained loss provides additional constraints for network training by utilizing preliminary 3D structure depth information.

4. The super-resolution-based 3D reconstruction method for the nipple protection area according to claim 3, wherein: Generating a dense high-precision point cloud includes performing multi-view stereo matching using camera parameters generated in the preliminary three-dimensional structure estimation and high-resolution images enhanced by a super-resolution network; The disparity estimation algorithm is used to calculate disparity and generate depth maps, and multi-view depth information is integrated to obtain a high-density 3D point cloud of the nipple protection area. The obtained high-density point cloud is denoised, filtered and optimized to remove clutter and isolated points and improve the clarity and accuracy of the point cloud.

5. The super-resolution-based 3D reconstruction method for the nipple protection area according to claim 4, wherein: The three-dimensional reconstruction includes meshing the optimized high-density point cloud using a Poisson reconstruction algorithm to generate a fine three-dimensional surface mesh model of the nipple protection area; Further optimization of the mesh model, including: Mesh smoothing and removal of redundant vertices, repairing holes and irregular edges, and streamlining model structure; For the high-resolution image obtained based on super-resolution enhancement, a multi-view texture mapping method is used to perform high-precision texture mapping optimization.

6. A system using the super-resolution-based 3D reconstruction method for the nipple protection area according to any one of claims 1 to 5, characterized in that: It includes data acquisition module, preliminary estimation module, 3D guided super-resolution processing module, and 3D reconstruction module; The data acquisition module includes a data acquisition unit and an image processing unit; Used to collect multi-angle low-resolution two-dimensional images and depth scanning data around the nipple protection area, and complete preliminary processing of the nipple protection area data; The preliminary estimation module includes a feature extraction unit, a parameter estimation unit and an initial structure generation unit; It is used to estimate the preliminary three-dimensional structure and output coarse-grained three-dimensional spatial prior information; The 3D guided super-resolution processing module is used to use the three-dimensional structural information as a guide to perform super-resolution processing on the two-dimensional image of the nipple protection area; The three-dimensional reconstruction module includes a reconstruction unit and a texture mapping unit, which is used to realize high-precision three-dimensional reconstruction of the nipple protection area based on the super-resolution image and output a refined three-dimensional model.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the super-resolution-based three-dimensional reconstruction method of the nipple protection zone are implemented as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the super-resolution-based three-dimensional reconstruction method of the nipple protection zone are implemented according to any one of claims 1 to 5.

Images