Artificial intelligence-based multi-scale optical sectioning liquid-repellent surface three-dimensional reconstruction method and system

Abstract

The application belongs to the technical field of liquid-repellent surface three-dimensional measurement, and discloses a multi-scale optical sectioning liquid-repellent surface three-dimensional reconstruction method based on artificial intelligence, a multi-scale optical sectioning center point extraction mechanism based on artificial intelligence, a Faster R-CNN interference area detection and U-Net de-interference basic model, a center point sequence fast extraction basic model established by fusing multi-scale features, and high-precision and high-robustness intelligent extraction of a center line; a spatial feature constraint multi-scale calibration mechanism based on artificial intelligence, an intelligent mapping law from an image domain to a spatial domain, a FFDNet de-noising basic model, a mapping calibration model established by fusing spatial multi-scale feature matching, and high-precision and high-robustness intelligent calibration of a system. The multi-scale optical sectioning three-dimensional reconstruction mechanism based on artificial intelligence can break through the liquid-repellent surface three-dimensional measurement bottleneck, provide key support for quantitative modeling and optimization of a biomimetic functional surface, and has important theoretical and application values.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional measurement technology of liquid-repellent surfaces, and particularly relates to a three-dimensional reconstruction method and system for multi-scale optical slice liquid-repellent surfaces based on artificial intelligence. Background Technology

[0002] Liquid-repellent surfaces possess excellent lubrication retention, droplet repulsion, and anti-fouling capabilities. Applying biomimetic liquid-repellent surfaces to key automotive components (such as bearings and pistons) can improve component performance and lifespan, achieving energy conservation and emission reduction in automobiles. Three-dimensional measurement of liquid-repellent surfaces can provide data support and theoretical basis for optimizing the structure of biomimetic liquid-repellent surfaces. Traditional contact or high-end microscopic measurement methods (such as coordinate measuring machines and profilometers) suffer from low efficiency, small field of view, complex operation, or susceptibility to damage, making it difficult to meet the practical needs of high-precision, high-efficiency, and non-contact measurement in engineering. To address these issues, this invention utilizes a multi-scale optical slicing liquid-repellent surface three-dimensional reconstruction mechanism based on artificial intelligence. Through artificial intelligence and multi-scale distribution patterns, it achieves interference removal, centerline extraction, and high-precision calibration, thereby overcoming the limitations of traditional optical slicing at the microscopic scale, such as low efficiency, poor centerline extraction accuracy, insufficient robustness, and poor calibration adaptability.

[0003] Optical slicing visual measurement technology, as a non-contact, high-precision, and interference-resistant 3D reconstruction method, has been widely used in aerospace, automotive manufacturing, and biomedicine. However, existing research mainly focuses on the macroscopic scale, and research on the imaging and reconstruction mechanisms at the microscopic scale remains insufficient, limiting its application in high-precision measurement of liquid-repellent surfaces.

[0004] Based on the above analysis, the problems and shortcomings of the existing technology are as follows:

[0005] (1) Insufficient accuracy and robustness of center point extraction. Traditional centroid method, fitting method and edge detection method have problems such as poor stability and low efficiency under conditions of uneven light intensity and complex structure; in actual measurement process, the diameter of a single micro-protrusion structure is close to the minimum width of the light slice, which causes a lot of interference and makes it difficult to achieve high-precision extraction.

[0006] (2) The calibration method at the microscopic scale has poor adaptability. Existing artificial feature point-dependent methods (such as three-point perspective and cross-ratio invariance method) are easily affected by distortion, feature blurring and noise, resulting in strong randomness and low robustness in matching. When facing three-dimensional measurement at the microscopic scale, the small field of view calibration process is cumbersome and difficult to operate. The angular change range of the calibration plane in space is small, which greatly reduces the calibration accuracy. Summary of the Invention

[0007] To address the problems existing in the prior art, this invention provides a three-dimensional reconstruction method for multi-scale optical slice liquid-repellent surfaces based on artificial intelligence.

[0008] This invention is implemented as follows: A method for three-dimensional reconstruction of a multi-scale optically sliced ​​liquid-repellent surface based on artificial intelligence includes:

[0009] Step 1: Establish a Faster R-CNN region detection model to accurately locate interference regions, construct a U-Net interference removal model to remove the influence of noise and stray light, and integrate the distribution law of center points of multi-scale light slices to establish a fast center point extraction model;

[0010] Step 2: Study the explicit noise mapping mechanism based on convolutional neural network (CNN) and construct a multi-scale region division and calibration model for microscopic structure light slices.

[0011] Furthermore, a Faster R-CNN region detection model is established to accurately locate interference regions, a U-Net interference removal model is constructed to remove the influence of noise and stray light, and the distribution law of multi-scale light slice center points is integrated to establish a fast center point extraction model:

[0012] Step 1: Implementation of Faster R-CNN for extracting interference regions and U-net for removing image interference.

[0013] First, images of light slices containing interference were collected. LabelImg was used to annotate the bounding boxes and categories of the interference regions, constructing a VOC format dataset, and the training and validation sets were divided in an 8:2 ratio. Based on this, using the Faster R-CNN framework and ResNet-50 as the backbone network, multi-scale anchor boxes were set to address the small-size characteristics of microscopic light slices, the RPN non-maximum suppression threshold was adjusted, and a joint loss function was employed.

[0014]

[0015] Where Lcls is the cross-entropy classification loss, Lreg is the smoothed L1 regression loss, λ is the weight coefficient, and the validation set is used; mAP≥0.9 is the training termination condition, to achieve high-precision localization of interference areas and ROI output;

[0016] Subsequently, interference regions are cropped from the ROIs output by Faster R-CNN to construct a ROI sub-image dataset, which is then input into an improved U-Net network for interference removal and restoration. U-Net introduces an attention gate module between the corresponding layers of the encoder and decoder.

[0017]

[0018] Where xe and xd are the feature maps of the encoder and decoder, respectively, W is the convolution weight, and σ is the sigmoid function; training uses a weighted combined loss:

[0019]

[0020] Where LBCE is pixel-level cross-entropy loss, LDice is Dice loss, and training is terminated after DSC ≥ 0.92 on the validation set;

[0021] After removing interference, the pixel is recovered using the neighborhood mean:

[0022]

[0023] The recovered ROI is then placed back into the original image to obtain an interference-free optical section image, thus providing a reliable foundation for subsequent extraction of the center line of the microscopic optical section and high-precision three-dimensional reconstruction.

[0024] Step 2: The single imaging plane dimension reduction decomposition method with the image layer space perpendicular to the principal direction of the light slice is implemented;

[0025] Step 3: Implementation of the gray-level vector index matching method within the gray-level vector pool of a single imaging plane;

[0026] Step 4: Implementation of a method for suppressing laser interference based on multi-scale frame field modulation.

[0027] Furthermore, the single-image plane dimensionality reduction decomposition method with the image layer space perpendicular to the principal direction of the light slice is implemented as follows:

[0028] By adjusting the spatial pose relationship between the light slice generator and the imaging system, the traveling direction of the light slice in the image layer space is made consistent with the row direction of the image layer space. The row direction of the image layer space is defined as the principal direction of the light slice. Assuming the resolution of the image layer space is... Then the image layer space can be decomposed into a dimensionless form by column vectors. common A grayscale vector pool consisting of grayscale vectors; using grayscale vectors For example, the coordinates of the center of the light slice corresponding to this grayscale vector are:

[0029]

[0030] In equation (1), The coordinates of the center column direction of the spatial light slice of the image layer; The coordinates of the center row direction of the spatial light slice of the image layer; The grayscale vector in the grayscale vector pool Subpixel coordinates of the grayscale extreme points in the image layer space column direction; grayscale vector Vector index in the row direction.

[0031] Furthermore, the method for matching the gray-level vector index within the single imaging plane gray-level vector pool is implemented as follows:

[0032] Based on the method in the first step, the extreme points of all gray-level vectors in the gray-level vector pool can be calculated by iterating through the gray-level vector pool. For example, after obtaining Subpixel coordinates of the center of the light slice At the same time, match the index number of the row direction of the current grayscale vector. By combining formula (5), the center coordinates of the optical slice can be obtained. Matching.

[0033] Furthermore, the method for suppressing laser interference based on multi-scale frame field modulation is implemented as follows:

[0034] This study investigates the collaborative distribution law of optical slice centers among multiple imaging planes during dynamic 3D scanning. The multi-scale frame field modulation reconstructed grayscale vector is obtained based on the collaborative distribution model of multiple grayscale vectors in the frame dimension. The calculation results of its grayscale extreme points can improve the laser interference phenomenon. Since the imaging of the frame field modulation group is directly realized in the dynamic scanning process, the scanning efficiency is guaranteed. The space of each image layer in the frame field modulation group is decomposed into an ordered grayscale vector pool group. The corresponding grayscale vectors in the grayscale vector pool group are collaboratively reconstructed in the frame dimension to form a grayscale collaborative distribution model in the frame dimension within the frame field modulation group.

[0035] Furthermore, the study constructs a multi-scale region segmentation and calibration model for microscopic structured light slices based on the explicit noise mapping mechanism of convolutional neural networks (CNNs):

[0036] Before and after optical slice calibration of the imaging system, the FFDnet model is proposed to denoise the acquired checkerboard calibration images. Drawing on the tunability principle of model-driven denoising methods, an explicit noise mapping mechanism based on convolutional neural networks (CNNs) is investigated. Traditional models typically achieve denoising by minimizing the following energy function:

[0037]

[0038] in, For data fidelity items, For image prior constraints, This is a balance factor used to adjust the relationship between noise suppression and detail preservation; because Absorbable to noise level In this model, the equation can be simplified to:

[0039]

[0040] This demonstrates that the denoising intensity can be flexibly controlled by setting the noise level; based on this theory, this invention constructs a CNN explicit mapping model and uses the scalar noise level... Expanding to a noise level map M of the same size as the input image enables joint modeling of noise and image, the expression is:

[0041]

[0042] Substitute the distance Rpij from the feature point to the imaging center in each CCD imaging plane into the following formula to allocate the region of the feature point group;

[0043]

[0044] Construct a high-order matrix collaborative model to realize a flexible mapping relationship between feature point groups, i.e. The function is as follows:

[0045]

[0046] The specific functional relationships between xij, yij and uij, vij in the above formula are as follows:

[0047]

[0048] Let s=f(Rpij) in formula (5) to determine the order of the matrix cooperative model used in the region where the current feature point group is located, as follows:

[0049]

[0050] Combining formulas (4), (5), (6), and (7), we can obtain the following mapping relationship:

[0051]

[0052] Where t=1, 2, 3, ..., t-1 is the minimum order of the matrix cooperative model used.

[0053] Another objective of this invention is to provide a multi-scale optical slicing liquid-repellent surface three-dimensional reconstruction system based on artificial intelligence, comprising:

[0054] The fast extraction model building module is used to build a Faster R-CNN region detection model to accurately locate interference regions, construct a U-Net interference removal model to remove noise and stray light effects, and integrate the distribution law of center points of multi-scale light slices to establish a fast center point extraction model.

[0055] The partitioning and calibration model module is used to study the explicit noise mapping mechanism based on convolutional neural networks (CNN) and to construct a multi-scale region partitioning and calibration model for microscopic structured light slices.

[0056] Another object of the present invention is to provide a computer device, the computer device including a memory and a processor, the memory storing a computer program, the computer program being executed by the processor causing the processor to perform the steps of the three-dimensional reconstruction method for a multi-scale optically sliced ​​liquid-repellent surface based on artificial intelligence.

[0057] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the three-dimensional reconstruction method for a multi-scale optically sliced ​​liquid-repellent surface based on artificial intelligence.

[0058] Another objective of this invention is to provide an information data processing terminal for implementing the artificial intelligence-based multi-scale optical slice liquid-repellent surface three-dimensional reconstruction system.

[0059] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0060] This invention is based on the multi-scale optical slice center point extraction mechanism of artificial intelligence, constructs a Faster R-CNN interference region detection and U-Net interference removal basic model, integrates multi-scale features to establish a basic model for fast center point extraction, and achieves high-precision and high-robust intelligent extraction of the center line;

[0061] This invention is based on the spatial feature constraint multi-scale calibration mechanism of artificial intelligence, reveals the intelligent mapping law from the image domain to the spatial domain, constructs the FFDNet denoising basic model, and integrates spatial multi-scale feature matching to establish a mapping calibration model, so as to realize the system's high-precision and high-robust intelligent calibration.

[0062] This invention is based on the multi-scale optical slicing three-dimensional reconstruction mechanism of artificial intelligence, and aims to break through the bottleneck of three-dimensional measurement of liquid-repellent surfaces, providing key support for quantitative modeling and optimization of biomimetic functional surfaces, and has important theoretical and application value.

[0063] The technical solution of this invention achieves a significant breakthrough based on existing optical slicing 3D reconstruction technology. This is primarily manifested in the collaborative introduction of Faster R-CNN and U-Net into the process of optical slicing interference identification and removal. It moves beyond traditional filtering, thresholding, or manual adjustment methods, instead using joint modeling of target detection and image segmentation to achieve precise localization and effective suppression of stray light, reflection interference, and noise regions at the microscopic scale, thereby significantly improving the accuracy and stability of centerline extraction. Simultaneously, this invention further solves the technical challenge of real-time adaptive processing of laser interference under dynamic scanning conditions. It can adaptively suppress interference based on changes in imaging characteristics, eliminating the need for repeated manual parameter tuning and enhancing the system's automation and engineering application capabilities. Furthermore, this invention breaks through the traditional understanding that deep learning methods must rely on large-scale labeled samples. Through a combination of transfer learning and multi-scale frame field manipulation, it achieves high accuracy even with small sample sizes, thus possessing outstanding innovation and practical value. Attached Figure Description

[0064] Figure 1 This is a flowchart of a multi-scale optical slice liquid-repellent surface three-dimensional reconstruction method based on artificial intelligence provided in an embodiment of the present invention.

[0065] Figure 2 This is a structural block diagram of a multi-scale optical slicing liquid-repellent surface three-dimensional reconstruction system based on artificial intelligence, provided in an embodiment of the present invention.

[0066] Figure 3 This is a diagram illustrating the liquid-repellent surface functional mechanism provided in an embodiment of the present invention.

[0067] Figure 4 This diagram illustrates the difficulties and breakthroughs in measuring liquid-repellent surfaces provided in this embodiment of the invention.

[0068] Figure 5 This is a diagram illustrating the technical implementation scheme provided in an embodiment of the present invention.

[0069] Figure 6 This is a technical roadmap of the technical implementation steps (1) provided in the embodiments of the present invention.

[0070] Figure 7 This is a schematic diagram of dimensionality reduction decomposition based on optical slices of image-space microstructure provided in an embodiment of the present invention.

[0071] Figure 8 This is a technical roadmap of step (3) of the technical implementation provided in the embodiments of the present invention.

[0072] Figure 9 This is a schematic diagram of chessboard feature point matching provided in an embodiment of the present invention. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0074] like Figure 1 As shown in the figure, the three-dimensional reconstruction method for a multi-scale optically sliced ​​liquid-repellent surface based on artificial intelligence provided by this invention includes the following steps:

[0075] S101: Establish a Faster R-CNN region detection model to accurately locate interference regions, construct a U-Net interference removal model to remove the influence of noise and stray light, and integrate the distribution law of multi-scale light slice center points to establish a fast center point extraction model.

[0076] S102: Research on explicit noise mapping mechanism based on convolutional neural network (CNN) and construct a multi-scale region division and calibration model for microstructured light slices.

[0077] like Figure 2 As shown in the figure, an embodiment of the present invention provides a multi-scale optical slicing liquid-repellent surface three-dimensional reconstruction system based on artificial intelligence, comprising:

[0078] The fast extraction model building module is used to build a Faster R-CNN region detection model to accurately locate interference regions, construct a U-Net interference removal model to remove noise and stray light effects, and integrate the distribution law of center points of multi-scale light slices to establish a fast center point extraction model.

[0079] The partitioning and calibration model module is used to study the explicit noise mapping mechanism based on convolutional neural networks (CNN) and to construct a multi-scale region partitioning and calibration model for microscopic structured light slices.

[0080] This invention addresses the challenges of stable extraction of light stripe centers, distorted spatial mapping, and limited 3D reconstruction accuracy in multi-scale optical slicing imaging of liquid-repellent surfaces due to stray light, strong local reflections, blurred boundaries, and differences in microstructural scale. It constructs a multi-stage collaborative reconstruction system based on artificial intelligence. First, a rapid extraction model building module uses the Faster R-CNN region detection model to accurately locate interference regions in the original optical slice image. Then, a U-Net interference removal model suppresses and separates noise, stray light, and abnormal bright spots, obtaining an effective optical slice image with clear edges and continuous stripes. Based on this, a rapid center point extraction model is established by combining the spatial distribution and sequential continuity characteristics of the center points of multi-scale optical slices, enabling stable identification and orderly output of light stripe centers at different scales and locations. Subsequently, the partitioning and calibration model module introduces an explicit noise mapping mechanism based on a convolutional neural network to model residual errors and scale variations in the image. Combined with the spatial partitioning features of microstructural light slices, a multi-scale spatial region partitioning and calibration model is established. This allows for adaptive correction of imaging distortion, noise effects, and mapping parameters in different regions, ultimately achieving high-precision reconstruction of the three-dimensional morphology of the liquid-repellent surface. Compared to existing technologies, the innovation of this invention is prominently reflected in its integration of target detection, image denoising, center sequence extraction, explicit noise modeling, and multi-scale region calibration. This forms a closed-loop intelligent processing mechanism from image perception to spatial reconstruction, simultaneously considering complex interference suppression capabilities, center extraction stability, and three-dimensional reconstruction accuracy. It is particularly suitable for liquid-repellent surface detection scenarios with fine structures, uneven reflectivity, and significant scale variations, thus possessing significant substantial features and advancements.

[0081] Specific implementation of the present invention:

[0082] This invention addresses the high precision and robustness requirements of 3D measurement of optical slices, focusing on the underlying algorithms and model architecture of artificial intelligence in the visual reconstruction of optical slices. Key breakthroughs are made in core technologies such as interference suppression, intelligent centerline extraction, and system calibration. A multi-task collaborative model integrating Faster R-CNN and U-Net is proposed to achieve adaptive identification of interference regions and multi-scale feature fusion. A self-learning denoising network based on FFDNet is established to analyze noise distribution and feature transformation patterns. A multi-scale adaptive mapping calibration method with spatial feature constraints is proposed to achieve end-to-end intelligent calibration from the image domain to the spatial domain. The research results will overcome the bottleneck of 3D measurement of liquid-repellent surfaces in optical slices, providing data support and theoretical basis for the optimization of biomimetic surface structures. Specific research content is as follows:

[0083] Research Content 1:

[0084] This invention addresses the problems of low accuracy, poor robustness, and numerous interferences in the extraction of center points from optical sections under microscopy. It investigates the multi-scale optical section center point extraction mechanism based on artificial intelligence and proposes an intelligent extraction method for optical section centerlines that integrates multi-scale features. A Faster R-CNN interference detection model and a U-Net interference removal model are constructed to effectively suppress noise and stray light. Furthermore, a rapid center point extraction model is established to achieve high-precision and robust extraction of optical section centerlines at the microscopic scale, providing a reliable foundation for the three-dimensional reconstruction of biomimetic liquid-repellent surfaces.

[0085] Research Content 2:

[0086] This invention addresses the problems of poor adaptability, low robustness, and noise interference in microscopic optical sectioning calibration. It investigates the multi-scale calibration mechanism based on spatial feature constraints using artificial intelligence and proposes an adaptive mapping intelligent calibration method using multi-scale feature matching. An FFDNet denoising model is constructed to suppress noise interference and extract stable structural features. By fusing spatial multi-scale features, an adaptive mapping calibration model is established, achieving high-precision and highly robust intelligent calibration of the optical sectioning system. This provides key technical support for improving the accuracy of three-dimensional reconstruction and measurement of biomimetic liquid-repellent surfaces.

[0087] Figure 3 Liquid-repellent surface functional mechanism

[0088] Figure 4 Challenges and Breakthroughs in Liquid-Repellent Surface Measurement

[0089] 2. Implementation plan:

[0090] To achieve the above research objectives, the following technical implementation scheme is proposed, as shown in the diagram below. Figure 5 As shown.

[0091] Technical implementation steps (1): Establish a Faster R-CNN region detection model to accurately locate the interference region, construct a U-Net interference removal model to remove the influence of noise and stray light, and integrate the distribution law of the center point of multi-scale light slices to establish a fast center point extraction model.

[0092] This invention introduces an AI-based noise reduction model to suppress noise and enhance features in the light slice image after a single imaging session and grayscale conversion, improving the signal-to-noise ratio and structural fidelity. It also constructs a dimensionality reduction decomposition model for the two-dimensional grayscale matrix of the light slice image layer space, enabling efficient feature extraction from the noise-reduced image data, such as... Figure 6 The diagram shows the technical roadmap for the technical implementation step (1).

[0093] Step 1: Implementation of Faster R-CNN for extracting interference regions and U-net for removing image interference.

[0094] This invention addresses the interference problems such as laser reflection and impurities in microscopic optical slice images by constructing a high-precision intelligent interference removal method. First, images of optical slices containing interference are collected. LabelImg is used to annotate the bounding boxes and categories of the interference regions, constructing a VOC format dataset, and dividing the training and validation sets in an 8:2 ratio. Based on this, using the Faster R-CNN framework and ResNet-50 as the backbone network, multi-scale anchor boxes are set to suit the small size characteristics of microscopic optical slices, the RPN non-maximum suppression threshold is adjusted, and a joint loss function is employed.

[0095]

[0096] Where Lcls is the cross-entropy classification loss, Lreg is the smoothed L1 regression loss, λ is the weight coefficient, and the validation set is used. mAP ≥ 0.9 is the training termination condition, achieving high-precision localization of interference areas and ROI output.

[0097] Subsequently, based on the ROIs output by Faster R-CNN, interference regions are cropped to construct a ROI sub-image dataset, which is then input into the improved U-Net network for interference removal and restoration. U-Net introduces an attention gate module between the corresponding layers of the encoder and decoder:

[0098]

[0099] Where xe and xd are the feature maps of the encoder and decoder, respectively, W is the convolution weight, and σ is the sigmoid function; training uses a weighted combined loss:

[0100]

[0101] Where LBCE is pixel-level cross-entropy loss, LDice is Dice loss, and training terminates after DSC ≥ 0.92 on the validation set.

[0102] After removing interference, the pixel is recovered using the neighborhood mean:

[0103]

[0104] The recovered ROI is then placed back into the original image to obtain an interference-free optical section image, thus providing a reliable foundation for subsequent extraction of the centerline of the optical section and high-precision three-dimensional reconstruction.

[0105] Step 2: Implementation of a single imaging plane dimensionality reduction decomposition method with the image space perpendicular to the principal direction of the light slice.

[0106] By adjusting the spatial pose relationship between the light slice generator and the imaging system, the traveling direction of the light slice in the image layer space is made consistent with the row direction in the image layer space, and the row direction in the image layer space is defined as the main direction of the light slice. For example... Figure 7 The diagram illustrates the dimensionality reduction decomposition of the image layer space along the column direction (perpendicular to the principal direction of the optical slice). Assume the resolution of the image layer space is... Then the image layer space can be decomposed into a dimensionless form by column vectors. common A grayscale vector pool consisting of grayscale vectors. For example, the coordinates of the center of the light slice corresponding to this grayscale vector are:

[0107]

[0108] In equation (1), The coordinates of the center column direction of the spatial light slice of the image layer; The coordinates of the center row direction of the spatial light slice of the image layer; The grayscale vector in the grayscale vector pool Subpixel coordinates of the grayscale extreme points in the image layer space column direction; grayscale vector Vector index in the row direction.

[0109] Step 3: Implementation of the gray-level vector index matching method within the single imaging plane gray-level vector pool

[0110] Based on the method in the first step, the extreme points of all gray-level vectors in the gray-level vector pool can be calculated by iterating through the gray-level vector pool. For example, after obtaining Subpixel coordinates of the center of the light slice At the same time, match the index number of the row direction of the current grayscale vector. By combining formula (5), the center coordinates of the optical slice can be obtained. Matching.

[0111] Step 4: Implementation of a method for suppressing laser interference based on multi-scale frame field modulation

[0112] This study investigates the collaborative distribution law of optical slice centers among multiple imaging planes during dynamic 3D scanning. The multi-scale frame field modulation reconstructed grayscale vectors are obtained based on a collaborative distribution model of multiple grayscale vectors along the frame dimension. The calculation results of grayscale extrema can improve laser interference. Since the imaging of the frame field modulation group is directly implemented during dynamic scanning, scanning efficiency is guaranteed. Each image layer in the frame field modulation group is decomposed into an ordered grayscale vector pool group. The corresponding grayscale vectors in the grayscale vector pool group are collaboratively reconstructed along the frame dimension to form a grayscale collaborative distribution model along the frame dimension within the frame field modulation group.

[0113] Technical implementation steps (2): Research the explicit noise mapping mechanism based on convolutional neural network (CNN), construct a multi-scale region division and calibration model for microstructured light slices, and realize a low-cost and highly robust microstructured light slice calibration method.

[0114] Before and after optical slice calibration of the imaging system, the FFDnet model is planned to be used to denoise the acquired checkerboard calibration images, such as... Figure 8 The diagram shows the technical roadmap for step (2) of the technical implementation. To improve the adaptability of the microscopic light slice image denoising model under different noise intensities and types, this invention draws on the tunability principle of model-driven denoising methods and studies an explicit noise mapping mechanism based on convolutional neural networks (CNNs). Traditional models typically achieve denoising by minimizing the following energy function:

[0115]

[0116] in, For data fidelity items, For image prior constraints, This is a balance factor used to adjust the relationship between noise suppression and detail preservation. Because... Absorbable to noise level In this model, the equation can be simplified to:

[0117]

[0118] This demonstrates that the denoising intensity can be flexibly controlled by setting the noise level. Based on this theory, this invention constructs a CNN explicit mapping model and uses the scalar noise level... Expanding to a noise level map M of the same size as the input image enables joint modeling of noise and image, the expression is:

[0119]

[0120] This mechanism can not only achieve adaptive denoising with different noise intensities in a single network, but also extend M to a multi-channel degradation map to adapt to multivariate Gaussian and spatially non-uniform noise models, thereby achieving synergistic optimization of noise suppression and detail preservation, providing a theoretical basis and methodological support for high-precision denoising of microscopic light section images.

[0121] In a CCD imaging plane, the central region experiences relatively small distortion due to its proximity to the lens optical axis. However, the distortion increases in the surrounding areas as the distance from the optical axis increases. Therefore, the CCD imaging plane is spatially divided, and a flexible mapping relationship Wi from the distorted CCD imaging plane (o-uv) to the optical slice plane (Ow-XwYw) is established at different spatial scales by constructing a multi-coefficient high-order matrix cooperative model. This optimizes the mapping accuracy between coordinate points and achieves high-precision calibration of the optical slice plane based on multiple spatial scales. Figure 9 As shown.

[0122] Substitute the distance Rpij from the feature point to the imaging center in each CCD imaging plane into the following formula to allocate the region of the feature point group.

[0123]

[0124] Construct a high-order matrix collaborative model to realize a flexible mapping relationship between feature point groups, i.e. The function is as follows:

[0125]

[0126] The specific functional relationships between xij, yij and uij, vij in the above formula are as follows:

[0127]

[0128] Let s=f(Rpij) in formula (5) to determine the order of the matrix cooperative model used in the region where the current feature point group is located, as follows:

[0129]

[0130] Combining formulas (4), (5), (6), and (7), we can obtain the following mapping relationship:

[0131]

[0132] Where t=1, 2, 3, ..., t-1 is the minimum order of the matrix cooperative model used.

[0133] The mapping coefficients of the three regions K1, K2 and K3 are obtained by using the least squares method, so as to achieve high-precision calibration of the optical slice plane. Since all the corner points of the chessboard are used as matching feature points in the optical slice plane calibration model, and the model does not depend on the extrinsic parameters of the imaging system and the center of the optical slice, the robustness of the calibration is improved.

[0134] Example 1: Noise Mapping and Multi-Scale Region Labeling Based on DnCNN (Static Light Slicing Scene) Technical Background

[0135] In static light slice imaging, noise mainly originates from sensor thermal noise, photon shot noise, and background stray light, manifesting as a mixture of high-frequency random noise and low-frequency structured noise. Traditional methods (such as Gaussian filtering) tend to blur edge details, while DnCNN (Denoising Convolutional Neural Network) directly predicts the noise distribution through residual learning, thus preserving more structural information.

[0136] Implementation steps:

[0137] Data preparation:

[0138] Multiscale optical slice images (e.g., 10×, 20×, 40× objectives) of the same liquid-repellent surface under different lighting conditions were collected to construct a training set containing clean image-noise image pairs.

[0139] Manually annotate noisy images and divide them into spatial regions (such as edge regions, center regions, and transition regions).

[0140] Model building:

[0141] A 17-layer DnCNN architecture is used, with each layer employing a 3×3 convolutional kernel. The difference between noise and clean images (i.e., noise mapping) is learned through residual connections.

[0142] Batch normalization (BN) layers are introduced to accelerate training, and the ReLU activation function is used to enhance nonlinear expressive power.

[0143] Multi-scale region calibration:

[0144] The trained DnCNN is applied to light slices of different scales to output noise maps.

[0145] Automatically divide regions based on noise intensity (e.g., standard deviation) and spatial distribution (e.g., gradient variation):

[0146] High-noise areas (edge ​​areas): Noise standard deviation > threshold, requiring enhanced noise reduction;

[0147] Low-noise zone (center zone): Noise standard deviation < threshold, details preserved;

[0148] Transition region: Noise gradient changes drastically, so adaptive filtering is used.

[0149] Effect verification:

[0150] In experiments on silicone oil-repellent surfaces, this method improved the signal-to-noise ratio (SNR) of the edge region by 12 dB and achieved a structural similarity (SSIM) of 0.92 in the central region, which is significantly better than the traditional method.

[0151] Example 2: Dynamic optical slicing noise suppression and region segmentation based on U-Net++ (real-time imaging scenario) Technical Background

[0152] In dynamic optical slicing imaging (such as flow field liquid repellency surface monitoring), noise varies over time and motion blur exists, requiring real-time processing of multi-scale data. U-Net++ enhances feature reuse through dense skip connections, making it suitable for handling complex noise and fast segmentation.

[0153] Implementation steps:

[0154] Dynamic data acquisition:

[0155] A high-speed camera was used to capture optical slice sequences of the liquid-repellent surface in a flowing state (frame rate ≥ 100fps), and the noise type (such as motion blur, Gaussian noise) and region label were labeled for each frame.

[0156] Model optimization:

[0157] Embed attention mechanisms (such as SE modules) in the U-Net++ encoder-decoder architecture to make the model focus on areas with significant noise.

[0158] A multi-scale loss function is introduced, combining pixel-level L1 loss and region-level cross-entropy loss to improve segmentation accuracy.

[0159] Real-time area calibration:

[0160] For the input light slice, the model simultaneously outputs:

[0161] Noise mapping diagram: identifies the type and intensity of noise;

[0162] Region segmentation diagram: Divides the flow core region, boundary layer region, and eddy current region.

[0163] Based on the calibration results, differentiated denoising parameters are applied to different regions (e.g., lightweight filtering is used in the core region, and deep denoising is used in the boundary layer region).

[0164] Effect verification:

[0165] In hydrogel flow experiments on repellent surfaces, this method achieves real-time processing (delay <10ms), region segmentation accuracy reaches 95%, and the PSNR of motion-blurred regions is improved by 8dB.

[0166] Example 3: Cross-scale optical slicing noise transfer and unified calibration based on GAN (multi-objective collaborative scenario) Technical Background

[0167] In cross-scale imaging (such as using 10× and 100× objectives simultaneously), the noise characteristics of light slices at different scales differ significantly (e.g., noise is widely distributed at low magnification and locally concentrated at high magnification). Generative adversarial networks (GANs) can achieve unified calibration across scales through noise transfer learning.

[0168] Implementation steps:

[0169] Cross-scale data generation:

[0170] A noisy transfer model was built using CycleGAN, with the training set containing:

[0171] Source region: Low-magnification microscopic section (wide noise distribution);

[0172] Target domain: High-magnification microscopic section (localized noise concentration).

[0173] By using cycle consistency loss, the model learns the noise transformation pattern from low to high magnification.

[0174] Unified calibration model:

[0175] The transferred noise map and the original optical slice are input into the calibration network (based on ResNet-50) to output cross-scale region segmentation results:

[0176] Globally consistent region: regions with similar noise characteristics (such as smooth surface regions);

[0177] Local difference region: The noise characteristics are different (such as the edge region of microstructure).

[0178] Calibration result fusion:

[0179] For globally consistent regions, high-powered microscope calibration parameters are used;

[0180] For areas of local difference, weighted calibration is performed by combining global information from low magnification with local details from high magnification.

[0181] Effect verification:

[0182] In experiments on liquid-repellent surfaces of polydimethylsiloxane (PDMS), this method reduced cross-scale calibration error to less than 2% and increased global processing speed by 3 times.

[0183] The above embodiments all utilize explicit noise mapping mechanisms, combined with CNNs and their variants (such as DnCNN, U-Net++, and GAN), to achieve spatial multi-scale region division and calibration of microstructure light slices, providing high-precision and robust technical support for three-dimensional reconstruction of liquid-repellent surfaces.

[0184] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0185] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for three-dimensional reconstruction of a multi-scale optically sliced ​​liquid-repellent surface based on artificial intelligence, characterized in that, Includes the following steps: Step 1: Establish a Faster R-CNN region detection model to accurately locate interference regions, construct a U-Net interference removal model to remove the influence of noise and stray light, and integrate the distribution law of center points of multi-scale light slices to establish a fast center point extraction model; Step 2: Study the explicit noise mapping mechanism based on convolutional neural networks and construct a multi-scale region division and calibration model for microstructured light slices.

2. The three-dimensional reconstruction method for multi-scale optically sliced ​​liquid-repellent surfaces based on artificial intelligence as described in claim 1, characterized in that, The Faster R-CNN region detection model is established to accurately locate interference regions, a U-Net interference removal model is constructed to remove the influence of noise and stray light, and the distribution law of center points of multi-scale light slices is integrated to establish a fast center point extraction model. Step 1: Implementation of Faster R-CNN for extracting interference regions and U-net for removing image interference. First, images of light slices containing interference were collected. LabelImg was used to annotate the bounding boxes and categories of the interference regions, constructing a VOC format dataset, and the training and validation sets were divided in an 8:2 ratio. Based on this, using the Faster R-CNN framework and ResNet-50 as the backbone network, multi-scale anchor boxes were set to address the small-size characteristics of microscopic light slices, the RPN non-maximum suppression threshold was adjusted, and a joint loss function was employed. ； Where Lcls is the cross-entropy classification loss, Lreg is the smoothed L1 regression loss, λ is the weight coefficient, and the validation set is used; mAP≥0.9 is the training termination condition, to achieve high-precision localization of interference areas and ROI output; Subsequently, interference regions are cropped from the ROIs output by Faster R-CNN to construct a ROI sub-image dataset, which is then input into an improved U-Net network for interference removal and restoration. U-Net introduces an attention gate module between the corresponding layers of the encoder and decoder. ； Where xe and xd are the feature maps of the encoder and decoder, respectively, W is the convolution weight, and σ is the sigmoid function; training uses a weighted combined loss: ； Where LBCE is pixel-level cross-entropy loss, LDice is Dice loss, and training is terminated after DSC ≥ 0.92 on the validation set; After removing interference, the pixel is recovered using the neighborhood mean: ； The recovered ROI is then placed back into the original image to obtain an interference-free optical section image, thus providing a reliable foundation for subsequent extraction of the center line of the microscopic optical section and high-precision three-dimensional reconstruction. Step 2: The single imaging plane dimension reduction decomposition method with the image layer space perpendicular to the principal direction of the light slice is implemented; Step 3: Implementation of the gray-level vector index matching method within the gray-level vector pool of a single imaging plane; Step 4: Implementation of a method for suppressing laser interference based on multi-scale frame field modulation.

3. The three-dimensional reconstruction method for multi-scale optically sliced ​​liquid-repellent surfaces based on artificial intelligence as described in claim 2, characterized in that, The single-image plane dimensionality reduction decomposition method, where the image layer space is perpendicular to the principal direction of the optical slice, is implemented as follows: By adjusting the spatial pose relationship between the light slice generator and the imaging system, the traveling direction of the light slice in the image layer space is made consistent with the row direction of the image layer space, and the row direction of the image layer space is defined as the main direction of the light slice. Assuming the resolution of the image layer space is Then the image layer space can be decomposed into a dimensionless form by column vectors. common A grayscale vector pool consisting of grayscale vectors; using grayscale vectors For example, the coordinates of the center of the light slice corresponding to this grayscale vector are: ； In equation (1), The coordinates of the center column direction of the spatial light slice of the image layer; The coordinates of the center row direction of the spatial light slice of the image layer; grayscale vectors in the grayscale vector pool Subpixel coordinates of the grayscale extreme points in the image layer space column direction; grayscale vector Vector index in the row direction.

4. The three-dimensional reconstruction method for multi-scale optically sliced ​​liquid-repellent surfaces based on artificial intelligence as described in claim 2, characterized in that, The method for matching gray-level vector indices within the single imaging plane gray-level vector pool is implemented as follows: Based on the method in the first step, the extreme points of all gray-level vectors in the gray-level vector pool can be calculated by iterating through the gray-level vector pool. For example, after obtaining Subpixel coordinates of the center of the light slice At the same time, match the index number of the row direction of the current grayscale vector. By combining formula (5), the center coordinates of the optical slice can be obtained. Matching.

5. The three-dimensional reconstruction method for multi-scale optically sliced ​​liquid-repellent surfaces based on artificial intelligence as described in claim 2, characterized in that, The method for suppressing laser interference based on multi-scale frame field modulation is implemented as follows: The study investigates the collaborative distribution law of optical slice centers among multiple imaging planes during dynamic 3D scanning. The multi-scale frame field modulation reconstructed grayscale vector is obtained based on the collaborative distribution model of multiple grayscale vectors in the frame dimension. Its grayscale extreme point calculation results can improve the laser interference phenomenon. Since the imaging of the frame field modulation group is directly realized during dynamic scanning, the scanning efficiency is guaranteed. Each image layer in the frame field control group is decomposed into an ordered gray-level vector pool group. The corresponding gray-level vectors in the gray-level vector pool group are collaboratively reconstructed in the frame dimension to form a gray-level collaborative distribution model in the frame dimension within the frame field control group.

6. The three-dimensional reconstruction method for multi-scale optically sliced ​​liquid-repellent surfaces based on artificial intelligence as described in claim 1, characterized in that, The study constructs a multi-scale region segmentation and calibration model for microscopic structured light slices based on the explicit noise mapping mechanism of convolutional neural networks (CNNs). Before and after optical slice calibration of the imaging system, the FFDnet model is proposed to denoise the acquired checkerboard calibration images. Drawing on the tunability principle of model-driven denoising methods, an explicit noise mapping mechanism based on convolutional neural networks (CNNs) is investigated. Traditional models typically achieve denoising by minimizing the following energy function: ； in, For data fidelity items, For image prior constraints, This is a balance factor used to adjust the relationship between noise suppression and detail preservation; because Absorbable to noise level In this model, the equation can be simplified to: ； This demonstrates that the denoising intensity can be flexibly controlled by setting the noise level; based on this theory, this invention constructs a CNN explicit mapping model and uses the scalar noise level... Expanding to a noise level map M of the same size as the input image enables joint modeling of noise and image, the expression is: ； Substitute the distance Rpij from the feature point to the imaging center in each CCD imaging plane into the following formula to allocate the region of the feature point group; ； Construct a high-order matrix collaborative model to realize a flexible mapping relationship between feature point groups, i.e. The function is as follows: ； The specific functional relationships between xij, yij and uij, vij in the above formula are as follows: ； Let s=f(Rpij) in formula (5) to determine the order of the matrix cooperative model used in the region where the current feature point group is located, as follows: ； Combining formulas (4), (5), (6), and (7), we can obtain the following mapping relationship: ； Where t=1, 2, 3, ..., t-1 is the minimum order of the matrix cooperative model used.

7. A three-dimensional reconstruction system for a multi-scale optically sliced ​​liquid-repellent surface based on artificial intelligence, implementing the method as described in any one of claims 1-6, characterized in that, The AI-based multi-scale optical slice liquid-repellent surface three-dimensional reconstruction system includes: The fast extraction model building module is used to build a Faster R-CNN region detection model to accurately locate interference regions, construct a U-Net interference removal model to remove noise and stray light effects, and integrate the distribution law of center points of multi-scale light slices to establish a fast center point extraction model. The partitioning and calibration model module is used to study the explicit noise mapping mechanism based on convolutional neural networks (CNN) and to construct a multi-scale region partitioning and calibration model for microscopic structured light slices.

8. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the three-dimensional reconstruction method for a multi-scale optically sliced ​​liquid-repellent surface based on artificial intelligence as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the three-dimensional reconstruction method for a multi-scale optically sliced ​​liquid-repellent surface based on artificial intelligence as described in any one of claims 1-6.

10. An information data processing terminal, characterized in that, The information data processing terminal is used to implement the three-dimensional reconstruction system for a multi-scale optical slice liquid-repellent surface based on artificial intelligence as described in claim 7.

Images