Fabric appearance three-dimensional reconstruction method under multi-view progressive matching

Abstract

The invention discloses a fabric apparent morphology three-dimensional reconstruction method under multi-view progressive matching. The method comprises the following steps: (1) building a closed shooting device; (2) acquiring a fabric image; (3) image preprocessing; (4) camera calibration and image correction: carrying out parameter calibration on the camera of the shooting device by adopting a checkerboard calibration method to obtain internal and external parameters of the camera, and carrying out correction operation on the acquired fabric image; (5) feature extraction and stereo matching: extracting appropriate feature points from the obtained left and right views of the fabric, and carrying out feature matching on the two images to obtain an optimal matching pixel pair; (6) generating a disparity map: calculating the fabric disparity map by utilizing the position difference between the left and right view pixels in the optimal matching pixel pair set; and (7) point cloud generation: calculating a depth map of each pixel value according to the view difference obtained in the step (6) so as to obtain a depth map of the fabric surface morphology and form a three-dimensional point cloud of the fabric apparent morphology.

Description

Technical Field

[0001] The invention belongs to the field of digital processing of fabric images, and in particular relates to a method for three-dimensional reconstruction of fabric surface morphology under multi-view progressive matching. Background Art

[0002] The testing of the fabric's appearance smoothness, pleat retention, seam smoothness, pilling, damage and defects after washing requires obtaining the true three-dimensional morphology after removing the fabric's apparent color, pattern and texture. Currently, the relevant evaluation still uses manual target evaluation, which is highly subjective, inefficient and non-reproducible, seriously restricting the development of the textile testing industry.

[0003] The evaluation of fabric surface performance, if only analyzed from two-dimensional images, will be mixed with the interference of fabric color, texture and pattern, which may easily lead to reconstruction distortion. The three-dimensional form of fabric without the interference of color, pattern and texture is more suitable for current detection needs. There is an urgent need for a method that can quickly reconstruct flexible fabrics in three dimensions. The existing laser triangulation method, grating projection method, photometric stereo vision method, binocular stereo vision method, etc., vary in cost, efficiency and accuracy, and it is difficult to meet the three-dimensional reconstruction of multiple types of fabrics. Binocular stereo vision will cause omissions in fabric image matching due to its harsh viewing angle and constraints. In addition, the reconstruction effect will also be affected by the pattern and color of the fabric surface.

[0004] Methods based on binocular stereoscopic imaging have made great progress in recent years. In theory, they can reconstruct the three-dimensional shape of objects under natural light. Binocular stereo vision is to imitate the human visual perception mechanism, use two identical cameras in parallel to obtain image pairs, eliminate the density feature matching set, calculate the relative position difference, and thus convert the height of the object. Xu Pinghua et al. made improvements on the basis of binocular stereo vision and proposed a rotational multi-eye stereo vision reconstruction method. Based on binocular reconstruction, the three-dimensional spatial information of multi-organization objects is obtained by rotating the camera group. Liu Xiaoli et al. disclosed a binocular three-dimensional reconstruction method and system based on line laser galvanometer scanning. The method is based on the corresponding matching points in the laser light strip image pair, and calculates the three-dimensional point cloud data of the object under a single perspective according to the calibration parameters and measurement model. Deng Liguo disclosed a multi-view three-dimensional reconstruction method based on a binocular stereo vision system, and proposed a spatial scanning voting and maximization strategy to estimate the semi-dense depth map of the selected viewpoint, and then merge the depth map to build a larger 3D model. Zhao Zhuangzhuang provided a binocular structured light 3D reconstruction method, reconstruction system and reconstruction equipment. Combined with the data of TOF equipment, the matching of points of the same name can be achieved, which significantly reduces the time of image acquisition and greatly reduces the data to be processed. It can significantly improve the speed of 3D reconstruction while maintaining accuracy. Yang Mengjie provided a large-scene binocular vision 3D reconstruction method. Through a lightweight closed-loop detection method, it can filter the new input position information and the information stored in the working memory, determine whether the closed-loop threshold is reached, and finally obtain the 3D model of the large scene. This invention can realize the rapid modeling of large outdoor scenes. Wu Jing provided a binocular image super-resolution reconstruction method, device and storage medium, including: obtaining low-resolution images of the left and right objects, and generating corresponding original left view feature maps and original right view feature maps. Yang Xu disclosed a binocular high-precision depth reconstruction method, which obtains the TOF depth data corresponding to each pixel on the binocular image and calculates the corresponding disparity; then sets a window for matching with the disparity as the center; finally, traverses all pixel points on the binocular image to obtain the matching results of all image pixels; this invention uses the binocular depth data corresponding to the TOF depth data for limitation, which greatly reduces the amount of binocular data calculation, can obtain binocular depth data more quickly, and improves the data quality while also improving the response speed.

[0005] Existing structured light 3D scanning is costly and time-consuming. Binocular stereo vision has strict viewing angles and constraints, which may lead to missed fabric image matches and partial missing of 3D point clouds. In addition, when the sample image is fabric pleats and the pleat height does not change significantly, the depth map generated by binocular stereo vision has low visualization and may even cause pleat deformation. These problems will directly affect the accuracy and effectiveness of 3D reconstruction, making it impossible to promote and popularize binocular stereo vision methods in the fabric appearance inspection industry. Summary of the invention

[0006] The purpose of the present invention is to provide a method for 3D reconstruction of fabric surface morphology under multi-view progressive matching. In view of the defects in the prior art, the present invention uses the fabric to progressively pair binocular views in a stable motion to iteratively match the high-precision reconstruction method of the fabric 3D morphology. By integrating the machine vision mechanism, a 3D reconstruction technology for fabric surface morphology is constructed for wrinkled fabric images, which avoids the interference of fabric texture, pattern and color, improves the recognition of fabric surface feature points and surface accuracy.

[0007] In order to solve the above technical problems, the following technical solutions are adopted:

[0008] A method for three-dimensional reconstruction of fabric surface morphology under multi-view progressive matching, characterized by comprising the following steps:

[0009] (1) Build a closed shooting device and unify the shooting environment;

[0010] (2) Fabric image acquisition: Collect fabric samples, process them uniformly, and then use a camera to complete fabric image acquisition;

[0011] (3) Image preprocessing: preprocessing the collected fabric images;

[0012] (4) Camera calibration and image correction: The camera of the shooting device is calibrated using the checkerboard calibration method to obtain the camera's internal and external parameters, and the collected fabric images are corrected;

[0013] (5) Feature extraction and stereo matching: Extract appropriate feature points from the acquired left and right views of the fabric, perform feature matching on the two images, and then progressively match the sequence pixel pairs, fuse them to form a fabric point cloud, and finally obtain the optimal matching pixel pair;

[0014] (6) Generate disparity map: Calculate the fabric disparity map using the position difference between the left and right view pixels in the set of optimal matching pixel pairs;

[0015] (7) Point cloud generation: Calculate the depth map of each pixel value based on the view difference obtained in step (6), thereby obtaining a depth map of the fabric surface morphology and forming a three-dimensional point cloud of the fabric surface morphology.

[0016] After optimization, in the step (1), the shooting device comprises a box body and a camera, an LED light source, a fabric stage and a control unit installed in the box body, the camera is a split dual camera or an integrated binocular camera, which respectively shoots left and right views of the fabric sample, the fabric stage adopts a drawer type, and the control unit comprises a power switch, a light source switch and a stage switch; its working process comprises: firstly connecting the box body to an external power source, turning on the light source switch, and connecting the camera to a computer via a data cable, and then placing the fabric sample into the fabric stage, controlling the fabric sample to enter the box body through the stage switch, and collecting and saving the fabric image on the computer side.

[0017] After optimization, in the step (2) of image acquisition, when the fabric carrier drives the fabric sample to move, the camera acquires multiple groups of left and right view image pairs at set intervals, and each group of image pairs is obtained by synchronous shooting of the camera to ensure the corresponding relationship of each group of image pairs.

[0018] After optimization, the step (3) of image preprocessing includes:

[0019] 1) Image cropping: Use image processing algorithms to identify and crop the effective area of ​​the fabric image and remove the background;

[0020] 2) Lighting equalization: Apply algorithms such as histogram equalization to adjust the brightness distribution of the fabric image to achieve lighting balance;

[0021] 3) Image enhancement: Use sharpening filters to improve the clarity of fabric image details;

[0022] 4) Image denoising: Use Gaussian filtering denoising algorithm to remove random noise in fabric images.

[0023] After optimization, the camera is calibrated in step (4), a standard checkerboard calibration plate is selected, the checkerboard calibration plate is placed at different positions and angles, and the image of the checkerboard calibration plate is photographed with a camera; the checkerboard corner points are extracted from each checkerboard calibration plate image by using image processing technology, and the sub-pixel corner point precision is performed; the coordinates of the corner points extracted from the checkerboard calibration plate image and the actual coordinates are used to establish an error function, with the goal of minimizing the projection error, and the intrinsic parameters of the camera are calculated by using an optimization algorithm such as the least squares method; the extrinsic parameters of the camera are calculated by using the three-dimensional coordinates of the checkerboard corner points in the calibration image and the corresponding image coordinates, using the PnP algorithm.

[0024] After optimization, the step (4) image correction is performed, and the camera distortion matrix and rotation matrix are calculated according to the camera calibration result; the collected fabric image is corrected using the distortion correction algorithm to eliminate the distortion and image offset; the fabric image is corrected using the intrinsic parameter matrix and the distortion correction result to eliminate the distortion and offset in the fabric image, and the corrected pixel coordinates are calculated.

[0025] After optimization, the step (5) is completed with the help of a feature extractor, a related pyramid and a GRU-based update operator network, including:

[0026] 1) Feature extraction: Two independent feature extractors are used, the feature encoder and the context encoder. The feature encoder is applied to the left view and the right view to extract feature maps from them. The context encoder enhances the context information of the feature map and generates a dense feature map. The feature extractor structure includes a residual block and a downsampling layer. The residual block is used to extract complex features, and the downsampling layer is used to reduce the image resolution layer by layer.

[0027] 2) Feature map generation: The extracted feature maps are processed by the context encoder to form dense feature maps, which are used to construct multi-dimensional correlation volumes. The correlation pyramid is constructed from the relevant features extracted from each biological image. The correlation pyramid contains feature maps of different scales, providing multi-level information for disparity estimation.

[0028] 3) Related feature calculation: Generate related features by calculating the similarity between the feature maps in the left and right views. The similarity calculation formula is as follows:

[0029] C(x,y,d)=||F L (x,y)-F R (xd,y)||

[0030] Among them, F L (x,y) and F R (xd, y) represent the feature vectors of the pixels in the left and right views respectively, and d represents the disparity value;

[0031] 4) Iterative update: In each iteration, the previous matching result is used as the initial condition, the matching cost is recalculated, and the matching pixel set is updated; it is iterated multiple times until the matching result converges and the optimal matching pixel pair is obtained; the update operator network consists of three fully connected layers, which are used to calculate the similarity between the extracted features;

[0032] 5) Progressive matching:

[0033] a) Preliminary matching: Using the feature map extracted by the feature extractor, the similarity between the features in the left and right views is calculated by updating the connection layer of the operator network to determine the preliminary matching pixel set;

[0034] b) Progressive matching of multiple image pairs: Match multiple image pairs, process each image group in turn, extract matching pixel pairs, add the matching results of each image pair to the matching pixel set, and record the confidence of each matching pixel pair; compare the confidence of the newly matched pixel pairs with the existing matching results, evaluate the reliability of each matching pixel pair, and update the matching pixel set; through the progressive matching mechanism, gradually optimize the matching results until the matching results converge and obtain the optimal matching pixel pairs.

[0035] After optimization, the step (6) comprises:

[0036] 1) Disparity calculation: Matching pixel pairs are obtained. Using the matching pixel pairs obtained in the above feature extraction and stereo matching steps, the pixels in the left view ((x L ,y L ) and the corresponding pixel in the right view (x R ,y R );The disparity value d is calculated by the horizontal position difference of the corresponding pixels in the left and right views:

[0037] d = x L -x R

[0038] Generate a disparity map, where each pixel position corresponds to a disparity value d, which represents the disparity of that point;

[0039] 2) Disparity map generation: Calculate the disparity values ​​of all pixels in the entire image to form a complete disparity map.

[0040] After optimization, the step (6) further comprises:

[0041] 3) Consistency check: reverse verify the disparity map, recalculate the disparity map d' by exchanging the positions of the left and right views, and compare it with the original disparity map d; for each pixel point, if the disparity values ​​calculated in the forward and reverse directions differ greatly, the match is considered unreliable; set a threshold for the consistency check, and screen out reliable matching points based on the results of the reverse verification; for unreliable matching points, use interpolation to correct them to ensure the overall consistency and accuracy of the disparity map; construct mind associations by calculating the visual similarities between all pixel pairs, and limit the calculation of associations to pixels with the same coordinates; given the feature maps extracted from the left and right views respectively, modify the construction of the mind volume and calculate the three-dimensional correlation volume by limiting the calculation of the inner product to feature vectors sharing the same first index.

[0042] After optimization, the step (7) comprises:

[0043] 1) Disparity map acquisition: Use the disparity map generated above to obtain the disparity value d of each pixel; calculate the depth value z of each pixel based on the disparity map and the parameters obtained by camera calibration; use the baseline distance B and focal length f of the camera, as well as the disparity d, and the calculation formula is as follows:

[0044]

[0045] According to the above formula, the depth value of each pixel is calculated to generate the corresponding depth map; each pixel value Z(x,y) in the depth map represents the depth information of the pixel point. The smaller the depth value, the closer it is to the camera. Conversely, the larger the depth value, the farther it is from the camera.

[0046] 2) 3D point cloud generation: Convert the depth map into a 3D point cloud to obtain the 3D morphology of the fabric surface; use the depth value z of each pixel and the camera calibration parameters to calculate the 3D coordinates (x, y, z); the point cloud contains the 3D coordinates of all pixels and reflects the 3D morphology of the fabric; optimize the generated 3D model to eliminate noise and outliers.

[0047] The above technical solution has the following beneficial effects:

[0048] High-precision stereo matching and depth calculation: This invention uses a multi-channel graph convolutional neural network and a GRU-based update operator to achieve high-precision feature extraction and stereo matching, which can effectively extract and match feature points in the left and right views, thereby generating an accurate disparity map. Combined with the camera calibration parameters, an accurate depth map is calculated to achieve high-precision 3D reconstruction of the fabric surface morphology.

[0049] Multi-view fusion and comprehensive 3D reconstruction: This invention adopts multi-view fusion technology, integrates depth maps from multiple perspectives, and generates a complete 3D point cloud and mesh model through point cloud registration and triangulation methods. This method can not only restore the surface morphology of fabrics of different appearance types such as solid color and printed, but also ensure the continuity and accuracy of the 3D model, and improve the stability and reliability of 3D reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The present invention will be further described below in conjunction with the accompanying drawings:

[0051] Figure 1 is a flow chart of the method of the present invention;

[0052] Figure 2 is a structural schematic diagram of a photographing device;

[0053] Figure 3 It is a schematic diagram of the structure of the shooting device when viewed from above;

[0054] Figure 4 is a schematic diagram of the effective area of ​​the fabric;

[0055] Figure 5 is the corrected image;

[0056] Figure 6 is the disparity map;

[0057] Figure 7 is the depth map;

[0058] Figure 8 For point cloud.

[0059] The accompanying drawings are marked as: box 1, camera 2, fabric carrier 3, power switch 4, light source switch 5, carrier switch 6, computer 7, LED light source 8. DETAILED DESCRIPTION

[0060] The present invention aims to provide a method for 3D reconstruction of fabric surface morphology under multi-view progressive matching, which uses the paired binocular views of the fabric in steady motion to iteratively match the high-precision reconstruction method of the 3D morphology of the fabric. By integrating the machine vision mechanism, a 3D reconstruction technology for fabric surface morphology is constructed for wrinkled fabric images, which avoids the interference of fabric texture, pattern and color, improves the recognition of fabric surface feature points and surface accuracy.

[0061] The technical solution of the present invention is described in detail below in conjunction with specific embodiments:

[0062] (1) Build a closed shooting device and unify the shooting environment;

[0063] like Figure 2 and Figure 3 As shown, the shooting device includes a box 1 and a camera 2, an LED light source 8, a fabric stage 3 and a control unit installed in the box 1. The camera 2 is a split dual digital camera 2 or an integrated binocular camera 2, which respectively shoots left and right views of the fabric sample. The fabric stage 3 adopts a drawer type, and the control unit includes a power switch 4, a light source switch 5 and a stage switch 6;

[0064] The working process includes: firstly connecting the box 1 to an external power supply, turning on the light source switch 5, and connecting the camera 2 to the computer 7 via a data cable, then placing the fabric sample on the fabric carrier 3, controlling the fabric sample to enter the box 1 through the carrier switch 6, and collecting and saving the fabric image on the computer 7.

[0065] In consideration of environmental consistency, a closed box 1 structure is selected to avoid external light interference; the inner wall is coated with a sub-gray coating or other anti-reflective materials to reduce the impact of internal reflections on imaging.

[0066] In terms of illumination uniformity, two parallel configured LED light sources 8 are used to ensure that the optical fibers are evenly distributed on the fabric surface; a diffuser plate can be installed in front of the light source to further homogenize the illumination;

[0067] In terms of test fixation, a drawer-type fabric carrier 3 is designed to ensure the stability of the fabric during the shooting process.

[0068] In terms of ensuring high definition of the image, a high-resolution digital camera 2 or an integrated binocular camera 2 is used to ensure a clear image. The optimal distance between the camera 2 and the fabric is set to avoid image blur.

[0069] (2) Fabric image acquisition:

[0070] 1) Sample preparation: collect fabric samples, number them, cut them, wash them, dry them, etc.

[0071] 2) Place the sample: Place the fabric flatly on the fabric carrier 3.

[0072] 3) Automatic entry and exit: Under the control of the control unit, the fabric carrier 3 drives the fabric sample to enter the box 1 smoothly.

[0073] 4) Image acquisition: During the movement of the stage, the camera 2 acquires multiple sets of left and right view image pairs at set intervals. Each set of image pairs is taken synchronously by the left and right cameras 2 to ensure the correspondence between each set of images.

[0074] (3) Image preprocessing: preprocessing the collected fabric images;

[0075] Image cropping: Use image processing algorithms to identify and crop the effective area of ​​the fabric and remove the background.

[0076] Lighting equalization: Apply algorithms such as histogram equalization to adjust the brightness distribution of the image to make the lighting more uniform.

[0077] Image enhancement: Use sharpening filters to improve the clarity of image details.

[0078] Image denoising: Use denoising algorithms such as Gaussian filtering to remove random noise in images and improve image quality.

[0079] (4) Camera 2 calibration and image correction:

[0080] 1) Camera 2 calibration: Use a standard checkerboard calibration plate to ensure that the calibration plate is flat, non-reflective and non-deformed. Place the checkerboard calibration plate at different positions and angles, and use two digital cameras 2 or integrated binocular cameras 2 to capture images of the checkerboard calibration plate, ensuring that different parts of the checkerboard in each image are fully captured and cover a wide range of viewing angles; use image processing technology to extract checkerboard corner points from each checkerboard calibration plate image, perform sub-pixel corner point precision, and improve calibration accuracy. Use the corner point coordinates extracted from the checkerboard calibration plate image and the actual coordinates to establish an error function, with the goal of minimizing the projection error, and calculate the intrinsic parameters of camera 2 through optimization algorithms such as the least squares method, including focal length, principal point position, and distortion coefficient; use the three-dimensional coordinates of the checkerboard corner points in the calibration image and the corresponding image coordinates, and use the PnP (Perspective-n-Point) algorithm to calculate the extrinsic parameters of camera 2.

[0081] 2) Image correction: According to the calibration results of camera 2, the distortion matrix and rotation matrix of camera 2 are calculated; the collected fabric image is corrected using the distortion correction algorithm to eliminate distortion and image offset; the fabric image is corrected using the intrinsic parameter matrix and the distortion correction results to eliminate distortion and offset in the fabric image, and the corrected pixel coordinates are calculated so that the straight lines in the image remain in a straight line shape, thereby improving the accuracy and visual effect of the image.

[0082] (5) Feature extraction and stereo matching: Extract appropriate feature points from the acquired left and right views of the fabric, perform feature matching on the two images, and then progressively match the sequence pixel pairs, fuse them to form a fabric point cloud, and finally obtain the optimal matching pixel pair;

[0083] This needs to be done with the help of a feature extractor, a related pyramid, and a GRU-based update operator network. The steps include:

[0084] 1) Feature extraction: Two independent feature extractors are used, the feature encoder and the context encoder. The feature encoder is applied to the left and right views to extract high-quality feature maps. The context encoder further enhances the context information of the feature map to generate a dense feature map. The feature extractor structure includes multiple residual blocks and downsampling layers, which together define the framework of feature extraction and information fusion. The residual block is used to extract complex features, and the downsampling layer is used to reduce the image resolution layer by layer to extract multi-scale features.

[0085] 2) Feature map generation: The extracted feature maps are processed by the context encoder to form dense feature maps containing rich image information. These dense feature maps are used to construct multi-dimensional correlation volumes, providing rich information for disparity estimation. Correlation pyramids are constructed from the relevant features extracted from each group of biological images. The correlation pyramid contains feature maps of different scales, providing multi-level information for disparity estimation.

[0086] 3) Related feature calculation: Generate related features by calculating the similarity between the feature maps in the left and right views. The similarity calculation formula is as follows:

[0087] C(x,y,d)=||F L (x,y)-F R (xd,y)||

[0088] Among them, F L (x,y) and F R (xd, y) identifies the feature vectors of the pixels in the left and right views respectively, and d represents the disparity value; the feature maps in the correlation pyramid jointly construct a multi-dimensional correlation volume, providing a comprehensive information basis for disparity estimation.

[0089] 4) Iterative update: In each iteration, the previous matching result is used as the initial condition, the matching cost is recalculated, and the matching pixel set is updated; it is iterated multiple times until the matching result converges and the optimal matching pixel pair is obtained; the update operator network consists of three fully connected layers, which are used to calculate the similarity between the extracted features;

[0090] 5) Progressive matching:

[0091] a) Preliminary matching: Using the feature map extracted by the feature extractor, the similarity between the features in the left and right views is calculated by updating the connection layer of the operator network to determine the preliminary matching pixel set;

[0092] b) Progressive matching of multiple image pairs: Match multiple image pairs, process each image group in turn, extract matching pixel pairs, add the matching results of each image pair to the matching pixel set, and record the confidence of each matching pixel pair; compare the confidence of the newly matched pixel pairs with the existing matching results, evaluate the reliability of each matching pixel pair, and update the matching pixel set; through the progressive matching mechanism, gradually optimize the matching results until the matching results converge and obtain the optimal matching pixel pairs.

[0093] (6) Generate disparity map: Calculate the fabric disparity map using the position difference between the left and right view pixels in the best matching pixel pair set, including:

[0094] 1) Disparity calculation: Matching pixel pairs are obtained. Using the matching pixel pairs obtained in the above feature extraction and stereo matching steps, the pixels in the left view ((xL ,y L ) and the corresponding pixel in the right view (x R ,y R );The disparity value d is calculated by the horizontal position difference of the corresponding pixels in the left and right views:

[0095] d = x L -x R

[0096] Generate a disparity map, where each pixel position corresponds to a disparity value d, which represents the disparity of that point;

[0097] 2) Disparity map generation: Calculate the disparity values ​​of all pixels in the entire image to form a complete disparity map.

[0098] 3) Consistency check: reverse verify the disparity map, recalculate the disparity map d' by exchanging the positions of the left and right views, and compare it with the original disparity map d; for each pixel point, if the disparity values ​​calculated in the forward and reverse directions differ greatly, the match is considered unreliable; set a threshold for the consistency check, and screen out reliable matching points based on the results of the reverse verification; for unreliable matching points, use interpolation to correct them to ensure the overall consistency and accuracy of the disparity map; construct mind associations by calculating the visual similarities between all pixel pairs, and limit the calculation of associations to pixels with the same coordinates; given the feature maps extracted from the left and right views respectively, modify the construction of the mind volume and calculate the three-dimensional correlation volume by limiting the calculation of the inner product to feature vectors sharing the same first index.

[0099] (7) Point cloud generation: Calculate the depth map of each pixel value according to the view difference obtained in step (6), thereby obtaining the depth map of the fabric surface morphology and forming a three-dimensional point cloud of the fabric surface morphology, including:

[0100] 1) Disparity map acquisition: Use the disparity map generated above to obtain the disparity value d of each pixel; calculate the depth value z of each pixel based on the disparity map and the parameters obtained by calibration of camera 2; use the baseline distance B and focal length f of camera 2, as well as the disparity d, and the calculation formula is as follows:

[0101]

[0102] According to the above formula, the depth value of each pixel is calculated to generate the corresponding depth map; each pixel value Z(x, y) in the depth map represents the depth information of the pixel point. The smaller the depth value, the closer the distance to the camera 2 is. On the contrary, the larger the depth value, the farther the distance to the camera 2 is.

[0103] 2) 3D point cloud generation: Convert the depth map into a 3D point cloud to obtain the 3D morphology of the fabric surface; use the depth value z of each pixel and the calibration parameters of camera 2 to calculate the 3D coordinates (x, y, z); the point cloud contains the 3D coordinates of all pixels and can accurately reflect the 3D morphology of the fabric; optimize the generated 3D model to eliminate noise and outliers and improve the accuracy and quality of the model.

[0104] The above are only specific embodiments of the present invention, but the technical features of the present invention are not limited thereto. Any simple changes, equivalent replacements or modifications made based on the present invention to solve basically the same technical problems and achieve basically the same technical effects are all included in the protection scope of the present invention.

Claims

1. A method for 3D reconstruction of fabric surface morphology under multi-view progressive matching, characterized in that The steps include: (1) Build a closed shooting device and unify the shooting environment; (2) Fabric image acquisition: Collect fabric samples, process them uniformly, and then use a camera to complete fabric image acquisition; (3) Image preprocessing: preprocessing the collected fabric images; (4) Camera calibration and image correction: The camera of the shooting device is calibrated using the checkerboard calibration method to obtain the camera's internal and external parameters, and the collected fabric images are corrected; (5) Feature extraction and stereo matching: Extract appropriate feature points from the acquired left and right views of the fabric, perform feature matching on the two images, and then progressively match the sequence pixel pairs, fuse them to form a fabric point cloud, and finally obtain the optimal matching pixel pair; (6) Generate disparity map: Calculate the fabric disparity map using the position difference between the left and right view pixels in the set of optimal matching pixel pairs; (7) Point cloud generation: Calculate the depth map of each pixel value based on the view difference obtained in step (6), thereby obtaining a depth map of the fabric surface morphology and forming a three-dimensional point cloud of the fabric surface morphology.

2. The method for three-dimensional reconstruction of fabric surface morphology under multi-view progressive matching according to claim 1, characterized in that: In the step (1), the photographing device comprises a box and a camera, an LED light source, a fabric stage and a control unit installed in the box, wherein the camera is a split dual camera or an integrated binocular camera, which respectively photographs the left and right views of the fabric sample, the fabric stage is a drawer type, and the control unit comprises a power switch, a light source switch and a stage switch; The working process includes: firstly connecting the box to an external power source, turning on the light source switch, and connecting the camera to a computer via a data cable, then placing a fabric sample on the fabric carrier, controlling the fabric sample to enter the box through the carrier switch, and collecting and saving fabric images on the computer.

3. The method for three-dimensional reconstruction of fabric surface morphology under multi-view progressive matching according to claim 2, characterized in that: In the step (2) of image acquisition, when the fabric carrier drives the fabric sample to move, the camera acquires multiple groups of left and right view image pairs at set intervals, and each group of image pairs is obtained by synchronously shooting by the camera to ensure the corresponding relationship between each group of image pairs.

4. The method for three-dimensional reconstruction of fabric surface morphology under multi-view progressive matching according to claim 1, characterized in that: The step (3) of image preprocessing comprises: 1) Image cropping: Use image processing algorithms to identify and crop the effective area of ​​the fabric image and remove the background; 2) Lighting equalization: Apply algorithms such as histogram equalization to adjust the brightness distribution of the fabric image to achieve lighting balance; 3) Image enhancement: Use sharpening filters to improve the clarity of fabric image details; 4) Image denoising: Use Gaussian filtering denoising algorithm to remove random noise in fabric images.

5. The method for three-dimensional reconstruction of fabric surface morphology under multi-view progressive matching according to claim 1, characterized in that: In the step (4), the camera is calibrated by selecting a standard checkerboard calibration plate, placing the checkerboard calibration plate at different positions and angles, and photographing images of the checkerboard calibration plate with a camera; extracting checkerboard corner points from each checkerboard calibration plate image using image processing technology, performing sub-pixel-level corner point precision, and establishing an error function using the coordinates of the corner points extracted from the checkerboard calibration plate image and the actual coordinates, with the goal of minimizing the projection error, and calculating the intrinsic parameters of the camera using an optimization algorithm such as the least squares method; The camera's extrinsic parameters are calculated by calibrating the three-dimensional coordinates of the checkerboard corner points in the image and the corresponding image coordinates using the PnP algorithm.

6. The method for three-dimensional reconstruction of fabric surface morphology under multi-view progressive matching according to claim 5, characterized in that: The step (4) image correction comprises calculating the camera distortion matrix and the rotation matrix according to the camera calibration result; using the distortion correction algorithm to correct the collected fabric image to eliminate the distortion and image offset; using the intrinsic parameter matrix and the distortion correction result to correct the fabric image to eliminate the distortion and offset in the fabric image, and calculating the pixel coordinates after correction.

7. The method for three-dimensional reconstruction of fabric surface morphology under multi-view progressive matching according to claim 1, characterized in that: The step (5) is completed with the help of a feature extractor, a related pyramid and a GRU-based update operator network, including: 1) Feature extraction: Two independent feature extractors are used, the feature encoder and the context encoder. The feature encoder is applied to the left view and the right view to extract feature maps from them. The context encoder enhances the context information of the feature map and generates a dense feature map. The feature extractor structure includes a residual block and a downsampling layer. The residual block is used to extract complex features, and the downsampling layer is used to reduce the image resolution layer by layer. 2) Feature map generation: The extracted feature maps are processed by the context encoder to form dense feature maps, which are used to construct multi-dimensional correlation volumes. The correlation pyramid is constructed from the relevant features extracted from each biological image. The correlation pyramid contains feature maps of different scales, providing multi-level information for disparity estimation. 3) Related feature calculation: Generate related features by calculating the similarity between the feature maps in the left and right views. The similarity calculation formula is as follows: C(x,y,d)=||F L (x,y)-F R (x-d,y)|| Among them, F L (x,y) and F R (xd, y) identifies the feature vectors of the pixels in the left and right views, respectively, indicating the disparity value; 4) Iterative update: In each iteration, the previous matching result is used as the initial condition, the matching cost is recalculated, and the matching pixel set is updated; it is iterated multiple times until the matching result converges and the optimal matching pixel pair is obtained; the update operator network consists of three fully connected layers, which are used to calculate the similarity between the extracted features; 5) Progressive matching: a) Preliminary matching: Using the feature map extracted by the feature extractor, the similarity between the features in the left and right views is calculated by updating the connection layer of the operator network to determine the preliminary matching pixel set; b) Progressive matching of multiple image pairs: Match multiple image pairs, process each image group in turn, extract matching pixel pairs, add the matching results of each image pair to the matching pixel set, and record the confidence of each matching pixel pair; compare the confidence of the newly matched pixel pairs with the existing matching results, evaluate the reliability of each matching pixel pair, and update the matching pixel set; through the progressive matching mechanism, gradually optimize the matching results until the matching results converge and obtain the optimal matching pixel pairs.

8. The method for three-dimensional reconstruction of fabric surface morphology under multi-view progressive matching according to claim 1, characterized in that: The step (6) comprises: 1) Disparity calculation: Matching pixel pairs are obtained. Using the matching pixel pairs obtained in the above feature extraction and stereo matching steps, the pixels in the left view ((x L ,y L ) and the corresponding pixel in the right view (x R ,y R ); The disparity value d is calculated from the horizontal position difference of corresponding pixels in the left and right views: d=x L -x R Generate a disparity map, where each pixel position corresponds to a disparity value d, which represents the disparity of that point; 2) Disparity map generation: Calculate the disparity values ​​of all pixels in the entire image to form a complete disparity map.

9. The method for three-dimensional reconstruction of fabric surface morphology under multi-view progressive matching according to claim 8, characterized in that: The step (6) further comprises: 3) Consistency check: reverse verify the disparity map, recalculate the disparity map d' by exchanging the positions of the left and right views, and compare it with the original disparity map d; for each pixel point, if the disparity values ​​calculated in the forward and reverse directions differ greatly, the match is considered unreliable; set a threshold for the consistency check, and screen out reliable matching points based on the results of the reverse verification; for unreliable matching points, use interpolation to correct them to ensure the overall consistency and accuracy of the disparity map; construct mind associations by calculating the visual similarities between all pixel pairs, and limit the calculation of associations to pixels with the same coordinates; given the feature maps extracted from the left and right views respectively, modify the construction of the mind volume and calculate the three-dimensional correlation volume by limiting the calculation of the inner product to feature vectors sharing the same first index.

10. The method for three-dimensional reconstruction of fabric surface morphology under multi-view progressive matching according to claim 8, characterized in that: The step (7) comprises: 1) Disparity map acquisition: Use the disparity map generated above to obtain the disparity value d of each pixel; calculate the depth value z of each pixel based on the disparity map and the parameters obtained by camera calibration; use the baseline distance B and focal length f of the camera, as well as the disparity d, and the calculation formula is as follows: According to the above formula, the depth value of each pixel is calculated to generate the corresponding depth map; each pixel value Z(x,y) in the depth map represents the depth information of the pixel point. The smaller the depth value, the closer it is to the camera. Conversely, the larger the depth value, the farther it is from the camera. 2) 3D point cloud generation: Convert the depth map into a 3D point cloud to obtain the 3D morphology of the fabric surface; use the depth value z of each pixel and the camera calibration parameters to calculate the 3D coordinates (x, y, z); the point cloud contains the 3D coordinates of all pixels and reflects the 3D morphology of the fabric; optimize the generated 3D model to eliminate noise and outliers.

Images