Hand-embroidery image analysis and digital reproduction system

Abstract

The application discloses a hand embroidery image analysis and digital reproduction system and belongs to the technical field of image processing.The system comprises an image acquisition module, a texture feature extraction module, a stitch identification module, a stitch three-dimensional reconstruction module, a digital representation module and an embroidery diagram generation module.The system realizes accurate identification of complex stitches by designing embroidery specific texture descriptors, accurately recovers the spatial hierarchy of embroidery lines by using three-dimensional reconstruction technology based on light and shadow, and can generate standardized embroidery diagrams for teaching and copying.Compared with the prior art, the stitch identification accuracy of the application is increased by more than 15%, and the three-dimensional reconstruction accuracy reaches 0.05mm.The application effectively solves the problems of insufficient texture analysis accuracy and inaccurate three-dimensional structure expression in embroidery digitization, and provides advanced technical support for the protection and inheritance of embroidery cultural heritage.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and image processing technology, specifically to a system for analyzing and digitally reproducing images of hand embroidery, which is particularly suitable for the digital preservation, skill inheritance, and modern design innovation of traditional hand embroidery works. Background Technology

[0002] Existing embroidery digitization technologies mainly include the following methods: The first is embroidery image classification and recognition methods based on deep learning. For example, Chinese invention CN120068023A discloses a full-process protection method for embroidery digitization that integrates deep learning and blockchain evidence storage. This method uses multimodal data acquisition technology to obtain three-dimensional point cloud data, needle tip movement trajectory, and material characteristics of embroidery works. It performs data preprocessing through an improved adaptive Kalman filter algorithm, uses 3D convolutional neural networks and generative adversarial networks to achieve needlework recognition and embroidery reproduction, and manages digital ownership through a blockchain protocol. However, this method has the following shortcomings: It focuses on the entire process management of data acquisition and blockchain notarization, but has limitations in the refined analysis of embroidery textures, especially for complex and varied embroidery stitch patterns. It lacks texture descriptors specifically designed for embroidery features, resulting in insufficient ability to identify subtle texture differences. Although this method uses 3D laser scanning to acquire point cloud data, in the reconstruction of the three-dimensional structure of the stitches, it mainly relies on the geometric information of the point cloud data, failing to fully utilize the optical properties and shadow information of the embroidery thread, making it difficult to accurately reconstruct the true three-dimensional hierarchical relationship of multiple layers of superimposed embroidery threads. The main purpose of this method is to achieve digital protection and blockchain notarization of embroidery; its function in generating practical embroidery diagrams from digital models is weak, and it cannot effectively support the teaching and inheritance of embroidery skills and modern design applications.

[0003] The second type is embroidery analysis based on traditional image processing. These methods typically employ traditional algorithms such as edge detection and texture analysis to process embroidery images. However, due to the complex and layered textures of embroidery, traditional methods struggle to accurately extract the unique texture features of embroidery, particularly in needlework recognition and thread direction analysis, where the results are unsatisfactory. Furthermore, these methods usually only perform two-dimensional image analysis and cannot reconstruct the three-dimensional structure of the embroidery, making it difficult to fully reproduce the true form of the embroidery work.

[0004] The third method is a 3D reconstruction method based on point cloud data. This method acquires 3D point cloud data of the embroidery work using 3D scanning equipment and then performs mesh reconstruction. However, the 3D model obtained by this method mainly reflects the overall shape of the embroidery surface, and often cannot accurately represent the fine stitch structure, needlework details, and the layering relationship of multiple stitches. In addition, the acquisition of point cloud data requires sophisticated equipment and the processing is complex, making it difficult to apply widely.

[0005] In summary, existing technologies for embroidery image analysis and digital reproduction suffer from the following main problems: a lack of analytical methods specifically designed for embroidery texture features, and limited ability to identify complex stitches; insufficient precision in expressing the microstructure and hierarchical relationships of embroidery threads using 3D reconstruction technology; and inadequate conversion capabilities from digital models to practical embroidery diagrams, limiting the technology's value in embroidery teaching and design applications. Therefore, there is an urgent need for a digital system capable of accurately analyzing embroidery texture features, reconstructing the 3D structure of stitches, and effectively generating practical embroidery diagrams to meet the needs of embroidery cultural heritage protection, skill transmission, and modern design innovation. Summary of the Invention

[0006] The purpose of this invention is to provide a system for analyzing and digitally reproducing hand embroidery images. By designing specific embroidery texture descriptors, it achieves accurate identification of complex stitches. Through three-dimensional reconstruction technology of stitches, it accurately restores the spatial hierarchical structure of embroidery threads. Through the embroidery diagram generation function, it transforms digital results into practical teaching and design resources, thereby solving the problems of insufficient accuracy in embroidery texture analysis, inaccurate three-dimensional structure reconstruction, and lack of practical diagram generation capabilities in existing technologies.

[0007] To achieve the above objectives, the present invention provides a system for analyzing and digitally reproducing hand embroidery images. The system includes an image acquisition module, a texture feature extraction module, a stitch recognition module, a three-dimensional reconstruction module for stitches, a digital representation module, and an embroidery pattern generation module.

[0008] The image acquisition module acquires high-resolution image data of the embroidery works, including visible light and multispectral images, providing high-quality raw data for subsequent analysis. The texture feature extraction module extracts multi-scale features of the embroidery texture based on the high-resolution image data, generating embroidery-specific texture descriptors. These descriptors include stitch direction features, density distribution features, and layering relationship features, accurately representing the uniqueness of the embroidery texture. The stitch recognition module matches the stitch types in a preset stitch template library based on the embroidery-specific texture descriptors, identifying the types and distribution areas of stitches in the embroidery work, achieving accurate classification of complex stitches. The stitch 3D reconstruction module, based on the lighting and shadow information in the high-resolution image data, calculates the 3D spatial position and hierarchical structure of the embroidery threads, generating a stitch 3D model to accurately restore the three-dimensional form of the embroidery. The digital representation module, based on the stitch 3D model, constructs a digital 3D representation of the embroidery work, including geometric models and texture maps, achieving complete digital preservation of the embroidery work. The embroidery pattern generation module generates standardized embroidery patterns based on stitch types, stitch distribution areas, and digital 3D representations. These patterns include stitch symbols, stitch direction indicators, and color markings, providing practical guidance for embroidery teaching and replication.

[0009] Through the above technical solutions, this invention achieves accurate image analysis and high-quality digital reproduction of handmade embroidery works, providing effective technical support for the protection of embroidery cultural heritage, the inheritance of skills, and modern design innovation.

[0010] Compared with the prior art, the present invention has the following beneficial effects:

[0011] This invention designs specific texture descriptors for embroidery, which can accurately capture the unique features of embroidery textures, including stitch direction, density distribution, and layering relationship. Compared with the existing technology that uses general texture analysis methods, this invention improves the accuracy of embroidery stitch recognition by more than 15%, especially for the subtle differences in traditional stitches such as flat stitch, satin stitch, and random stitch, and can achieve more accurate differentiation and recognition.

[0012] This invention employs a three-dimensional reconstruction technique based on lighting and shadow information to accurately calculate the three-dimensional spatial position and hierarchical structure of embroidery threads. Compared with geometric reconstruction methods that rely on point cloud data, this invention can better restore the true hierarchical relationship of multi-layered embroidery threads, improving the accuracy of the three-dimensional model by about 20%. Especially when dealing with complex multi-layered embroidery threads, it can clearly show the spatial distribution and interrelationship of each layer of threads.

[0013] This invention integrates an embroidery pattern generation module, which can convert identified stitches and reconstructed 3D models into standardized embroidery patterns—a function lacking in existing technologies. The generated patterns include stitch symbols, stitch direction indicators, and color markings, and can be directly used for embroidery teaching and artwork reproduction, greatly promoting the inheritance and dissemination of traditional embroidery techniques, while also providing modern designers with abundant creative materials and reference resources.

[0014] The system architecture of this invention features excellent modularity and scalability, with clear interfaces between functional modules, facilitating functional expansion and performance optimization based on actual application needs. The system can be flexibly deployed on cloud servers or edge computing devices, supporting remote image uploading and digital result downloading, adapting to application needs of different scales and scenarios, and providing a solid technical foundation for the promotion and application of embroidery digitization technology. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the overall architecture of the hand embroidery image analysis and digital reproduction system of the present invention;

[0016] Figure 2 This is a schematic diagram of the texture feature extraction module of the present invention;

[0017] Figure 3 This is a schematic diagram of the workflow of the needle recognition module of the present invention;

[0018] Figure 4This is a schematic diagram of the structure of the three-dimensional reconstruction module for line traces of the present invention;

[0019] Figure 5 This is a schematic diagram of the workflow of the embroidery pattern generation module of the present invention.

[0020] Figure 6 This is a schematic diagram of the system processing flow of the present invention. Detailed Implementation

[0021] Please refer to the attached document. Figures 1-6 The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments, but is not limited thereto.

[0022] like Figure 1 As shown, the hand embroidery image analysis and digital reproduction system provided by the present invention includes an image acquisition module 1, a texture feature extraction module 2, a stitch recognition module 3, a three-dimensional reconstruction module 4, a digital representation module 5, and an embroidery pattern generation module 6.

[0023] Image acquisition module 1 is used to acquire high-resolution image data of the embroidery work. In one embodiment of the present invention, image acquisition module 1 uses an industrial camera with a resolution of no less than 48 million pixels for image acquisition, equipped with a macro lens to capture the detailed texture of the embroidery. Image acquisition includes two types: visible light images and multispectral images. Visible light images use standard RGB three-channel imaging to record the color and basic texture information of the embroidery. Multispectral images use spectral imaging in the 400nm to 2500nm wavelength band, and obtain material information by analyzing the reflection characteristics of the embroidery thread and fabric in different wavelength bands.

[0024] A multi-angle shooting strategy was employed during image acquisition. The embroidery artwork was photographed from multiple angles, including 0°, 15°, 30°, and 45°, by rotating the shooting platform or moving the camera. Three to five images were captured from each angle for subsequent 3D reconstruction. Adjustable LED ring light sources were used for illumination, with a color temperature set to 5500K. Illumination uniformity was controlled within ±5% to ensure consistent image quality.

[0025] In a preferred embodiment, the image acquisition module 1 is further equipped with a calibration module for distortion correction, color calibration, and scale calibration of the acquired images. Distortion correction uses a checkerboard calibration board for camera calibration, obtaining the camera's intrinsic parameters and distortion coefficients, and then performing distortion correction on the images. Color calibration uses a standard color chart for white balance adjustment and color restoration to ensure color consistency across different batches of images. Scale calibration establishes the correspondence between pixels and actual physical dimensions by placing a standard-sized reference object next to the embroidery work, facilitating subsequent quantitative analysis.

[0026] like Figure 2As shown, the texture feature extraction module 2 includes a multi-scale analysis unit, a direction feature extraction unit, a density calculation unit, and a layer recognition unit. The texture feature extraction module 2 receives high-resolution image data provided by the image acquisition module 1, extracts multi-scale features of the embroidery texture, and generates a specific embroidery texture descriptor.

[0027] A multi-scale analysis unit performs multi-scale decomposition on high-resolution image data to obtain texture information at different scales. In an embodiment of the invention, a Gabor filter bank is used for multi-scale texture analysis. The Gabor filter bank includes filters in 8 directions (0°, 22.5°, 45°, 67.5°, 90°, 112.5°, 135°, 157.5°) and 5 scales (frequency 0.05, 0.1, 0.2, 0.4, 0.8 cycles / pixel, respectively), for a total of 40 filters. The Gabor filter bank is applied to the input image to obtain 40 feature response maps, each reflecting the texture information of the image at a specific direction and scale.

[0028] In one embodiment of the present invention, the Gabor filter is generated according to the principle of two-dimensional Gaussian function modulation of sine waves. The size of the filter is adaptively adjusted according to the frequency, with a larger size for low-frequency filters (e.g., 31×31 pixels) and a smaller size for high-frequency filters (e.g., 15×15 pixels). By comprehensively analyzing the filter responses at different scales and directions, the diverse characteristics of embroidery textures can be effectively captured.

[0029] The directional feature extraction unit is used to extract the directional features of the embroidery stitches. Based on the response results of the Gabor filter bank, the principal direction at each pixel position is calculated. The principal direction is determined as follows: for each pixel, the response intensity of the eight directional filters is statistically analyzed, and the direction with the strongest response is the principal direction of that pixel. Simultaneously, the direction corresponding to the second strongest response is calculated as an auxiliary direction to describe the complexity of the stitch. For stitches with multi-directional features, such as random stitch embroidery, the difference between the principal and auxiliary directions is relatively small; while for single-directional stitches, such as flat stitch embroidery, the principal direction dominates.

[0030] The directional features also include a directional consistency index, which is measured by calculating the standard deviation of directional changes within a local neighborhood. Regions with high directional consistency correspond to regular stitches such as flat stitch and nested stitch, while regions with low directional consistency correspond to irregular stitches such as random stitch and skipped stitch. In a preferred embodiment, the local neighborhood is set as a 15×15 pixel window, and the sliding window traverses the entire image to generate a directional consistency distribution map.

[0031] The density calculation unit is used to calculate the stitch density distribution of the embroidery area. Stitch density is defined as the total length of embroidery threads per unit area, reflecting the density of the embroidery. The density calculation employs a local energy statistical method, performing energy integration on the Gabor filter response map. Specifically, the amplitudes of 40 characteristic response maps are squared and summed to obtain a local texture energy map. Regions with high texture energy correspond to densely stitched embroidery areas, while regions with low texture energy correspond to sparsely stitched or exposed fabric areas.

[0032] By setting energy thresholds, the image is segmented into three categories: dense embroidery areas, medium-density areas, and sparse areas. In a preferred embodiment, the threshold for dense areas is set to 1.5 times the average energy value, and the threshold for sparse areas is set to 0.5 times the average energy value. Density information is used not only for needlework recognition but also to guide the generation of embroidery patterns, ensuring that the patterns accurately reflect the density characteristics of the original work.

[0033] The layering recognition unit is used to identify the overlapping relationships of multiple layers of embroidery threads. Embroidery works often contain multiple overlapping stitches, which appear as complex textures in the image. Layering recognition employs frequency domain analysis, performing a Fourier transform on the image to analyze the energy distribution of the spectrum. Overlapping regions are represented by multiple energy peaks in the spectrum; the number and location of these peaks reflect the number of layers and the dominant direction of each layer.

[0034] In one embodiment of the present invention, a short-time Fourier transform is used to perform local spectral analysis on the image, with a window size of 32×32 pixels and a step size of 16 pixels. Peak detection is performed on the spectrum of each local window, and regions with more than two peaks are marked as multi-layered overlapping regions. Combining directional and density features, the number of overlapping layers and the texture features of each layer are further determined, providing hierarchical information for subsequent 3D reconstruction.

[0035] Through the collaborative work of the aforementioned multi-scale analysis unit, orientation feature extraction unit, density calculation unit, and layering recognition unit, texture feature extraction module 2 generates an embroidery-specific texture descriptor that includes stitch orientation features, density distribution features, and layering relationship features. This descriptor is represented as a feature vector with a dimension of 128, including 40 Gabor response features, 8 orientation histogram features, 16 density distribution features, and 64 layering relationship features, enabling a comprehensive and accurate characterization of the uniqueness of the embroidery texture.

[0036] like Figure 3 As shown, the stitch recognition module 3 includes a feature matching unit and a classification decision unit, used to identify the stitch type based on the specific texture descriptor of the embroidery. The stitch recognition module 3 is connected to a stitch template library, which stores standard texture features and stitch parameters of various traditional stitches.

[0037] The needlework template library is constructed based on expert-annotated embroidery samples. In embodiments of this invention, the needlework template library includes eight common traditional needlework techniques: flat stitch, satin stitch, random stitch, joining stitch, tassel stitch, net stitch, seed stitch, and rush stitch. For each needlework technique, 50 to 100 standard samples are collected, covering different colors, densities, and fabric materials. For each sample, a specific embroidery texture descriptor is extracted, and the mean and covariance of all samples are calculated to form a needlework template. Each needlework template includes a feature mean vector (128-dimensional), a covariance matrix (128×128-dimensional), and typical parameters (such as the range of stitch density and the distribution range of the main direction).

[0038] The feature matching unit calculates the similarity between the texture descriptor of the embroidery area to be identified and the standard stitch templates in the stitch template library. The similarity calculation uses the Mahalanobis distance metric, taking into account the statistical distribution characteristics of the features. For the texture descriptor of the area to be identified… and the Mean vector of the needle pattern template With covariance matrix The Mahalanobis distance is calculated as follows:

[0039] ,

[0040] in, For the region to be identified and the first Mahalanobis distance for the needle method This is the texture descriptor vector of the region to be identified. For the first The characteristic mean vector of the needle method, For the first The characteristic covariance matrix of the needle method, superscript Indicates vector transpose, superscript This represents finding the inverse of a matrix. A smaller Mahalanobis distance indicates a higher similarity.

[0041] The classification decision unit determines the needlework type based on similarity calculation results. For each region to be identified, the Mahalanobis distance between it and eight needlework templates is calculated, and the needlework with the smallest distance is selected as the identification result. Simultaneously, a threshold is set. =3.0. If the minimum Mahalanobis distance is greater than this threshold, the area is considered not to belong to any known needling method and is marked as an unknown needling method or a mixed needling method, requiring further manual confirmation.

[0042] In a preferred embodiment, the classification decision unit also incorporates a confidence assessment mechanism. Confidence is defined as the ratio of the minimum Mahalanobis distance to the second minimum Mahalanobis distance. If the minimum distance is much smaller than the second minimum distance (ratio less than 0.6), the classification confidence is considered high; if the two are close (ratio greater than 0.8), the classification confidence is considered low, indicating potential confusion or transitional regions, and a review is recommended.

[0043] The stitch recognition module 3 performs region segmentation and region-by-region recognition on the entire embroidery image. Region segmentation employs a texture consistency-based clustering method, grouping pixels with similar texture features into the same region. The clustering algorithm uses the K-means method, with the number of clusters adaptively determined based on image complexity, ranging from 10 to 50 clusters. For each clustered region, an overall texture descriptor is extracted for stitch recognition, generating a stitch distribution map that labels the stitch type and confidence level for each region.

[0044] like Figure 4 As shown, the three-dimensional reconstruction module 4 for embroidery threads includes a lighting analysis unit, a depth estimation unit, a spatial positioning unit, and a hierarchical sorting unit, which are used to reconstruct the three-dimensional structure of the embroidery thread based on the lighting and shadow information of the image.

[0045] The illumination analysis unit is used to analyze illumination conditions and shadow distribution in high-resolution image data. Because embroidery threads have a certain height, they produce subtle shadows and highlights under illumination; these illumination features contain three-dimensional information about the thread. The illumination analysis unit first performs illumination normalization processing on multi-angle images to eliminate the influence of global illumination variations and preserve local illumination details. Illumination normalization uses a bilateral filtering method to separate the low-frequency illumination component and high-frequency detail component of the image, retaining the high-frequency component for subsequent analysis.

[0046] In one embodiment of the invention, the illumination analysis unit extracts the illumination gradient for each pixel. The illumination gradient reflects the change in the surface normal vector, and thus reflects the surface undulations. For raised portions of the embroidery thread, the illumination gradient direction points towards the highlight area; for recessed portions, the illumination gradient direction points towards the shadow area. By analyzing the direction and magnitude of the illumination gradient, the geometry of the surface can be inferred.

[0047] The depth estimation unit estimates the height information of the embroidery thread relative to the embroidery fabric surface based on lighting conditions and shadow distribution. Depth estimation employs a photometric stereo vision method, which uses images from multiple lighting directions to recover the three-dimensional shape by solving for surface normal vectors. In this embodiment, three images are captured using lighting from three different angles (vertical light from the top, 45° light from the left, and 45° light from the right) to form the input for the photometric stereo vision.

[0048] The innovative photometric stereo depth estimation algorithm is as follows: For each pixel in the image, let the image grayscale values ​​under the three illumination directions be respectively... , , The unit vectors of the illumination direction are respectively , , The surface normal vector is Surface reflectivity According to the Lambert reflection model, we have:

[0049] ,

[0050] in, For the first Pixel grayscale values ​​under each lighting direction For surface reflectivity, It is the surface normal vector (unit vector). For the first A unit vector in the direction of illumination, with superscript This represents the transpose of a vector. Rewrite the three equations in matrix form:

[0051] ,

[0052] in, For image grayscale vectors, The illumination direction matrix is ​​3×3. Let be the surface normal vector. Solving this equation yields:

[0053] ,

[0054] in, This represents the inverse matrix of the transpose of the illumination direction matrix. From this, the reflectivity of each pixel can be obtained. and normal vector Reflectivity Through vectors Calculation of modulus:

[0055] ,

[0056] in, The Euclidean norm of a vector. , , These are the three components of the normal vector. The normal vector is obtained through normalization:

[0057] ,

[0058] After obtaining the normal vector, the surface depth is recovered using integration. Depth integration employs a row-by-row or column-by-column accumulation method, based on the normal vector. Quantity and , The depth gradient is calculated by the ratio of the components, and then integrated along the path to obtain the depth value. To improve integration accuracy, a strategy of multi-path integration and averaging is adopted, which effectively reduces the accumulation of integration error.

[0059] In a preferred embodiment, the Lambertian reflection model is modified to account for the unique reflective properties of the embroidery thread. Embroidery threads possess certain specular reflective properties, and strictly adhering to the Lambertian model would lead to estimation errors. By introducing the specular reflection term from the Phong reflection model, the modified reflection equation is:

[0060] ,

[0061] in, The diffuse reflectance coefficient is... The specular reflection coefficient, For the first The direction of specular reflection from each direction of illumination. Direction of sight The specular reflectance index. For silk embroidery thread, A value between 8 and 16 indicates strong directional reflection characteristics. An iterative optimization method is used to simultaneously solve... , and This allows for more accurate estimation of the normal vector and depth.

[0062] The spatial positioning unit determines the position coordinates of each embroidery thread in three-dimensional space based on depth information. This is combined with the pixel coordinates of the image. and depth value Converted into three-dimensional spatial coordinates using camera intrinsic parameters Camera parameters include focal length. and principal point coordinates The transformation relationship is as follows:

[0063] ,

[0064] in, , , In three-dimensional space coordinates, , Image pixel coordinates, , Principal point coordinates (image center), , For focal length at and Components of direction (usually) ), This represents the depth value. This transformation converts the entire depth map into a 3D point cloud, where each point represents a location on the embroidered surface.

[0065] To address the slender nature of embroidery threads, the spatial localization unit further extracts the centerline of the stitch. A skeleton extraction algorithm is then used to refine the binarized embroidery area, resulting in the stitch skeleton. Each point on the skeleton is mapped to a 3D point cloud, forming the 3D centerline of the stitch. This 3D centerline contains both the direction and height information of the stitch, providing a compact representation of the 3D structure of the embroidery thread.

[0066] Hierarchical sorting units are used to determine the hierarchical relationship between multiple layers of embroidery threads. In a 3D point cloud, embroidery threads in different layers have different depth values. A depth clustering method is used to divide the point cloud into multiple layers based on depth. The depth clustering employs the density-based DBSCAN algorithm, grouping points with a depth difference of less than 0.2 mm into the same layer. The clustering results provide the number of layers and the point cloud set for each layer.

[0067] For each layer of embroidery thread, the image area it covers is extracted. Combining the needlework recognition results and texture features, the occlusion relationship between layers is analyzed. If a certain area is visible in the image but occluded by other layers in the 3D point cloud, it indicates that the layer is located below. Through occlusion relationship analysis, a hierarchical topology is established, clarifying the sequential order of each layer of embroidery thread.

[0068] In a preferred embodiment, hierarchical ranking also incorporates needlework semantic information. Some needlework techniques (such as the serger needle) naturally possess upper-level features, while others (such as the plain needle) typically serve as lower-level features. Integrating prior knowledge of needlework techniques with deep clustering results improves the reliability of hierarchical ranking. The resulting 3D stitch model not only contains the 3D geometric information of each layer but also the topological information of the hierarchical relationships, providing complete 3D structural data for digital representation and diagram generation.

[0069] The Digital Representation Module 5 constructs a digital 3D representation of the embroidery artwork based on the 3D model of the stitches. The digital representation includes two parts: a geometric model and a texture map.

[0070] The geometric model is represented by a triangular mesh, generated by reconstructing a 3D point cloud. The mesh reconstruction employs a Poisson surface reconstruction algorithm, which uses the normal vector information of the point cloud to solve the Poisson equation to obtain an implicit surface, and then uses the Marching Cubes algorithm to extract isosurfaces to generate the triangular mesh. In this embodiment, the resolution of the mesh reconstruction is set to 0.1 mm, which is sufficient to capture the detailed features of the embroidery thread. The generated triangular mesh contains vertex coordinates, facet connectivity, and vertex normal vectors, forming a complete geometric model.

[0071] Texture mapping is generated by mapping the original image onto the surface of a triangular mesh. Texture mapping employs a multi-view fusion method, utilizing images captured from multiple angles to select the image with the best viewing angle as the texture source for each facet of the mesh. The criterion for the best viewing angle is that the angle between the facet's normal vector and the camera's line of sight is minimized; at this angle, the facet faces the camera directly, resulting in minimal texture distortion. For the same facet, if multiple images have similar viewing angles, a weighted fusion process is performed, with the weight proportional to the cosine of the angle, resulting in a more uniform and natural fused texture.

[0072] In a preferred embodiment, the texture map further includes a normal map and a specular map. The normal map records the normal vector perturbation of the surface's micro-bumps and can represent rich details without increasing geometric complexity. The normal map is generated by encoding high-frequency depth information and stored as an RGB image with R-channel encoding. Directional normal vector components, G-channel encoding Directional component, B-channel encoding Orientation component. The specular map records the intensity of specular reflection on the surface and is used to simulate the sheen effect of embroidery threads. The specular map is generated by extracting the specular regions from multi-angle images.

[0073] The digital representation module 5 outputs a digital 3D representation stored in the common OBJ or FBX format, containing geometric meshes, texture images, normal maps, and specular maps. This representation can be imported into mainstream 3D software (such as Blender, 3ds Max, and Maya) for further editing, rendering, and display. The digital 3D representation not only accurately preserves the appearance of the embroidery work but also retains its 3D structural information, providing high-quality digital assets for virtual museums and the preservation of digital cultural heritage.

[0074] like Figure 5 As shown, the embroidery pattern generation module 6 includes a path planning unit, a symbol mapping unit, and a pattern rendering unit, which are used to generate standardized embroidery patterns.

[0075] The path planning unit plans the stitch path during embroidery production based on the type of stitch and the 3D model of the stitch. Path planning considers the technological characteristics of the stitch and the spatial direction of the stitch. For linear stitches such as straight stitch and satin stitch, the path is generated along the center line of the stitch, and the path direction is consistent with the main direction of the stitch. For multi-directional stitches such as random stitch and quick stitch, the path adopts a region filling strategy, planning multiple interwoven paths within the stitch area according to the density requirements.

[0076] The path planning also considers the order in which the embroidery is done. Based on the hierarchical order, the lower layers are embroidered first, followed by the upper layers, ensuring the correct representation of the hierarchical relationship. For complex multi-layered structures, a layered path plan is generated, with each layer planned separately and labeled with a layer number. The path planning result is represented as a sequence of path points, with each path point containing its position coordinates, direction angle, and stitch type information.

[0077] The symbol mapping unit maps the identified stitch types to standard embroidery symbols. Embroidery symbols are the standard representation of embroidery patterns, with different symbols corresponding to different stitches. In embodiments of this invention, the symbol mapping follows international embroidery symbol standards: flat stitches are represented by short straight lines, satin stitches by overlapping arcs, random stitches by intersecting diagonal lines, satin stitches by a T-shape, net stitches by a grid shape, and seed stitches by small dots. The size and orientation of the symbols are adjusted according to the actual size and direction of the stitch to ensure that the symbols accurately reflect the characteristics of the stitch.

[0078] In a preferred embodiment, the symbol mapping further includes color annotation. Based on the color information provided by the color analysis module, the embroidery thread color is labeled for each stitch area. The color annotation uses a standard color code system (such as DMC color codes) to facilitate the selection of the appropriate embroidery thread during embroidery. For gradient color areas, multiple color codes and mixing ratios are labeled to guide the achievement of the gradient effect.

[0079] The illustration rendering unit generates a visual embroidery illustration that includes stitch symbols, direction arrows, and color annotations. Illustration rendering uses vector graphics to ensure the illustration can be scaled arbitrarily without distortion. The rendering process first draws the outline of the embroidery fabric and divides the area, then draws the corresponding stitch symbols within each area, arranging the symbols along the path point sequence, with the symbols facing the same direction as the path. The direction of the stitches is indicated by arrows, with the starting point of the arrow indicating the beginning position of the stitch and the ending point indicating the end position. The arrows curve along the path, visually showing the direction of the stitches.

[0080] Color markings are superimposed on the diagram as labels, indicating the area number, stitch type, and thread color. For large areas, the label is placed in the center; for small areas, the label is connected to the perimeter by a guide line to avoid obscuring the stitch symbols. The diagram is rendered as an SVG vector graphic file, which can be printed or viewed on a screen, providing clear guidance for embroidery teaching and reproduction.

[0081] In a preferred embodiment of the present invention, the embroidery pattern generation module 6 also provides an interactive pattern editing function. Users can manually adjust the path planning, modify the symbol type, and edit the color labels to meet personalized needs. The edited pattern can be saved and re-rendered, supporting continuous optimization and improvement of the pattern.

[0082] In an extended embodiment of the present invention, the system further includes a material recognition module and a color analysis module, providing richer information for embroidery analysis.

[0083] The material identification module uses multispectral image recognition to identify the material type of embroidery thread and fabric. Different materials (such as silk, cotton thread, and metallic thread) have different spectral reflectance characteristics in different wavelength bands. Material identification uses a spectral feature matching method to compare the acquired spectral curves with a standard material spectral library to identify the material type. The spectral library contains standard spectral reflectance curves of common embroidery materials (silk, rayon, cotton thread, polyester thread, and metallic thread) in the 400nm to 2500nm wavelength band.

[0084] Material similarity calculation employs a spectral angle mapping method, treating spectral curves as vectors in a high-dimensional space and calculating the angle between these vectors. The smaller the angle, the higher the material similarity. The identification results provide the most probable material type and confidence level, offering a basis for material analysis and conservation of embroidery works.

[0085] The color analysis module extracts color information from the embroidery artwork. Color analysis includes dominant hue extraction, color scheme analysis, and color transition region detection. Dominant hue extraction uses a color clustering method, clustering pixel colors in the image within the HSV color space; the cluster centers are the dominant hues. Typically, 3 to 5 dominant hues are extracted, covering the main color composition of the embroidery artwork.

[0086] Color scheme analysis statistically analyzes the matching relationships and proportional distribution between primary color tones, generating color scheme statistics charts. These color schemes can guide color application in modern design and provide designers with references to traditional color schemes. Color transition area detection identifies areas of color gradient; by calculating color gradients, areas with larger gradients correspond to color transitions. Color transition information is used for color mixing annotations in embroidery diagrams, guiding the creation of gradient effects.

[0087] In the application layer extension of this invention, the system also includes an interactive display module and a teaching assistance module to enhance user experience and application value.

[0088] The interactive display module provides multi-view 3D displays of embroidery works. Based on the 3D model generated by the digital representation module, the interactive display module enables real-time rendering and interactive operation. Users can rotate, zoom, and pan the 3D model using a mouse or touch device to observe the details of the embroidery work from different angles. The interactive display supports a local zoom function; when a user clicks on an area of ​​interest, the system automatically zooms in on that area and displays a high-resolution texture, facilitating careful observation of stitch details and line direction.

[0089] The interactive display module uses a WebGL-based 3D rendering engine, which can run in a web browser without requiring additional software installation. The rendering engine supports real-time lighting and shadow effects, allowing users to adjust the direction and intensity of the lighting to showcase the visual effects of the embroidery under different lighting conditions. Furthermore, the interactive display offers a layered viewing function, allowing users to choose to show or hide specific layers of embroidery, analyzing the structure and stitches layer by layer.

[0090] The teaching support module generates step-by-step embroidery tutorials based on embroidery diagrams. The tutorials break down the embroidery process into multiple steps, each corresponding to a specific stitch area or layer of embroidery. Each step provides a diagram, text description, and operational guidance. The diagram highlights the area to be embroidered in the current step and marks the stitch symbol and direction arrow. The text description outlines the key points and precautions for each stitch, such as using the nested stitch technique with a 2mm stitch spacing, overlapping stitches along the outline. The operational guidance demonstrates the stitch techniques through videos or animations, helping learners intuitively understand the embroidery methods.

[0091] The teaching support module also provides a progress tracking function. Learners can mark completed steps during the actual embroidery process, and the system automatically updates the progress and displays the next step. For complex embroidery works, the tutorials generated by the teaching support module can have dozens or even hundreds of steps, ensuring that every detail is clearly guided. The tutorials can be exported as PDF documents or interactive web pages, facilitating sharing and dissemination, and promoting the inheritance of embroidery skills.

[0092] like Figure 6 As shown, the hand embroidery image analysis and digital reproduction system of the present invention can be flexibly deployed on cloud servers or edge computing devices. Cloud deployment is suitable for large-scale batch processing and shared services. Users upload embroidery images through web pages or mobile applications, and the system completes the analysis and digital processing in the cloud, returning the results to the user. The cloud server is equipped with high-performance GPUs to accelerate computing, supporting concurrent processing of multiple user requests. Processing a 20-megapixel embroidery image, from upload to generating a complete digital result, takes approximately 5 to 10 minutes.

[0093] Edge computing device deployments are suitable for localized applications such as museums, cultural heritage preservation institutions, and embroidery workshops. Edge devices can be high-performance workstations or embedded computing platforms that operate independently without a network connection. The advantages of edge deployment include strong data privacy protection and processing speeds not limited by network bandwidth, making it suitable for scenarios with high privacy requirements or poor network conditions.

[0094] The system provides standard API interfaces, supporting integration with other applications. For example, integration with a digital museum system enables online display and digital archiving of embroidery works; integration with cultural and creative design software provides designers with tools for extracting and applying embroidery elements; and integration with an embroidery CAD system converts digital models into a format recognizable by embroidery machines, enabling automated embroidery production.

[0095] Taking the Suzhou embroidery masterpiece "Cat" as an example, the application effect of the system of this invention is illustrated. This work measures 40cm × 30cm and employs various needlework techniques to depict the texture of the cat's fur and its expression. The specific steps for digital processing using the system of this invention are as follows:

[0096] Using image acquisition module 1, a 50-megapixel industrial camera with a macro lens was used to capture 5 images each from three angles: 0°, 15°, and 30°. The lighting was provided by a top vertical light source and 45° light sources on both sides. The shooting time for each angle was about 2 minutes, resulting in a total of 15 high-resolution images and corresponding multispectral images.

[0097] The texture feature extraction module 2 performs multi-scale analysis on the image and applies 40 Gabor filters to extract texture features. Dense directional features were detected in the cat fur area, with a directional consistency index of 0.75, indicating nested stitches or quick stitches. Multi-layered stacking features were detected in the eyes, with spectral analysis showing three energy peaks, indicating three layers of embroidery stacked. Sparse stitches were detected in the background, indicating a flat stitch underlay. Texture descriptor extraction took approximately 3 minutes.

[0098] The stitch recognition module 3 matches the extracted texture descriptors with the stitch template library, identifying that the cat fur uses random stitch embroidery (confidence 0.92), the eyes use a combination of satin stitch and serif stitch (confidence 0.88), and the background uses plain stitch (confidence 0.95). The stitch recognition divides the artwork into 12 areas, labeling each area with the stitch type, density, and main direction. The recognition process takes approximately 2 minutes.

[0099] The 3D line reconstruction module 4 estimates depth based on illumination images from three angles using an improved photometric stereo vision method. The height of the cat fur is approximately 0.5 mm, the eye height is approximately 0.8 mm (multiple layers stacked), and the background height is approximately 0.1 mm. The 3D point cloud contains approximately 15 million points, which are reconstructed through meshing to generate a geometric model containing 3 million triangular faces, taking approximately 8 minutes. Hierarchical sorting identifies the three-layer structure of the eye area: the first layer is a base layer with stitches, the second layer is fine line outlining, and the third layer is highlighting with nail lines.

[0100] The digital representation module 5 generates a digital 3D representation including a geometric model and texture maps. The texture maps have a resolution of 4096×4096 pixels, as do the normal maps and specular maps. The digital model file is approximately 150MB in size and can be smoothly viewed and edited in 3D software. The representation generation takes approximately 5 minutes.

[0101] The embroidery pattern generation module 6 plans the embroidery paths, totaling 450 paths, covering all stitch areas. The generated embroidery pattern clearly marks the stitch symbols, stitch directions, and thread color numbers (using DMC color numbers) for 12 areas. The pattern is in SVG vector format, clearly visible when printed at A3 size, and takes approximately 3 minutes to generate.

[0102] The entire digitization process took approximately 23 minutes, generating a high-precision 3D model, standard embroidery diagrams, and a detailed needlework analysis report. These digital results have been applied to virtual museum displays, allowing visitors to view the artwork in 360° through an interactive interface and understand the embroidery structure layer by layer. In embroidery education, learners practice copying embroidery diagrams, with needlework recognition accuracy verified at 93%, highly consistent with expert manual identification results. Furthermore, in cultural and creative design, designers extracted random needlework elements resembling cat fur and incorporated them into modern clothing designs, achieving a fusion of tradition and modernity.

[0103] The hand embroidery image analysis and digital reproduction system of the present invention has the following performance indicators and advantages:

[0104] The texture analysis is highly accurate; the embroidery-specific texture descriptor can accurately capture the stitch direction, density, and layering relationship, achieving a stitch recognition accuracy of over 93%, with excellent recognition performance for eight common traditional stitches. Compared to general texture analysis methods, this invention improves the accuracy of embroidery texture recognition by more than 15%.

[0105] The 3D reconstruction is highly accurate. The depth estimation method based on photometric stereo vision can accurately calculate the height information of embroidery threads, with a depth accuracy of 0.05mm. The layer recognition accuracy of multi-layer embroidery reaches over 90%. Compared with point cloud-based reconstruction methods, this invention provides a clearer and more accurate representation of multi-layered structures.

[0106] The generated embroidery diagrams are highly practical, conforming to international standards with clear symbols, well-defined directions, and accurate color codes. They can be directly used for embroidery teaching and reproduction. The step-by-step tutorials provided in the teaching support module enable beginners to master complex embroidery techniques, promoting the inheritance of embroidery skills.

[0107] The system features a modular design with clearly defined interfaces for each functional module, allowing for flexible combination and expansion as needed. It supports both cloud and edge deployment, adapting to application requirements of varying scales and scenarios. The system provides standard API interfaces for easy integration with other systems, expanding its application scope.

[0108] The system boasts high processing efficiency; for a typical 40cm x 30cm embroidery piece, the total time from image acquisition to generating a complete digital result is approximately 20 to 30 minutes, meeting the efficiency requirements for batch processing. The system supports GPU acceleration, fully utilizing parallel computing capabilities to enhance processing speed.

[0109] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A system for analyzing and digitally reproducing hand embroidery images, characterized in that, include: An image acquisition module is used to acquire high-resolution image data of the embroidery work, including visible light images and multispectral images; The texture feature extraction module, connected to the image acquisition module, is used to extract multi-scale features of the embroidery texture based on the high-resolution image data and generate an embroidery-specific texture descriptor, which includes stitch direction features, density distribution features, and layering relationship features. The needlework recognition module, connected to the texture feature extraction module, is used to match the needlework type in the preset needlework template library according to the specific embroidery texture descriptor, and identify the needlework type and needlework distribution area in the embroidery work; The stitch 3D reconstruction module is connected to the image acquisition module and the needle recognition module. It is used to calculate the 3D spatial position and hierarchical structure of the embroidery thread based on the illumination and shadow information in the high-resolution image data, and generate a 3D stitch model. The digital representation module, connected to the stitch 3D reconstruction module, is used to construct a digital 3D representation of the embroidery work based on the stitch 3D model. The digital 3D representation includes a geometric model and a texture map. The embroidery pattern generation module is connected to the stitch recognition module and the digital representation module. It is used to generate standardized embroidery patterns based on the stitch type, the stitch distribution area, and the digital three-dimensional representation. The embroidery pattern includes stitch symbols, stitch direction indicators, and color markings.

2. The system according to claim 1, characterized in that, The texture feature extraction module includes: A multi-scale analysis unit is used to perform multi-scale decomposition on the high-resolution image data to obtain texture information at different scales. A directional feature extraction unit, connected to the multi-scale analysis unit, is used to extract the directional features of embroidery stitches, the directional features including the main direction and auxiliary direction; The density calculation unit, connected to the multi-scale analysis unit, is used to calculate the stitch density distribution of the embroidery area, which reflects the number of stitches per unit area. The layering recognition unit, connected to the multi-scale analysis unit, is used to identify the layering relationship of multiple embroidery threads and determine the upper and lower layer order of the stitches.

3. The system according to claim 2, characterized in that, The multi-scale analysis unit uses a Gabor filter bank for texture analysis. The Gabor filter bank includes filters of different directions and frequencies to capture the characteristic responses of the embroidery texture at multiple scales and directions.

4. The system according to claim 1, characterized in that, The needle pattern recognition module includes: A feature matching unit is used to calculate the similarity between the embroidery-specific texture descriptor and the standard stitch templates in the stitch template library; A classification decision unit, connected to the feature matching unit, is used to determine the stitch type of each area in the embroidery work based on the similarity calculation results; The needlework template library stores standard texture features and needlework parameters for flat stitch, satin stitch, random stitch, joining stitch, tassel stitch, net embroidery stitch, seed stitch, and rush stitch.

5. The system according to claim 1, characterized in that, The line trace 3D reconstruction module includes: The illumination analysis unit is used to analyze the illumination conditions and shadow distribution in the high-resolution image data; A depth estimation unit, connected to the illumination analysis unit, is used to estimate the height information of the embroidery thread relative to the surface of the embroidery fabric based on the illumination conditions and the shadow distribution. A spatial positioning unit, connected to the depth estimation unit, is used to determine the position coordinates of each embroidery thread in three-dimensional space based on the height information; The hierarchical sorting unit, connected to the spatial positioning unit, is used to determine the front-to-back hierarchical relationship of the multi-layer embroidery threads based on the position coordinates.

6. The system according to claim 5, characterized in that, The depth estimation unit uses a photometric stereo vision method to calculate the three-dimensional shape of the embroidery thread by analyzing its reflective properties and surface normal vector.

7. The system according to claim 1, characterized in that, The embroidery pattern generation module includes: The path planning unit is used to plan the stitch path during embroidery production based on the stitch type and the three-dimensional stitch model. A symbol mapping unit, connected to the path planning unit, is used to map the identified stitch type to a standard embroidery symbol; The illustration rendering unit, connected to the symbol mapping unit, is used to generate a visual embroidery illustration that includes stitch symbols, direction arrows, and color annotations.

8. The system according to claim 1, characterized in that, Also includes: A material identification module, connected to the image acquisition module, is used to identify the material type of embroidery thread and embroidery fabric based on the multispectral image; The color analysis module, connected to the image acquisition module, is used to extract color information from the embroidery work. The color information includes the main color tone, color scheme, and color transition areas.

9. The system according to claim 1, characterized in that, Also includes: The interactive display module provides a multi-view 3D display of embroidery works, allowing users to view details and zoom in on specific parts of the embroidery works through interactive operations; The teaching assistance module is connected to the embroidery pattern generation module and is used to generate step-by-step embroidery tutorials based on the embroidery patterns.

10. The system according to claim 1, characterized in that, The system is deployed on a cloud server or edge computing device and supports remote image uploading and digital result downloading.

Images