An edge extraction method, system, device and medium of a dual-module enhanced adaptive descriptor

Abstract

The application discloses a kind of double module enhanced self-adaptive descriptor edge extraction method, system, equipment and medium, method includes: after relic fragment cleaning and numbering, scanning, obtaining the two-dimensional image and three-dimensional model of relic fragment, and carrying out label association annotation, construct original data set;Original data set is handled, realizes two-dimensional image data enhancement based on geometric transformation by multi-angle image screenshot, and the three-dimensional model of fragment is converted into point cloud format mapping corresponding two-dimensional image by 3D scanning analysis software;The corresponding two-dimensional image of enhanced two-dimensional image and three-dimensional model mapping is commonly composed of fragment two-dimensional image data set;According to the obtained fragment two-dimensional image data set, construct the multi-modal descriptor of extracting feature;According to descriptor, obtain the edge extraction result of relic fragment two-dimensional image, relic fragment three-dimensional model respectively, complete edge extraction task;The double module enhanced self-adaptive descriptor provided by the application can comprehensively capture edge information.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of image processing, and relates to an edge feature extraction algorithm for mapping two-dimensional images and three-dimensional point clouds, in particular to an edge extraction method, system, device and medium of a double-module enhanced adaptive descriptor. BACKGROUND

[0002] With the development of visualization technology in the digital protection of cultural heritage, the edge extraction of cultural relic fragments is particularly important. However, traditional edge extraction methods for terracotta figures have problems such as low efficiency, strong subjectivity, and lack of standardization. The edge extraction method based on visualization technology for three-dimensional models can automatically extract the edge features of terracotta fragments, thereby reducing the manual burden, improving the efficiency and accuracy, effectively protecting cultural relics, and avoiding secondary damage to them. In addition, this method not only accurately identifies the edge features of terracotta fragments, but also analyzes shape and texture information in different dimensions. For example, when digitally repairing broken terracotta figures, more accurate matching and splicing can be performed based on their two-dimensional or three-dimensional edge features, providing technical support for subsequent cultural relic restoration work.

[0003] In the current research, some edge extraction methods have been applied to the study of cultural relic fragments. For example, an explicit ridge-based cultural relic line generation algorithm is used to solve the problem of edge interruption in the contour extraction process of three-dimensional terracotta models; a broken cultural relic automatic splicing algorithm based on fracture surface topological features is used to extract fracture feature points by defining a saliency index function and constructing a topological graph, finally realizing the effective splicing of the edge-damaged cultural relic model; the selected region is segmented based on the SLIC0 algorithm, the number of superpixels is iterated, and the AP clustering method is used to merge the superpixels of the cultural relic region to obtain edge information; a multi-scale generation and discrimination method based on cyclic consistent generative adversarial network is used to make significant progress in the edge extraction of painting cultural relics. However, due to the problems of edge blur, complex surface texture, and slight geometric differences between fragments, the descriptor needs to capture edge information comprehensively, including global and local, two-dimensional and three-dimensional. However, the descriptor in the prior art cannot capture edge information comprehensively, mainly having the following problems:

[0004] (1) Edge blur and incompleteness

[0005] The edges of cultural relic fragments often become blurred or incomplete due to long-term physical wear, natural corrosion or other factors. Traditional edge descriptors, such as Canny edge detection or Sobel operator, usually rely on the gradient of pixel gray level change to identify edges. However, for blurred edges or irregularly shaped objects, the gradient change may not be obvious or lack clear boundaries, causing these algorithms to fail to effectively identify complex edges.

[0006] (2) Confusion between texture and edge

[0007] Many cultural relics fragments have complex surface textures, such as carvings, paintings, wear marks, etc. These texture features can be mixed with the edges of the object, making it difficult for the descriptor to distinguish between edges and surface textures. Traditional descriptors such as SIFT, SURF, etc. can extract local feature points in the image, but they often fail in complex texture or blurred edge cases, making it difficult to distinguish and process edges and textures.

[0008] (3) Difficulty in integrating local and global information

[0009] Existing edge descriptors often focus on extracting local information (such as edge features in a small area) when extracting features, while ignoring the global structure of the image or object.

[0010] (4) Difficulty in processing multi-scale, multi-angle and three-dimensional structures

[0011] The geometric structure of cultural relics fragments is usually a complex three-dimensional shape, especially with complex structures such as curved surfaces, concave and convex, detailed carvings, etc. Traditional edge detection methods (such as Canny, Sobel, etc. based on images) may not be able to effectively capture the changes in object edges at different angles and depths when processing cultural relics fragments with curved surfaces.

[0012] (5) High noise and false edge problem

[0013] Cultural relics fragments may have high noise, especially after long-term preservation or repair, noise points and false edges in the image can affect the effectiveness of the edge extraction algorithm. SUMMARY

[0014] To overcome the shortcomings of the prior art, the purpose of the present application is to propose a dual-module enhanced adaptive descriptor edge extraction method, system, device and medium, by extracting global edge features and local edge features, and using a dynamic weight fusion strategy to fuse global edge features and local edge features, improving the accuracy of cultural relic edge extraction.

[0015] The technical solution adopted by the present application is:

[0016] A dual-module enhanced adaptive descriptor edge extraction method, comprising the following steps:

[0017] Step 1, after cleaning and numbering the cultural relics fragments, scan to obtain the two-dimensional image and three-dimensional model of the cultural relics fragments, and perform label association annotation to construct the original data set;

[0018] Step 2: Process the original dataset, performing geometric transformation-based 2D image data enhancement by capturing multi-angle image screenshots. Using 3D scanning analysis software, the 3D model is converted into a point cloud format and mapped to a corresponding 2D image. The enhanced 2D image and the corresponding 2D image mapped from the 3D model together form a fragmented 2D image dataset.

[0019] Step 3: Based on the fragmented two-dimensional image dataset obtained in step 2, a dual-module enhanced adaptive descriptor GLSoble for feature extraction is constructed, specifically:

[0020] Step 3.1, extract global edge features through frequency domain transformation enhancement method;

[0021] Step 3.2, extract local edge features using the adaptive domain method;

[0022] Step 3.3: The global edge features and local edge features are fused through a dynamic weight fusion strategy to obtain a dual-module enhanced adaptive descriptor.

[0023] Step 4: Based on the dual-module enhanced adaptive descriptor obtained in step 3, edge extraction results of the two-dimensional image of the cultural relic fragment and the three-dimensional model of the cultural relic fragment are obtained respectively, completing edge extraction based on the dual-module enhanced adaptive descriptor.

[0024] In step 1, the direction, height and distance of the light source are adjusted to determine the optimal light source angle, reduce reflection, and obtain a two-dimensional image of the cultural relic fragment; and the laboratory's EVA scanner is used to perform three-dimensional scanning to obtain a three-dimensional model of the cultural relic fragment.

[0025] Step 2 converts the 3D model into a point cloud format and maps it using 3D scanning analysis software, specifically:

[0026] Step 2.1: Use Geomagic Studio software to flip and rotate the 3D model at different angles and then take a two-dimensional screenshot to obtain 2D images corresponding to different angles of the 3D model.

[0027] Step 2.2, traverse each three-dimensional coordinate (x, y, z), use the minimum and maximum values ​​of the point cloud to normalize x and y to the image pixel range (a, b), if the image pixel range (a, b) is within the valid pixel range of the image, then normalize the z value to the brightness value, update the corresponding pixel position to ensure that the highest brightness value is stored, and map the corresponding two-dimensional image; if the image pixel range (a, b) exceeds the valid pixel range of the image, then discard the point.

[0028] In step 3.1, global edge feature extraction is performed by frequency domain transformation enhancement method, specifically:

[0029] Step 3.11, input the processed fragment two-dimensional image data set obtained in step 2, calculate the Fourier transform to convert the image from the spatial domain to the frequency domain, and perform the shift of the transform to move the zero frequency component to the center of the frequency, after the shift, the center of the spectrum is the low frequency information and the edge is the high frequency information, and the calculation process is as follows:

[0030] F(u,v) = ∫∫f(x,y)e -2πi(ux+vy) dxdy

[0031]

[0032] wherein x is the horizontal coordinate of the image, y is the vertical coordinate of the image; u is the horizontal frequency of the frequency domain, indicating the frequency component in the x direction; v is the vertical frequency of the frequency domain, indicating the frequency component in the y direction; i is the imaginary unit; f(x,y) represents the pixel value of the image at the coordinate (x,y); F(u,v) represents the frequency domain image, that is, the complex value of the frequency component;

[0033] Step 3.12, according to the shift F shift (u,v) obtained in step 3.11, calculate the amplitude spectrum to obtain the image frequency information, and perform logarithmic transformation to enhance the contrast;

[0034]

[0035] M(u,v) = log(1 + S(u,v))

[0036] wherein Re is the real part of the Fourier transform result, and Im is the imaginary part of the Fourier transform result;

[0037] Step 3.13, apply a Gaussian low-pass filter to the logarithm transformed amplitude spectrum M(u,v) obtained in step 3.12 to smooth the image to reduce high frequency noise;

[0038]

[0039] F filtered = M(u,v)·G(c,d)

[0040] wherein σ is the standard deviation for controlling the width of the filter; c is the horizontal distance from the center of the frequency domain to the current frequency u; d is the vertical distance from the center of the frequency domain to the current frequency v; G is the value of the Gaussian filter;

[0041] Step 3.14, perform inverse Fourier transform on the amplitude spectrum F filtered obtained in step 3.13 to convert the frequency domain image back to the spatial domain;

[0042]

[0043] Step 3.15, f back (x, y) is extracted using the Sobel operator, and the global edge feature F global is output.

[0044]

[0045]

[0046]

[0047] F global = normalize(S)

[0048] where S x and S y are the edge detection results of the image in the x direction and the y direction, respectively, S x contains a matrix for detecting horizontal edges, and S y contains a matrix for detecting vertical edges; * represents convolution operation, and normalize represents normalization operation.

[0049] The local edge feature extraction in step 3.2 is performed by an adaptive field method, specifically:

[0050] Step 3.21, input the processed fragment two-dimensional image data set obtained in step 2, set the sliding window size and step size, traverse the image by sliding window, and apply Gaussian blur in each sliding window;

[0051]

[0052] where is the Gaussian blur processing operation, W is the image block in the current sliding window, and σ is the sliding window step size;

[0053] Step 3.22, using the Sobel operator to extract the local edge feature E blurred obtained in step 3.21 and perform threshold processing, filter insignificant edges, and retain important edge features;

[0054] The calculation process of the local edge feature E local is as follows:

[0055]

[0056]

[0057]

[0058] The calculation process of threshold processing is as follows:

[0059] E local (E local (threshold) = 0

[0060] Wherein, threshold is a threshold value, located in the transition zone between low gray value and high gray value;

[0061] Step 3.23, compare the edge features in the window with the local edge features E local , keep the maximum value, and output as local edge features F local :

[0062] L(y, x) = max(L(y, x), E local )

[0063] F local = normalize(L)

[0064] Wherein, L(y, x) represents a local maximum matrix, indicating the edge intensity of each window, and the edge information of multiple local windows can be combined to capture global edge features, F local is the normalized local edge features F local .

[0065] The specific process of the step 3.3 is as follows:

[0066] Step 3.31, calculate the global feature dynamic weight W global and the local feature dynamic weight W local , the calculation process is as follows:

[0067]

[0068]

[0069] Wherein, ∈ is a small constant, used to prevent the denominator from being zero;

[0070] Step 3.32, normalize the global feature dynamic weight and the local feature dynamic weight obtained in step 3.31, the calculation process is as follows:

[0071]

[0072] Step 3.33, according to the normalized weight W norm obtained in step 3.32, the feature fusion of global edge features and local edge features is carried out, and the double module enhanced adaptive descriptor F fused is obtained:

[0073] F fused= W norm · F global +(1-W norm )· F local .

[0074] The sliding window step is half of the sliding window size, and each region of the image is traversed using double loops, the outer loop traverses the height of the image, and the inner loop traverses the width of the image, and in each loop, a window of a specified size is extracted.

[0075] An edge extraction system of a dual-module enhanced adaptive descriptor comprises:

[0076] An original data set acquisition module: after cleaning and numbering the cultural relic fragments, scanning the cultural relic fragments to obtain two-dimensional images and three-dimensional models of the cultural relic fragments, and performing label association annotation, an original data set is constructed;

[0077] An original data set processing module: the original data set is processed, wherein two-dimensional image data enhancement based on geometric transformation is realized through multi-angle image screenshots; the three-dimensional model is converted into a point cloud format and mapped through 3D scanning analysis software, and the corresponding two-dimensional image is mapped; the enhanced two-dimensional image and the three-dimensional model mapped into the corresponding two-dimensional image together constitute a fragment two-dimensional image data set;

[0078] A dual-module enhanced adaptive descriptor construction module: according to the obtained fragment two-dimensional image data set, a dual-module enhanced adaptive descriptor GLSoble for extracting features is constructed, specifically:

[0079] Global edge feature extraction is performed through a frequency domain transformation enhancement method;

[0080] Local edge feature extraction is performed through an adaptive field method;

[0081] Feature fusion of global edge features and local edge features is performed through a dynamic weight fusion strategy, and a dual-module enhanced adaptive descriptor is obtained;

[0082] An edge extraction module: according to the dual-module enhanced adaptive descriptor, edge extraction results of the cultural relic fragment two-dimensional image and the cultural relic fragment three-dimensional model are obtained respectively, and edge extraction based on the dual-module enhanced adaptive descriptor is completed.

[0083] An edge extraction device of a dual-module enhanced adaptive descriptor comprises:

[0084] A memory: storing a computer program of an edge extraction method of a dual-module enhanced adaptive descriptor, the computer program being readable by a computer;

[0085] A processor: used for executing the computer program to realize an edge extraction method of a dual-module enhanced adaptive descriptor.

[0086] A computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement a double-module enhanced adaptive descriptor edge extraction method.

[0087] The beneficial effects of the present application are:

[0088] The present application integrates global and local edge features, retains the maximum edge value at each pixel position, forms a more comprehensive feature description, effectively integrates information of different scales and directions, improves the accuracy of cultural relic fragment reconstruction, and ensures the identification and preservation of important features of cultural relic fragments in archaeological analysis.

[0089] The present application applies an adaptive Gaussian low-pass filter, dynamically adjusts the standard deviation according to the image features, more accurately smooths the image and retains important low-frequency information, so that the global feature extraction can more comprehensively capture the image information, and improves the overall image quality and analysis accuracy; through the adaptive threshold strategy, the Sobel operator is applied at different scales, the small edge changes in the damaged and blurred areas are effectively detected, and each local feature is fully mined, thereby improving the identification ability of the fine carvings and decorations of cultural relic fragments. BRIEF DESCRIPTION OF DRAWINGS

[0090] Figure 1 The flowchart of the method of the present application is shown in the figure;

[0091] Figure 2 The Qin figurine fragment two-dimensional image dataset used in the present application is shown in the figure;

[0092] Figure 3 The Qin figurine fragment three-dimensional point cloud dataset used in the present application is shown in the figure;

[0093] Figure 4 The structure diagram of the algorithm in the present application is shown in the figure;

[0094] Figure 5 The real cultural relic scene feature extraction result figure in the present application is shown in the figure;

[0095] Figure 6 The two-dimensional edge feature extraction result figure in the present application is shown in the figure;

[0096] Figure 7 The three-dimensional edge feature extraction result figure in the present application is shown in the figure. DETAILED DESCRIPTION

[0097] The specific content of the present application is further explained and described in detail in combination with the embodiments.

[0098] A double-module enhanced adaptive descriptor edge extraction method comprises the following steps:

[0099] Step 1, after cleaning and numbering the cultural relic fragments, scan the cultural relic fragments to obtain two-dimensional images and three-dimensional models of the cultural relic fragments, and perform label association annotation to construct an original data set;

[0100] In step 1, the best light source angle is determined by adjusting the direction, height and distance of the light source to reduce the reflection phenomenon and obtain the two-dimensional images of the cultural relic fragments; the three-dimensional model of the cultural relic fragments is obtained by using the EVA scanner in the laboratory.

[0101] Step 2, processing the original data set, wherein the two-dimensional image data enhancement based on geometric transformation is realized by multi-angle image screenshot; the three-dimensional model is converted into point cloud format and mapped by 3D scanning analysis software, and the corresponding two-dimensional image is mapped;

[0102] In step 2, the three-dimensional model is converted into point cloud format and mapped by 3D scanning analysis software, specifically:

[0103] Step 2.1, using GeomagicStudio software, the three-dimensional model is flipped and rotated at different angles, and then planar screenshot is performed to obtain two-dimensional images corresponding to different angles from the three-dimensional model;

[0104] Step 2.2, traverse each three-dimensional coordinate (x, y, z), use the minimum and maximum values of the point cloud to normalize x and y to the image pixel range (a, b), if the image pixel range (a, b) is within the effective pixel range of the image, then the z value is normalized to the brightness value, update the corresponding pixel position to ensure that the highest brightness value is stored, and the corresponding two-dimensional image is mapped; if the image pixel range (a, b) is out of the effective pixel range of the image, discard the point.

[0105] Step 3, according to the fragment two-dimensional image data set obtained in step 2, construct a double-module enhanced adaptive descriptor GLSoble for feature extraction, specifically:

[0106] Step 3.1, global edge feature extraction by frequency domain transformation enhancement method;

[0107] Step 3.2, local edge feature extraction by adaptive domain method;

[0108] Step 3.3, feature fusion of global edge features and local edge features by dynamic weight fusion strategy to obtain a double-module enhanced adaptive descriptor;

[0109] In step 3.1, global edge feature extraction by frequency domain transformation enhancement method, specifically:

[0110] In step 3.11, input the processed fragmented 2D image dataset obtained in step 2 and calculate the Fourier transform to convert the image from the spatial domain to the frequency domain. Perform a transform shift to move the zero-frequency component to the center of the frequency harmonic. After the shift, the center of the spectrum is low-frequency information and the edges are high-frequency information. The calculation process is as follows:

[0111] F(u, v) = ∫∫f(x, y)e -2πi(ux+vy) dxdy

[0112]

[0113] Where x is the horizontal coordinate of the image, y is the vertical coordinate of the image; u is the horizontal frequency in the frequency domain, which represents the frequency component in the x direction; v is the vertical frequency in the frequency domain, which represents the frequency component in the y direction; i is the imaginary unit; f(x, y) represents the pixel value of the image at the coordinate (x, y); F(u, v) represents the frequency domain image, that is, the complex value of the frequency component;

[0114] Step 3.12, based on the change in displacement F obtained in step 3.11 shift (u, v), calculate the amplitude spectrum to obtain the image frequency information, and perform logarithmic transformation to enhance the contrast;

[0115]

[0116] M(u, v) = log(1 + S(u, v))

[0117] Where Re is the real part of the Fourier transform result, and Im is the imaginary part of the Fourier transform result;

[0118] Step 3.13, applying a Gaussian low-pass filter to the magnitude spectrum M(u, v) obtained after logarithmic transformation in step 3.12 to smooth the image and reduce high-frequency noise;

[0119]

[0120] F filtered =M(u,v)·G(c,d)

[0121] Where σ is the standard deviation of the control filter width; c is the horizontal distance from the center of the frequency domain to the current frequency u; d is the vertical distance from the center of the frequency domain to the current frequency v; G is the value of the Gaussian filter;

[0122] Step 3.14: The amplitude spectrum F obtained in step 3.13 after passing through the Gaussian filter is filtered Perform inverse Fourier transform to convert the frequency domain image back to the spatial domain;

[0123]

[0124] Step 3.15, f back (x, y) is subjected to global edge feature extraction using a Sobel operator, and global edge features F global are outputted.

[0125]

[0126]

[0127]

[0128] F global = normalize(S)

[0129] wherein S x , S y are edge detection results of the image in the x direction and the y direction respectively, S x contains a matrix for detecting horizontal edges, and S y contains a matrix for detecting vertical edges; * represents convolution operation, and normalize represents normalization operation.

[0130] The local edge feature extraction in step 3.2 is performed by an adaptive field method, specifically as follows:

[0131] Step 3.21, input the processed fragment two-dimensional image data set obtained in step 2, set the sliding window size and step size, and traverse the image by sliding window, and apply Gaussian blur in each sliding window;

[0132]

[0133] wherein, is a Gaussian blur processing operation, W is an image block in the current sliding window, and σ is the sliding window step size; the sliding window step size is half of the sliding window size, and each region of the image is traversed using double loops, the outer loop traverses the height of the image, and the inner loop traverses the width of the image, and in each loop, a window of a specified size is extracted;

[0134] Step 3.22, W blurred obtained in step 3.21 is subjected to local edge feature extraction using a Sobel operator and threshold processing, to filter insignificant edges and retain important edge features;

[0135] The calculation process of the local edge feature E local is as follows:

[0136]

[0137]

[0138]

[0139] The calculation process of threshold processing is as follows:

[0140] E local (E local <threshold)=0

[0141] Wherein, threshold is a threshold value, located in the transition zone between low gray value and high gray value;

[0142] Step 3.23, compare the edge features in the window with the local edge features E local , keep the maximum value, and output as the local edge features F local :

[0143] L(y, x) = max(L(y, x), E local )

[0144] F local = normalize(L)

[0145] Wherein, L(y, x) represents a local maximum matrix, indicating the edge intensity of each window, and the edge information of multiple local windows can be combined to capture global edge features, F local is the normalized local edge features F local .

[0146] The specific process of step 3.3 is as follows:

[0147] Step 3.31, calculate the global feature dynamic weight W global and the local feature dynamic weight W local , the calculation process is as follows:

[0148]

[0149]

[0150] Wherein, ∈ is a small constant, used to prevent the denominator from being zero;

[0151] Step 3.32, normalize the global feature dynamic weight and the local feature dynamic weight obtained in step 3.31, the calculation process is as follows:

[0152]

[0153] Step 3.33, according to the normalized weight W normTo perform feature fusion of global edge features and local edge features, we obtain the dual-module enhanced adaptive descriptor F fused :

[0154] F fused =W norm ·F global +(1-W norm )·F local .

[0155] Step 4: Based on the dual-module enhanced adaptive descriptor obtained in step 3, edge extraction results of the two-dimensional image of the cultural relic fragment and the three-dimensional model of the cultural relic fragment are obtained respectively, completing edge extraction based on the dual-module enhanced adaptive descriptor.

[0156] A dual-module enhanced adaptive descriptor edge extraction system, comprising:

[0157] Original dataset acquisition module: After cleaning and numbering the cultural relic fragments, they are scanned to obtain two-dimensional images and three-dimensional models of the cultural relic fragments, and then associated and annotated with labels to construct the original dataset;

[0158] Raw Dataset Processing Module: Processes the raw dataset, including geometric transformation-based 2D image data enhancement through multi-angle image capture. 3D scanning analysis software converts the 3D model into a point cloud format and maps it to a corresponding 2D image. The enhanced 2D image and the corresponding 2D image mapped from the 3D model together form a fragmented 2D image dataset.

[0159] Dual-module enhanced adaptive descriptor construction module: Based on the obtained fragmented two-dimensional image dataset, a dual-module enhanced adaptive descriptor GLSoble for feature extraction is constructed, specifically:

[0160] Global edge feature extraction is performed through frequency domain transformation enhancement method;

[0161] Local edge feature extraction through adaptive domain method;

[0162] The dynamic weight fusion strategy is used to fuse global edge features and local edge features, and a dual-module enhanced adaptive descriptor is obtained.

[0163] Edge extraction module: According to the dual-module enhanced adaptive descriptor, the edge extraction results of the two-dimensional image of the cultural relic fragment and the three-dimensional model of the cultural relic fragment are obtained respectively, completing the edge extraction based on the dual-module enhanced adaptive descriptor.

[0164] An edge extraction device with dual-module enhanced adaptive descriptor, comprising:

[0165] Memory: store a computer program of a double-module enhanced adaptive descriptor edge extraction method, for a computer readable device;

[0166] Processor: for executing the computer program to realize a double-module enhanced adaptive descriptor edge extraction method.

[0167] A computer readable storage medium, the computer readable storage medium stores a computer program, the computer program is executed by the processor to realize a double-module enhanced adaptive descriptor edge extraction method.

[0168] Embodiments

[0169] Reference Figure 1 A double-module enhanced adaptive descriptor edge extraction method, comprising the following steps:

[0170] Step 1, after cleaning and numbering the unearthed Qinshihuang terracotta warriors and horses fragments, scanning is performed to obtain fragment images and part of the model, during the shooting process, by adjusting the direction, height and distance of the light source, the best light source angle is determined, the reflection phenomenon is reduced, and the original data set of the Qin terracotta sample is constructed;

[0171] The process of constructing the original data set of the Qin terracotta sample in step 1 specifically includes: fragment cleaning, artificial classification and numbering, light source adjustment, three-dimensional scanning using the laboratory EVA scanner, label association annotation and other links.

[0172] As Figure 2 And Figure 3 As shown in step 2, the two and three-dimensional data are respectively effectively formatted, which includes: through multi-angle image interception, the fragment image data enhancement based on geometric transformation is realized, through 3D scanning analysis software, the model is converted into point cloud format and mapping is performed;

[0173] The fragment image data enhancement based on geometric transformation in step 2, the model is converted into point cloud format and mapping includes: ① through GeomagicStudio software, after the three-dimensional model of the fragment is flipped and rotated at different angles, plane screenshot is performed, the two-dimensional image corresponding to different angles from the three-dimensional model of the Qin terracotta fragment is obtained; ② traverse each three-dimensional coordinate (x, y, z), using the minimum value and the maximum value of the point cloud, x and y are normalized to the image pixel range (a, b). Check whether (a, b) is in the effective range:

[0174] If within the valid range, the processing is: Keep. If the (a, b) coordinate value after checking is within the valid pixel range of the image, then the brightness value (z value) of the corresponding 3D point can be mapped to the image position corresponding to (a, b). The brightness value of the pixel is updated to ensure that only the maximum brightness value is retained at the same pixel position. In other words, the corresponding pixel position is updated to ensure that the highest brightness value is stored, and the corresponding 2D image is mapped.

[0175] Processing that is not within the valid range: discard. If the (a, b) coordinate exceeds the valid range of the image, that is, (a, b) is not between the width and height of the image (usually (0, 0) to (W-1, H-1), where W and H are the width and height of the image respectively), then discard the point and skip it directly without any mapping.

[0176] like Figure 4 As shown, step 3 is to construct a multimodal descriptor GLSoble (Global-local Sobel descriptor) for extracting edge features of the terracotta warriors, which includes: extracting global features through frequency domain transformation enhancement method; extracting local features through adaptive domain method; and fusing features through dynamic weight fusion strategy to obtain the final result;

[0177] The process of extracting global features by frequency domain transformation enhancement method in step 3 includes: ① inputting Qin Terracotta Warriors data; ② calculating Fourier transform to convert the image from spatial domain to frequency domain; performing transformation displacement to move the zero-frequency component to the frequency harmonic center, so that the center of the spectrum after shifting is low-frequency information and the edge is high-frequency information, making the visualization of the frequency domain more intuitive; ③ calculating the amplitude spectrum to obtain image frequency information, and performing logarithmic transformation to enhance contrast; ④ applying a Gaussian low-pass filter to smooth the image to reduce high-frequency noise; ⑤ performing inverse Fourier transform to convert the frequency domain image back to the spatial domain; ⑥ using the Sobel operator to extract global features; ⑦ outputting global edge features.

[0178] The calculation process of Fourier transform and displacement is as follows:

[0179] F(u, v) = ∫∫f(x, y)e -2πi(ux+vy) dxdy

[0180]

[0181] where x is the horizontal coordinate (column) of the image, y is the vertical coordinate (row) of the image; u is the horizontal frequency in the frequency domain, representing the frequency component in the x direction; v is the vertical frequency in the frequency domain, representing the frequency component in the y direction; i is the imaginary unit; f(x, y) represents the pixel value of the image at coordinate (x, y); F(u, v) represents the frequency domain image, i.e. the complex value of the frequency component.

[0182] The calculation process of the amplitude spectrum and the enhanced contrast is as follows:

[0183]

[0184] M(u, v) = log(1 + S(u, v))

[0185] where Re is the real part of the Fourier transform result, and Im is the imaginary part of the Fourier transform result.

[0186] The calculation process of applying the Gaussian low-pass filter is as follows:

[0187]

[0188] F filtered = M(u, v) · G(c, d)

[0189] where σ is the standard deviation that controls the width of the filter, and the larger the value represents the stronger the filtering effect; c is the horizontal distance from the center of the frequency domain to the current frequency u; d is the vertical distance from the center of the frequency domain to the current frequency v; G is the value of the Gaussian filter.

[0190] The calculation process of the inverse Fourier transform is as follows:

[0191]

[0192] where x and y are the coordinates in the spatial domain; X and Y are the size or scale of the time domain signal F(x, y) in the x direction and y direction respectively, usually representing the spatial size of the image or signal.

[0193] After adding the normalization coefficient of 1 / XY, the Fourier transform and the inverse Fourier transform are symmetrical, i.e. the signal size before and after the transformation is equal.

[0194] The calculation process of the global edge feature is as follows:

[0195]

[0196]

[0197]

[0198] F global= normalize(S)

[0199] where S x , S y are the edge detection results of the image in x direction and y direction respectively, S x contains a matrix which is a convolution kernel for detecting horizontal edges, S y contains a matrix which is a convolution kernel for detecting vertical edges; * represents convolution operation, and normalize represents normalization operation.

[0200] The process of local feature extraction in step 3 by adaptive field method includes: ① inputting the Qin figure data; ② setting the field, i.e. the size and step of the sliding window, and performing Gaussian blur in the window; ③ using Sobel operator to extract local features; ④ threshold processing to filter out insignificant edges and retain important edge features; ⑤ local maximum value retention: comparing the edges in the window with the previous local edges and retaining the maximum value; ⑥ outputting the local edge features.

[0201] The processing process of Gaussian blur in the sliding window is as follows:

[0202]

[0203] where is the Gaussian blur processing operation, W is the image block in the current sliding window, and σ is the sliding window step.

[0204] The calculation process of local feature extraction is as follows:

[0205]

[0206]

[0207]

[0208] The calculation process of threshold processing is as follows:

[0209] E local (E local < threshold) = 0

[0210] where threshold is the threshold value, which is set according to the characteristics of the data set. For example, after several rounds of testing, threshold is set to 30 in this experiment, which can basically determine that it is located in the transition zone between low gray value and high gray value; it can retain most of the important texture details of the Qin figure fragments, and avoid losing, missing edge information or misjudging noise as much as possible.

[0211] The process of local maximum value retention and normalization is as follows:

[0212] L(y,x)=max(L(y,x),E local )

[0213] F local =normalize(L)

[0214] The specific process corresponding to the adaptive domain is as follows: ① Set the window size and step size, with the step size being half the window size, to ensure a certain overlap between windows and enhance the continuity and integrity of edge information; ② Slide the window to traverse the image

[0215] A double loop is used to iterate over each region of the image: the outer loop iterates over the image's height, and the inner loop iterates over its width. In each iteration, a window of a specified size is extracted. This divides the entire image into multiple overlapping patches, each of which is processed independently to extract local features. (3) Gaussian blur is applied within each window to reduce image noise. (4) For each blurred window, the Sobel operator is used for edge detection. (5) A window threshold is set, and edge intensities below this threshold are set to zero to preserve significant edge information.

[0216] The process of feature fusion through the dynamic weight fusion strategy in step 3 includes: ① dynamic weight calculation, including the weights of global features and local features; ② weight normalization, calculating the comprehensive weight to represent the proportion of global features in the final fusion, reflecting the relative importance of global and local features; ③ feature fusion.

[0217] The calculation process of dynamic weight is as follows:

[0218]

[0219]

[0220]

[0221] where ∈ is a small constant used to prevent the denominator from being zero.

[0222] The calculation process of fusion features is as follows:

[0223] F fused =W norm ·F global +(1-W norm )·F local

[0224] like Figure 6 and Figure 7As shown in step 4, the edge extraction results for two-dimensional and three-dimensional data are obtained respectively, the task of edge extraction of Qin Shi Huang's Terracotta Warriors and Horses fragments based on the dual-module enhanced adaptive descriptor is completed, and as can be seen from the figure, the overall image quality and analysis accuracy of the restored Qin Shi Huang's Terracotta Warriors and Horses fragments are high, the identification ability of the detailed carvings and decorations of the Qin Shi Huang's Terracotta Warriors and Horses is strong, and the accuracy of the reconstruction of cultural relic fragments is improved.

[0225] As shown in step 4, the edge extraction results for two-dimensional and three-dimensional data are obtained respectively, the task of edge extraction of Qin Shi Huang's Terracotta Warriors and Horses fragments based on the dual-module enhanced adaptive descriptor is completed, and as can be seen from the figure, the overall image quality and analysis accuracy of the restored Qin Shi Huang's Terracotta Warriors and Horses fragments are high, the identification ability of the detailed carvings and decorations of the Qin Shi Huang's Terracotta Warriors and Horses is strong, and the accuracy of the reconstruction of cultural relic fragments is improved. Figure 5 Figure 5 As shown in step 4, the edge extraction results for two-dimensional and three-dimensional data are obtained respectively, the task of edge extraction of Qin Shi Huang's Terracotta Warriors and Horses fragments based on the dual-module enhanced adaptive descriptor is completed, and as can be seen from the figure, the overall image quality and analysis accuracy of the restored Qin Shi Huang's Terracotta Warriors and Horses fragments are high, the identification ability of the detailed carvings and decorations of the Qin Shi Huang's Terracotta Warriors and Horses is strong, and the accuracy of the reconstruction of cultural relic fragments is improved.

[0226] In summary, the present application constructs a laboratory cultural relic fragment dataset based on Qin Shi Huang's Terracotta Warriors and Horses, provides a training basis for subsequent edge extraction experiments, designs a global feature extraction module based on frequency domain transformation enhancement, a local feature extraction module based on adaptive domain, and an edge feature fusion module based on a dynamic weight fusion strategy, applies an adaptive Gaussian low-pass filter to dynamically adjust the standard deviation according to the image features, more accurately smooths the image and preserves important low-frequency information. This improvement enables the global feature extraction to more comprehensively capture image information, improves the overall image quality and analysis accuracy, applies the Sobel operator at different scales through an adaptive threshold strategy, effectively detects small edge changes in damaged and blurred areas, ensures that each local feature is fully mined, thereby improving the identification ability of the detailed carvings and decorations of the Qin Shi Huang's Terracotta Warriors and Horses, integrates the global and local edge features, preserves the maximum edge value at each pixel position, and forms a more comprehensive feature description. Effectively integrates information of different scales and directions, improves the accuracy of the reconstruction of cultural relic fragments, and ensures the identification and preservation of important features in archaeological analysis.

[0227] The present application provides a dual-module enhanced adaptive descriptor for edge extraction of cultural relic fragments, which is superior to traditional edge extraction operators in datasets containing special materials, textures, fracture surfaces and other objects, provides technical support for the edge extraction of Qin Shi Huang's Terracotta Warriors and Horses fragments, and plays a strong promoting role in the digital protection of cultural heritage.

[0228] The operation process of the three-dimensional model edge extraction method of the cross-latitude global-local descriptor provided by the present application is described in detail above. The Qin Shi Huang's Terracotta Warriors and Horses fragment dataset constructed in the laboratory is applied in the present application to demonstrate the technology. The above description of the embodiments is only used to help understand the core idea of the present application, and the method can be extended to an image edge extraction public dataset.

Claims

1. A dual-module enhanced adaptive descriptor edge extraction method, characterized in that: The following steps are involved: Step 1: After cleaning and numbering the cultural relic fragments, scan them to obtain 2D images and 3D models of the cultural relic fragments, perform label association annotation, and construct the original dataset; Step 2: Process the original dataset, performing geometric transformation-based 2D image data enhancement by capturing multi-angle image screenshots. Using 3D scanning analysis software, the 3D model is converted into a point cloud format and mapped to a corresponding 2D image. The enhanced 2D image and the corresponding 2D image mapped from the 3D model together form a fragmented 2D image dataset. Step 3: Based on the fragmented two-dimensional image dataset obtained in step 2, a dual-module enhanced adaptive descriptor GLSoble for feature extraction is constructed, specifically: Step 3.1, extract global edge features through frequency domain transformation enhancement method; In step 3.1, global edge feature extraction is performed by frequency domain transformation enhancement method, specifically: In step 3.11, input the processed fragmented 2D image dataset obtained in step 2 and calculate the Fourier transform to convert the image from the spatial domain to the frequency domain. Perform a transform shift to move the zero-frequency component to the center of the frequency harmonic. After the shift, the center of the spectrum is low-frequency information and the edges are high-frequency information. The calculation process is as follows: F(u,v)=∫∫f(x,y)e -2πi(ux+vy) dxdy Where x is the horizontal coordinate of the image, y is the vertical coordinate of the image; u is the horizontal frequency in the frequency domain, which represents the frequency component in the x direction; v is the vertical frequency in the frequency domain, which represents the frequency component in the y direction; i is the imaginary unit; f(x,y) represents the pixel value of the image at the coordinate (x,y); F(u,v) represents the frequency domain image, that is, the complex value of the frequency component; Step 3.12, based on the change in displacement F obtained in step 3.11 shift (u, v), calculate the amplitude spectrum to obtain the image frequency information, and perform logarithmic transformation to enhance the contrast; M(u,v)=log(1+S(u,v)) Where Re is the real part of the Fourier transform result, and Im is the imaginary part of the Fourier transform result; Step 3.13, applying a Gaussian low-pass filter to the logarithmic transformed magnitude spectrum M(u,v) obtained in step 3.12 to smooth the image and reduce high-frequency noise; F filtered =M(u,v)·G(c,d) Where σ is the standard deviation of the control filter width; c is the horizontal distance from the center of the frequency domain to the current frequency u; d is the vertical distance from the center of the frequency domain to the current frequency v; G is the value of the Gaussian filter; Step 3.14: The amplitude spectrum F obtained in step 3.13 after passing through the Gaussian filter is filtered Perform inverse Fourier transform to convert the frequency domain image back to the spatial domain; Step 3.15, for the f obtained in step 3.14 back (x,y) uses the Sobel operator to extract global edge features and output the global edge feature F global ; F global =normalize(S) Among them, S x 、S y They are the edge detection results of the image in the x direction and y direction, S x The matrix contained in is the convolution kernel used to detect horizontal edges, S y The matrix contained in is the convolution kernel used to detect vertical edges; * represents the convolution operation, and normalize represents the normalization operation; Step 3.2, extract local edge features using the adaptive domain method; The domain is the sliding window size and step size, and Gaussian blur is performed within the window; Step 3.3: The global edge features and local edge features are fused through a dynamic weight fusion strategy to obtain a dual-module enhanced adaptive descriptor. The specific process of step 3.3 is as follows: Step 3.31, calculate the global feature dynamic weight W global and local feature dynamic weight W local , the calculation process is as follows: Where ∈ is a small constant used to prevent the denominator from being zero; Step 3.32, normalize the global feature dynamic weight and local feature dynamic weight obtained in step 3.

31. The calculation process is as follows; Step 3.33, the normalized weight W obtained in step 3.32 norm To perform feature fusion of global edge features and local edge features, we obtain the dual-module enhanced adaptive descriptor F fused : F fused =W norm ·F global +(1-W norm )·F local ; Step 4: Based on the dual-module enhanced adaptive descriptor obtained in step 3, edge extraction results of the two-dimensional image of the cultural relic fragment and the three-dimensional model of the cultural relic fragment are obtained respectively, completing edge extraction based on the dual-module enhanced adaptive descriptor.

2. The edge extraction method of the dual-module enhanced adaptive descriptor according to claim 1, characterized in that: In step 1, the direction, height and distance of the light source are adjusted to determine the optimal light source angle, reduce reflection, and obtain a two-dimensional image of the cultural relic fragment; and the laboratory's EVA scanner is used to perform three-dimensional scanning to obtain a three-dimensional model of the cultural relic fragment.

3. The edge extraction method of the dual-module enhanced adaptive descriptor according to claim 1, characterized in that: Step 2 converts the 3D model into a point cloud format and maps it using 3D scanning analysis software, specifically: Step 2.1: Use Geomagic Studio software to flip and rotate the 3D model at different angles and then take a two-dimensional screenshot to obtain 2D images corresponding to different angles of the 3D model. Step 2.2, traverse each three-dimensional coordinate (x, y, z), use the minimum and maximum values ​​of the point cloud to normalize x and y to the image pixel range (a, b), if the image pixel range (a, b) is within the valid pixel range of the image, then normalize the z value to the brightness value, update the corresponding pixel position to ensure that the highest brightness value is stored, and map the corresponding two-dimensional image; if the image pixel range (a, b) exceeds the valid pixel range of the image, then discard the point.

4. The edge extraction method of the dual-module enhanced adaptive descriptor according to claim 1, characterized in that: In step 3.2, local edge feature extraction is performed using an adaptive domain method, specifically: Step 3.21: Input the processed fragmented 2D image dataset obtained in step 2, set the sliding window size and step size, traverse the image through the sliding window, and apply Gaussian blur within each sliding window; in, is the Gaussian blur operation, W is the image block in the current sliding window, and σ is the sliding window step size; Step 3.22, after Gaussian blurring, the W blurred , use the Sobel operator to extract local edge features and perform threshold processing to filter out insignificant edges and retain important edge features; Local edge feature E local The calculation process of extraction is as follows: The calculation process of threshold processing is as follows: AND local (AND local <threshold)=0 Among them, threshold is the threshold value, which is located in the transition zone between low grayscale value and high grayscale value; Step 3.23, the edge features in the window are combined with the local edge features E local Compare, retain the maximum value, and output as local edge feature F local : L(y,x)=max(L(y,x),E local ) F local =normalize(L) Among them, L(y,x) represents the local maximum matrix, which indicates the edge strength of each window. The edge information of multiple local windows can be combined to capture the global edge features. local is the normalized local edge feature F local .

5. The edge extraction method of the dual-module enhanced adaptive descriptor according to claim 4 is characterized in that: The sliding window step size is half of the sliding window size, and a double loop is used to traverse each area of ​​the image. The outer loop traverses the height of the image and the inner loop traverses the width of the image. In each loop, a window of a specified size is extracted.

6. A dual-module enhanced adaptive descriptor edge extraction system, characterized in that: include: Original dataset acquisition module: After cleaning and numbering the cultural relic fragments, they are scanned to obtain two-dimensional images and three-dimensional models of the cultural relic fragments, and then associated and annotated with labels to construct the original dataset; Raw Dataset Processing Module: Processes the raw dataset, including geometric transformation-based 2D image data enhancement through multi-angle image capture. 3D scanning analysis software converts the 3D model into a point cloud format and maps it to a corresponding 2D image. The enhanced 2D image and the corresponding 2D image mapped from the 3D model together form a fragmented 2D image dataset. Dual-module enhanced adaptive descriptor construction module: Based on the obtained fragmented two-dimensional image dataset, a dual-module enhanced adaptive descriptor GLSoble for feature extraction is constructed, specifically: The global edge feature extraction is performed by frequency domain transformation enhancement method. The specific process of global edge feature extraction by frequency domain transformation enhancement method is as follows: In step 3.11, input the processed fragmented 2D image dataset and calculate the Fourier transform to convert the image from the spatial domain to the frequency domain. Perform a transform shift to move the zero-frequency component to the center of the frequency harmonic. After the shift, the center of the spectrum is low-frequency information and the edges are high-frequency information. The calculation process is as follows: F(u,v)=∫∫f(x,y)e -2πi(ux+vy) dxdy Where x is the horizontal coordinate of the image, y is the vertical coordinate of the image; u is the horizontal frequency in the frequency domain, which represents the frequency component in the x direction; v is the vertical frequency in the frequency domain, which represents the frequency component in the y direction; i is the imaginary unit; f(x,y) represents the pixel value of the image at the coordinate (x,y); F(u,v) represents the frequency domain image, that is, the complex value of the frequency component; Step 3.12, based on the change in displacement F obtained in step 3.11 shift (u, v), calculate the amplitude spectrum to obtain the image frequency information, and perform logarithmic transformation to enhance the contrast; M(u,v)=log(1+S(u,v)) Where Re is the real part of the Fourier transform result, and Im is the imaginary part of the Fourier transform result; Step 3.13, applying a Gaussian low-pass filter to the logarithmic transformed magnitude spectrum M(u,v) obtained in step 3.12 to smooth the image and reduce high-frequency noise; F filtered =M(u,v)·G(c,d) Where σ is the standard deviation of the control filter width; c is the horizontal distance from the center of the frequency domain to the current frequency u; d is the vertical distance from the center of the frequency domain to the current frequency v; G is the value of the Gaussian filter; Step 3.14: The amplitude spectrum F obtained in step 3.13 after passing through the Gaussian filter is filtered Perform inverse Fourier transform to convert the frequency domain image back to the spatial domain; Step 3.15, for the f obtained in step 3.14 back (x,y) uses the Sobel operator to extract global edge features and output the global edge feature F global ; F global =normalize(S) Among them, S x 、S y They are the edge detection results of the image in the x direction and y direction, S x The matrix contained in is the convolution kernel used to detect horizontal edges, S y The matrix contained in is the convolution kernel used to detect vertical edges; * represents the convolution operation, and normalize represents the normalization operation; Local edge feature extraction through adaptive domain method; The domain is the sliding window size and step size, and Gaussian blur is performed within the window; The dynamic weight fusion strategy is used to fuse global edge features and local edge features, and a dual-module enhanced adaptive descriptor is obtained. The dynamic weight fusion strategy is used to fuse the global edge features and local edge features to obtain a dual-module enhanced adaptive descriptor. The specific process is as follows: Step 3.31, calculate the global feature dynamic weight W global and local feature dynamic weight W local , the calculation process is as follows: Where ∈ is a small constant used to prevent the denominator from being zero; Step 3.32, normalize the global feature dynamic weight and local feature dynamic weight obtained in step 3.

31. The calculation process is as follows; Step 3.33, the normalized weight W obtained in step 3.32 norm To perform feature fusion of global edge features and local edge features, we obtain the dual-module enhanced adaptive descriptor F fused : F fused =W norm ·F global +(1-W norm )·F local ; Edge extraction module: According to the dual-module enhanced adaptive descriptor, the edge extraction results of the two-dimensional image of the cultural relic fragment and the three-dimensional model of the cultural relic fragment are obtained respectively, completing the edge extraction based on the dual-module enhanced adaptive descriptor.

7. A dual-module enhanced adaptive descriptor edge extraction device, characterized in that: include: Memory: a computer-readable device storing a computer program for the edge extraction method of a dual-module enhanced adaptive descriptor according to any one of claims 1 to 5; Processor: configured to implement the edge extraction method of the dual-module enhanced adaptive descriptor described in any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it can implement the edge extraction method of the dual-module enhanced adaptive descriptor described in any one of claims 1 to 5.

Images