An image recognition-based cultural relic information extraction system

Abstract

The application relates to the technical field of image recognition, in particular to a cultural relic information extraction system based on image recognition; the system comprises a cultural relic image enhancement module, a point cloud reconstruction and elimination module, a model fusion and detailing module and a cultural relic model correction module; firstly, an image enhancement network model is established based on a double-branch neural network, and cultural relic images are subjected to image enhancement processing; secondly, image distortion correction is carried out, a point cloud fusion reconstruction method is used to screen matching points, and an outlying point detection algorithm based on a dynamic standard deviation is used to eliminate outlying points; then, a model surface is reconstructed, internal and external model fusion is carried out, a fused three-dimensional cultural relic model is obtained, and detailed processing is carried out to obtain a detailed cultural relic surface image; finally, the detailed cultural relic surface image is corrected, and the fused three-dimensional cultural relic model is subjected to mapping to obtain a final three-dimensional cultural relic model; the application realizes the purpose of cultural relic information extraction by processing cultural relic multi-source images, and the method is objective and accurate.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, specifically to a system for extracting cultural relic information based on image recognition. Background Technology

[0002] In recent years, deep learning-driven image recognition technology has made breakthroughs in fields such as security, healthcare, and autonomous driving. It achieves automatic learning and recognition of complex visual features through models such as convolutional neural networks (CNNs). In the field of cultural heritage protection, extracting surface information (such as inscriptions, patterns, materials, and damage characteristics) from cultural relics using image recognition has become a research hotspot. However, facing the complex scenarios of actual cultural relic protection, such as fading murals, corrosion of bronzes, adherence of fabric textures, and interference from cracks in oracle bone inscriptions, existing models fail to recognize artifacts due to insufficient feature generalization ability. This is especially true for irregular characters (such as oracle bone inscriptions and variant inscriptions) and microscopic damage, where recognition accuracy is insufficient. Current technologies still rely on manual correction, have limited intelligence, and cannot meet the needs of large-scale cultural relic digitization.

[0003] Traditional methods for extracting information from cultural relics generally rely on manual rubbing, which involves physically rubbing the surface of stone carvings, bronzes, etc., using Xuan paper and ink. Local features are then observed using a microscope or magnifying glass, and basic image processing is performed to extract information such as the outline of the cultural relics using traditional computer vision methods such as edge detection (Canny operator) and threshold segmentation (Otsu algorithm).

[0004] Traditional methods for extracting information about cultural relics may accelerate their aging due to physical contact, violating the principle of prioritizing protection. Traditional computer vision methods used may fail to recognize relics due to insufficient feature generalization ability. When faced with cultural relics that are difficult to handle, such as incomplete or deformed text or irregular patterns, the extraction speed lags behind the protection needs, and the extracted information about the cultural relics is incomplete. Summary of the Invention

[0005] In view of the problems in related technologies, the present invention provides a cultural relic information extraction system based on image recognition to overcome the technical problems existing in the existing related technologies.

[0006] To solve the aforementioned technical problem, the present invention is achieved through the following technical solution:

[0007] This invention is a system for extracting cultural relic information based on image recognition, specifically including: a cultural relic image enhancement module, a point cloud reconstruction and removal module, a model fusion and detailing module, and a cultural relic model correction module;

[0008] The cultural relic image enhancement module is used to simultaneously acquire multi-source cultural relic images, establish an image enhancement network model based on a dual-branch neural network, perform image enhancement processing, and output enhanced cultural relic images.

[0009] The point cloud reconstruction and removal module is used to correct image distortion in the enhanced cultural relic image, use the point cloud fusion reconstruction method to screen matching points, and then remove outliers based on the outlier detection algorithm of dynamic standard deviation to extract the final set of matching points.

[0010] The model fusion and detailing module is used to reconstruct the model surface based on the final set of matching points, obtain a panoramic view of the inside of the cultural relic and fuse the internal and external models to generate a fused 3D cultural relic model, and then perform model planar detailing to obtain a detailed image of the cultural relic surface.

[0011] The artifact model correction module is used to refine the surface image of the artifact based on the color difference of the model, and then apply textures to the fused 3D artifact model to obtain the final 3D artifact model.

[0012] Preferably, the image enhancement network model based on a dual-branch neural network includes:

[0013] Collect images of cultural relics from multiple sources and align them with spatial coordinates to obtain processed images of cultural relics from multiple sources;

[0014] The image enhancement network model is set to adopt a dual-branch neural network, which includes a first branch and a second branch. The first branch receives surface texture images and depth images, extracts multi-layer features using an encoder with a convolution kernel size of 3×3 and a stride of 2, and then introduces a deconvolution layer and residual connections to form a decoder to recover a high-resolution image and output a uniform cultural relic image.

[0015] The second branch receives near-infrared imaging, uses stacked residual blocks with 2 convolutional layers per level to obtain the image noise distribution, uses the ReLU function as the activation function, and introduces a spatial attention mechanism to suppress noise in the highlight area, outputting a denoised near-infrared image, thus constructing an image enhancement network model.

[0016] Preferably, the output enhanced artifact image includes:

[0017] A new set of multi-source cultural relic image samples is collected and input into an image enhancement network model for training to obtain the final image enhancement network model. The processed multi-source cultural relic images are then input into the final image enhancement network model for image enhancement processing, and the enhanced cultural relic images are output.

[0018] Preferably, the image distortion correction for the enhanced cultural relic image includes:

[0019] Obtain the correction coefficient, calculate the bending radius, apply the correction force according to the correction coefficient, obtain the corrected pixel coordinates, and then remap them to the pixel positions of the enhanced cultural relic image to generate the corrected cultural relic image.

[0020] Preferably, the step of using point cloud fusion reconstruction to filter matching points includes:

[0021] The corrected depth image in the corrected cultural relic image is converted into point cloud data, and the point cloud data feature set is output.

[0022] Define the neighborhood range, calculate the neighborhood features, and sum the neighborhood features within the neighborhood range to obtain the final features;

[0023] Calculate the similarity threshold based on the final features, filter valid matching points in the point cloud data feature set, and generate an initial set of matching points;

[0024] Record the three-dimensional direction of the matching point. Search for matching points in the three-dimensional direction of the initial matching point set to obtain the three-dimensional direction matching points. Build a K-dimensional binary tree and calculate the distance between matching points in the K-dimensional binary tree. Set a distance threshold. When the distance between matching points is less than the distance threshold, retain the corresponding two matching points to obtain the precise matching point set.

[0025] Preferably, the outlier detection algorithm based on dynamic standard deviation removes outliers by:

[0026] Calculate the distance between matching points in the set of precisely matched points to obtain the average distance;

[0027] Calculate the global mean and global standard deviation. Divide the global standard deviation by the global mean to obtain the standard deviation coefficient. When the absolute value of the difference between the distance of the matching point to the mean and the global mean is greater than the standard deviation coefficient multiplied by the global standard deviation, the matching point is identified as an outlier. Iterate through the set of exact matching points in turn and remove all outliers to obtain the final set of matching points.

[0028] Preferably, the generation and fusion of the three-dimensional cultural relic model includes:

[0029] Generate a minimum bounding box based on the final set of matching points, and recursively divide the minimum bounding box into equal parts using the root node to generate an octree; calculate the implicit surface function value and store the implicit surface function value using the leaf nodes of the octree.

[0030] The eight corner points of the leaf nodes of the octree are marked with 1 or 0 to determine the connection order of the corner points, generate triangular patches, and obtain a three-dimensional cultural relic model.

[0031] Images of the internal structure of the cultural relic are collected, and after distortion correction, an internal ring image of the cultural relic is output. The image is then expanded and mapped along the axial line to obtain a panoramic view of the inside of the cultural relic. After reconstructing the internal model, the three-dimensional cultural relic model is fused to generate a fused three-dimensional cultural relic model.

[0032] Preferably, the process of performing planar detailing of the model includes:

[0033] Ultraviolet light was used to extract fluorescent ink marks from cultural relics and convert them into grayscale images. Weighted fusion was then used to output the fused image of the cultural relics.

[0034] The fused image of the cultural relic is subjected to wavelet transform to decompose it into a background image and a detail image. After detail reconstruction, a detailed image of the surface of the cultural relic is obtained.

[0035] Preferably, the texturing of the fused 3D cultural relic model includes:

[0036] Set a standard color chart, calculate the color difference between the color model of the detailed artifact surface image and the standard color chart, obtain the model color difference, determine whether to perform linear compensation on the detailed artifact surface image, and perform planar projection or curved surface projection, mapping it onto the surface of the fused 3D artifact model to obtain the final 3D artifact model.

[0037] The present invention has the following beneficial effects:

[0038] 1. This invention ensures spatial consistency of multi-source data by synchronously acquiring images of cultural relics, restoring details from surface texture images, enhancing three-dimensional structural information from depth images, revealing hidden internal features through near-infrared imaging, and establishing an image enhancement network model based on a dual-branch neural network to perform image enhancement processing on cultural relic images. The dual-branch network eliminates uneven lighting and shadows, improves texture clarity, significantly reduces noise interference, and provides high-quality input for subsequent information extraction.

[0039] 2. This invention solves the problem of surface distortion of cultural relics through high-precision geometric correction, uses point cloud fusion reconstruction to screen matching points, uses K-dimensional binary tree space partitioning to efficiently extract high-confidence matching points from complex point cloud data, reducing the false matching rate, and then uses an outlier detection algorithm based on dynamic standard deviation to remove outliers. The threshold of dynamic standard deviation coefficient is used to determine outliers, automatically adapting to the complexity of the bronze tripod surface and ensuring the reliability of the matching point set.

[0040] 3. This invention accurately restores the curved topological structure of cultural relics by reconstructing the model surface and using recursive segmentation and implicit surface function calculation. It obtains images of the internal structure of cultural relics to generate panoramic images of the internal structure of cultural relics, breaks through the bottleneck of modeling the internal structure of closed cultural relics, and seamlessly aligns and merges the internal and external models. It comprehensively extracts information about cultural relics and then uses multispectral layering to enhance and restore some difficult-to-distinguish text, irregular patterns, artistic details, and material deterioration traces.

[0041] 4. This invention is based on model color difference correction to refine the surface image of cultural relics, dynamically calculates color difference to perform linear compensation for non-protected areas, scientifically and accurately restores the color of cultural relics, and then applies textures to the fused 3D cultural relic model. The adaptive textures in this process achieve seamless integration, meet museum-level visualization standards, and ensure the texture integrity and comprehensiveness of highly complex structures. Attached Figure Description

[0042] To more clearly illustrate the technical solutions of the embodiments of the invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For those skilled in the art, the drawings can be obtained from these drawings without creative effort.

[0043] Figure 1 A flowchart of an image recognition-based cultural relic information extraction system is provided for this invention;

[0044] Figure 2 This invention provides a flowchart illustrating a method for extracting cultural relic information based on image recognition. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Traditional methods for extracting information about cultural relics may accelerate their aging due to physical contact, violating the principle of prioritizing protection. Traditional computer vision methods used may fail to recognize relics due to insufficient feature generalization ability. When faced with cultural relics that are difficult to handle, such as incomplete or deformed text or irregular patterns, the extraction speed lags behind the protection needs, and the extracted information about the cultural relics is incomplete.

[0047] To solve the above technical problems, such as Figure 1 As shown in the figure, this invention provides a system for extracting cultural relic information based on image recognition, specifically including: a cultural relic image enhancement module, a point cloud reconstruction and removal module, a model fusion and detailing module, and a cultural relic model correction module; the cultural relic image enhancement module is used to simultaneously acquire multi-source cultural relic images, establish an image enhancement network model based on a dual-branch neural network, perform image enhancement processing, and output an enhanced cultural relic image; the point cloud reconstruction and removal module is used to correct image distortion in the enhanced cultural relic image, use a point cloud fusion reconstruction method to screen matching points, and then remove outliers based on a dynamic standard deviation outlier detection algorithm to extract the final set of matching points; the model fusion and detailing module is used to reconstruct the model surface based on the final set of matching points, obtain a panoramic view of the interior of the cultural relic and perform internal and external model fusion to generate a fused 3D cultural relic model, and then perform model planar detailing processing to obtain a detailed cultural relic surface image; the cultural relic model correction module is used to correct the detailed cultural relic surface image based on model color difference, and then apply textures to the fused 3D cultural relic model to obtain the final 3D cultural relic model.

[0048] In a specific embodiment, in an archaeological research laboratory, researchers need to extract detailed information about a Han Dynasty bronze ding (a type of ancient Chinese cooking vessel). The ding has complex patterns and corrosion on its surface, but the original image quality is poor due to uneven lighting, surface highlights, and background noise. Three dedicated cameras (a high-resolution visible light camera, a near-infrared camera, and a structured light scanner) are used to simultaneously acquire multi-source images, providing a good image foundation for the embodiments of the present invention.

[0049] In the specific implementation process of the above embodiments, firstly, multi-source images of cultural relics are simultaneously acquired and spatially aligned. An image enhancement network model is established based on a dual-branch neural network to enhance the images of cultural relics. This method ensures the spatial consistency of multi-source data by simultaneously acquiring surface texture, near-infrared, and depth images. Surface texture images restore details (such as bronze ding ornamentation and corrosion), depth images enhance three-dimensional structural information, and near-infrared imaging reveals internal hidden features. At the same time, dual-branch imaging is used to eliminate uneven lighting and shadows, improve texture clarity, and significantly reduce noise interference, providing high-quality input for subsequent information extraction. Secondly, image distortion correction is performed. A point cloud fusion reconstruction method is used to screen matching points, and then an outlier detection algorithm based on dynamic standard deviation is used to remove outliers, resulting in the final set of matching points. This method solves the problem of surface distortion of cultural relics through high-precision geometric correction. K-dimensional binary tree spatial partitioning efficiently extracts high-confidence matching points from complex point cloud data, reducing the false matching rate. At the same time, outliers are determined based on the threshold of dynamic standard deviation coefficient, automatically adapting to the complexity of the bronze ding surface. (e.g., rust holes, damaged areas) to ensure the reliability of the matching point set; then reconstruct the model surface, obtain images of the internal structure of the cultural relic to generate a panoramic view of the inside of the cultural relic, and perform internal and external model fusion to generate a fused 3D cultural relic model, and then perform planar detailing of the model to obtain a detailed image of the cultural relic surface; this method accurately restores the curved surface topology of the cultural relic through recursive segmentation and implicit surface function calculation, and innovatively uses endoscopic axial unfolding combined with internal and external fusion to break through the bottleneck of modeling the inside of closed cultural relics, seamlessly aligning and fusing the internal and external models, comprehensively extracting cultural relic information, and then using multispectral layering to enhance and restore some difficult-to-distinguish text, irregular patterns, artistic details and material deterioration traces; finally, based on the model color difference correction of the detailed cultural relic surface image, and then apply texture to the fused 3D cultural relic model to obtain the final 3D cultural relic model; in this process, the color difference is dynamically calculated to perform linear compensation for non-protected areas (such as the gilded patterns on the tripod feet), scientifically and accurately restore the color of the cultural relic, and adaptive texture achieves seamless fusion, meeting museum-level visualization standards and ensuring the texture integrity and comprehensiveness of highly complex structures.

[0050] Furthermore, to better illustrate the technical solutions of the embodiments of the present invention, such as... Figure 2As shown, this paper describes in detail an image recognition-based system for extracting cultural relics information, using an image recognition-based method. The specific details include the following:

[0051] S1. Simultaneously acquire images of cultural relics from multiple sources and align them with spatial coordinates. Establish an image enhancement network model based on a dual-branch neural network, perform image enhancement processing on the cultural relics images, and output enhanced cultural relics images.

[0052] S1 includes the following steps:

[0053] S11. Obtain cultural relic samples. Use a camera to collect surface texture images, near-infrared images, and depth images of the cultural relic samples to obtain multi-source cultural relic images, and uniformly scale them to a standard size. Set up a calibration board in the multi-source cultural relic images, detect the corner points of the calibration board's checkerboard grid, solve the rotation matrix and translation matrix of the camera relative to the world coordinate system, establish coordinate system transformation relationships, transform the multi-source cultural relic images to the world coordinate system, complete spatial coordinate alignment, and obtain the processed multi-source cultural relic images.

[0054] S12. Establish an image enhancement network model based on a dual-branch neural network and train the model to perform image enhancement processing on the cultural relic image. Input the processed multi-source cultural relic image and output the enhanced cultural relic image. The specific steps are as follows:

[0055] S121. The image enhancement network model is set to adopt a dual-branch neural network, which includes a first branch and a second branch. The first branch receives the surface texture image and the depth image, extracts multi-layer features using an encoder with a convolution kernel size of 3×3 and a stride of 2, and then introduces a deconvolution layer and residual connections to form a decoder to recover the high-resolution image and output a uniform cultural relic image.

[0056] The second branch receives near-infrared imaging, uses stacked residual blocks with 2 convolutional layers per level to obtain the image noise distribution, uses the ReLU function as the activation function, and introduces a spatial attention mechanism to suppress noise in the highlight area, outputs the denoised near-infrared image, and constructs the image enhancement network model.

[0057] S122. Reacquire new multi-source cultural relic images to generate a multi-source cultural relic image sample set. Divide the multi-source cultural relic image sample set into a sample test set and a sample validation set. Input the sample test set into the image enhancement network model for training until the model converges to obtain a trained image enhancement network model. Then, input the sample validation set into the trained image enhancement network model and output the model output image. At this time, calculate the structural similarity index and peak signal-to-noise ratio of the model output image, and compare them with a set threshold. When the structural similarity index and peak signal-to-noise ratio of the model output image are less than the set threshold, the final image enhancement network model is obtained; otherwise, trigger secondary enhancement and input the sample validation set into the trained image enhancement network model again.

[0058] S123. Input the processed multi-source cultural relic images into the final image enhancement network model, perform image enhancement processing, and output the enhanced cultural relic images;

[0059] In this embodiment, multi-source images of cultural relics are acquired simultaneously and aligned in spatial coordinates. An image enhancement network model is established based on a dual-branch neural network to enhance the images of the cultural relics. This method ensures the spatial consistency of multi-source data by simultaneously acquiring surface texture, near-infrared, and depth images. Surface texture images restore details (such as the patterns and corrosion of bronze tripods), depth images enhance three-dimensional structural information, and near-infrared imaging reveals hidden internal features. At the same time, dual-branch processing is used to eliminate uneven lighting and shadows, improve texture clarity, and significantly reduce noise interference. Specifically, for example, during image acquisition, a standard checkerboard calibration board (10cm x 10cm) is fixed next to the tripod. After acquisition, the images are uniformly scaled to a standard size (e.g., 512x512 pixels) to maintain consistency. The checkerboard corner points of the calibration board are detected, and the rotation and translation matrices of the camera relative to the world coordinate system are calculated to align all images (surface texture, near-infrared, and depth images) to a unified world coordinate system. Dual-branch processing: The first branch (processing surface texture and depth images) is encoded... The first branch (convolution kernel size 3x3, stride 2) extracts multi-layer features, which are then decoded (containing deconvolution layers and residual connections) to restore high resolution, enhancing the texture image by removing uneven lighting and shadows (e.g., making the patterns on the tripod clearer and the surface smoother). The second branch (processing near-infrared imaging) extracts noise distribution through 5 levels of residual blocks (2 convolutional layers per level, with ReLU activation function), and then introduces a spatial attention mechanism to weighted suppress noise in highlight areas (e.g., weakening the highlight areas on the edges of the tripod). The output is a denoised near-infrared image, making the internal... The structure is easier to identify; Training process: A multi-source image set is generated using other bronze artifact samples, divided into a test set and a validation set. The model is trained on the test set until convergence, and then the structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) are evaluated on the validation set. If the model meets the threshold requirements (e.g., SSIM>0.9 and PSNR>30dB) and no secondary enhancement is triggered, the final model is obtained; It can automatically handle common problems in cultural relic images (such as uneven lighting or specular noise), providing high-quality input for cultural relic information extraction;

[0060] S2. Perform image distortion correction on the enhanced cultural relic image, use point cloud fusion reconstruction method to screen matching points, and then remove outliers based on dynamic standard deviation outlier detection algorithm to extract the final set of matching points.

[0061] S2 includes the following steps:

[0062] S21. The enhanced cultural relic image includes enhanced surface texture image, enhanced near-infrared imaging, and enhanced depth image; obtain the camera focal length, the coordinates of the center pixel of the enhanced cultural relic image, and the distortion coefficient, etc.; transfer all pixels in the enhanced cultural relic image to the camera coordinate system, calculate the bending radius, apply a correction force according to the correction coefficient, obtain the corrected pixel coordinates, and then remap them to the pixel positions of the enhanced cultural relic image to generate the corrected cultural relic image;

[0063] S22. Based on the corrected image of the cultural relic, the matching points are selected using the point cloud fusion reconstruction method. The specific steps are as follows:

[0064] S221. In the corrected cultural relic image, the corrected depth image is obtained. The depth values ​​of the pixels in the corrected depth image are statistically analyzed. The pixels in the corrected depth image are converted into three-dimensional spatial coordinates to form an initial point cloud dataset. A neighborhood radius is set to form a neighborhood range. The least squares plane of each point cloud data is fitted in the initial point cloud dataset to calculate the curvature value. A curvature threshold is set, and the point cloud data with curvature values ​​greater than the curvature threshold are retained. The point cloud data feature set is output.

[0065] For the point cloud data feature set, the central point cloud data is determined within the neighborhood range, and other point cloud data are recorded as neighborhood points. Geometric features such as the angle between the normal of the central point cloud data and the neighborhood points, and the angle between the connecting line and the normal are calculated to form neighborhood features. The neighborhood features within the neighborhood range are accumulated with the distance from the neighborhood points to the central point cloud data as the weight to obtain the final features.

[0066] Select the final features of two point cloud data in the feature set of point cloud data, calculate the distance between the two point cloud data, and calculate the similarity threshold equal to 0.25 times the distance between the two point cloud data. When the final features of the two point cloud data are less than the similarity threshold, the two point cloud data are recorded as valid matching points, and an initial matching point set is generated.

[0067] S222. Select matching points from the initial matching point set, record the three-dimensional direction of the matching points, calculate the average value of the three-dimensional direction of the matching points, and search for the matching point in the initial matching point set that is closest to the average value of the three-dimensional direction of the matching point in the three-dimensional direction of the matching point to obtain the three-dimensional direction matching points. The three-dimensional direction matching points include x-direction matching points, y-direction matching points, and z-direction matching points. Draw a plane passing through the three-dimensional direction matching points and perpendicular to the three-dimensional direction of the matching points, and divide the spatial coordinate system into eight sub-regions. Repeat the division on the eight sub-regions until all sub-regions contain a matching point to obtain a K-dimensional binary tree. Calculate the distance between the matching points in the K-dimensional binary tree, set a distance threshold, and when the distance between the matching points is less than the distance threshold, retain the corresponding two matching points to obtain precise matching points and a set of precise matching points.

[0068] S23. Find the i-th matching point in the set of precise matching points, set a k-neighborhood, calculate the distance between the i-th matching point and other matching points in the k-neighborhood, and then use the sum of the distances between the i-th matching point and the k-neighborhood matching points to divide by the number of k-neighborhood matching points to obtain the average distance.

[0069] Calculate the mean distances of all other matching points in the exact matching point set within their k-neighborhoods to obtain a set of distance means. Then, divide the mean distance in the distance mean set by the number of exact matching points to obtain the global mean. Finally, calculate the global standard deviation. Where m represents the number of exact matching points, d i This represents the average distance to the i-th matching point. Let i represent the global mean, i = 1, 2, 3, ..., m;

[0070] The standard deviation coefficient is obtained by dividing the global standard deviation by the global mean. When the absolute value of the difference between the distance to the mean and the global mean of the i-th matching point is greater than the standard deviation coefficient multiplied by the global standard deviation, the i-th matching point is identified as an outlier. The set of precise matching points is traversed sequentially, and all outliers are removed to obtain the final set of matching points.

[0071] In this embodiment, image distortion correction is performed. A point cloud fusion reconstruction method is used to screen matching points, and then an outlier detection algorithm based on dynamic standard deviation is used to remove outliers, resulting in the final set of matching points. This method solves the problem of surface distortion on cultural relics through high-precision geometric correction. K-dimensional binary tree space partitioning efficiently extracts high-confidence matching points from complex point cloud data, reducing the false matching rate. Simultaneously, outliers are determined based on a threshold of the dynamic standard deviation coefficient, automatically adapting to the complexity of the bronze tripod surface (such as rust holes and damaged areas), ensuring the reliability of the matching point set. Specifically, for example, camera intrinsic parameters are obtained: focal length 24mm, image center pixel coordinates (512, 512), radial distortion coefficients k1 = -0.2, k2 = 0.1. A point (600, 700) on the tripod belly in the depth image is corrected. The image is then transformed to the camera coordinate system → the bending radius is calculated → a correction force is applied → the image is remapped back to pixel coordinates. The stretching deformation of the bulging area of ​​the tripod belly is corrected (original elliptical patterns). (Restored to a circle), the edge pixel displacement error decreased from ±15 pixels to ±3 pixels; a neighborhood radius of 5mm was set, and the least squares plane of each point was fitted. The curvature was calculated (points with curvature > 0.05 were retained, and planar regions were filtered out). For a feature point on one of the ding ears, the angle between the normals of the neighboring points (threshold ±10°) and the distance weight were calculated. It was compared with a symmetrical point on another ding ear. The distance between the two points was 300mm, the similarity threshold was 75mm, and the weighted feature difference was <75mm, so it was determined to be a valid matching point; segmentation Spatial analysis: A horizontal plane is drawn along the z-axis, dividing the ding body into 8 sub-regions (upper and lower). This subdivision is recursively reduced to a single point per region (e.g., point C at the end of a leg). A distance threshold of 2mm is set. At this point, C and its neighboring point D are retained, outputting a set of precisely matched points containing 1200 points (e.g., turning points of the ding's ear ornaments, support points on the bottom of the legs). For a point P on the ding body, with k = 20 neighbors, the mean distance between neighboring points is calculated to be 8.2mm, the global mean to be 7.9mm, the standard deviation to be 1.2mm, and the coefficient of variation to be approximately 0.15. P was identified as an outlier (due to abnormal distance caused by localized corrosion holes); 38 outliers were removed throughout the process (mostly rusted or damaged areas), and the final set of matching points retained 1162 high-confidence points;

[0072] S3. Reconstruct the model surface based on the final matching point set, obtain the internal structure image of the cultural relic to generate a panoramic view of the inside of the cultural relic, and perform internal and external model fusion to generate a fused three-dimensional cultural relic model. Then, perform model planar detailing to obtain a detailed image of the cultural relic surface.

[0073] S3 includes the following steps:

[0074] S31. Based on the final set of matching points, generate the minimum bounding box that surrounds all matching points, determine the boundary of the minimum bounding box, take the minimum bounding box as the root node, divide the minimum bounding box into eight identical sub-bounding boxes to form sub-nodes, and then recursively divide the sub-nodes, setting a maximum depth until the maximum depth is reached, then stop dividing to obtain an octree; the leaf nodes of the octree are all sub-bounding boxes, introduce the Laplace operator to convert the matching points into an indicator function gradient field, solve for the implicit surface function value, and store the implicit surface function value in the leaf nodes of the octree;

[0075] A surface boundary is defined. When the implicit surface function value of the eight corner points of the leaf node of the octree is greater than the surface boundary, the corresponding corner point is marked as 1, otherwise it is marked as 0. Then it is converted into an 8-bit binary index to obtain a decimal value. A triangulation template is preset. The connection order of the corner points is determined according to the decimal value to generate triangular patches. The normal interpolation of the triangular patches is performed to output a triangular mesh. Small holes are automatically filled to obtain a three-dimensional cultural relic model.

[0076] S32. Using an ultra-fine endoscope to collect images of the internal structure of the cultural relic through the opening, the original internal image of the cultural relic is obtained. The focal length and distortion coefficient of the ultra-fine endoscope are obtained. After distortion correction, the internal ring image of the cultural relic is output. The internal ring image of the cultural relic is unfolded and mapped along the axial line to obtain a panoramic view of the inside of the cultural relic. Then, matching points are extracted, the interior of the model is reconstructed, and a three-dimensional internal model of the cultural relic is obtained. The three-dimensional cultural relic model is fused to generate a fused three-dimensional cultural relic model.

[0077] S33. Multispectral fusion is used to perform planar detailing on the fused 3D cultural relic model to obtain an enhanced 3D cultural relic model. The specific steps are as follows:

[0078] S331. Use ultraviolet light to extract fluorescent ink marks from cultural relics and convert them into grayscale images. Combine the corrected surface texture image and calibrated near-infrared image in the corrected cultural relic image, assign weights to the grayscale image, the corrected surface texture image and the calibrated near-infrared image, and use weighted fusion to output the fused cultural relic image.

[0079] S332. Perform wavelet transform on the fused cultural relic image to decompose it into a background image and a detail image. Perform nonlinear sharpening on the background image and histogram equalization on the detail image. After detail reconstruction, complete the model plane detailing process to obtain a detailed cultural relic surface image.

[0080] In this embodiment, the model surface is reconstructed, an image of the internal structure of the artifact is obtained to generate a panoramic view of the artifact's interior, and the internal and external models are fused to generate a fused 3D artifact model. Then, the model's planar details are further refined to obtain a detailed image of the artifact's surface. This method accurately restores the artifact's curved surface topology through recursive segmentation and implicit surface function calculation. It innovatively uses endoscopic axial unfolding combined with internal and external fusion to overcome the bottleneck of modeling the interior of closed artifacts. The internal and external models are seamlessly aligned and fused, comprehensively extracting artifact information. Finally, multispectral layering enhancement is used to restore artistic details such as some difficult-to-decipher text and irregular patterns. And material degradation traces; specifically, for example, the final set of matching points (1162 points, including key points of the tripod legs and ears), generates the minimum bounding box of matching points (size 80cm×70cm×60cm), which is used as the root node of the octree, and recursively divided to a maximum depth of 6, generating 4096 sub-bounding boxes (leaf nodes); for a kui dragon pattern area on the belly of the tripod, the Laplace operator calculates the gradient field of the indicator function, solves the implicit surface function value, sets the surface boundary as 0, and marks the corner function value > 0 as 1, otherwise as 0; for example, the 8-bit binary index 01011010→90, from The system extracts the corresponding topology from a preset table of 256 patterns, matches it with a preset triangulation template, generates continuous triangular patches, and automatically fills small holes caused by corrosion. An ultra-fine endoscope (2.8mm diameter, 1.5mm focal length) is used to open the belly of the tripod. The endoscope penetrates deep into the tripod cavity to collect 360° annular images (12 images per circle, 3 circles in total). Based on the endoscope's distortion coefficient of 0.15, the image stretching of the inner wall of the tripod belly is corrected (the original elliptical rivets are restored to perfect circles). The annular image is then unfolded along the axis of the tripod cavity into a rectangular panoramic view to reconstruct a 3D model of the artifact's interior. The internal model is then aligned with the external model at the opening of the tripod belly. Fusion: Gaussian filtering is used to smooth the transition region to generate a fused 3D cultural relic model; Ultraviolet light fluorescent ink grayscale image: A multi-source image set is generated using other bronze samples, and the weights are adjusted to calculate the resolution of the cultural relic surface image. When the weights are assigned as follows: fluorescent ink grayscale image (0.6), surface texture (0.3), and near-infrared (0.1), the resolution of the cultural relic surface image reaches 100μm / pixel, which meets the best visual perception; Background image (low frequency): Nonlinear sharpening → enhances the overall contrast of the pattern; Detail image (high frequency): Histogram equalization → strengthens the micro-cracks in the oxide layer;

[0081] S4. Based on the color difference correction of the model, the detailed surface image of the cultural relic is corrected, and then the fused three-dimensional cultural relic model is textured to obtain the final three-dimensional cultural relic model;

[0082] S4 includes the following steps:

[0083] S41. Set the color model of the detailed cultural relic surface image as the RGB model, where the RGB model includes hue, saturation, and lightness; set a standard color card, calculate the color difference between the color model of the detailed cultural relic surface image and the standard color card, and obtain the model color difference.

[0084] Set a hue protection area. After removing the hue protection area from the detailed cultural relic surface image, set a color difference threshold. When the model color difference is less than the color difference threshold, there is no need to correct the detailed cultural relic surface image; otherwise, perform linear compensation on the detailed cultural relic surface image to obtain the corrected cultural relic surface image.

[0085] S42. Project the corrected cultural relic surface image onto the surface of the fused three-dimensional cultural relic model either in a planar projection or a curved surface projection, keeping the aspect ratio unchanged, filling the empty areas transparently, and placing the texture seams in the non-display area on the back to obtain the final three-dimensional cultural relic model.

[0086] In this embodiment, the detailed cultural relic surface image is corrected based on the model color difference, and then the fused three-dimensional cultural relic model is textured to obtain the final three-dimensional cultural relic model; in this process, the color difference is dynamically calculated to perform linear compensation on the non-protected areas (such as the gilded patterns on the tripod feet), scientifically and accurately restoring the colors of the cultural relics, and achieving seamless fusion through adaptive texture mapping, meeting the museum-level visualization standards and ensuring the integrity and comprehensiveness of the textures of high-complexity structures; specifically, for example, place an X-Rite standard color card in the rust-free area of the tripod belly, select a gilded pattern on one of the tripod feet, and extract the RGB values (hue, saturation, and lightness) of the image as 180, 100, and 80 (reddish-brown), and the corresponding color block of the standard color card is gilded (gold) (210, 180, 60), and the model color difference is calculated. Set the area with the inscription "Yi Zisun" as the protected area and do not participate in the correction; set the color difference threshold to 15 (the maximum color difference between the rust-free area of the tripod belly and the standard color card), and 87.3 > 15 triggers compensation; for the corrected cultural relic surface image and the fused three-dimensional cultural relic model, planar projection is used for the large-area curved surface of the tripod belly, and curved surface projection is used for the complex structures of the tripod ears / tripod feet; the image is divided into 4 quadrants for separate projection, the seams are placed on the non-display surface behind the ears, the empty areas (holes) are filled transparently, and adjacent texture blocks are fused with Gaussian weighting to eliminate the seam color difference, and the original gilding process details of the bronze tripod are displayed at a high resolution.

[0087] In the description of this specification, the descriptions referring to terms such as "one embodiment", "example", "specific example", etc. mean that the specific features, structures, materials, or feature points described in connection with the embodiment or example are included in at least one embodiment or example of the invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or feature points described can be combined in a suitable manner in any one or more embodiments or examples.

[0088] The preferred embodiments of the invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention.

Claims

1. A system for extracting cultural relic information based on image recognition, characterized in that, include: The cultural relic image enhancement module is used to simultaneously acquire images of cultural relics from multiple sources, establish an image enhancement network model based on a dual-branch neural network, perform image enhancement processing, and output enhanced images of cultural relics. The point cloud reconstruction and removal module is used to correct image distortion in the enhanced cultural relic image. It uses the point cloud fusion reconstruction method to screen matching points, and then removes outliers based on the outlier detection algorithm based on dynamic standard deviation to extract the final set of matching points. The model fusion and detailing module is used to reconstruct the model surface based on the final set of matching points, obtain a panoramic view of the inside of the cultural relic and fuse the internal and external models to generate a fused 3D cultural relic model. Then, the model plane detailing is performed to obtain a detailed image of the cultural relic surface. The artifact model correction module is used to refine the surface image of the artifact based on the color difference of the model, and then apply textures to the fused 3D artifact model to obtain the final 3D artifact model. The image enhancement network model based on a dual-branch neural network includes: Collect images of cultural relics from multiple sources and align them with spatial coordinates to obtain processed images of cultural relics from multiple sources; The image enhancement network model is set to adopt a dual-branch neural network, which includes a first branch and a second branch. The first branch receives surface texture images and depth images, extracts multi-layer features using an encoder with a convolution kernel size of 3×3 and a stride of 2, and then introduces a deconvolution layer and residual connections to form a decoder to recover a high-resolution image and output a uniform cultural relic image. The second branch receives near-infrared imaging, uses stacked residual blocks with 2 convolutional layers per level to obtain the image noise distribution, uses the ReLU function as the activation function, and introduces a spatial attention mechanism to suppress noise in the highlight area, outputs the denoised near-infrared image, and constructs the image enhancement network model. The generated and fused 3D cultural relic model includes: Generate a minimum bounding box based on the final set of matching points, and recursively divide the minimum bounding box into equal parts using the root node to generate an octree; calculate the implicit surface function value and store the implicit surface function value using the leaf nodes of the octree. The eight corner points of the leaf nodes of the octree are marked with 1 or 0 to determine the connection order of the corner points, generate triangular patches, and obtain a three-dimensional cultural relic model. Images of the internal structure of the cultural relic are collected, and after distortion correction, an internal ring image of the cultural relic is output. The image is then expanded and mapped along the axial line to obtain a panoramic view of the inside of the cultural relic. After reconstructing the internal model, the three-dimensional cultural relic model is fused to generate a fused three-dimensional cultural relic model.

2. The image recognition-based cultural relic information extraction system according to claim 1, characterized in that, The output enhanced artifact image includes: A new set of multi-source cultural relic image samples is collected and input into an image enhancement network model for training to obtain the final image enhancement network model. The processed multi-source cultural relic images are then input into the final image enhancement network model for image enhancement processing, and the enhanced cultural relic images are output.

3. The image recognition-based cultural relic information extraction system according to claim 2, characterized in that, The image distortion correction for enhanced cultural relic images includes: Obtain the correction coefficient, calculate the bending radius, apply the correction force according to the correction coefficient, obtain the corrected pixel coordinates, and then remap them to the pixel positions of the enhanced cultural relic image to generate the corrected cultural relic image.

4. The image recognition-based cultural relic information extraction system according to claim 3, characterized in that, The method of using point cloud fusion and reconstruction to screen matching points includes: The corrected depth image in the corrected cultural relic image is converted into point cloud data, and the point cloud data feature set is output. Define the neighborhood range, calculate the neighborhood features, and sum the neighborhood features within the neighborhood range to obtain the final features; Calculate the similarity threshold based on the final features, filter valid matching points in the point cloud data feature set, and generate an initial set of matching points; Record the three-dimensional direction of the matching point. Search for matching points in the three-dimensional direction of the initial matching point set to obtain the three-dimensional direction matching points. Build a K-dimensional binary tree and calculate the distance between matching points in the K-dimensional binary tree. Set a distance threshold. When the distance between matching points is less than the distance threshold, retain the corresponding two matching points to obtain the precise matching point set.

5. The image recognition-based cultural relic information extraction system according to claim 4, characterized in that, The outlier detection algorithm based on dynamic standard deviation removes outliers by including: Calculate the distance between matching points in the set of precisely matched points to obtain the average distance; Calculate the global mean and global standard deviation. Divide the global standard deviation by the global mean to obtain the standard deviation coefficient. When the absolute value of the difference between the distance of the matching point to the mean and the global mean is greater than the standard deviation coefficient multiplied by the global standard deviation, the matching point is identified as an outlier. Iterate through the set of exact matching points in turn and remove all outliers to obtain the final set of matching points.

6. The image recognition-based cultural relic information extraction system according to claim 5, characterized in that, The process of performing planar detailing of the model includes: Ultraviolet light was used to extract fluorescent ink marks from cultural relics and convert them into grayscale images. Weighted fusion was then used to output the fused image of the cultural relics. The fused image of the cultural relic is subjected to wavelet transform to decompose it into a background image and a detail image. After detail reconstruction, a detailed image of the surface of the cultural relic is obtained.

7. The image recognition-based cultural relic information extraction system according to claim 5, characterized in that, The texturing of the fused 3D cultural relic model includes: Set a standard color chart, calculate the color difference between the color model of the detailed artifact surface image and the standard color chart, obtain the model color difference, determine whether to perform linear compensation on the detailed artifact surface image, and perform planar projection or curved surface projection, mapping it onto the surface of the fused 3D artifact model to obtain the final 3D artifact model.

8. A method for extracting cultural relic information based on image recognition, characterized in that, Specifically, it includes: S1. Simultaneously acquire images of cultural relics from multiple sources and align them with spatial coordinates. Establish an image enhancement network model based on a dual-branch neural network, perform image enhancement processing on the cultural relics images, and output enhanced cultural relics images. S2. Perform image distortion correction on the enhanced cultural relic image, use point cloud fusion reconstruction method to screen matching points, and then remove outliers based on dynamic standard deviation outlier detection algorithm to extract the final set of matching points. S3. Reconstruct the model surface based on the final matching point set, obtain the internal structure image of the cultural relic to generate a panoramic view of the inside of the cultural relic, and perform internal and external model fusion to generate a fused three-dimensional cultural relic model. Then, perform model planar detailing to obtain a detailed image of the cultural relic surface. S4. Based on the color difference correction of the model, the detailed surface image of the cultural relic is corrected, and then the fused three-dimensional cultural relic model is textured to obtain the final three-dimensional cultural relic model; The image enhancement network model based on a dual-branch neural network includes: Collect images of cultural relics from multiple sources and align them with spatial coordinates to obtain processed images of cultural relics from multiple sources; The image enhancement network model is set to adopt a dual-branch neural network, which includes a first branch and a second branch. The first branch receives surface texture images and depth images, extracts multi-layer features using an encoder with a convolution kernel size of 3×3 and a stride of 2, and then introduces a deconvolution layer and residual connections to form a decoder to recover a high-resolution image and output a uniform cultural relic image. The second branch receives near-infrared imaging, uses stacked residual blocks with 2 convolutional layers per level to obtain the image noise distribution, uses the ReLU function as the activation function, and introduces a spatial attention mechanism to suppress noise in the highlight area, outputs the denoised near-infrared image, and constructs the image enhancement network model. The generated and fused 3D cultural relic model includes: Generate a minimum bounding box based on the final set of matching points, and recursively divide the minimum bounding box into equal parts using the root node to generate an octree; calculate the implicit surface function value and store the implicit surface function value using the leaf nodes of the octree. The eight corner points of the leaf nodes of the octree are marked with 1 or 0 to determine the connection order of the corner points, generate triangular patches, and obtain a three-dimensional cultural relic model. Images of the internal structure of the cultural relic are collected, and after distortion correction, an internal ring image of the cultural relic is output. The image is then expanded and mapped along the axial line to obtain a panoramic view of the inside of the cultural relic. After reconstructing the internal model, the three-dimensional cultural relic model is fused to generate a fused three-dimensional cultural relic model.

Images