A non-contact multimodal data acquisition device for artworks and its joint calibration method, multimodal data acquisition and fusion method

By using a non-contact multimodal data acquisition device for artworks and a joint calibration method, the problem of joint calibration of equipment in the digital preservation of cultural relics and artworks has been solved, achieving high-precision, non-destructive multimodal data acquisition and registration, and improving acquisition efficiency.

CN123111981APending Publication Date: 2026-11-17WUHAN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611253100.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-18
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing technologies make it difficult to achieve joint calibration of 3D scanning, multispectral imaging, and microscopic imaging equipment under non-contact conditions, resulting in poor registration accuracy, low efficiency, and a large amount of manual intervention in the digital preservation of cultural relics and artworks, which cannot meet the requirements of non-destructive testing.

Method used

A non-contact art multimodal data acquisition device is adopted. A texture camera is used as a coordinate transmission bridge to perform joint external parameter calibration of multiple cameras. A structured light 3D scanner, a multispectral camera and a microscope camera are rigidly fixed to an integrated bracket to form a common imaging area. Internal parameters are calibrated by a hybrid checkerboard calibration plate and a dot calibration plate to establish the spatial relationship between the devices.

Benefits of technology

It enables non-contact, non-destructive multimodal data acquisition and registration of cultural relics and artworks, improving registration accuracy and acquisition efficiency, reducing manual intervention, and meeting the requirements of cultural relic protection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123111981A_ABST
    Figure CN123111981A_ABST
Patent Text Reader

Abstract

This invention discloses a non-contact multimodal data acquisition device for artworks, comprising: a multispectral camera for acquiring multispectral images of the surface of the artwork being measured; a microscope camera for acquiring microscope images of a local area of ​​the surface of the artwork being measured; a structured light 3D scanner, including an infrared projection module for reconstructing a 3D point cloud and an infrared camera for acquiring texture images of the surface of the artwork being measured; the structured light 3D scanner has pre-calibrated intrinsic parameters of the texture camera and extrinsic parameters between the texture camera coordinate system and the 3D point cloud coordinate system; a multi-device integrated bracket, including a mounting plate and an arc-shaped adjustment hole, wherein the structured light 3D scanner, the multispectral camera, and the microscope camera are rigidly fixed to the mounting plate, so that the three maintain a fixed relative pose; the arc-shaped adjustment hole is used to adjust the optical axis tilt angle of the multispectral camera, so that at a preset working distance, the fields of view of the multispectral camera and the texture camera have a common imaging area; the field of view of the microscope camera is at least partially located within the common imaging area.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of digital preservation of cultural relics and artworks and photoelectric measurement technology, specifically to a non-contact multimodal data acquisition device for artworks and its joint calibration method, as well as a multimodal data acquisition and fusion method. Background Technology

[0002] Digital preservation of traditional artworks (paintings, murals, sculptures, and other artifacts) requires acquiring information in multiple modalities, including their three-dimensional morphology, material spectral characteristics, and surface microscopic details. Three-dimensional point clouds are used to record the artwork's geometric form; multispectral images are used to analyze pigment composition, reveal hidden layers and repair traces; and microscopic images are used to record microscopic features such as brushstrokes, cracks, and pigment particles. Due to the preciousness and fragility of cultural relics and artworks, the above data acquisition must meet the basic requirements of being non-contact and non-destructive.

[0003] Among existing publicly available technologies, one approach uses single-modal devices such as 3D scanners, multispectral cameras, and microscopic instruments to independently acquire data. Each modal data resides in its own sensor coordinate system, and subsequent registration relies on manual selection of corresponding points or registration based on image features. This approach suffers from poor registration accuracy and low success rate on artwork surfaces with weak or repetitive textures, and involves significant manual intervention and is inefficient. Another approach establishes coordinate relationships between devices by attaching markers to the surface of the object being measured. However, this marking process can contact with or even damage the surface of the artifact, failing to meet the non-destructive requirements for artifact preservation.

[0004] At the same time, existing multi-device integration and joint calibration technologies still have the following shortcomings when directly applied to the acquisition of multimodal data of artworks:

[0005] Existing multi-device integration solutions typically focus on the mechanical combination of different acquisition devices, lacking collaborative design to address the differences in working distance, field of view, spatial resolution, and imaging area between 3D scanning, multispectral imaging, and microscopic imaging. This makes it difficult to ensure that each device can stably cover the same measured area under non-contact conditions and form a common observation space suitable for joint calibration and collaborative acquisition.

[0006] The imaging scale, imaging mechanism and field of view of 3D scanners, multispectral cameras and microscopes differ significantly. Traditional calibration methods that use a unified calibration board or require complete observation calibration patterns are difficult to adapt to various types of equipment at the same time, and therefore it is difficult to stably establish spatial transformation relationships between different sensor coordinate systems.

[0007] Existing technologies lack a complete technical system that combines rigid integration of multiple devices, cross-scale parameter calibration, multi-coordinate system integration, error evaluation, and direct registration of multimodal data. The acquisition results still rely heavily on contact markers or manual registration in the later stages, making it difficult to simultaneously meet the requirements of digitalization of cultural relics and artworks for non-contact nature, registration accuracy, system stability, and acquisition efficiency. Summary of the Invention

[0008] To address the challenges of establishing a shared observation space among 3D scanning, multispectral imaging, and microscopic imaging devices, the difficulty in joint calibration of heterogeneous cameras across scales, and the challenge of stable registration of multimodal data in a unified coordinate system, this invention provides a non-contact multimodal data acquisition device for artworks, along with its joint calibration method and multimodal data acquisition and fusion method. This device uses a texture camera as a coordinate transfer bridge for joint external parameter calibration of multiple cameras, eliminating the need for contact markers on the artwork surface. It is suitable for non-contact, non-destructive multimodal data acquisition and registration of cultural relics and artworks.

[0009] According to one aspect of the present invention, a non-contact multimodal data acquisition device for artworks is provided, comprising: a multispectral camera for acquiring multispectral images of the surface of the artwork to be measured; a microscope camera for acquiring microscope images of a local area of ​​the surface of the artwork to be measured; a structured light 3D scanner including an infrared projection module for reconstructing a 3D point cloud and an infrared camera for acquiring texture images of the surface of the artwork to be measured; the structured light 3D scanner having pre-calibrated intrinsic parameters of the texture camera and extrinsic parameters between the texture camera coordinate system and the 3D point cloud coordinate system; and a multi-device integrated bracket including a mounting plate, wherein the structured light 3D scanner, the multispectral camera, and the microscope camera are rigidly fixed to the mounting plate, so that the three maintain a fixed relative pose during joint calibration and data acquisition; the mounting plate is provided with an arc-shaped adjustment hole for adjusting and locking the optical axis tilt angle of the multispectral camera, such that at a preset working distance, the field of view of the multispectral camera and the texture camera have a common imaging area; wherein the field of view of the microscope camera is at least partially located within the common imaging area.

[0010] Furthermore, the multi-device integrated bracket also includes a multispectral camera tilt mount and a microscope camera side mount; the mounting plate is vertically arranged, the structured light 3D scanner is fixed to the lower part of the mounting plate, and the microscope camera is fixed to the side of the mounting plate via the microscope camera side mount; the multispectral camera is fixed to the upper part of the mounting plate via the multispectral camera tilt mount, and the multispectral camera tilt mount cooperates with the arc-shaped adjustment hole to adjust and lock the optical axis tilt angle of the multispectral camera, so that the optical axis of the multispectral camera and the optical axis of the texture camera intersect at the preset working distance, thereby making the multispectral camera and the texture camera form a common imaging area at the preset working distance.

[0011] Furthermore, the multi-device integrated bracket is provided with a connecting flange, which is configured to connect to a robotic arm, gimbal, or support platform via a replaceable adapter to adjust the overall position of the non-contact artwork multimodal data acquisition device.

[0012] According to one aspect of the present invention, a joint calibration method for a non-contact multimodal data acquisition device for artworks is provided, comprising: maintaining a fixed relative pose between a structured light 3D scanner, a multispectral camera, and a microscope camera; acquiring the intrinsic parameters of a texture camera, a multispectral camera, and a microscope camera; and acquiring the extrinsic parameters between the texture camera coordinate system and the 3D point cloud coordinate system; placing a hybrid checkerboard calibration board within the common observation area of ​​the texture camera, the multispectral camera, and the microscope camera; acquiring images of the calibration board by the texture camera, the multispectral camera, and the microscope camera respectively at each fixed pose of the hybrid checkerboard calibration board; changing the pose of the hybrid checkerboard calibration board and repeating the acquisition to obtain multiple sets of images in the same pose; and according to each set... Using the corner points of the mixed checkerboard in the pose image and the calibrated intrinsic parameters of each camera, the pose of the mixed checkerboard calibration board relative to the texture camera, the multispectral camera, and the microscope camera is calculated respectively. Based on the pose of the mixed checkerboard calibration board relative to each camera, and using the texture camera as a coordinate transfer bridge, the extrinsic parameters of the multispectral camera coordinate system and the microscope camera coordinate system relative to the texture camera coordinate system are determined. Based on the extrinsic parameters of the multispectral camera coordinate system and the microscope camera coordinate system relative to the texture camera coordinate system, and the extrinsic parameters between the texture camera coordinate system and the 3D point cloud coordinate system, the extrinsic parameters of the multispectral camera coordinate system and the microscope camera coordinate system relative to the 3D point cloud coordinate system are determined, and the joint calibration result is output.

[0013] Furthermore, the acquisition of the intrinsic parameters of the microscope camera includes: selecting a dot calibration plate with a diameter and center-to-center spacing of dots that are compatible with the field of view, magnification, and depth of field of the microscope camera; acquiring multiple dot calibration plate images in different poses at different distances and tilt angles within the depth of field of the microscope camera; extracting the sub-pixel coordinates of the center of each dot in each dot calibration plate image, and establishing a correspondence between the dots in the dot calibration plate image and the dots in the dot calibration plate based on the sub-pixel coordinates of the center of each dot; and calculating the intrinsic parameters and distortion parameters of the microscope camera based on the camera imaging model and distortion model constructed from the correspondence between the dots in the dot calibration plate image and the dots in the dot calibration plate, with reprojection error as the optimization objective.

[0014] Furthermore, the acquisition of the intrinsic parameters of the multispectral camera includes: selecting a hybrid checkerboard calibration board whose grid side length and encoding size are adapted to the field of view of the multispectral camera; selecting a reference band for detecting the corner points of the hybrid checkerboard calibration board, and acquiring hybrid checkerboard calibration board images in multiple poses at different distances and tilt angles under the reference band; detecting the sub-pixel coordinates of each corner point in each hybrid checkerboard calibration board image, and establishing a correspondence between the corner points in the hybrid checkerboard calibration board image and the corner points in the hybrid checkerboard calibration board based on the sub-pixel coordinates of each corner point; and calculating the intrinsic parameters and distortion parameters of the multispectral camera based on the camera imaging model and distortion model constructed from the correspondence between the corner points in the hybrid checkerboard calibration board image and the corner points in the hybrid checkerboard calibration board, with reprojection error as the optimization objective.

[0015] Further, based on the pose of the hybrid checkerboard calibration board relative to each camera, and using the texture camera as a coordinate transfer bridge, the extrinsic parameters of the multispectral camera coordinate system and the microscopic camera coordinate system relative to the texture camera coordinate system are determined. This includes: for each set of in-pose images, based on the pose of the hybrid checkerboard calibration board relative to the texture camera and the pose of the hybrid checkerboard calibration board relative to the multispectral camera or the microscopic camera, the extrinsic parameters of the multispectral camera coordinate system and the microscopic camera coordinate system relative to the texture camera coordinate system are obtained respectively; the extrinsic parameters corresponding to multiple sets of in-pose images are fused, and the corresponding extrinsic parameters are nonlinearly optimized with the hybrid checkerboard corner reprojection error in the multispectral camera and the microscopic camera as the optimization target.

[0016] Furthermore, the microscope camera acquires a local area of ​​the hybrid checkerboard calibration plate, the local area including codes and corner points used to determine global corner point numbers and meet pose calculation requirements.

[0017] Furthermore, the method also includes: calculating the total reprojection error of the intrinsic and extrinsic parameter calibrations of each camera and the reprojection error of each pose image group; comparing the total reprojection error and the reprojection error of each image group with a preset accuracy threshold; when the total reprojection error exceeds the preset accuracy threshold, identifying and removing gross error image groups based on the reprojection error of each pose image group, and recalculating and optimizing the corresponding calibration parameters.

[0018] According to one aspect of the present invention, a method for multimodal data acquisition and fusion of a non-contact multimodal data acquisition device for artworks is provided, comprising: performing joint calibration using the joint calibration method of the non-contact multimodal data acquisition device for artworks; acquiring a three-dimensional point cloud of the artwork under test using a structured light three-dimensional scanner, acquiring a multispectral image of the artwork under test using a multispectral camera, and acquiring a microscopic image of a local area of ​​the artwork under test using a microscope camera; acquiring the extrinsic parameters of the multispectral camera coordinate system and the microscope camera coordinate system relative to the three-dimensional point cloud coordinate system; transforming the three-dimensional points in the three-dimensional point cloud to the multispectral camera coordinate system and the microscope camera coordinate system respectively according to the extrinsic parameters, and projecting the transformed three-dimensional points onto the corresponding multispectral image and the microscope image respectively according to the intrinsic parameters of the multispectral camera and the microscope camera; assigning multispectral information to the corresponding visible points in the three-dimensional point cloud according to the projection results, and assigning microscope image information to the corresponding local points covered by the field of view of the microscope camera, thereby generating a multimodal digital model of the artwork in the three-dimensional point cloud coordinate system.

[0019] The above technical solution discloses a non-contact multimodal data acquisition device for artworks and its joint calibration method, as well as a multimodal data acquisition and fusion method. The device includes a structured light 3D scanner, a multispectral camera, and a microscope camera rigidly fixed to a multi-device integrated bracket. Each camera forms a common imaging area at a common working distance. The method uses a dot calibration plate and a ChArUco calibration plate adapted to the camera's field of view scale to complete intrinsic parameter calibration. Through observation with multiple sets of in-position calibration plates, and using a texture camera as a coordinate transfer bridge, the extrinsic parameters of the multispectral camera and microscope camera relative to the 3D point cloud coordinate system are calculated and optimized. The calibration results are evaluated based on reprojection error, and the multispectral and microscope images are mapped to the 3D point cloud to generate a multimodal digital model of the artwork in a unified coordinate system. This solution eliminates the need for contact markers on the artwork surface and is suitable for non-contact, non-destructive multimodal data acquisition and registration of cultural relics and artworks.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0021] 1) This invention rigidly integrates three-dimensional scanning, multispectral imaging and microscopic imaging equipment into one unit, and establishes the spatial relationship between the various devices through joint calibration. It eliminates the need to set contact markers on the surface of the artwork, reduces the reliance on manual registration in the later stage, and meets the requirements of non-contact and non-destructive digital acquisition of cultural relics and artworks.

[0022] 2) This invention improves the consistency of joint calibration observation data and the efficiency of multimodal data acquisition by matching the working distance, optical axis direction and field of view of different imaging devices, so that each device forms a common observation space at a common working distance;

[0023] 3) This invention adopts a calibration method adapted to different camera field-of-view scales and enables small-field-of-view cameras to participate in joint calibration through local observation, thereby improving the applicability of joint calibration of heterogeneous cameras across scales;

[0024] 4) By establishing coordinate transfer relationships between different devices and combining the fusion of multiple sets of observation results, parameter optimization, and error evaluation, this invention unifies three-dimensional point clouds, multispectral images, and microscopic images into the same coordinate system, thereby improving the accuracy and reliability of multimodal data registration. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A flowchart of the joint calibration method provided in an embodiment of the present invention.

[0027] Figure 2 This is a first schematic diagram of the structure of the multimodal data acquisition device provided in an embodiment of the present invention.

[0028] Figure 3 This is a second schematic diagram of the structure of the multimodal data acquisition device provided in an embodiment of the present invention.

[0029] Figure 4 This is a schematic diagram of the coordinate system transfer relationship in the multi-camera joint extrinsic parameter calibration provided in an embodiment of the present invention, using a texture camera as a bridge.

[0030] Figure 2The components include: 1. Structured light 3D scanner; 1.1 Infrared camera; 1.2 Texture camera; 1.3 Infrared projection module; 2. Multispectral camera; 3. Microscope camera; 4. Multi-device integrated bracket; 5. Connecting flange; 6. Multispectral camera tilt mount; 7. Microscope camera side mount; 8. ChArUco calibration plate; 9. Dot calibration plate; 10. Mounting plate; 10.1 Arc-shaped adjustment hole; 10.2 Circular positioning hole. Detailed Implementation

[0031] It should be noted that:

[0032] The terms “comprising” and “having”, and any variations thereof, in the specification, claims, and accompanying drawings of this invention are intended to cover a non-exclusive inclusion, such as a process, method, system, product, or apparatus that includes a series of steps or units, not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices. The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be decomposed, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined to form new technical solutions. Such combinations are not bound by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0035] Please refer to Figure 2This invention proposes a non-contact art multimodal data acquisition device, including a structured light 3D scanner 1, a multispectral camera 2, a microscope camera 3, and a multi-device integrated bracket 4.

[0036] The multi-device integrated bracket 4 includes a connecting flange 5, a multispectral camera tilt mount 6, a microscope camera side mount 7, and a vertically mounted plate 10. The structured light 3D scanner 1 is fixed to the lower part of the mounting plate. The multispectral camera 2 is fixed to the upper part of the mounting plate via the multispectral camera tilt mount 6. The multispectral camera tilt mount 6, in conjunction with the arc-shaped adjustment hole 10.1 on the mounting plate 10, allows for fine-tuning of the optical axis tilt angle of the multispectral camera 2 during assembly, and is locked in place after adjustment. The microscope camera 3 is fixed to the side of the mounting plate via the microscope camera side mount 7. The connecting flange 5 is located at the top of the mounting plate and is configured to connect to different types of robotic arms, pan-tilt units, or other support platforms via replaceable adapters, facilitating the overall pose adjustment and automated acquisition expansion of the non-contact art multimodal data acquisition device. Furthermore, the structured light 3D scanner 1, multispectral camera 2, and microscope camera 3 are rigidly fixed to the multi-device integrated bracket to form a rigid whole, ensuring that their relative poses remain constant during joint calibration and data acquisition.

[0037] The structured light 3D scanner 1 integrates an infrared projection module 1.3, an infrared camera 1.1, and a texture camera 1.2. The infrared projection module 1.3 projects infrared structured light stripes onto the surface of the artwork being measured. The infrared camera 1.1, a grayscale camera, is used to acquire the infrared structured light stripes reflected from the surface to obtain a structured light image, and then reconstructs a 3D point cloud based on the structured light image. In this embodiment, the coordinate system of the infrared camera 1.1 is used as the 3D reconstruction coordinate system of the structured light 3D scanner 1, and the 3D point cloud output by the structured light 3D scanner 1 lies within this structured light 3D scanner coordinate system.

[0038] Please refer to the appendix for details. Figure 3 and Figure 4The mounting plate 10 has a circular positioning hole 10.2 and two arc-shaped adjustment holes 10.1. The two arc-shaped adjustment holes 10.1 are arranged symmetrically about the circular positioning hole 10.2. The arc-shaped bending directions of the two arc-shaped adjustment holes 10.1 are opposite and both face the circular positioning hole 10.2. The multispectral camera tilt mounting base 6 has a positioning component and two adjustment components (screws in this embodiment). The positioning component is embedded in the circular positioning hole 10.2, and the two adjustment components are embedded in the two arc-shaped adjustment holes 10.1 respectively. The multispectral camera tilt mount 6 is used in conjunction with the arc-shaped adjustment hole 10.1 as follows: with the circular positioning hole 10.2 as the center, the multispectral camera 2 is rotated, which can drive the positioning component to rotate in the circular positioning hole 10.2. At the same time, it can drive the two adjustment components to slide in opposite directions along the extension trajectory of the two arc-shaped adjustment holes 10.1 until the optical axis of the multispectral camera 2 and the optical axis of the texture camera 1.2 intersect at the preset working distance. Fixing the two adjustment components can fix the optical axis tilt angle of the multispectral camera 2.

[0039] It should be noted that, considering the imaging mode and output data format of the infrared camera 1.1 used in this embodiment, the infrared camera 1.1 is not used as a common observation camera for joint calibration with external cameras. Instead, the texture camera 1.2, whose intrinsic parameters and extrinsic parameters with the infrared camera 1.1 are provided by the factory calibration, is selected as the coordinate transfer bridge. The texture camera 1.2 is a visible light color camera used to acquire texture images of the surface of the artwork being measured. The multispectral camera 2 is used to acquire multispectral images (i.e., images of each band) of several discrete bands (e.g., multiple channels covering the visible to near-infrared 400-1000nm range) of the surface of the artwork being measured. The images of each band output by the multispectral camera 2 have undergone inter-channel geometric registration through internal calibration or pre-calibration, and the pixel correspondence between each band image and the reference band image is known. The microscopic camera 3 is used to acquire high-resolution microscopic images of local areas of the surface of the artwork being measured, and its field of view is significantly smaller than that of the texture camera 1.2 and the multispectral camera 2.

[0040] It should also be noted that, to ensure that each camera can clearly observe the calibration plate and the same measured area under the same fixed pose, this invention features a matching design for the working distance and field of view of each camera: at a preset working distance, the fields of view of texture camera 1.2 and multispectral camera 2 share a common imaging area, and the field of view of microscope camera 3 is at least partially located within this common imaging area. The tilt mount of multispectral camera 2, in conjunction with an arc-shaped adjustment hole, is used to adjust and lock the optical axis tilt angle of multispectral camera 2, so that the optical axis of multispectral camera 2 intersects with the optical axis of texture camera 1.2 at the preset working distance, thereby forming a common imaging area between multispectral camera 2 and texture camera 1.2 at the preset working distance. The working distance of microscope camera 3 is adapted to the working distances of texture camera 1.2 and multispectral camera 2, enabling texture camera 1.2, multispectral camera 2, and microscope camera 3 to acquire calibration plate images respectively under the same fixed pose of the calibration plate.

[0041] In this embodiment, the working distance and field of view of each camera are matched as follows: the working distances of all three cameras are adjusted to a preset working distance, which is described below as an example of a preset working distance of 300mm. At this preset working distance, the imaging area of ​​multispectral camera 2 is approximately 225mm × 180mm, and the imaging area of ​​texture camera 1.2 is approximately 200mm × 100mm. Both have a common imaging area that meets the requirements for joint calibration. Multispectral camera 2 is tilted via multispectral camera tilt mount 6, and its optical axis intersects with the optical axis of texture camera 1.2 at this preset working distance, causing the imaging areas of the two cameras to essentially overlap. The overlapping area of ​​the two camera imaging areas is the common imaging area. The imaging area of ​​microscope camera 3 at this preset working distance is approximately 80mm × 64mm, significantly smaller than the former two, but its field of view falls within this common imaging area. Therefore, when the calibration board or the workpiece under test is placed at a preset working distance of approximately 300mm, all three cameras can clearly image the data, providing hardware support for multi-camera joint extrinsic parameter calibration and multimodal data acquisition.

[0042] Please see Figure 1 This invention also proposes a joint calibration method for a non-contact art multimodal data acquisition device, which specifically includes the following steps:

[0043] Step S1: Keep the relative poses of the structured light 3D scanner, multispectral camera and microscope camera fixed, obtain the intrinsic parameters of the texture camera, multispectral camera and microscope camera, and obtain the extrinsic parameters between the texture camera coordinate system and the 3D point cloud coordinate system.

[0044] Step S1 aims to assemble the device and acquire known parameters. First, the structured light 3D scanner, multispectral camera, and microscope camera are fixed to a multi-device integrated bracket. Then, the optical axis tilt angle of the multispectral camera is finely adjusted using the multispectral camera tilt mount and arc-shaped adjustment hole, ensuring that the field of view of the multispectral camera and the field of view of the texture camera are substantially coincident at a preset working distance. It is also confirmed that the field of view of the microscope camera is within the common imaging area. After adjustment, the brackets are locked to form a rigid assembly. The intrinsic parameters of the texture camera are then read from the device parameters of the structured light 3D scanner. Distortion parameters And the extrinsic parameters from the texture camera coordinate system to the 3D point cloud coordinate system These parameters are used as known quantities in subsequent calibrations.

[0045] In this invention, the 3D point cloud coordinate system is equivalent to the structured light 3D scanner coordinate system. Since the structured light 3D scanner uses the infrared camera coordinate system as its reference, the 3D point cloud coordinate system is also equivalent to the infrared camera coordinate system. The external parameters from the texture camera coordinate system to the 3D point cloud coordinate system are also considered. This is equivalent to a known coordinate transformation relationship between the texture camera coordinate system and the 3D point cloud coordinate system. Furthermore, the steps for obtaining the intrinsic parameters of multispectral cameras and microscope cameras are as follows:

[0046] Step S11, intrinsic parameter calibration of the microscope camera. This step uses a dot calibration plate 9 adapted to the small field of view of the microscope camera to complete the intrinsic parameter calibration of the microscope camera. Further, step S11 includes the following sub-steps:

[0047] Step S111: Based on the field of view, magnification, and depth of field of the microscope camera, select a dot calibration plate with a dot diameter and dot center-to-center spacing that are compatible with it. In this embodiment, a high-precision dot array calibration plate with an aluminum substrate is selected. The dot diameter is 3.5 mm, the dot center-to-center spacing is 7 mm, the overall plate size is approximately 70 mm × 70 mm, and the dot array area is approximately 56 mm × 56 mm, which is compatible with the imaging area of ​​the microscope camera, which is approximately 80 mm × 64 mm, so that a single frame image of the microscope camera can contain dozens of complete dots.

[0048] Step S112: Within the depth of field of the microscope camera, images of the dot calibration plate in multiple poses are acquired at different distances and tilt angles. In this embodiment, microscope images in 15-25 poses are acquired.

[0049] Step S113: Extract the sub-pixel coordinates of the center of each dot in the dot calibration plate image, and establish the correspondence between the dots in the dot calibration plate image and the dots in the dot calibration plate based on the sub-pixel coordinates of the center of each dot. In this embodiment, the gray-scale centroid method or ellipse fitting method is used to extract the sub-pixel coordinates of the center of each dot, and the correspondence between the dots in the dot calibration plate image and the dots in the dot calibration plate is established based on the sub-pixel coordinates of the center of each dot and the arrangement rules of the dot array.

[0050] Step S114: Based on the camera imaging model and distortion model constructed from the correspondence between the dots in the dot calibration plate image and the dots in the dot calibration plate, the intrinsic parameters and distortion parameters of the microscope camera are calculated with reprojection error as the optimization target.

[0051] In this embodiment, the initial values ​​are obtained using the Zhang Zhengyou calibration method, and then nonlinear optimization is performed using the Levenberg-Marquardt algorithm to output the intrinsic parameters of the microscope camera. and distortion parameters .

[0052] This embodiment calculates the intrinsic parameters of the microscope camera based on a pinhole imaging model and a distortion model. The pinhole imaging model is as follows:

[0053]

[0054] In formulas (1) and (2), To determine the coordinates of a three-dimensional point in the calibration plate coordinate system, These are the pixel coordinates of a 3D point on the image. As a scale factor, and These are the rotation matrix and translation vector from the calibration board coordinate system to the camera coordinate system, respectively. The camera intrinsic parameter matrix, , For equivalent focal length, The coordinates of the main point.

[0055] In this embodiment, the distortion model adopts a combination of radial and tangential distortion:

[0056]

[0057] In formula (3), To normalize the ideal coordinates on the image plane, The coordinates are after distortion. The distance from the image point to the center of the image. , , , For radial distortion parameters, , Let be the tangential distortion parameters, and let the distortion parameter vector be denoted as . The intrinsic parameter solution aims to minimize the reprojection error.

[0058]

[0059] In formula (4), The camera intrinsic parameter matrix, For distortion parameters, For the first Rotation matrix for each pose For the first Translation vector at each pose The square of the L2 norm, To calibrate the number of board poses, For the first The number of circles or corners involved in the solution at each pose. For the first The first position under the first position The measured pixel coordinates of the center or corner of each dot. For the first The three-dimensional coordinates of a point or corner point in the calibration plate coordinate system. This is a projection function that reprojects the 3D points of the calibration board onto the image plane based on the camera's intrinsic parameters, distortion parameters, and the calibration board's pose.

[0060] Step S12, Multispectral Camera Intrinsic Parameter Calibration. This step utilizes a hybrid checkerboard calibration board 8 (i.e., the ChArUco calibration board) to calibrate the intrinsic parameters of the multispectral camera. The ChArUco calibration board combines a checkerboard pattern with ArUco two-dimensional coding, with each corner point having a unique number. Even when the calibration board is partially occluded or only partially falls within the field of view, corner point correspondences can still be established. Further, step S12 includes the following sub-steps:

[0061] Step S121: Select a ChArUco calibration board whose grid side length and ArUco coding size are compatible with the field of view of the multispectral camera. In this embodiment, a ChArUco calibration board with 12×9 grids, a grid side length of 15mm, and an ArUco coding side length of 11.25mm is selected. The overall board size is approximately 200mm×150mm, and the pattern area is approximately 180mm×135mm, which is compatible with the imaging area of ​​the multispectral camera at a working distance of approximately 300mm, which is approximately 225mm×180mm.

[0062] Step S122: Select a reference band for detecting the corner points of the ChArUco calibration board, and acquire ChArUco calibration board images in multiple poses at different distances and tilt angles under the reference band. In this embodiment, the band with the best signal-to-noise ratio and contrast in the multispectral camera is selected as the reference band, and ChArUco calibration board images in 15-25 poses are acquired at different distances and tilt angles.

[0063] Step S123: Detect the sub-pixel coordinates of each corner point in each hybrid checkerboard calibration board image, and establish the correspondence between the corner points in the hybrid checkerboard calibration board image and the corner points in the hybrid checkerboard calibration board based on the sub-pixel coordinates of each corner point. In this embodiment, for each frame of the ChArUco calibration board image, first detect the ArUco encoding to obtain the unique ChArUco corner point number, then perform sub-pixel refinement on the ChArUco corner points to obtain the sub-pixel coordinates of the corner points with unique numbers, and establish the correspondence between the sub-pixel coordinates of the corner points and the corner points in the ChArUco calibration board.

[0064] Step S124: Based on the camera imaging model and distortion model constructed from the correspondence between corner points in the mixed checkerboard calibration board image and corner points in the mixed checkerboard calibration board, the intrinsic parameters and distortion parameters of the multispectral camera are calculated with reprojection error as the optimization objective. In this embodiment, the intrinsic parameters of the multispectral camera are calculated according to the model and objective function described in formulas (1) to (4). and distortion parameters .

[0065] Step S2: Place the ChArUco calibration plate in the common observation area of ​​the texture camera, the multispectral camera, and the microscope camera. Under each fixed pose of the ChArUco calibration plate, the texture camera, the multispectral camera, and the microscope camera respectively acquire images of the calibration plate. Change the pose of the ChArUco calibration plate and repeat the acquisition to obtain multiple sets of images in the same pose.

[0066] In step S2, the common observation area refers to the overlapping imaging area among the texture camera, multispectral camera, and microscope camera. In this step, the ChArUco calibration plate is placed at the common working distance (approximately 300 mm) of the texture camera, multispectral camera, and microscope camera. Because the working distances of the three cameras are matched and their fields of view converge in the same imaging area, the three cameras can clearly image the calibration plate under the same fixed pose. For the microscope camera, whose field of view is significantly smaller than the other two cameras, its local field of view contains a fully identifiable ArUco code, and at least four ChArUco corner points with identifiable numbers, non-collinearity, and reasonable distribution are detected to participate in pose calculation. When the texture camera only observes a local area of ​​the calibration plate, the same method can be used to establish corner point correspondences and calculate the pose. Specifically, the microscope camera acquires a local area of ​​the ChArUco calibration plate, which includes ArUco markers and ChArUco corner points used to determine global corner point numbers and meet the pose calculation requirements.

[0067] Furthermore, while keeping the calibration plate pose fixed, images of the calibration plate in that pose are acquired by a texture camera, a multispectral camera, and a microscope camera, respectively, forming a set of images in the same pose; the calibration plate pose is then changed, and the acquisition is repeated. Groups (e.g.) Take 10-20 groups), and the poses of each group should be different in distance and tilt angle to ensure the good state of the solution.

[0068] Step S3: Based on the ChArUco corner points in each group of pose images and the calibrated intrinsic parameters of each camera, calculate the pose of the ChArUco calibration plate relative to the texture camera, the multispectral camera, and the microscope camera.

[0069] In step S3, for the first Using the same pose images, and leveraging the calibrated intrinsic parameters of each camera and the detected ChArUco corners, the pose transformation matrix from the calibration board coordinate system to each camera coordinate system is calculated using the PnP method. , ,in, Indicates a texture camera. Indicates camera, , , These represent a multispectral camera and a microscopic camera, respectively.

[0070] Step S4: Based on the pose of the ChArUco calibration plate relative to each camera, and using the texture camera as a coordinate transfer bridge, determine the extrinsic parameters of the multispectral camera coordinate system and the microscopic camera coordinate system relative to the texture camera coordinate system.

[0071] In step S4, for each set of pose images, based on the pose of the hybrid checkerboard calibration plate relative to the texture camera and the pose of the hybrid checkerboard calibration plate relative to the multispectral camera or the microscope camera, the extrinsic parameters of the multispectral camera coordinate system and the microscope camera coordinate system relative to the texture camera coordinate system are obtained respectively; the extrinsic parameters corresponding to multiple sets of pose images are fused, and the corresponding extrinsic parameters are nonlinearly optimized with the hybrid checkerboard corner reprojection error in the multispectral camera and the microscope camera as the optimization target.

[0072] In this embodiment, a texture camera is used as a bridge, and the camera is solved according to the following formula. ( Extrinsic parameters from the coordinate system to the texture camera coordinate system:

[0073]

[0074] In formula (5), , The first The pose transformation matrices from the calibration board to the texture camera and from the calibration board to camera C in the group ( (homogeneous transformation matrix) For the first The external parameters calculated from the group of observations.

[0075] Furthermore, on The results of the out-of-group parameters are fused: the first group of parameters is fused. The out-of-group parameters are expressed as and the rotation matrix Convert to unit quaternion .by To standardize the notation of quaternions, the quaternion with standardized notation is obtained using the following formula. :

[0076]

[0077] Calculate the initial values ​​of the extrinsic parameters by performing a normalized average on the quaternions and an arithmetic average on the translation vectors. ;

[0078]

[0079]

[0080] in, The quaternion after normalization and averaging. The translation vector is the result of the arithmetic mean, from which the initial values ​​of the extrinsic parameters are constructed. :

[0081]

[0082] For cameras ( ),by As initial values ​​for iteration, the optimized extrinsic parameters are obtained by minimizing the reprojection error of the corner points involved in the solution across all image groups. :

[0083]

[0084] In formula (10), The number of image groups in the same pose. For the first Cameras in the group The number of corner points involved in the solution. For the first Cameras in the group The observed first Measured pixel coordinates of each corner point , For camera The calibrated intrinsic parameters and distortion parameters, For the first The three-dimensional coordinates of each corner point in the calibration plate coordinate system For the first Pose transformation from the calibration board coordinate system to the camera C coordinate system. This is a projection function that reprojects the 3D points of the calibration board onto the image plane based on the camera's intrinsic parameters, distortion parameters, and the calibration board's pose.

[0085] Step S5: Based on the extrinsic parameters of the multispectral camera coordinate system and the microscope camera coordinate system relative to the texture camera coordinate system, as well as the extrinsic parameters between the texture camera coordinate system and the three-dimensional point cloud coordinate system, determine the extrinsic parameters of the multispectral camera coordinate system and the microscope camera coordinate system relative to the three-dimensional point cloud coordinate system, and output the joint calibration result.

[0086] Step S5 aims to perform joint extrinsic parameter calibration using multiple cameras. Specifically, using the texture camera as a coordinate transfer bridge, the extrinsic parameters from the multispectral camera, the microscope camera, and the 3D point cloud coordinate system are calculated by jointly observing the ChArUco calibration board in the same pose using three cameras (see coordinate system transfer relationship). Figure 4 ).

[0087] In this embodiment, based on the extrinsic parameters of the multispectral camera coordinate system and the microscope camera coordinate system relative to the texture camera coordinate system... (That is, the optimized extrinsic parameters in step S4) ), and the known extrinsic parameters between the texture camera coordinate system and the 3D point cloud coordinate system. The following formula is used to convert the multispectral camera and the microscope camera to a unified three-dimensional point cloud coordinate system:

[0088]

[0089] In formula (11), For camera The extrinsic parameters from the coordinate system to the 3D point cloud coordinate system, including the extrinsic parameters from the multispectral camera coordinate system to the 3D point cloud coordinate system. The external parameters from the camera microscope coordinate system to the 3D point cloud coordinate system This establishes a spatial mapping relationship between the multispectral camera, the microscope camera, and the three-dimensional point cloud coordinate system.

[0090] In some feasible embodiments, the method further includes:

[0091] Step S6, Calibration Accuracy Evaluation and Iterative Optimization. This step is used to quantitatively evaluate and remove gross errors from the joint calibration results output in step S5. Further, step S6 includes the following sub-steps:

[0092] Step S61: Calculate the total weight projection error of the intrinsic and extrinsic parameter calibrations of each camera, as well as the reprojection error of each image group.

[0093] In this embodiment, the root mean square error of total weight projection for each camera's intrinsic parameter calibration and extrinsic parameter calibration, as well as the root mean square error of reprojection for each pose-in-place image group, are calculated using the following formulas:

[0094]

[0095] In formula (12), when calculating the overall reprojection error, This refers to the total number of corner or dot observations included in the statistics; when calculating the reprojection error of a single in-pose image group, This refers to the number of corner points or dots included in the statistics within the same pose image group; For the first One measured pixel coordinate, To obtain the first three-dimensional point by reprojecting it onto the image plane using calibration parameters. Predicted pixel coordinates.

[0096] Step S62: Compare the overall reprojection error and the reprojection error of each pose image group with the corresponding preset accuracy threshold. When the overall reprojection error exceeds the preset accuracy threshold, identify and remove the gross error image group based on the reprojection error of each pose image group, and recalculate and optimize the corresponding calibration parameters.

[0097] In this embodiment, the preset accuracy threshold can be set according to the camera resolution and multimodal data registration accuracy requirements. For example, the root mean square error of the overall reprojection error for intrinsic parameter calibration is no greater than 0.2 pixels, and the root mean square error of the overall reprojection error for extrinsic parameter calibration is no greater than 1.0 pixel. If the overall reprojection error exceeds the corresponding preset accuracy threshold, then the gross error image groups are identified and eliminated based on the reprojection error of each pose image group. If necessary, the calibration board image of the corresponding pose is re-acquired, and the corresponding calibration steps are returned to recalculate and optimize until the preset accuracy requirements are met. The preset accuracy requirements are not limited here.

[0098] Step S63: Output the intrinsic parameters and distortion parameters of each camera, the extrinsic parameters of each camera to the 3D point cloud coordinate system, and the corresponding reprojection error statistics to complete the joint calibration of the device. The joint calibration of the device is now complete.

[0099] Based on the same technical concept as the foregoing embodiments, this invention also proposes a multimodal data acquisition and fusion method for a non-contact multimodal data acquisition device for artworks. This method aims to perform non-contact multimodal data acquisition of a local area of ​​an artwork under the condition of a fixed device pose, utilizing a device that has completed joint calibration. Specifically, it includes the following steps:

[0100] Step S71: Obtain the three-dimensional point cloud of the artwork under test using a structured light 3D scanner, obtain the multispectral image of the artwork under test using a multispectral camera, and obtain the microscopic image of a local area of ​​the artwork under test using a microscope camera.

[0101] Step S72: Obtain the extrinsic parameters of the multispectral camera coordinate system and the microscope camera coordinate system relative to the three-dimensional point cloud coordinate system.

[0102] Step S73: Based on the extrinsic parameters, the three-dimensional points in the three-dimensional point cloud are transformed to the coordinate system of the multispectral camera and the coordinate system of the microscopic camera, respectively. Based on the intrinsic parameters of the multispectral camera and the microscopic camera, the transformed three-dimensional points are projected onto the corresponding multispectral image and microscopic image, respectively.

[0103] Step S74: Based on the projection results, multispectral information is assigned to the corresponding visible points in the three-dimensional point cloud, and microscopic image information is assigned to the corresponding local points covered by the field of view of the microscope camera, thereby generating a multimodal digital model of the artwork in the three-dimensional point cloud coordinate system.

[0104] In this embodiment, for three-dimensional points in a three-dimensional point cloud coordinate system According to the camera output in step S6 ( )External parameters from the coordinate system to the 3D point cloud coordinate system First, transform the 3D points to the camera. The coordinate system is then projected onto the corresponding image plane:

[0105]

[0106] In formula (13), These are the homogeneous coordinates of a 3D point in a 3D point cloud coordinate system. For this point at the camera Homogeneous coordinates in a coordinate system , Cameras Intrinsic parameters and distortion parameters, For this 3D point on the camera Predicted pixel coordinates in the image. After determining the positive depth, image boundary, and occlusion relationship of the projected points, multispectral information or microscopic image information is read from the corresponding pixel positions; other band images from the multispectral camera are first registered to the reference band according to the known inter-channel geometric registration relationship. Thus, the multispectral and microscopic image information are assigned to the corresponding points in the 3D point cloud, generating a unified registered multimodal digital model of the artwork in the 3D point cloud coordinate system, providing data support for applications such as cultural relic damage analysis, pigment analysis, and the establishment of restoration archives.

[0107] In summary, this invention relates to the field of digital preservation technology for cultural relics and artworks, and discloses a non-contact multimodal data acquisition device for artworks and its joint calibration method, as well as a multimodal data acquisition and fusion method. The device includes a structured light 3D scanner, a multispectral camera, and a microscope camera rigidly fixed to a multi-device integrated bracket. Each camera forms a common imaging area at a common working distance. The method uses a dot calibration plate and a ChArUco calibration plate adapted to the camera's field of view to complete the intrinsic parameter calibration; through observation using multiple sets of in-position calibration plates, and with a texture camera as a coordinate transfer bridge, the extrinsic parameters of the multispectral camera and microscope camera relative to the 3D point cloud coordinate system are calculated and optimized; the calibration results are evaluated based on reprojection error, and the multispectral image and microscope image are mapped to the 3D point cloud to generate a multimodal digital model of the artwork in a unified coordinate system. This solution eliminates the need for contact markers on the artwork surface and is suitable for non-contact, non-destructive multimodal data acquisition and registration of cultural relics and artworks.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A non-contact art multi-modal data acquisition device, characterized in that, include: A multispectral camera is used to acquire multispectral images of the surface of the artwork being measured. A microscope camera is used to acquire microscopic images of local areas on the surface of an artwork being tested. A structured light 3D scanner includes an infrared projection module and an infrared camera for reconstructing 3D point clouds, and for acquiring texture images of the surface of the artwork being measured; the structured light 3D scanner has pre-calibrated intrinsic parameters of the texture camera, extrinsic parameters between the texture camera coordinate system and the 3D point cloud coordinate system; and A multi-device integrated bracket includes a mounting plate, on which the structured light 3D scanner, multispectral camera, and microscope camera are rigidly fixed, maintaining their relative pose during joint calibration and data acquisition. The mounting plate is provided with an arc-shaped adjustment hole for adjusting and locking the optical axis tilt angle of the multispectral camera, so that at a preset working distance, the field of view of the multispectral camera and the texture camera share a common imaging area. The field of view of the microscope camera is at least partially located within the common imaging area.

2. A non-contact art multi-modal data acquisition device as claimed in claim 1, wherein, The multi-device integrated bracket also includes a multispectral camera tilt mount and a microscope camera side mount; the mounting plate is vertically arranged, the structured light 3D scanner is fixed to the lower part of the mounting plate, and the microscope camera is fixed to the side of the mounting plate via the microscope camera side mount; the multispectral camera is fixed to the upper part of the mounting plate via the multispectral camera tilt mount, and the multispectral camera tilt mount cooperates with the arc-shaped adjustment hole to adjust and lock the optical axis tilt angle of the multispectral camera, so that the optical axis of the multispectral camera and the optical axis of the texture camera intersect at the preset working distance, thereby making the multispectral camera and the texture camera form a common imaging area at the preset working distance.

3. A non-contact art multi-modal data acquisition device as claimed in claim 1, wherein, The multi-device integrated bracket is equipped with a connecting flange, which is configured to connect to a robotic arm, gimbal, or support platform via a replaceable adapter to adjust the overall position of the non-contact artwork multimodal data acquisition device.

4. A joint calibration method for a non-contact multimodal data acquisition device for artworks, characterized in that, include: Keep the relative poses of the structured light 3D scanner, multispectral camera and microscope camera fixed, obtain the intrinsic parameters of the texture camera, multispectral camera and microscope camera, and obtain the extrinsic parameters between the texture camera coordinate system and the 3D point cloud coordinate system. The hybrid checkerboard calibration board is placed in the common observation area of ​​the texture camera, the multispectral camera and the microscope camera. At each fixed pose of the hybrid checkerboard calibration board, the texture camera, the multispectral camera and the microscope camera respectively acquire images of the calibration board. The pose of the hybrid checkerboard calibration board is changed and the acquisition is repeated to obtain multiple sets of images in the same pose. Based on the corner points of the mixed checkerboard in each set of in-pose images and the calibrated intrinsic parameters of each camera, the pose of the mixed checkerboard calibration plate relative to the texture camera, the multispectral camera and the microscope camera are calculated respectively. Based on the pose of the hybrid checkerboard calibration plate relative to each camera, and using the texture camera as a coordinate transfer bridge, the extrinsic parameters of the multispectral camera coordinate system and the microscopic camera coordinate system relative to the texture camera coordinate system are determined. Based on the extrinsic parameters of the multispectral camera coordinate system and the microscope camera coordinate system relative to the texture camera coordinate system, as well as the extrinsic parameters between the texture camera coordinate system and the 3D point cloud coordinate system, the extrinsic parameters of the multispectral camera coordinate system and the microscope camera coordinate system relative to the 3D point cloud coordinate system are determined, and the joint calibration result is output.

5. The joint calibration method for a non-contact artwork multimodal data acquisition device as described in claim 4, characterized in that, The acquisition of the intrinsic parameters of the microscope camera includes: Based on the field of view, magnification, and depth of field of the microscope camera, a dot calibration plate with matching dot diameter and dot center-to-center spacing is selected. Within the depth of field of the microscope camera, images of the dot calibration plate in multiple poses are acquired at different distances and tilt angles. Extract the sub-pixel coordinates of the center of each dot in the dot calibration plate image, and establish the correspondence between the dots in the dot calibration plate image and the dots in the dot calibration plate based on the sub-pixel coordinates of the center of each dot. Based on the camera imaging model and distortion model constructed from the correspondence between the dots in the dot calibration plate image and the dots in the dot calibration plate, the intrinsic parameters and distortion parameters of the microscope camera are calculated with reprojection error as the optimization objective.

6. The joint calibration method for a non-contact artwork multimodal data acquisition device as described in claim 4, characterized in that, The acquisition of the intrinsic parameters of the multispectral camera includes: Select a hybrid checkerboard calibration plate whose grid side length and coding size are adapted to the field of view of the multispectral camera; A reference band for detecting the corner points of the hybrid checkerboard calibration board is selected, and images of the hybrid checkerboard calibration board in multiple poses under the reference band are acquired at different distances and tilt angles. The sub-pixel coordinates of each corner point in each hybrid checkerboard calibration board image are detected, and the correspondence between the corner points in the hybrid checkerboard calibration board image and the corner points in the hybrid checkerboard calibration board is established based on the sub-pixel coordinates of each corner point. Based on the camera imaging model and distortion model constructed from the correspondence between the corner points in the mixed checkerboard calibration board image and the corner points in the mixed checkerboard calibration board, the intrinsic parameters and distortion parameters of the multispectral camera are calculated with reprojection error as the optimization objective.

7. The joint calibration method for a non-contact artwork multimodal data acquisition device as described in claim 4, characterized in that, Based on the pose of the hybrid checkerboard calibration board relative to each camera, and using the texture camera as a coordinate transfer bridge, the extrinsic parameters of the multispectral camera coordinate system and the microscopic camera coordinate system relative to the texture camera coordinate system are determined, including: For each set of in-pose images, based on the pose of the hybrid checkerboard calibration plate relative to the texture camera and the pose of the hybrid checkerboard calibration plate relative to the multispectral camera or the microscope camera, the extrinsic parameters of the multispectral camera coordinate system and the microscope camera coordinate system relative to the texture camera coordinate system are obtained respectively. The extrinsic parameters corresponding to multiple sets of in-pose images are fused, and the reprojection error of the mixed checkerboard corner points in the multispectral camera and the microscope camera are used as optimization targets to perform nonlinear optimization on the corresponding extrinsic parameters.

8. The joint calibration method for a non-contact artwork multimodal data acquisition device as described in claim 4, characterized in that, The microscope camera acquires a local area of ​​the hybrid checkerboard calibration plate, and the local area includes codes and corner points used to determine global corner point numbers and meet pose calculation requirements.

9. The joint calibration method for a non-contact artwork multimodal data acquisition device as described in claim 4, characterized in that, The method further includes: Calculate the total weight projection error of each camera's intrinsic and extrinsic parameter calibrations, as well as the reprojection error of each pose image group. The overall reprojection error and the reprojection error of each image group are compared with a preset accuracy threshold. When the overall reprojection error exceeds the preset accuracy threshold, gross error image groups are identified and eliminated based on the reprojection error of each pose image group, and the corresponding calibration parameters are recalculated and optimized.

10. A method for multimodal data acquisition and fusion in a non-contact multimodal data acquisition device for artworks, characterized in that, include: Joint calibration is performed using the joint calibration method for a non-contact artwork multimodal data acquisition device as described in any one of claims 4 to 9; The three-dimensional point cloud of the artwork under test is obtained by a structured light three-dimensional scanner, the multispectral image of the artwork under test is obtained by a multispectral camera, and the microscopic image of a local area of ​​the artwork under test is obtained by a microscopic camera. Obtain the extrinsic parameters of the multispectral camera coordinate system and the microscope camera coordinate system relative to the 3D point cloud coordinate system; Based on the extrinsic parameters, the three-dimensional points in the three-dimensional point cloud are transformed to the coordinate system of the multispectral camera and the coordinate system of the microscopic camera, respectively. Based on the intrinsic parameters of the multispectral camera and the microscopic camera, the transformed three-dimensional points are projected onto the corresponding multispectral image and microscopic image, respectively. Based on the projection results, multispectral information is assigned to the corresponding visible points in the three-dimensional point cloud, and microscopic image information is assigned to the corresponding local points covered by the field of view of the microscope camera, thereby generating a multimodal digital model of the artwork in the three-dimensional point cloud coordinate system.