Method for constructing scarce sample pigment database based on transfer learning

Abstract

The invention relates to a scarce sample pigment database construction method based on transfer learning, and the method comprises the specific steps: collecting modern synthetic pigment data as source domain data including RGB values, LAB values and spectral reflectivity, collecting Han cultural relic pigment samples for XRF and Raman spectrum analysis, and carrying out data correction and denoising through a white balance algorithm and wavelet transform; the method comprises the following steps of: pre-training a convolutional neural network by using modern synthetic pigment data, inputting RGB (Red, Green, Blue) values and spectral reflectivity data of pigments, and training 100 epochs by adopting an Adam optimizer and a cross entropy loss function so as to learn color and spectral characteristics of the pigments; finely adjusting the model; a prototype Networks algorithm is applied to the model obtained after fine tuning is carried out on the model obtained after fine tuning is carried out on the model; and constructing and verifying a database. According to the invention, the model is pre-trained by using modern synthetic pigment data, and fine tuning is carried out in combination with a small amount of cultural relic samples, so that a more accurate and reliable pigment database is constructed.

Description

Technical Field

[0001] The present invention relates to the technical field of cultural relics science, and in particular to a method for constructing a scarce sample pigment database based on transfer learning. Background Art

[0002] In the field of cultural relic science, building an accurate pigment database is crucial for the conservation and restoration of cultural relics. This work relies on an accurate understanding of the pigments used in the artifacts. Only with an accurate pigment database can we select appropriate restoration materials and methods to maximize the restoration of the artifacts' original appearance and preserve their cultural value. However, particularly when dealing with Han Dynasty artifacts, due to their age, many pigment samples have suffered varying degrees of damage, degradation, or contamination over time, making usable pigment samples extremely scarce. This directly leads to insufficient data in the database, as without sufficient sample data, it is difficult to fully and accurately depict the characteristics and distribution of Han Dynasty pigments.

[0003] Traditional data processing methods, such as cluster analysis, rely on large amounts of sample data to ensure their accuracy and reliability. Cluster analysis calculates the similarity between samples and groups similar samples into the same category. When the number of samples is sufficient, it can relatively accurately discover the underlying structure and patterns in the data. However, when samples are scarce, these methods often fail to provide satisfactory results. Because the number of samples is too small to represent the characteristics of the entire Han Dynasty pigment collection, it is impossible to effectively extract and generalize features from small samples. For example, when only a small number of Han Dynasty pigment samples are available, cluster analysis may mistakenly classify pigments belonging to different categories into the same category, or fail to identify some unique pigment categories.

[0004] In recent years, transfer learning, an emerging machine learning technique, has demonstrated its potential in few-shot learning across multiple fields. Transfer learning applies knowledge learned in one domain (the source domain) to a different but related domain (the target domain), enabling effective learning even when data in the target domain is scarce. In computer vision, by transferring models pre-trained on large-scale image datasets to specific few-shot image classification tasks, classification accuracy can be significantly improved. In particular, in the field of few-shot learning, the Prototypical Networks algorithm has been shown to outperform traditional clustering methods in few-shot classification tasks by computing prototype representations for each category and utilizing these prototypes for classification. This algorithm is better able to handle sample scarcity and learn the essential characteristics of a category from a limited number of samples. Summary of the Invention

[0005] The present invention aims to solve the problem of insufficient data caused by the scarcity of Han Dynasty pigment samples, which has brought great obstacles to the research and application in the field of cultural relics science, and provides a method for constructing a scarce sample pigment database based on transfer learning.

[0006] In order to achieve the above-mentioned purpose, the present invention adopts the following technical solutions:

[0007] The method for constructing a scarce sample pigment database based on transfer learning is as follows:

[0008] S1. Data collection and preprocessing:

[0009] Modern synthetic pigment data was collected as source data, including RGB values, LAB values, and spectral reflectance. Han Dynasty cultural relic pigment samples were also collected for XRF and Raman spectroscopy analysis. Data correction and denoising were performed using white balance algorithms and wavelet transforms.

[0010] S2. Model pre-training:

[0011] The convolutional neural network is pre-trained using modern synthetic pigment data. The input is the RGB value and spectral reflectance data of the pigment. The Adam optimizer and cross entropy loss function are used to train the neural network for 100 epochs to learn the color and spectral characteristics of the pigment.

[0012] S3. Model fine-tuning:

[0013] Based on the pre-trained model, we used pigment samples from Han Dynasty cultural relics for fine-tuning. We adjusted the last fully connected layer, reduced the learning rate to 0.0001, and trained for 50 epochs to adapt to the spectral characteristics of Han Dynasty pigment samples and improve classification accuracy.

[0014] S4, Prototypical Networks Application:

[0015] The Prototypical Networks algorithm is applied to the fine-tuned model to calculate the prototype representation of each category. The samples to be classified are assigned to the category of the nearest prototype using Euclidean distance, effectively handling small sample classification tasks.

[0016] S5. Database construction and verification:

[0017] The trained model is used to classify unknown pigment samples, the results are stored in the database, and the accuracy and reliability of the database are verified through cross-validation to ensure that it meets the needs of cultural relics scientific research and application.

[0018] In step S1, the specific steps of data collection and preprocessing are:

[0019] S11. Collect modern synthetic pigment data as source data. This data includes the pigment's RGB value, LAB value, and spectral reflectance. RGB values ​​represent the intensity of the pigment in the red, green, and blue color channels. LAB values ​​are a color space based on human eye perception, where L represents brightness, and A and B represent the green-red and blue-yellow components of the color, respectively. Spectral reflectance describes the reflectance characteristics of the pigment at different wavelengths and is measured using a spectrometer.

[0020] S12. Collect pigment samples from Han Dynasty cultural relics as target domain data and perform X-ray fluorescence spectroscopy and Raman spectroscopy on these samples to obtain their chemical composition and spectral characteristics. XRF analysis can detect the elemental composition of the pigment, and Raman spectroscopy can identify the molecular structure of the pigment.

[0021] S13. Preprocess these two types of data:

[0022] S131: Correct the color data using a white balance algorithm. The specific implementation process is as follows:

[0023] First, a standard white reference point is selected and its average value in the RGB color space is calculated. Then, the RGB value of each pigment is normalized to keep it consistent with the RGB value of the standard white reference point. The formula is expressed as:

[0024]

[0025] in,

[0026] R original , G original 、B original is the original RGB value;

[0027] R white , G white 、B white is the RGB value of the standard white reference point;

[0028] R corrected , G corrected 、B corrected is the corrected RGB value;

[0029] S132: Remove noise from the spectral data by wavelet transform. The specific implementation process is as follows:

[0030] First, the appropriate wavelet basis function is selected to perform multi-scale decomposition on the spectral data. Then, the noise in the high-frequency components is removed by threshold processing, and the useful information in the low-frequency components is retained. Finally, the denoised spectral data is reconstructed by inverse wavelet transform. The formula is expressed as:

[0031]

[0032] in,

[0033] f(t) is the original spectral data;

[0034] j is the scale parameter;

[0035] J is the maximum scale parameter;

[0036] k is the position parameter;

[0037] t is time or spatial position;

[0038] ψ j，k (t) and φ J，k (t) are wavelet function and scaling function respectively;

[0039] c j,k and d J,k are the wavelet coefficients and scaling coefficients respectively;

[0040] Through the preprocessing steps, the quality of the source and target domain data is ensured, providing a reliable data foundation for subsequent model training and fine-tuning.

[0041] In step S2, the specific steps of model pre-training are:

[0042] S21. Model architecture design:

[0043] Choose a convolutional neural network architecture suitable for processing multimodal data. This network consists of two parallel input branches, one for processing the RGB values ​​of the pigment and the other for processing the spectral reflectance data. The RGB input branch consists of three convolutional layers, each followed by a ReLU activation function and a max pooling layer to extract color features. The spectral reflectance input branch consists of a one-dimensional convolutional layer to extract spectral features. The outputs of the two branches are fused through a fully connected layer, and finally the classification result is output through a softmax layer.

[0044] S22. Data input and normalization:

[0045] The RGB value and spectral reflectance data of modern synthetic pigments are used as input. The RGB value is a three-dimensional vector that represents the intensity of the pigment in the red, green, and blue channels. The spectral reflectance data is a one-dimensional vector that represents the reflectance of the pigment at different wavelengths. In order to ensure the consistency of the data, the RGB value and spectral reflectance data are normalized so that their value range is between [0,1]. The normalization formula is:

[0046]

[0047] in,

[0048] x is the original data;

[0049] x min and x max are the minimum and maximum values ​​of the data respectively;

[0050] x normalized is the normalized data;

[0051] S23, optimizer and loss function:

[0052] The Adam optimizer is used for model training, and the initial learning rate is set to 0.001. The Adam optimizer combines the momentum method and the adaptive learning rate adjustment strategy, which can dynamically adjust the learning rate during training and improve the convergence speed. The loss function uses the cross-entropy loss to measure the difference between the model prediction result and the true label. The cross-entropy loss formula is:

[0053]

[0054] in,

[0055] is the cross entropy loss;

[0056] N is the number of categories;

[0057] y i is the one-hot encoding of the true label;

[0058] is the probability distribution predicted by the model;

[0059] S24, training process:

[0060] The modern synthetic pigment data was divided into a training set and a validation set. The training set was used for model training, and the validation set was used to monitor the generalization ability of the model. During the training process, each epoch contained multiple iterations. Each iteration randomly sampled a small batch of data from the training set with a batch size of 32, and performed forward propagation and backpropagation for 100 epochs.

[0061] S25. Model preservation and evaluation:

[0062] After training is completed, the model weight parameters are saved, and the model is evaluated using indicators such as confusion matrix and classification accuracy to ensure that its performance on the source domain data meets the requirements.

[0063] In step S3, the specific steps of model fine-tuning are:

[0064] S31. Data preparation:

[0065] Pigment samples from Han Dynasty cultural relics were collected. Their chemical composition and spectral characteristics were analyzed by X-ray fluorescence spectroscopy and Raman spectroscopy, and data enhancement technology was used to enhance the samples.

[0066] S32. Model structure adjustment:

[0067] Based on the pre-trained model, the last fully connected layer of the model is adjusted, and the output dimension of the fully connected layer is modified to the number of categories of the target domain data. To preserve the feature extraction capability of the pre-trained model, the parameters of all layers except the last fully connected layer are frozen, and only the weights of the last layer are updated during fine-tuning.

[0068] S33, learning rate adjustment:

[0069] In order to prevent the model from overfitting during fine-tuning, the learning rate is reduced to 0.0001, and a learning rate decay strategy is adopted. As the training progresses, the learning rate is gradually reduced. The learning rate decay formula is:

[0070] lr new =lr initial ×γ epoch

[0071] in,

[0072] lr new is the updated learning rate;

[0073] lr initial is the initial learning rate;

[0074] γ is the attenuation coefficient (set to 0.9);

[0075] epoch is the current training round number;

[0076] S34. Loss function and optimizer:

[0077] During fine-tuning, the cross entropy loss function is used to measure the difference between the model prediction results and the true labels. The cross entropy loss formula is:

[0078]

[0079] in,

[0080] is the cross entropy loss;

[0081] N is the number of categories;

[0082] y i is the one-hot encoding of the true label;

[0083] is the probability distribution predicted by the model;

[0084] S35, training process:

[0085] The pigment samples from Han Dynasty cultural relics were divided into a training set and a validation set. The training set was used for model fine-tuning, and the validation set was used to monitor the generalization ability of the model. During the training process, each epoch contained multiple iterations. Each iteration randomly sampled a small batch of data from the training set with a batch size of 16, and performed forward propagation and backward propagation for 50 epochs.

[0086] S36. Model evaluation and saving:

[0087] After fine-tuning is completed, the model is evaluated using indicators such as confusion matrix and classification accuracy to ensure that its performance on the target domain data meets the requirements. If the model performs poorly on the validation set, try further adjusting the learning rate or increasing the intensity of data augmentation. Finally, save the fine-tuned model weight parameters to provide a basis for subsequent Prototypical Networks applications.

[0088] In step S31 , the data enhancement technique includes randomly shifting, scaling, and adding noise to the spectral data.

[0089] In step S4, the specific steps of the Prototypical Networks application are:

[0090] S41. Feature extraction:

[0091] First, use the fine-tuned model to extract features from samples of each category. Assume that the number of samples in each category is n. k , where k represents the category index, n k Represents the number of samples in the kth class. For each sample, the model outputs a feature vector f i , where i represents the sample index, the feature extraction process can be expressed as:

[0092] f i =CNN(x i )

[0093] in,

[0094] x i is the input sample;

[0095] CNN is a fine-tuned convolutional neural network model;

[0096] S42, prototype calculation:

[0097] For each category, the prototype representation is calculated, and its prototype is obtained by taking the average of all sample features of the category. The prototype c of the kth category kIt can be expressed as:

[0098]

[0099] in,

[0100] f i is the feature vector of the i-th sample in the k-th class;

[0101] n k is the number of samples in the kth class;

[0102] Prototype c k represents the central characteristics of the category;

[0103] S43. Distance calculation:

[0104] For the sample f to be classified test , calculate its prototype c for each category k The Euclidean distance between them is:

[0105]

[0106] in,

[0107] m is the dimension of the feature vector;

[0108] f tes,j and c k,j are the j-th eigenvalues ​​of the sample to be classified and the k-th prototype respectively.

[0109] S44. Classification decision:

[0110] Assign the sample to be classified to the category to which the nearest prototype belongs. The classification decision can be expressed as:

[0111]

[0112] in,

[0113] is the predicted class label;

[0114] d k is the Euclidean distance between the sample to be classified and the k-th prototype;

[0115] S45, algorithm optimization:

[0116] In order to further improve the performance of Prototypical Networks in small sample classification tasks, the following optimizations are performed on the algorithm:

[0117] S451, feature normalization:

[0118] Before calculating the Euclidean distance, the feature vector is normalized to eliminate the influence of different feature scales. The normalization formula is:

[0119]

[0120] in,

[0121] f norm,i is the normalized eigenvector;

[0122] μ and σ are the mean and standard deviation of the eigenvector, respectively;

[0123] S452, weighted prototype:

[0124] When calculating the prototype, the samples are weighted according to their importance. The formula is:

[0125]

[0126] in,

[0127] w i is the weight of the i-th sample.

[0128] In step S5, the specific steps of database construction and verification are:

[0129] S51. Sample classification and storage:

[0130] First, the trained model is used to classify unknown Han Dynasty pigment samples. The model extracts features from each sample and calculates the Euclidean distance between it and the prototype of each category. Finally, the sample is assigned to the category of the prototype with the closest distance. The sample is stored in the pigment database to form a structured data record.

[0131] S52, cross validation:

[0132] In order to verify the accuracy and reliability of the database, the cross-validation method is used to evaluate the model. The specific steps are as follows:

[0133] S521, dividing the existing sample data into k subsets;

[0134] S522, sequentially use k-1 subsets as training sets and the remaining subset as a test set;

[0135] S523. Train the model on the training set, perform classification on the test set, and record the classification accuracy;

[0136] S524. Repeat the above steps k times, using a different subset as the test set each time;

[0137] S525. Calculate the average of k classification accuracy rates as the final performance indicator of the model. The cross-validation formula is:

[0138]

[0139] in,

[0140] TP i TN i , FP i 、FN i They represent true positive examples, true negative examples, false positive examples, and false negative examples in the i-th verification respectively;

[0141] S53. Database optimization:

[0142] Based on the results of cross-validation, the database is optimized. If the classification accuracy of certain categories is low, more samples need to be collected for retraining or the model parameters need to be adjusted.

[0143] S54, Application Verification:

[0144] The constructed pigment database will be applied to actual cultural relics restoration and research work to verify its effectiveness in practical applications.

[0145] In step S51, the classification result includes the sample's category label, feature vector, and classification confidence; the fields of the database include sample number, category label, RGB value, LAB value, spectral reflectance, chemical composition, and classification confidence.

[0146] The beneficial effects of the present invention are as follows: the present invention constructs a more accurate and reliable pigment database by utilizing a pre-training model based on modern synthetic pigment data and fine-tuning it in combination with a small number of cultural relic samples. It adopts the Prototypical Networks algorithm in Few-shot learning, which can effectively extract and generalize key features from limited samples, thereby improving the quality and efficiency of database construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0147] Figure 1 is a flow chart of the steps of the present invention;

[0148] Figure 2 This is a flowchart of step S2 of the present invention;

[0149] Figure 3 This is a flowchart of step S3 of the present invention;

[0150] Figure 4 This is a flowchart of step S4 of the present invention;

[0151] Figure 5This is a flowchart of step S5 of the present invention;

[0152] The following is a detailed description of the embodiments of the present invention with reference to the accompanying drawings. DETAILED DESCRIPTION

[0153] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not intended to limit the scope of the present invention. The following paragraphs describe the present invention in more detail by way of example with reference to the accompanying drawings. The advantages and features of the present invention will become more apparent from the following description. It should be noted that the drawings are all in a very simplified form and are not in exact proportions, and are only used to facilitate and clearly illustrate the purpose of the embodiments of the present invention.

[0154] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention pertains. The terms used in this specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0155] The present invention will be further described below with reference to the accompanying drawings and examples:

[0156] A method for constructing a scarce sample pigment database based on transfer learning, such as Figure 1 As shown, the specific steps are:

[0157] S1. Data collection and preprocessing:

[0158] Modern synthetic pigment data was collected as source data, including RGB values, LAB values, and spectral reflectance. Han Dynasty cultural relic pigment samples were also collected for XRF and Raman spectroscopy analysis. Data correction and denoising were performed using white balance algorithms and wavelet transforms.

[0159] S2. Model pre-training:

[0160] The convolutional neural network is pre-trained using modern synthetic pigment data. The input is the RGB value and spectral reflectance data of the pigment. The Adam optimizer and cross entropy loss function are used to train the neural network for 100 epochs to learn the color and spectral characteristics of the pigment.

[0161] S3. Model fine-tuning:

[0162] Based on the pre-trained model, we used pigment samples from Han Dynasty cultural relics for fine-tuning. We adjusted the last fully connected layer, reduced the learning rate to 0.0001, and trained for 50 epochs to adapt to the spectral characteristics of Han Dynasty pigment samples and improve classification accuracy.

[0163] S4, Prototypical Networks Application:

[0164] The Prototypical Networks algorithm is applied to the fine-tuned model to calculate the prototype representation of each category. The samples to be classified are assigned to the category of the nearest prototype using Euclidean distance, effectively handling small sample classification tasks.

[0165] S5. Database construction and verification:

[0166] The trained model is used to classify unknown pigment samples, the results are stored in the database, and the accuracy and reliability of the database are verified through cross-validation to ensure that it meets the needs of cultural relics scientific research and application.

[0167] 1. Data collection and preprocessing:

[0168] In step S1, the specific steps of data collection and preprocessing are:

[0169] S11. Collect a large amount of modern synthetic pigment data as source data. This data contains various information such as the pigment's RGB values, LAB values, and spectral reflectance. RGB values ​​represent the intensity of the pigment in the red, green, and blue color channels. LAB values ​​are a color space based on human perception, where L represents brightness, and A and B represent the green-red and blue-yellow components of the color, respectively. Spectral reflectance describes the pigment's reflectivity at different wavelengths and is typically measured using a spectrometer.

[0170] S12. Collect a small number of pigment samples from Han Dynasty artifacts as target domain data. Perform X-ray fluorescence (XRF) and Raman spectroscopy on these samples to determine their chemical composition and spectral characteristics. XRF analysis can detect the elemental composition of the pigment, while Raman spectroscopy can identify the molecular structure of the pigment.

[0171] S13. Preprocess these two types of data:

[0172] S131. Correct the color data using a white balance algorithm. The purpose of the white balance algorithm is to eliminate the influence of the light source color on the pigment color and ensure the accuracy of the color data. The specific implementation process is as follows:

[0173] First, a standard white reference point is selected and its average value in the RGB color space is calculated. Then, the RGB value of each pigment is normalized to keep it consistent with the RGB value of the standard white reference point. The formula is expressed as:

[0174]

[0175] in,

[0176] R original , G original 、B original is the original RGB value;

[0177] R white , G white 、B white is the RGB value of the standard white reference point;

[0178] R corrected , G corrected 、B corrected is the corrected RGB value;

[0179] S132. Remove noise from the spectral data by wavelet transform. Wavelet transform is a time-frequency analysis method that can decompose the signal into components of different frequencies, thereby effectively removing noise. The specific implementation process is as follows:

[0180] First, a suitable wavelet basis function (such as Daubechies wavelet) is selected to perform multi-scale decomposition on the spectral data. Then, the noise in the high-frequency components is removed by threshold processing, while the useful information in the low-frequency components is retained. Finally, the denoised spectral data is reconstructed by inverse wavelet transform. The formula is expressed as:

[0181]

[0182] in,

[0183] f(t) is the original spectral data;

[0184] j is the scale parameter;

[0185] J is the maximum scale parameter;

[0186] k is the position parameter;

[0187] t is time or spatial position;

[0188] ψ j，k (t) and φ J，k (t) are wavelet function and scaling function respectively;

[0189] c j,k and d J,k are the wavelet coefficients and scaling coefficients respectively;

[0190] Through the above preprocessing steps, the quality of source and target domain data is ensured, providing a reliable data foundation for subsequent model training and fine-tuning.

[0191] Specific implementation: Suppose we have collected RGB values, LAB values, and spectral reflectance data for 1,000 modern synthetic pigments, as well as XRF and Raman spectral data for 10 Han Dynasty cultural relic pigment samples. First, we perform white balance correction on the RGB values ​​of the modern synthetic pigments. For example, if the original RGB values ​​of a certain pigment are (150, 100, 200), and the RGB values ​​of a standard white reference point are (255, 255, 255), then the corrected RGB values ​​are (150 / 255, 100 / 255, 200 / 255) ≈ (0.59, 0.39, 0.78). Then, we perform wavelet denoising on the spectral data of the Han Dynasty cultural relic pigment samples. For example, the original reflectance of the spectral data of a certain pigment at a wavelength of 500nm is 0.85. After wavelet denoising, the reflectance is adjusted to 0.83. This process ensures data accuracy and reliability, providing high-quality data input for subsequent model training and fine-tuning.

[0192] 2. Model Pre-training:

[0193] The core of model pre-training is to pre-train the convolutional neural network (CNN) using modern synthetic pigment data. The purpose is to enable the model to learn the basic color and spectral characteristics of pigments from rich source domain data, laying the foundation for the subsequent fine-tuning stage.

[0194] In step S2, if Figure 2 As shown in the figure, the specific steps of model pre-training are:

[0195] S21. Model architecture design:

[0196] Choose a convolutional neural network architecture suitable for processing multimodal data. This network consists of two parallel input branches, one for processing the RGB values ​​of the pigment and the other for processing the spectral reflectance data. The RGB input branch consists of three convolutional layers, each followed by a ReLU activation function and a max pooling layer to extract color features. The spectral reflectance input branch consists of a one-dimensional convolutional layer to extract spectral features. The outputs of the two branches are fused through a fully connected layer, and finally the classification result is output through a softmax layer.

[0197] S22. Data input and normalization:

[0198] The RGB value and spectral reflectance data of modern synthetic pigments are used as input. The RGB value is a three-dimensional vector that represents the intensity of the pigment in the red, green, and blue channels. The spectral reflectance data is a one-dimensional vector that represents the reflectance of the pigment at different wavelengths. In order to ensure the consistency of the data, the RGB value and spectral reflectance data are normalized so that their value range is between [0,1]. The normalization formula is:

[0199]

[0200] in,

[0201] x is the original data;

[0202] x min and x max are the minimum and maximum values ​​of the data respectively;

[0203] x normalized is the normalized data;

[0204] S23, optimizer and loss function:

[0205] The Adam optimizer is used for model training, and the initial learning rate is set to 0.001. The Adam optimizer combines the momentum method and the adaptive learning rate adjustment strategy, which can dynamically adjust the learning rate during training and improve the convergence speed. The loss function uses the cross-entropy loss to measure the difference between the model prediction result and the true label. The cross-entropy loss formula is:

[0206]

[0207] in,

[0208] is the cross entropy loss;

[0209] N is the number of categories;

[0210] y i is the one-hot encoding of the true label;

[0211] is the probability distribution predicted by the model;

[0212] S24, training process:

[0213] The modern synthetic pigment data was divided into a training set and a validation set. The training set was used for model training, and the validation set was used to monitor the generalization ability of the model. During the training process, each epoch contained multiple iterations. Each iteration randomly sampled a small batch of data from the training set with a batch size of 32, and performed forward propagation and backpropagation for 100 epochs.

[0214] S25. Model preservation and evaluation:

[0215] After training is completed, the model weight parameters are saved, and the model is evaluated using indicators such as confusion matrix and classification accuracy to ensure that its performance on the source domain data meets the requirements.

[0216] Specific implementation: Suppose RGB values ​​and spectral reflectance data for 1,000 modern synthetic pigments are collected and divided into a training set (800) and a validation set (200). First, the RGB values ​​and spectral reflectance data are normalized. For example, the RGB values ​​of a pigment are (150, 100, 200), which are normalized to (0.59, 0.39, 0.78). Then, a convolutional neural network is trained using the training set. Each epoch consists of 25 iterations (800 / 32 = 25), with 32 pigments randomly sampled from the training set for training each iteration. After 100 epochs of training, the model achieves 95% classification accuracy on the validation set, demonstrating that the model is able to effectively learn the color and spectral characteristics of the pigments. Finally, the model's weight parameters are saved to provide a foundation for subsequent fine-tuning.

[0217] 3. Model fine-tuning:

[0218] Based on the pre-trained model, a small number of Han Dynasty cultural relic pigment samples are used for fine-tuning to make it better adapt to the characteristics of the target domain data.

[0219] In step S3, Figure 3 As shown in Figure 2, the specific steps of model fine-tuning are:

[0220] S31. Data preparation:

[0221] A small number of pigment samples from Han Dynasty artifacts are collected. These samples are typically analyzed through X-ray fluorescence (XRF) and Raman spectroscopy to determine their chemical composition and spectral characteristics. Given the limited number of samples, data augmentation techniques can be used to enhance data diversity, such as random shifting, scaling, and adding noise to the spectral data. Although the number of samples after data augmentation is limited, it can alleviate overfitting to a certain extent.

[0222] S32. Model structure adjustment:

[0223] Based on the pre-trained model, the model's last fully connected layer is adjusted. Specifically, the output dimension of the fully connected layer is modified to the number of categories in the target domain data. For example, if the Han Dynasty pigment samples are classified into five categories, the output dimension of the fully connected layer is set to 5. Furthermore, to preserve the feature extraction capabilities of the pre-trained model, the parameters of all layers except the last fully connected layer are frozen, and only the weights of the last layer are updated during fine-tuning.

[0224] S33, learning rate adjustment:

[0225] To prevent overfitting of the model during fine-tuning, the learning rate is reduced to 0.0001. A lower learning rate allows the model to be more cautious when updating weights, avoiding overfitting to a small number of samples. At the same time, a learning rate decay strategy is adopted, gradually reducing the learning rate as training progresses. The learning rate decay formula is:

[0226] lr new =lr initial ×γ epoch

[0227] in,

[0228] lr new is the updated learning rate;

[0229] lr initial is the initial learning rate;

[0230] γ is the attenuation coefficient (set to 0.9);

[0231] epoch is the current training round number;

[0232] S34. Loss function and optimizer:

[0233] During fine-tuning, the cross entropy loss function is used to measure the difference between the model prediction results and the true labels. The cross entropy loss formula is:

[0234]

[0235] in,

[0236] is the cross entropy loss;

[0237] N is the number of categories;

[0238] y i is the one-hot encoding of the true label;

[0239] is the probability distribution predicted by the model;

[0240] The optimizer still uses the Adam optimizer, but due to the lower learning rate, the speed at which the model updates weights will be correspondingly slower.

[0241] S35, training process:

[0242] The pigment samples from Han Dynasty artifacts were divided into a training set and a validation set. The training set was used for model fine-tuning, while the validation set was used to monitor the model's generalization ability. Each epoch of training consisted of multiple iterations, with each iteration randomly sampling a mini-batch of data (batch size = 16) from the training set for forward and backward propagation. After 50 epochs of training, the model was well adapted to the spectral characteristics of Han Dynasty pigment samples and achieved high classification accuracy on the validation set.

[0243] S36. Model evaluation and saving:

[0244] After fine-tuning, evaluate the model using metrics such as the confusion matrix and classification accuracy to ensure that its performance on the target domain data meets requirements. If the model performs poorly on the validation set, try further adjusting the learning rate or increasing the strength of data augmentation. Finally, save the fine-tuned model weights to provide a foundation for subsequent PrototypicalNetworks applications.

[0245] Specific implementation method: Assume that 50 samples of Han Dynasty cultural relics pigments are collected and divided into a training set (40 types) and a validation set (10 types). First, perform data augmentation on the spectral data, such as random translation and adding noise, to increase the number of samples to 200. Then, adjust the last fully connected layer of the pre-trained model, and set the output dimension to 5 (assuming that the Han Dynasty pigment samples are divided into 5 categories). During the fine-tuning process, the learning rate is initially set to 0.0001, and a learning rate decay strategy is adopted, multiplying the learning rate by 0.9 every 10 epochs. After training for 50 epochs, the classification accuracy of the model on the validation set reached 90%, indicating that the model can adapt well to the spectral characteristics of Han Dynasty pigment samples. Finally, save the fine-tuned model weight parameters to provide a basis for subsequent Prototypical Networks applications.

[0246] 4. Application of Prototypical Networks

[0247] In step S4, Figure 4 As shown, the specific steps of Prototypical Networks application are:

[0248] S41. Feature extraction:

[0249] First, use the fine-tuned model to extract features from samples of each category. Assume that the number of samples in each category is n. k , where k represents the category index, n k Represents the number of samples of the kth class. For each sample, the model outputs a feature vector f i , where i represents the sample index, the feature extraction process can be expressed as:

[0250] f i =CNN(x i )

[0251] in,

[0252] x i is the input sample;

[0253] CNN is a fine-tuned convolutional neural network model;

[0254] S42, prototype calculation:

[0255] For each category, the prototype representation is calculated, and its prototype is obtained by taking the average of all sample features of the category. The prototype c of the kth category k It can be expressed as:

[0256]

[0257] in,

[0258] f i is the feature vector of the i-th sample in the k-th class;

[0259] n k is the number of samples in the kth class;

[0260] Prototype c k represents the central characteristics of the category;

[0261] S43. Distance calculation:

[0262] For the sample f to be classified test , calculate its prototype c for each category k The Euclidean distance between them is:

[0263]

[0264] in,

[0265] m is the dimension of the feature vector;

[0266] f tes,j and c k,j are the j-th eigenvalues ​​of the sample to be classified and the k-th prototype respectively.

[0267] S44. Classification decision:

[0268] Assign the sample to be classified to the category to which the nearest prototype belongs. The classification decision can be expressed as:

[0269]

[0270] in,

[0271] is the predicted class label;

[0272] d k is the Euclidean distance between the sample to be classified and the k-th prototype;

[0273] S45, algorithm optimization:

[0274] In order to further improve the performance of Prototypical Networks in small sample classification tasks, the following optimizations are performed on the algorithm:

[0275] S451, feature normalization:

[0276] Before calculating the Euclidean distance, the feature vector is normalized to eliminate the influence of different feature scales. The normalization formula is:

[0277]

[0278] in,

[0279] f norm,i is the normalized eigenvector;

[0280] μ and σ are the mean and standard deviation of the eigenvector, respectively;

[0281] S452, weighted prototype:

[0282] When calculating the prototype, the samples are weighted according to their importance. For example, they can be weighted according to their confidence or representativeness. The formula is:

[0283]

[0284] in,

[0285] w i is the weight of the i-th sample.

[0286] Specific implementation: Assume that there is a fine-tuned model and there are 5 categories of Han Dynasty pigment samples, with 10 samples in each category. First, use the model to extract features for each sample and obtain the feature vector f i Then, calculate the prototype c of each category k , for example, the prototype c1 of category 1 is:

[0287]

[0288] Next, for the sample f to be classified text , calculate its prototype c for each category k The Euclidean distance d between kFor example, the distance to the prototype of category 1 is:

[0289]

[0290] Finally, the sample to be classified is assigned to the category of the prototype with the closest distance. If d1 is the smallest, the sample is predicted to belong to category 1. In this way, Prototypical Networks can effectively handle small sample classification tasks and improve the quality and efficiency of pigment database construction.

[0291] 5. Database construction and verification:

[0292] In step S5, Figure 5 As shown in the figure, after completing the training and fine-tuning of the model, the next step is to build a pigment database and verify it. The specific steps of database construction and verification are:

[0293] S51. Sample classification and storage:

[0294] First, the trained model is used to classify unknown Han Dynasty pigment samples. These unknown samples can be newly collected cultural relic pigment samples or previously unlabeled samples. The model extracts features from each sample and calculates the Euclidean distance between it and each category prototype, ultimately assigning the sample to the category of the prototype with the closest distance. The classification results include information such as the sample's category label, feature vector, and classification confidence. This information is stored in a pigment database as structured data records. The database fields may include sample number, category label, RGB value, LAB value, spectral reflectance, chemical composition, and classification confidence.

[0295] S52, cross validation:

[0296] In order to verify the accuracy and reliability of the database, the model is evaluated using the cross-validation method. Cross-validation is a statistical method used to evaluate the performance of the model on an independent data set. The specific steps are as follows:

[0297] S521, dividing the existing sample data into k subsets, for example, k=5;

[0298] S522, sequentially use k-1 subsets as training sets and the remaining subset as a test set;

[0299] S523. Train the model on the training set, perform classification on the test set, and record the classification accuracy;

[0300] S524. Repeat the above steps k times, using a different subset as the test set each time;

[0301] S525. Calculate the average of k classification accuracy rates as the final performance indicator of the model. The cross-validation formula is:

[0302]

[0303] in,

[0304] TP i TN i , FP i 、FN i They represent true positive examples, true negative examples, false positive examples, and false negative examples in the i-th verification respectively;

[0305] S53. Database optimization:

[0306] Based on the results of cross-validation, optimize the database. If the classification accuracy of certain categories is low, you may need to collect more samples for retraining or adjust the model parameters. Additionally, you can enrich the database by introducing more features (such as pigment texture and microstructure) to improve classification accuracy.

[0307] S54, Application Verification:

[0308] Finally, the constructed pigment database will be applied to actual cultural relic restoration and research to verify its effectiveness in practical applications. For example, restorers can use the information in the database to select restoration materials that match the original pigments of Han Dynasty cultural relics and observe whether the restoration results meet expectations. Feedback from practical applications will further optimize the database's content and structure.

[0309] Specific implementation: Assume a model has been trained and 100 unlabeled Han Dynasty pigment samples need to be classified and stored. First, the model is used to classify these samples, obtaining each sample's class label, feature vector, and classification confidence. For example, sample 1 is classified as "Cinnabar" with RGB values ​​of (255, 0, 0), LAB values ​​of (53.24, 80.09, 67.20), spectral reflectances of [0.85, 0.80, 0.75, ...], and a classification confidence of 0.92. This information is stored in the database as a record. Next, cross-validation is performed, dividing the existing 500 samples into five subsets, which are then trained and tested sequentially. The average classification accuracy is calculated to be 95%. Based on the cross-validation results, the classification accuracy for the "Malachite" category is low, at only 85%, so it is decided to collect more "Malachite" samples for retraining. Finally, the database was applied to cultural relics restoration work. Based on the information in the database, the restorers selected restoration materials that matched the original pigments of the Han Dynasty cultural relics. After observing the restoration effects, they found that the colors and textures after restoration were highly consistent with the original cultural relics, verifying the accuracy and reliability of the database.

[0310] The purpose of this invention is to address the data shortage problem caused by the scarcity of Han Dynasty pigment samples. Due to the scarcity of Han Dynasty pigment samples, traditional methods have made it difficult to construct an accurate and reliable pigment database, which has greatly hindered research and application in the field of cultural relic science. For example, in cultural relic restoration work, the lack of an accurate pigment database makes it difficult for restorers to select materials that match the original pigments of Han Dynasty relics. This can lead to poor restoration results and affect the historical and artistic value of the relics.

[0311] This paper constructs a more accurate and reliable pigment database by pre-training the model using modern synthetic pigment data and fine-tuning it with a small number of cultural relic samples. Modern synthetic pigment data is relatively abundant, and the model can learn the basic characteristics and classification knowledge of pigments from this data. Then, fine-tuning the model with a small number of Han Dynasty cultural relic samples allows it to adapt to the specific characteristics of the target domain.

[0312] The innovation of this invention lies in its use of the Prototypical Networks algorithm from Few-shot Learning, which is more suitable for processing small sample scenarios than traditional clustering methods. It can effectively extract and generalize key features from limited samples, improving the quality and efficiency of database construction. By constructing an accurate and reliable pigment database, it can provide strong data support for research and applications in the field of cultural relic science, such as helping cultural relic restorers accurately select restoration materials and promoting in-depth research on pigments in Han Dynasty cultural relics.

[0313] The present invention is described above by way of example in conjunction with the accompanying drawings. It is obvious that the specific implementation of the present invention is not limited to the above-mentioned method. As long as various improvements are made using the method concept and technical solution of the present invention, or they are directly applied to other occasions without improvement, they are all within the scope of protection of the present invention.

Claims

1. A method for constructing a scarce sample pigment database based on transfer learning, characterized in that: The specific steps are: S1. Data collection and preprocessing: Modern synthetic pigment data was collected as source data, including RGB values, LAB values, and spectral reflectance. Han Dynasty cultural relic pigment samples were also collected for XRF and Raman spectroscopy analysis. Data correction and denoising were performed using white balance algorithms and wavelet transforms. S2. Model pre-training: The convolutional neural network is pre-trained using modern synthetic pigment data. The input is the RGB value and spectral reflectance data of the pigment. The Adam optimizer and cross entropy loss function are used to train the neural network for 100 epochs to learn the color and spectral characteristics of the pigment. S3. Model fine-tuning: Based on the pre-trained model, we used pigment samples from Han Dynasty cultural relics for fine-tuning. We adjusted the last fully connected layer, reduced the learning rate to 0.0001, and trained for 50 epochs to adapt to the spectral characteristics of Han Dynasty pigment samples and improve classification accuracy. S4, Prototypical Networks Application: The Prototypical Networks algorithm is applied to the fine-tuned model to calculate the prototype representation of each category. The samples to be classified are assigned to the category of the nearest prototype using Euclidean distance, effectively handling small sample classification tasks. S5. Database construction and verification: The trained model is used to classify unknown pigment samples, the results are stored in the database, and the accuracy and reliability of the database are verified through cross-validation to ensure that it meets the needs of cultural relics scientific research and application.

2. The method for constructing a scarce sample pigment database based on transfer learning according to claim 1, characterized in that: In step S1, the specific steps of data collection and preprocessing are: S11. Collect modern synthetic pigment data as source data. This data includes the pigment's RGB value, LAB value, and spectral reflectance. RGB values ​​represent the intensity of the pigment in the red, green, and blue color channels. LAB values ​​are a color space based on human eye perception, where L represents brightness, and A and B represent the green-red and blue-yellow components of the color, respectively. Spectral reflectance describes the reflectance characteristics of the pigment at different wavelengths and is measured using a spectrometer. S12. Collect pigment samples from Han Dynasty cultural relics as target domain data, perform X-ray fluorescence spectroscopy and Raman spectroscopy on these samples to obtain their chemical composition and spectral characteristics; XRF analysis can detect the elemental composition of the pigment, and Raman spectroscopy can identify the molecular structure of the pigment; S13. Preprocess these two types of data: S131: Correct the color data using a white balance algorithm. The specific implementation process is as follows: First, a standard white reference point is selected and its average value in the RGB color space is calculated. Then, the RGB value of each pigment is normalized to keep it consistent with the RGB value of the standard white reference point. The formula is expressed as: in, R original , G original 、B original is the original RGB value; R white , G white 、B white is the RGB value of the standard white reference point; R corrected , G corrected 、B corrected is the corrected RGB value; S132: Remove noise from the spectral data by wavelet transform. The specific implementation process is as follows: First, the appropriate wavelet basis function is selected to perform multi-scale decomposition on the spectral data. Then, the noise in the high-frequency components is removed by threshold processing, and the useful information in the low-frequency components is retained. Finally, the denoised spectral data is reconstructed by inverse wavelet transform. The formula is expressed as: in, f(t) is the original spectral data; j is the scale parameter; J is the maximum scale parameter; k is the position parameter; t is time or spatial position; ψ j，k (t) and φ J,k (t) are wavelet function and scaling function respectively; c j,k and d J,k are the wavelet coefficients and scaling coefficients respectively; Through the preprocessing steps, the quality of the source and target domain data is ensured, providing a reliable data foundation for subsequent model training and fine-tuning.

3. The method for constructing a scarce sample pigment database based on transfer learning according to claim 1, characterized in that: In step S2, the specific steps of model pre-training are: S21. Model architecture design: Choose a convolutional neural network architecture suitable for processing multimodal data. This network consists of two parallel input branches, one for processing the RGB values ​​of the pigment and the other for processing the spectral reflectance data. The RGB input branch consists of three convolutional layers, each followed by a ReLU activation function and a max pooling layer to extract color features. The spectral reflectance input branch consists of a one-dimensional convolutional layer to extract spectral features. The outputs of the two branches are fused through a fully connected layer, and finally the classification result is output through a softmax layer. S22. Data input and normalization: The RGB value and spectral reflectance data of modern synthetic pigments are used as input. The RGB value is a three-dimensional vector that represents the intensity of the pigment in the red, green, and blue channels. The spectral reflectance data is a one-dimensional vector that represents the reflectance of the pigment at different wavelengths. In order to ensure the consistency of the data, the RGB value and spectral reflectance data are normalized so that their value range is between [0,1]. The normalization formula is: in, x is the original data; x min and x max are the minimum and maximum values ​​of the data respectively; x normalized is the normalized data; S23, optimizer and loss function: The Adam optimizer is used for model training, and the initial learning rate is set to 0.

001. The Adam optimizer combines the momentum method and the adaptive learning rate adjustment strategy, which can dynamically adjust the learning rate during training and improve the convergence speed. The loss function uses the cross-entropy loss to measure the difference between the model prediction result and the true label. The cross-entropy loss formula is: in, is the cross entropy loss; N is the number of categories; y i is the one-hot encoding of the true label; is the probability distribution predicted by the model; S24, training process: The modern synthetic pigment data was divided into a training set and a validation set. The training set was used for model training, and the validation set was used to monitor the generalization ability of the model. During the training process, each epoch contained multiple iterations. Each iteration randomly sampled a small batch of data from the training set with a batch size of 32, and performed forward propagation and backpropagation for 100 epochs. S25. Model preservation and evaluation: After training is completed, the model weight parameters are saved, and the model is evaluated using indicators such as confusion matrix and classification accuracy to ensure that its performance on the source domain data meets the requirements.

4. The method for constructing a scarce sample pigment database based on transfer learning according to claim 1, characterized in that: In step S3, the specific steps of model fine-tuning are: S31. Data preparation: Pigment samples from Han Dynasty cultural relics were collected. Their chemical composition and spectral characteristics were analyzed by X-ray fluorescence spectroscopy and Raman spectroscopy, and data enhancement technology was used to enhance the samples. S32. Model structure adjustment: Based on the pre-trained model, the last fully connected layer of the model is adjusted, and the output dimension of the fully connected layer is modified to the number of categories of the target domain data. To preserve the feature extraction capability of the pre-trained model, the parameters of all layers except the last fully connected layer are frozen, and only the weights of the last layer are updated during fine-tuning. S33, learning rate adjustment: In order to prevent the model from overfitting during fine-tuning, the learning rate is reduced to 0.0001, and a learning rate decay strategy is adopted. As the training progresses, the learning rate is gradually reduced. The learning rate decay formula is: lr new =lr initial ×γ epoch in, lr new is the updated learning rate; lr initial is the initial learning rate; γ is the attenuation coefficient (set to 0.9); epoch is the current training round number; S34. Loss function and optimizer: During fine-tuning, the cross entropy loss function is used to measure the difference between the model prediction results and the true labels. The cross entropy loss formula is: in, is the cross entropy loss; N is the number of categories; y i is the one-hot encoding of the true label; is the probability distribution predicted by the model; S35, training process: The pigment samples from Han Dynasty cultural relics were divided into a training set and a validation set. The training set was used for model fine-tuning, and the validation set was used to monitor the generalization ability of the model. During the training process, each epoch contained multiple iterations. Each iteration randomly sampled a small batch of data from the training set with a batch size of 16, and performed forward propagation and backward propagation for 50 epochs. S36. Model evaluation and saving: After fine-tuning is completed, the model is evaluated using indicators such as confusion matrix and classification accuracy to ensure that its performance on the target domain data meets the requirements. If the model performs poorly on the validation set, try further adjusting the learning rate or increasing the intensity of data augmentation. Finally, save the fine-tuned model weight parameters to provide a basis for subsequent Prototypical Networks applications.

5. The method for constructing a scarce sample pigment database based on transfer learning according to claim 4, characterized in that: In step S31 , the data enhancement technique includes randomly shifting, scaling, and adding noise to the spectral data.

6. The method for constructing a scarce sample pigment database based on transfer learning according to claim 1, characterized in that: In step S4, the specific steps of the Prototypical Networks application are: S41. Feature extraction: First, use the fine-tuned model to extract features from samples of each category. Assume that the number of samples in each category is n. k , where k represents the category index, n k Represents the number of samples in the kth class. For each sample, the model outputs a feature vector f i , where i represents the sample index, the feature extraction process can be expressed as: f i =CNN(x i ) in, x i is the input sample; CNN is a fine-tuned convolutional neural network model; S42, prototype calculation: For each category, the prototype representation is calculated, and its prototype is obtained by taking the average of all sample features of the category. The prototype c of the kth category k It can be expressed as: in, f i is the feature vector of the i-th sample in the k-th class; n k is the number of samples in the kth class; Prototype c k represents the central characteristics of the category; S43. Distance calculation: For the sample f to be classified test , calculate its prototype c for each category k The Euclidean distance between them is: in, m is the dimension of the feature vector; f tes,j and c k,j are the j-th eigenvalues ​​of the sample to be classified and the k-th prototype respectively. S44. Classification decision: Assign the sample to be classified to the category to which the nearest prototype belongs. The classification decision can be expressed as: in, is the predicted class label; d k is the Euclidean distance between the sample to be classified and the k-th prototype; S45, algorithm optimization: In order to further improve the performance of Prototypical Networks in small sample classification tasks, the following optimizations are performed on the algorithm: S451, feature normalization: Before calculating the Euclidean distance, the feature vector is normalized to eliminate the influence of different feature scales. The normalization formula is: in, f norm,i is the normalized eigenvector; μ and σ are the mean and standard deviation of the eigenvector, respectively; S452, weighted prototype: When calculating the prototype, the samples are weighted according to their importance. The formula is: in, w i is the weight of the i-th sample.

7. The method for constructing a scarce sample pigment database based on transfer learning according to claim 1, characterized in that: In step S5, the specific steps of database construction and verification are: S51. Sample classification and storage: First, the trained model is used to classify unknown Han Dynasty pigment samples. The model extracts features from each sample and calculates the Euclidean distance between it and the prototype of each category. Finally, the sample is assigned to the category of the prototype with the closest distance. The sample is stored in the pigment database to form a structured data record. S52, cross validation: In order to verify the accuracy and reliability of the database, the cross-validation method is used to evaluate the model. The specific steps are as follows: S521, dividing the existing sample data into k subsets; S522, sequentially use k-1 subsets as training sets and the remaining subset as a test set; S523. Train the model on the training set, perform classification on the test set, and record the classification accuracy; S524. Repeat the above steps k times, using a different subset as the test set each time; S525. Calculate the average of k classification accuracy rates as the final performance indicator of the model. The cross-validation formula is: in, TP i TN i , FP i 、FN i They represent true positive examples, true negative examples, false positive examples, and false negative examples in the i-th verification respectively; S53. Database optimization: Based on the results of cross-validation, the database is optimized. If the classification accuracy of certain categories is low, more samples need to be collected for retraining or the model parameters need to be adjusted. S54, Application Verification: The constructed pigment database will be applied to actual cultural relics restoration and research work to verify its effectiveness in practical applications.

8. The method for constructing a scarce sample pigment database based on transfer learning according to claim 7, characterized in that: In step S51, the classification result includes the sample's category label, feature vector, and classification confidence; the fields of the database include sample number, category label, RGB value, LAB value, spectral reflectance, chemical composition, and classification confidence.

Images