Oracle auxiliary deciphering method based on conditional diffusion model
Patent Information
- Application Number
- CN202311295878.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-07
- Publication Date
- 2026-11-03
- Estimated Expiration
- 2043-10-07
AI Technical Summary
The main difficulties in deciphering oracle bone script in existing technologies lie in the difficulty of constructing datasets, the large differences in font styles, and the lack of data on the evolution stages of the characters. The lack of effective and universal deciphering methods makes it difficult to conduct in-depth research on oracle bone script.
A conditional diffusion model-based approach is adopted. By collecting multi-stage character evolution data of deciphered oracle bone script, a U-Net denoising neural network is constructed. The conditional diffusion model is trained using the supervised information of the deciphered characters to generate character evolution images of each stage of oracle bone script, which assists in deciphering.
It has enabled the accurate generation of characters from various stages of oracle bone script, assisting experts in deciphering them and improving the efficiency and accuracy of oracle bone script research.
Smart Images
Figure CN117333881B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and computer vision, and in particular to an oracle bone script-assisted deciphering method based on a conditional diffusion model. Background Technology
[0002] Writing is a symbol of civilization and an important identifier of a nation. Oracle bone script, as the earliest systematic writing system in China, is an invaluable archaeological resource.
[0003] Although the country has vigorously promoted the research of oracle bone inscriptions, and more and more experts and scholars have devoted themselves to the research, and social attention has gradually increased, the deciphering rate of oracle bone inscriptions is still very small, making it difficult to carry out subsequent research. The deciphering of oracle bone inscriptions remains the core of oracle bone inscription research.
[0004] Currently, the main challenges in deciphering oracle bone inscriptions lie in the difficulty of constructing datasets, the significant differences in writing styles, and the lack of data on the evolutionary stages of the characters. The interpretation of oracle bone inscriptions is complex, requiring in-depth, multi-dimensional research and substantial evidence to gain the acceptance of philologists and truly achieve their decipherment. At present, there is essentially no universally effective method for deciphering oracle bone inscriptions.
[0005] Therefore, this invention proposes an oracle bone script deciphering method based on a conditional diffusion model, which can generate text images of each character evolution stage corresponding to the oracle bone script, assisting oracle bone script experts in deciphering the oracle bone script. Summary of the Invention
[0006] To address the above problems, this invention proposes an oracle bone script-aided deciphering method based on a conditional diffusion model, comprising the following steps:
[0007] Step 1: Data collection. Collect the deciphered oracle bone script characters and organize the data on the evolution of these characters in five stages: oracle bone script, bronze script, seal script, clerical script, and regular script.
[0008] Step 2: Data preprocessing. Pair two text images of the same character from different periods, perform random cropping at a fixed size, and then combine the cropped pairs of images.
[0009] Step 3: Train the network and build a conditional diffusion model neural network. Feed pairs of images into the neural network and use the supervision information of the deciphered oracle bone inscriptions from different periods. Use the Gaussian noise regression loss at multiple moments in the forward diffusion process as the loss function of the network to train the conditional diffusion model neural network until it converges.
[0010] Step 4: Test the network, save the trained conditional diffusion model, input an image of an oracle bone script character, use the trained conditional diffusion model to perform reverse diffusion on the image, and use the weighted sliding method to generate an image of the evolution of the oracle bone script character.
[0011] A further improvement lies in the fact that the deciphered oracle bone script in step one includes five stages of character evolution data: "oracle bone script", "bronze script", "great seal script", "clerical script" and "regular script". Not all five stages of character forms may exist, but at least the forms of the "oracle bone script" and "regular script" stages are required.
[0012] A further improvement is made in the following: In step two, the text-image pairing can arbitrarily select two different glyph stages of the same character for matching. First, all images are unified to a fixed resolution of 100*100. Then, a fixed-size random cropping operation is performed at the same position of the two paired images. Finally, the randomly cropped paired images are combined, ensuring that the combined images are also paired.
[0013] A further improvement is made: the randomly cropped image size is fixed at 64*64*3, where 3 represents the original RGB channels of the image. For a pair of character images in two stages, the spatial position of the random crop must be the same.
[0014] A further improvement is that the backbone structure of the conditional diffusion model neural network in step three is a U-Net denoising neural network structure, which consists of a downsampling network, an upsampling network, and skip connections.
[0015] Further improvements are made as follows: The downsampling network consists of 5 convolutional modules. The first 4 modules each consist of 2 residual modules, 2 self-attention modules, and 1 downsampling convolution, while the 5th convolutional module consists of only 2 residual modules and 1 self-attention module. The upsampling network consists of 4 upconvolutional modules, each consisting of 3 residual modules, 3 self-attention modules, and 1 upsampling convolution. Downsampling and upsampling modules of the same resolution are connected using skip connections, that is, the feature maps of the downsampling convolution and the upsampling convolution are concatenated along the channel dimension before being fed into the upsampling network for residual convolution and self-attention calculations.
[0016] A further improvement lies in the training process of the U-Net denoising neural network: Given an RGB oracle bone script image with a height of 64, a width of 64, and 3 color channels, and an RGB noisy regular script image with a height of 64, a width of 64, and 3 color channels; the two images are concatenated along the channel dimension to obtain a fused feature map with a height of 64, a width of 64, and 6 channels, which is then fed into U-Net. A predicted noise with a height of 64, a width of 64, and 3 channels is input. The loss function is obtained by calculating the distance between the predicted noise and the added noise in the regular script image. Gradient descent is then used to train the U-Net denoising neural network.
[0017] A further improvement lies in the following: the U-Net denoising neural network is specifically implemented as follows:
[0018] Oracle bone script images To conditionally train the U-Net denoising neural network model, gradually transforming purely noisy images... The noise of varying degrees is removed until the regular script image Y′ is obtained. The diffusion model has two steps: forward noise addition and backward noise reduction. The forward noise addition process q is regarded as a Markov chain model that continuously adds Gaussian noise to the original image, i.e., the regular script image y0=Y′, until it becomes a pure noise image. This process is represented by the following formula:
[0019]
[0020] Where T is the number of steps in the diffusion model, and the noise addition for each iteration of the forward process is performed according to the following formula:
[0021]
[0022] Where α 1：T The hyperparameters, ranging from 0 to 1, determine the noise variable at each step. I is the standard identity matrix. The forward noise addition process is performed on a given original image y. o Under the condition that sampling can be performed at any time step t, it can be expressed by the following formula:
[0023]
[0024] in This formula is applicable to fast sampling y in step t. t Very helpful, expressed by the following formula:
[0025]
[0026] Given the oracle bone script images and noisy target image y t Train a U-Net neural network as a denoising model, represented as: Used to predict the noise vector ∈, where Given the conditional oracle bone script image, γ is the statistic of noise variance, and the final goal is to minimize the diffusion loss term:
[0027]
[0028] Where γ ~ p(γ) and set During training, a time step t ~ {0, ..., T} is first uniformly sampled, and then γ is used to perform the sampling. t-1 and γ t Uniform distribution U(γ) t-1 γ t Sampling γ in ); furthermore, using Bayes' theorem, the given (y o y t The posterior distribution y t-1 The derivation is shown in the following formula:
[0029]
[0030]
[0031]
[0032]
[0033]
[0034]
[0035] The inverse denoising process is given by the parameter θ, and is defined as follows:
[0036]
[0037]
[0038]
[0039] Among them, the inverse noise addition process will distribute the latent variable p θ (y T Transform into data distribution p θ (y0), where x is an oracle bone script image. We combine the above equations and substitute y0 into the posterior distribution q(y0). t-1 |y0,y t Parameterization of p in ) θ (y t-1 |y t The mean of x) is given by the following formula:
[0040]
[0041] Finally, in the inference phase, reparameterizing the reverse process yields the following equation:
[0042]
[0043] in The model ultimately uses the predictions. As output.
[0044] A further improvement lies in the fact that the U-Net denoising neural network can perform multi-stage oracle bone script inference, meaning it can generate character images at a specific stage from oracle bone script. This process can be viewed as a category condition, with a total of four categories: oracle bone script to bronze script, oracle bone script to seal script, oracle bone script to clerical script, and oracle bone script to regular script. These four categories, along with temporal sampling, are transformed into embedding information and embedded into the feature map of each convolutional module, thereby achieving the goal of specifying the character generation stage.
[0045] A further improvement is made in step four, where an oracle bone script image is cropped using a sliding window. The cropped images are then input into a conditional diffusion model. After specifying the glyph stage category for generating the image, the diffusion model outputs the predicted generation result for each cropped image. The image output after weighted sliding of these generation results is the final predicted generation result.
[0046] Beneficial effects:
[0047] By embedding oracle bone inscription images and character evolution stages as conditional information into a conditional diffusion model, and using images of deciphered oracle bone inscriptions from different periods as supervisory information, the model is trained to predict and generate character evolution images for each stage corresponding to a given oracle bone inscription image, thus assisting oracle bone experts in deciphering oracle bone inscriptions. Attached image description:
[0048] Figure 1 This is a schematic diagram of the oracle bone script-assisted deciphering method based on the conditional diffusion model of the present invention;
[0049] Figure 2 This is a schematic diagram of the structure of the U-Net denoising neural network in the method of the present invention;
[0050] Figure 3 This is a schematic diagram of the forward diffusion and backward reasoning of the conditional diffusion model in the method of the present invention;
[0051] Figure 4 This is a schematic diagram illustrating the training and inference testing of the conditional diffusion model in the method of this invention. Detailed implementation method:
[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.
[0053] like Figure 1 As shown in the figure, this invention proposes an oracle bone script-assisted deciphering method based on a conditional diffusion model, comprising the following steps:
[0054] Step 1: Data Collection. Collect the deciphered oracle bone script characters and organize their evolutionary data across five stages: oracle bone script, bronze script, seal script, clerical script, and regular script. The deciphered oracle bone script characters encompass the evolutionary data of these five stages: oracle bone script, bronze script, seal script, clerical script, and regular script. Not all characters from all five stages may exist, but at least characters from the oracle bone script and regular script stages are required.
[0055] Step Two: Data Preprocessing. Pair two text images of the same character from different periods and perform a fixed-size random cropping operation. Combine the cropped pairs of images. Text image pairing can be performed on any two different glyph stages of the same character. First, all images are standardized to a fixed resolution of 100*100. Then, a fixed-size random cropping operation is performed on the same position in both pairs of images. Finally, the randomly cropped pairs of images are combined, ensuring that the combined images are also pairs. The size of the randomly cropped images is fixed at 64*64*3, where 3 represents the original RGB channels of the image. For the two glyph images in a pair, the spatial position of the random cropping must be identical.
[0056] Step 3: Train the network and build a conditional diffusion model neural network. Feed pairs of images into the neural network and use the supervision information of the deciphered oracle bone inscriptions from different periods. Use the Gaussian noise regression loss at multiple moments in the forward diffusion process as the loss function of the network to train the conditional diffusion model neural network until it converges.
[0057] like Figure 2As shown, the backbone structure of the conditional diffusion model neural network in the method of this invention is a U-Net denoising neural network structure, which consists of a downsampling network, an upsampling network, and skip connections. The downsampling network consists of 5 convolutional modules. The first 4 modules each consist of 2 residual modules, 2 self-attention modules, and 1 downsampling convolution. The 5th convolutional module consists of only 2 residual modules and 1 self-attention module. The upsampling network consists of 4 upconvolutional modules. Each module consists of 3 residual modules, 3 self-attention modules, and 1 upsampling convolution. Skip connections are used between downsampling and upsampling modules of the same resolution. That is, the feature maps of the downsampling convolution and the upsampling convolution are concatenated along the channel dimension before being fed into the upsampling network for residual convolution and self-attention calculation.
[0058] The training process of the U-Net denoising neural network is as follows: Given an RGB oracle bone script image with a height of 64, a width of 64, and 3 color channels, and an RGB noisy regular script image with a height of 64, a width of 64, and 3 color channels, the two images are concatenated along the channel dimension to obtain a fused feature map with a height of 64, a width of 64, and 6 channels. This fused feature map is then fed into U-Net with a predicted noise input of 64 height, a width of 64, and 3 channels. The loss function is obtained by calculating the distance between the predicted noise and the added noise in the regular script image. Gradient descent is then used to train the U-Net denoising neural network.
[0059] like Figure 3 As shown, the forward diffusion and backward inference of the conditional diffusion model in the method of this invention are specifically implemented as follows:
[0060] Oracle bone script images To conditionally train the U-Net denoising neural network model, gradually transforming purely noisy images... The noise of varying degrees is removed until the regular script image Y′ is obtained. The diffusion model has two steps: forward noise addition and backward noise reduction. The forward noise addition process q is regarded as a Markov chain model that continuously adds Gaussian noise to the original image, i.e., the regular script image y0=Y′, until it becomes a pure noise image. This process is represented by the following formula:
[0061]
[0062] Where T is the number of steps in the diffusion model, and the noise addition for each iteration of the forward process is performed according to the following formula:
[0063]
[0064] Where α 1：T The hyperparameters, ranging from 0 to 1, determine the noise variable at each step. I is the standard identity matrix. The forward noise addition process is performed on a given original image y. oUnder the condition that sampling can be performed at any time step t, it can be expressed by the following formula:
[0065]
[0066] in This formula is applicable to fast sampling y in step t. t Very helpful, expressed by the following formula:
[0067]
[0068] Given the oracle bone script images Noise target image y t Train a U-Net neural network as a denoising model, represented as: Used to predict the noise vector ∈, where Given the conditional oracle bone script image, γ is the statistic of noise variance, and the final goal is to minimize the diffusion loss term:
[0069]
[0070] Where γ ~ p(γ) and set During training, a time step t ~ {0, ..., T} is first uniformly sampled, and then γ is used to... t-1 and γ t Uniform distribution U(γ) t-1 γ t Sampling γ in ); furthermore, using Bayes' theorem, the given (y o y t The posterior distribution y t-1 The derivation is shown in the following formula:
[0071]
[0072]
[0073]
[0074] The inverse denoising process is given by the parameter θ, and is defined as follows:
[0075]
[0076]
[0077]
[0078] Among them, the inverse noise addition process will distribute the latent variable p θ (y T Transform into data distribution p θ(y0), where x is an oracle bone script image. We combine the above equations and substitute y0 into the posterior distribution q(y0). t-1 |y0,y t ) parameterization p θ (y t-1 |y t The mean of x) is given by the following formula:
[0079]
[0080] Finally, in the inference phase, reparameterizing the reverse process yields the following equation:
[0081]
[0082] in The model ultimately uses the predictions. As output.
[0083] The U-Net denoising neural network can perform multi-stage oracle bone script inference, meaning it can generate character images at a specific stage from oracle bone script. This process can be viewed as a categorical condition, with a total of four categories: oracle bone script to bronze script, oracle bone script to seal script, oracle bone script to clerical script, and oracle bone script to regular script. These four categories, along with temporal sampling, are transformed into embedding information and embedded into the feature map of each convolutional module, thereby achieving the goal of specifying the character generation stage.
[0084] Step Four: As Figure 4 The reasoning and testing process of the conditional diffusion model in the method of this invention is as follows:
[0085] The test network uses a pre-trained conditional diffusion model. An image of oracle bone script is input, and the model performs reverse diffusion on the image, generating an image showing the evolution of the oracle bone script characters using a weighted sliding method. Specifically, an oracle bone script image is cropped using a sliding window. A series of cropped images are input into the conditional diffusion model. After specifying the character stage category for the generated image, the model outputs the predicted generation result for each cropped image. These results are then weighted and slid together to output the final predicted image.
[0086] By embedding oracle bone script images and character evolution stage categories as conditional information into a conditional diffusion model, the model can predict and generate character evolution images for each stage corresponding to a given oracle bone script image, thus assisting oracle bone script experts in deciphering oracle bone script.
[0087] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.
Claims
1. A method for assisting in the deciphering of oracle bone inscriptions based on a conditional diffusion model, characterized in that, Includes the following steps: Step 1: Data collection. Collect the deciphered oracle bone script characters and organize the data on the evolution of these characters in five stages: oracle bone script, bronze script, seal script, clerical script, and regular script. Step 2: Data preprocessing. Pair two text images of the same character from different periods, perform random cropping at a fixed size, and then combine the cropped pairs of images. Step 3: Train the network and construct a conditional diffusion model neural network. Pairs of images are fed into the neural network. Using the supervisory information from deciphered oracle bone inscriptions from different periods, the Gaussian noise regression loss at multiple time points during the forward diffusion process is used as the network's loss function. The conditional diffusion model neural network is trained until convergence. The backbone structure of the conditional diffusion model neural network is a U-Net denoising neural network structure, which consists of a downsampling network, an upsampling network, and skip connections. The specific implementation of the U-Net denoising neural network is as follows: Oracle bone script images To conditionally train the U-Net denoising neural network model, gradually transforming purely noisy images... Remove noise of varying degrees until the image of the regular script is obtained. The diffusion model has two steps: forward denoising and backward denoising. The forward denoising process... Viewed as a Markov chain model, it continuously processes the original image, i.e., the calligraphy image. Adding Gaussian noise until the image becomes purely noisy is represented by the following formula: in For the number of steps in the diffusion model, noise is added for each iteration of the forward process according to the following formula: in The hyperparameters, ranging from 0 to 1, determine the noise variable at each step. Given a standard identity matrix, the forward noise addition process is performed on a given original image. Under the condition of supporting any time step Sampling is performed, as expressed by the following formula: in This formula is applicable to... Step-by-step fast sampling Very helpful, expressed by the following formula: Given the oracle bone script images Noise target image Train a U-Net neural network as a denoising model, represented as: Used to predict noise vectors ,in For the conditional oracle bone script images, Let be a statistic of the noise variance, and the final goal is to minimize the diffusion loss term: in And set During training, a time step is first sampled uniformly. , and then from and uniform distribution Medium sampling Furthermore, using Bayes' theorem, a given ( , posterior distribution of ) The derivation is shown in the following formula: The inverse denoising process is determined by parameters Given, defined as follows: The reverse noise addition process distributes the latent variables. Transform into data distribution ,in For the image of oracle bone script, combine the above equations and... Substitute into the posterior distribution In-parameterization The mean is given by the following formula: Finally, in the inference phase, reparameterizing the reverse process yields the following equation: in The model ultimately uses the predicted values. As output; Step 4: Test the network, save the trained conditional diffusion model, input an image of an oracle bone script character, use the trained conditional diffusion model to perform reverse diffusion on the image, and use the weighted sliding method to generate an image of the evolution of the oracle bone script character.
2. The oracle bone script-assisted deciphering method based on the conditional diffusion model according to claim 1, characterized in that: The deciphered oracle bone script in step one includes five stages of character evolution data: "oracle bone script", "bronze script", "great seal script", "clerical script" and "regular script". Not all five stages of character forms may exist, but at least the forms of "oracle bone script" and "regular script" stages are required.
3. The oracle bone script-assisted deciphering method based on the conditional diffusion model according to claim 1, characterized in that: In step two, the text-image pairing process involves arbitrarily selecting two different glyph stages of the same character for matching. First, all images are standardized to a fixed resolution of 100. 100, then perform a fixed-size random cropping operation on the same position of the two paired images, and finally combine the randomly cropped paired images, ensuring that the combined images are also paired.
4. The oracle bone script-assisted deciphering method based on the conditional diffusion model according to claim 3, characterized in that: The randomly cropped image size is fixed at 64. 64 Size 3, where 3 represents the original RGB channels of the image; for two pairs of glyph images, the spatial positions of the random cropping must be the same.
5. The oracle bone script-assisted deciphering method based on the conditional diffusion model according to claim 1, characterized in that: The downsampling network consists of 5 convolutional modules. The first 4 modules each consist of 2 residual modules, 2 self-attention modules, and 1 downsampling convolution. The 5th convolutional module consists of only 2 residual modules and 1 self-attention module. The upsampling network consists of 4 upconvolutional modules. Each module consists of 3 residual modules, 3 self-attention modules, and 1 upsampling convolution. The downsampling and upsampling modules of the same resolution are connected by skip connections. That is, the feature maps of the downsampling convolution and the feature maps of the upsampling convolution are connected along the channel dimension, and then fed into the upsampling network for residual convolution and self-attention calculation.
6. The oracle bone script-assisted deciphering method based on the conditional diffusion model according to claim 1, characterized in that: The training process of the U-Net denoising neural network is as follows: Given an RGB oracle bone script image with a height of 64, a width of 64, and 3 color channels, and an RGB noisy regular script image with a height of 64, a width of 64, and 3 color channels; the two images are concatenated along the channel dimension to obtain a fused feature map with a height of 64, a width of 64, and 6 channels, which is then fed into U-Net. A predicted noise with a height of 64, a width of 64, and 3 channels is input, and the loss function is obtained by calculating the distance between the predicted noise and the added noise in the regular script image; then gradient descent is used to train the U-Net denoising neural network.
7. The oracle bone script-assisted deciphering method based on the conditional diffusion model according to claim 1, characterized in that: U-Net denoising neural network can perform multi-stage oracle bone script inference, that is, generate character images of a certain stage from oracle bone script. This process is regarded as a category condition, with a total of 4 categories: oracle bone script to bronze script, oracle bone script to seal script, oracle bone script to clerical script, and oracle bone script to regular script. These 4 categories are converted into embedding information along with temporal sampling and embedded into the feature map of each convolutional module, thereby achieving the purpose of specifying the character generation stage.
8. The oracle bone script-assisted deciphering method based on the conditional diffusion model according to claim 1, characterized in that: In step four, a oracle bone script image is cropped using a sliding window. The cropped images are then input into a conditional diffusion model. After specifying the glyph stage category for the generated images, the diffusion model outputs the predicted generation result for each cropped image. The image output after weighted sliding of these generation results is the final predicted generation result.