Automatic oracle bone copy generation method based on improved CycleGAN network

Abstract

The invention provides an improved CycleGAN network-based automatic oracle bone copy generation method, which comprises the following steps of: scanning a book by using a non-contact file book scanner to obtain an oracle bone description image, preprocessing the oracle bone description image to obtain an oracle bone image, obtaining an oracle bone data set, and dividing the oracle bone data set into a training set and a test set; constructing an improved CycleGAN network model; adding a multi-scale feature extraction module and a style enhanced attention residual module in a generator network; training an improved CycleGAN network model by using the oracle bone images in the training set to obtain a trained oracle bone copy generation model; and inputting the oracle bone image in the test set into the trained oracle bone copy generation model to obtain a translated oracle bone copy image. According to the method, the structure and details of the oracle can be accurately captured and reproduced, the high-quality copy is generated, an efficient image generation method is provided for digital research of the oracle, and a new thought is provided for generating the high-quality oracle copy.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence and computer vision, and in particular to a method for automatically generating oracle bone copies. Background Art

[0002] As an important carrier for recording the form and details of oracle bone inscriptions, oracle bone copies have important academic value in the study of the origin and evolution of Chinese characters. Oracle bone inscriptions are the key content of the development of Chinese characters and are of great value in exploring the origin, development and evolution of ancient characters. Due to the long history of oracle bone inscriptions and the common phenomenon of loss of original pieces, copies have become extremely important materials for oracle bone inscription research. However, the existing oracle bone copies are rough and erroneous to a certain extent, which not only reduces the accuracy of oracle bone inscription research, but also may lead to deviations in the interpretation of ancient history and culture.

[0003] The production of traditional oracle bone copies mainly uses two technologies: line drawing extraction and image style transfer. Line drawing extraction focuses on the extraction of image edge contours, extracting lines and contours from oracle bone images to generate corresponding copies, while image style transfer focuses on applying the style of one image to another image using machine learning algorithms to generate a new image. The generated image combines the content of the original image and the style of the target image. The above methods have achieved good results, but traditional methods can only handle simple texture level conversions, highly relying on manual processing by oracle bone experts and scholars, and the recognition efficiency is relatively low. In addition, traditional copy generation methods have limitations in detail reproduction, which is particularly obvious when faced with a large number of samples.

[0004] With the rapid development of deep learning technology, line drawing extraction and image style transfer technology have made significant progress, providing strong technical support for the digitization of oracle bone research and significantly improving the efficiency of copy generation. However, there is still room for improvement in terms of the stability of style transfer and the preservation of semantic content. One of the main methods of image style transfer is based on generative adversarial networks (GAN). However, during the training process, the GAN network is unstable, which may cause the generated oracle bone copies to show discontinuity in the morphological contours and edge areas, as well as insufficient detail restoration. In addition, in response to interference factors such as background noise and blur that are prevalent in oracle bone images, cycle-consistent generative adversarial networks (CycleGAN) show obvious limitations in feature recognition and noise suppression. The traditional CycleGAN network has limited ability to capture fine-grained style features and structural information, and the traditional single attention module is difficult to balance the unity of global style and local details.

[0005] The invention patent with application number 202110888155.3 discloses a method for automatically extracting oracle bone outlines and character traces based on a generative adversarial network, including: organizing paired oracle bone copy images and oracle bone rubbing images, constructing a copy generation data set, designing and constructing a generative adversarial network model OBTGAN, training the generation model using the cyclic training method of the CycleGAN model, and obtaining the generated copy image using the trained generative adversarial network. The generator in OBTGAN uses a residual network module, inserts a SpatialAttention layer before the third downsampling, and uses a residual network embedded with a ChannelAttention structure to extract features again after the downsampling is completed. The upsampling module uses a Unet structure to jump-connect the low-level semantic features in the downsampling process with the upsampling generated feature matrix. The above method uses computer technology to replace the manual tracing steps of copy production, and the generated copies have higher accuracy in glyph details, greatly reducing the production time of oracle bone copies, and providing the possibility for batch production of oracle bone copies. The model of the above patent has limited generalization ability and cannot accurately extract the contours and character traces of different types of oracle bone tablets; the model has limited ability to extract details and cannot completely restore all the details on the rubbings, such as scratches and incompleteness on the rubbings; the extraction effect may be affected for rubbings with poor image quality such as noise and blur. Summary of the invention

[0006] In view of the limitations of traditional copy generation methods in detail reproduction and low recognition efficiency, the present invention proposes an automatic generation method of oracle bone copies based on an improved CycleGAN network, which can accurately capture and reproduce the structure and details of oracle bone inscriptions and generate more accurate and clear oracle bone copies.

[0007] In order to achieve the above object, the technical solution of the present invention is implemented as follows: a method for automatically generating oracle bone copies based on an improved CycleGAN network, the steps of which are as follows:

[0008] Step 1: Scan the books with a non-contact file and book scanner to obtain oracle bone inscription images, pre-process the oracle bone inscription images to obtain oracle bone images, obtain an oracle bone dataset, and divide the oracle bone dataset into a training set and a test set;

[0009] Step 2: Build an improved CycleGAN network model: add a multi-scale feature extraction module and a style enhancement attention residual module to the generator network;

[0010] Step 3: Use the oracle bone images in the training set to train the improved CycleGAN network model to obtain a trained oracle bone copy generation model;

[0011] Step 4: Input the oracle bone images in the test set into the trained oracle bone copy generation model to obtain the translated oracle bone copy images.

[0012] Preferably, the method for obtaining the oracle bone data set is: respectively intercepting two types of oracle bone images, namely, the color version and the copy, from the oracle bone inscription images, and the two types of oracle bone images, namely, the color version and the copy, constitute the oracle bone data set;

[0013] The oracle bone dataset is divided into a training set and a test set in a ratio of 7:3. The oracle bone images in the training set are then subjected to data enhancement methods such as geometric transformation, contrast enhancement, sharpness enhancement or Gaussian filtering to obtain an extended training set. The improved CycleGAN network model is trained using the extended training set.

[0014] The evaluation indicators SSIM, PSNR, FID and KID are used to evaluate the image generation quality of the oracle bone model generation model.

[0015] Preferably, the improved CycleGAN network model includes two groups of generator networks G and generator network F and a discriminator network D x and the discriminator network D y The two generator networks F are responsible for generating color oracle bone images from the replica oracle bone images, the generator network G is responsible for generating replica oracle bone images from the color oracle bone images, and the discriminator network D x The adversarial loss LD is used to compare the real sample images of the color plates in the oracle bone dataset with the color plate oracle bone images generated by the generator network F. X , discriminator network D y The adversarial loss LD is used to compare the real sample images of the replicas in the oracle bone dataset with the replica oracle bone images generated by the generator network G. Y .

[0016] Preferably, the color version oracle bone image in the oracle bone data set is used as the real sample image real_A to obtain the pseudo sample image fake_B of the replica through the generator network G, and the pseudo sample image fake_B obtains the bidirectional cycle consistency path image Cycle_A of the color version through the generator network F, and the difference between the bidirectional cycle consistency path image Cycle_A and the real sample image real_A is measured by calculating the mean square error between the bidirectional cycle consistency path image Cycle_A and the real sample image real_A; the real sample image real_B of the replica in the oracle bone data set is obtained through the generator network F to obtain the pseudo sample image fake_A of the color version, and the pseudo sample image fake_A obtains the bidirectional cycle consistency path image Cycle_B of the replica through the generator network G, and the distinction between the real sample image real_B and the generated image is learned by comparing the real sample image real_B with the bidirectional cycle consistency path image Cycle_B;

[0017] The discriminator network D y The part receives the fake sample image fake_B and the real sample image real_B as input, extracts and distinguishes features through multiple convolutional layers, and finally outputs the true and false discrimination result; the discriminator network D x The part receives the fake sample image fake_A and the real sample image real_A as input, performs feature extraction and discrimination through multiple convolutional layers, and finally outputs the true and false discrimination result.

[0018] Preferably, the generator network G and the generator network F both adopt an encoder-decoder architecture, the encoder converts the real sample image into a feature representation through a convolution layer and downsampling, the decoder reconstructs the features into a pseudo sample image using a transposed convolution layer and upsampling, and the generator adds the designed multi-scale feature extraction module and style-enhanced attention residual module in parallel to achieve progressive feature conversion.

[0019] Preferably, the style-enhanced attention residual module is arranged between the encoder and the decoder of the generator network G and the generator network F, and the multi-scale feature extraction module is arranged between the second convolution layer of the encoder and the first convolution layer of the decoder.

[0020] Preferably, the input of the multi-scale feature extraction module is the feature map F after the first two convolutional layers. Conv2 ∈R B ×C×H×W , the feature map F Conv2 It is divided into four convolution branches, which perform four parallel convolution operations with different convolution kernel sizes and different numbers of groups. The feature maps obtained by the four convolution branches are spliced ​​to obtain the feature map F Conv2 Output feature maps of the same size; where B is the batch size, C is the number of channels, and H and W are the height and width of the feature map.

[0021] Preferably, the input of the style enhanced attention residual module is the feature representation output by the encoder. The feature representation output by the encoder is used as the input of the style enhanced attention residual module and then passed through two 3×3 convolutional layers for convolution operation. There is a batch normalization operation after each convolutional layer, and a ReLU activation function is processed before the second convolutional layer. The result obtained by the second convolutional layer is residually connected with the feature representation output by the input encoder as the input of the ESA module. The ESA module obtains the corresponding feature map by respectively passing the input feature map through global context encoding, local structure encoding, edge attention module, channel attention module and style statistical feature module. The corresponding feature maps are fused to obtain a fused feature map, which is residually connected with the feature representation output by the encoder again as the output of the ESARM module.

[0022] Preferably, the input of the ESA module is the downsampled feature map F Downsampling ∈R B×C×H×W , where B is the batch size, C is the number of channels, H and W are the height and width of the feature map respectively; the feature map F Downsampling In the edge attention module, the feature map F1 is obtained by sequentially processing through a 3×3 convolution layer and a ReLU activation function, a 1×1 convolution layer and a Sigmoid activation function. The feature map F Downsampling In the channel attention module, the feature map F3 is obtained by adaptive flat pooling, 1×1 convolution layer and ReLU activation function, 1×1 convolution layer, and Sigmoid activation function. Downsampling In the style statistical feature module, convolution, average pooling, and standardization operations are performed in sequence to extract the feature map F. Downsampling The visual style features in the feature map F2 are obtained, and the feature map F Downsampling , feature map F2 and feature map F3 are multiplied element by element to obtain feature map F6, feature map F6 and feature map F1 are multiplied element by element to obtain feature map F7; feature map F Downsampling In the local structure encoding, the feature map F4 is obtained by sequentially passing through a 3×3 convolution layer, a ReLU activation function, and a 3×3 convolution layer. The feature map F Downsampling In the global context coding, the feature map F5 is obtained by adaptive flat pooling, 1×1 convolution layer and ReLU activation function, and 1×1 convolution layer in sequence. The corresponding position elements of feature map F7, feature map F4 and feature map F5 are added to obtain a fused feature map.

[0023] Preferably, in step 4, the color oracle bone images in the test set are tested using the replica generation weights trained in the oracle bone replica generation model, and the color oracle bone images are converted into replica oracle bone images using the generator network G to obtain translated oracle bone replica images;

[0024] The loss function during training includes adversarial loss and cycle consistency loss, the generator network G:x→y and its discriminator network D Y The corresponding adversarial loss LD Y for:

[0025]

[0026] Generator network F:y→x and its discriminator network D X The corresponding adversarial loss LD X for:

[0027]

[0028] The cycle consistency loss L2 Loss is:

[0029]

[0030] Among them, S GAN (G,D Y ,X,Y) is the loss function of converting the color sample image x in the color image dataset to the replica sample image y in the replica image dataset. represents sampling from the replica image dataset, log D Y (y) represents the discriminator network D Y The logarithm of the predicted result for the true replica image; Denotes sampling from the color image dataset, D Y (G(x)) represents the discriminator network D Y The predicted probability value of the replica image generated by the generator network G, S GAN (G,D X , X, Y) represents the loss function of converting the replica sample image y into the color plate sample image x in the replica image dataset, D X (x) represents the discriminator network D Y The predicted probability value of the real color plate image, D X (G(y)) represents the discriminator network D X The predicted probability value of the color image generated by the generator network F, S cyc (G, F) represents the sum of the losses between the color plate sample images of the color plate image dataset, which are generated by the generator network G and then regenerated by the generator network F, and the real color plate images, and the color plate images generated by the generator network F and then regenerated by the generator network G. F(G(x)) represents the samples in the color plate image dataset, which are generated by the generator network G and then regenerated by the generator network F; G(F(y)) represents the samples in the replica image dataset, which are generated by the generator F and then regenerated by the generator G.

[0031] The total loss function is:

[0032] S(G, F, D X , D Y )=S GAN (G,D Y ,x,y)+S GAN (F,D X , y, x)+λ cyc S cyc (G, F)

[0033] Among them, λ cyc is the cycle consistency loss weight.

[0034] Compared with the prior art, the present invention has the following beneficial effects: based on the idea of ​​image style transfer, the present invention uses an improved CycleGAN network to construct an oracle bone copy style transfer model, and optimizes the network structure. By adding a multi-scale feature extraction module (Split Conv Concat, SCC) to the original CycleGAN model, it is possible to simultaneously capture local detail features and global semantic information at different scales, and achieve the synchronous capture of cross-scale local details and global semantic information, thereby significantly improving the quality of the generated oracle bone copy. In addition, the present invention also adds an Enhanced Style Attention Residual Module (ESARM) to the network generator to achieve selective enhancement of the key features of the oracle bone and effectively suppress the background noise interference of the oracle bone image. The present invention can accurately capture and reproduce the structure and details of oracle bone inscriptions, generate high-quality copies, provide an efficient image generation method for digitization research of oracle bone inscriptions, and provide a new idea for generating high-quality oracle bone copies.

[0035] The experimental results show that compared with other methods, the present invention has significant improvements in four quantitative evaluation indicators: SSIM, PSNR, FID and KID. Further analysis of the qualitative results shows that the present invention effectively maintains the topological integrity of the characters and achieves highly accurate reconstruction of the details, with edge clarity and continuity closest to the standard copy. The present invention is suitable for the high-standard generation of oracle bone copies, providing strong support for the study of ancient characters. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 It is a flow chart of the present invention.

[0038] Figure 2 These are some sample images of high-definition description images.

[0039] Figure 3 These are example images of some oracle bone images in the oracle bone dataset.

[0040] Figure 4 This is a framework diagram of the improved CycleGAN network of the present invention.

[0041] Figure 5 This is a single-branch framework diagram of the improved CycleGAN network of the present invention.

[0042] Figure 6 This is a detailed diagram of data processing of the SCC module of the present invention.

[0043] Figure 7 It is a structural diagram of the ESARM module of the present invention.

[0044] Figure 8 It is a structural diagram of the ESA module of the present invention.

[0045] Fig. 9 This is a detailed diagram of data processing of the ESA module of the present invention.

[0046] Fig.10 It is a comparison diagram of the copy generation results of the present invention and the copy generation results of other methods, wherein (a) is a color plate, (b) is a copy, (c) is a Canny algorithm, (d) is a HED algorithm, (e) is a pix2pix algorithm, (f) is a CycleGAN network, and (g) is the present invention.

[0047] Fig.11 The figures are comparison diagrams of the copy generation effects of the experiments with and without the SCC module, where (a) is the color version, (b) is the copy, (c) is without the SCC module, and (d) is the present invention.

[0048] Fig.12 The figures are comparison diagrams of the copy generation effects of the experiments with and without the ESA module, wherein (a) is the color version, (b) is the copy, (c) is without the ESA module, and (d) is the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] like Figure 1 As shown, a method for automatically generating oracle bone copies based on an improved CycleGAN network specifically includes the following steps:

[0051] Step 1: The researchers used a professional non-contact file and book scanner to scan the books to obtain high-definition oracle bone inscription images. The obtained oracle bone inscription images were preprocessed to obtain oracle bone images. The oracle bone images constituted an oracle bone dataset, which included a test set and a training set. The oracle bone images in the training set were enhanced to obtain an extended training set.

[0052] For details, see Figure 2Obtain high-definition oracle bone inscription images, and extract two types of oracle bone images, color version and copy, from the high-definition oracle bone inscription images. Figure 3 The oracle bone images in the color version and the replica are shown in the figure. The oracle bone dataset is divided into a training set and a test set in a ratio of 7:3. Then, the oracle bone images in the training set are subjected to data enhancement methods such as geometric transformation, contrast enhancement, sharpness enhancement, and Gaussian filtering to finally obtain an extended training set. The above data enhancement methods can improve the accuracy and robustness of the model, reduce overfitting, and enable it to adapt to different application scenarios and task requirements.

[0053] Step 2: Build an improved CycleGAN network model: Add a multi-scale feature extraction module and a style enhancement attention residual module to the generator network.

[0054] The improved CycleGAN network model introduces a multi-scale feature extraction module (SCC) to simultaneously capture the local detail features and global semantic information of the oracle bone color plate image at different scales. The style-enhanced attention residual module (ESARM) is used to achieve the selective enhancement of the key features of the oracle bone and effectively suppress the background noise interference of the oracle bone color plate image, generating a more accurate and clear oracle bone copy, which provides important clues for in-depth research on the origin and development of ancient Chinese characters.

[0055] like Figure 4 As shown, the improved CycleGAN network model consists of two groups of networks, the generator network G and the generator network F and the discriminator network D x and the discriminator network D y . The data in the improved CycleGAN network model include real sample images (real_A, real_B) and generated pseudo sample images (fake_A, fake_B). The real sample images are oracle bone images of the oracle bone dataset, and the pseudo samples are images generated by the generator. The real sample image real_A obtains the pseudo sample image fake_B through the generator network G, and the pseudo sample image fake_B obtains the bidirectional cycle consistency path image Cycle_A through the generator network F. The difference between the bidirectional cycle consistency path image Cycle_A and the real sample image real_A is measured by calculating the mean square error between the bidirectional cycle consistency path image Cycle_A and the real sample image real_A. The real sample image real_B obtains the pseudo sample image fake_A through the generator network F, and the pseudo sample image fake_A obtains the bidirectional cycle consistency path image Cycle_B through the generator network G. The discriminator D y By comparing the real sample image real_B and the bidirectional cycle consistency path image Cycle_B, we learn to distinguish between the real sample image and the generated image.

[0056] The real sample image real_B generates a fake sample image fake_A through the generator network F and the discriminator network D x The adversarial loss LD is obtained by comparing the real sample image real_A and the fake sample image fake_A X The real sample image real_A is generated through the generator network G to generate the fake sample image fake_B, and the discriminator network D y The adversarial loss LD is obtained by comparing the real sample image real_B with the fake sample image fake_B Y The two generator networks F are responsible for generating color images, and the generator network G is responsible for generating replica images.

[0057] like Figure 5 As shown, the generator network G adopts an encoder-decoder architecture. The encoder converts the real sample image real_A into feature representation through convolution layers and downsampling. The decoder reconstructs the features into a pseudo sample image fake_B using transposed convolution layers and upsampling. The generator adopts a parallel method to add the designed SCC module and ESARM module to achieve a progressive feature conversion from the input image to the output image. The ESARM module is set between the encoder and the decoder, and the SCC module is set between the second convolution layer of the encoder and the first convolution layer of the decoder. The generator network F is used for the conversion of the replica image to the color plate image. The generator network F generates a pseudo sample image fake_A based on the real sample image real_B. At the same time, the pseudo sample image fake_B can be converted into a color plate image through the generator network F, which is called a bidirectional cycle consistency path image Cycle_A. The generator network G is used for the conversion of the color plate image to the replica image. The pseudo sample image fake_A can be converted into a replica image through the generator network G, which is called a bidirectional cycle consistency path image Cycle_B. The improved CycleGAN network model ensures the reversibility of the transformation through bidirectional cycle consistency path images (Cycle_A and Cycle_B), and combines L2 reconstruction loss and adversarial loss to ensure the authenticity and accuracy of the generated images.

[0058] The discriminator network D y The part receives the fake sample image fake_B and the real sample image real_B as input, extracts features and distinguishes through multiple convolutional layers, and finally outputs the true and false discrimination result. Discriminator network D x The part receives the fake sample image fake_A and the real sample image real_A as input, performs feature extraction and discrimination through multiple convolutional layers, and finally outputs the true and false discrimination result.

[0059] like Figure 6 As shown, the input of the SCC module is the feature map F after the first two convolutional layers. Conv2∈R B×C×H×W , where B is the batch size, C is the number of channels, H and W are the height and width of the feature map, and Split refers to performing four parallel convolution operations on the original feature map, dividing it into four convolution branches. Four convolution kernel sizes (3×3, 5×5, 7×7, 9×9) and four different numbers of groups (1, 4, 8, 16) are used for parallel convolution operations. Because the convolution operation divides the original number of channels C by the number of groups, the number of channels of the feature maps of the four branches is C / 4, but it does not change the spatial dimensions H and W of the feature maps of the four branches. Therefore, the size of the feature maps calculated by the four convolution branches is C / 4×H×W. The feature maps calculated by the four convolution branches are connected to obtain a feature map of size C×H×W. Splicing the feature maps of the four convolution branches can fuse multi-scale information, enhance the expression ability and generalization performance of the model, and improve the image generation effect.

[0060] See also Figures 7 to 9 , the ESARM module contains the ESA module, and the input of the ESA module is the feature map F after the downsampling process Downsampling ∈R B×C×H×W , B is the batch size, C is the number of channels, H and W are the height and width of the feature map, respectively, the feature map F Downsampling ∈R B×C×H×W The corresponding features are obtained through five core components: global context encoding, local structure encoding, edge attention module, channel attention module and style statistical feature module. The global context encoding is responsible for capturing the global features in the feature map and extracting the overall semantic information and structural information. The local structure encoding focuses on capturing the local details and texture information in the feature map. The edge attention module emphasizes the edge information in the feature map to enhance the clarity and structural features of the generated image. The channel attention module adaptively adjusts the weights of each channel to highlight useful features and suppress redundant features. The style statistical feature module extracts style features (such as color distribution, texture, etc.) and applies them to image generation, and describes the style through statistical information (such as mean and variance), which helps to generate more consistent style features. Finally, the five features obtained are combined and output through the feature fusion module, which can significantly improve the effects in image generation and style transfer tasks and improve the results from different dimensions. Fig. 9 As shown, the feature map F Downsampling In the edge attention module, the feature map F1 is obtained by sequentially processing through a 3×3 convolution layer and a ReLU activation function, a 1×1 convolution layer and a Sigmoid activation function. The feature map F Downsampling In the channel attention module, the feature map F3 is obtained by adaptive flat pooling, 1×1 convolution layer and ReLU activation function, 1×1 convolution layer, and Sigmoid activation function. DownsamplingIn the style statistical feature module, after convolution, average pooling, standardization and other operations, the feature map F is extracted. Downsampling The visual style features in the feature map F2 are obtained, and the feature map F Downsampling , feature map F2 is multiplied element by element with feature map F3 to obtain feature map F6, which enhances key features, integrates content and style, and establishes associations between features. Feature map F6 is multiplied element by element with feature map F1 to obtain feature map F7, which further combines the image's semantic information (feature map F1) with the style information (feature map F6) to produce a feature representation that integrates content and style, allowing the model to better balance the expression of content and style when generating images. Feature map F Downsampling In the local structure encoding, the feature map F4 is obtained by sequentially passing through a 3×3 convolution layer, a ReLU activation function, and a 3×3 convolution layer. The feature map F Downsampling In the global context coding, the feature map F5 is obtained by adaptive flat pooling, 1×1 convolution layer and ReLU activation function, and 1×1 convolution layer. Since the number of channels of feature maps F7, feature maps F4 and feature maps F5 has not changed and is C, and the spatial size is also the same, the corresponding position elements of the three feature map matrices can be directly added, and the number of channels and size of the output feature map are the same as those of feature map F. Downsampling of consistency.

[0061] The input of the ESARM module is the feature representation output by the encoder. The feature representation output by the encoder is passed through two 3×3 convolutional layers as input. There is a batch normalization operation after each convolutional layer. The ReLU activation function is processed before the second convolutional layer. Finally, it is residually connected with the feature representation output by the input encoder as the input of the ESA module to enhance the nonlinear ability of the feature representation: through continuous convolution and ReLU activation, more complex and abstract features are extracted. The feature map obtained after the ESA module is residually connected with the feature representation output by the encoder again as the output of the ESARM module, ensuring that the key information of the original encoder output is not lost, so that the feature information at different levels can be further interacted and integrated.

[0062] Step 3: Use the oracle bone images in the extended training set to train the improved CycleGAN network model to obtain a trained oracle bone copy generation model.

[0063] The loss function in the training process mainly includes two types of losses, namely adversarial loss and cycle consistency loss. Adversarial loss is used to match the distribution of the generated replica image with the data distribution of the real replica, so as to ensure that the style of the generated replica image is consistent with the real replica image. Adversarial loss can only ensure that the replica image generated by the generator is distributed in the same way as the real replica image, but the generated replica cannot be distinguished from the real replica. Cycle consistency loss is needed to make the replicas generated by the two generators consistent.

[0064] The improved CycleGAN network model contains two generators and discriminators, corresponding to two adversarial losses. Generator network G: x→y and its discriminator D Y The corresponding adversarial loss LD Y for:

[0065]

[0066] Generator network F: y→x and its discriminator D X The corresponding adversarial loss LD X for:

[0067]

[0068] The L2 loss of cycle consistency loss is:

[0069]

[0070] Among them, S GAN (G,D Y , X, Y) is the loss function of converting the color image dataset sample x into the replica image y, represents sampling from the replica image dataset, log D Y (y) represents the model domain discriminator network D Y The logarithm of the predicted result for the true replica image. Denotes sampling from the color image dataset, D Y (G(x)) represents the model domain discriminator network D Y The predicted probability value of the replica image generated by the generator network G. S GAN (G,D X , X, Y) represents the loss function of converting the sample y of the replica image dataset into the color version image x. X (x) represents the color plate domain discriminator network D Y The predicted probability value for the real color plate image. X (G(y)) represents the color plate domain discriminator D X The predicted probability value of the color image generated by generator F. S cyc(G, F) represents the sum of the loss between the samples in the color image dataset, which are generated by the generator G and then regenerated into color images by generator F, and the loss between the samples in the replica image dataset, which are generated by the generator F and then regenerated into color images by generator G. F(G(x)) represents the samples in the color image dataset, which are generated by the generator G and then regenerated into color images by generator F. G(F(y)) represents the samples in the replica image dataset, which are generated by the generator F and then regenerated into replica images by generator G. x represents the samples in the color image dataset, and y represents the samples in the replica image dataset.

[0071] L2 loss S of cycle consistency loss cyc (G, F) consists of two parts: for a sample image x from a color image domain dataset X, the learned mapping function should be able to return the sample image x to the original image, that is: x→G(x)→F(G(x))≈x, which is called the forward cycle consistency loss; for a sample image y from a replica image domain dataset Y, the learned mapping function should be able to return the sample image y to the original image, that is: y→F(y)→G(F(y))≈y, which is called the backward cycle consistency loss.

[0072] The total loss function is expressed as follows:

[0073] S(G, F, D X , D Y )=S GAN (G,D Y ,x,y)+S GAN (F,D X , y, x)+λ cyc S cyc (G, F)

[0074] Among them, λ cyc is the cycle consistency loss weight, which is generally between 1.0 and 10.0.

[0075] Step 4: Input the oracle bone images in the test set into the trained oracle bone copy generation model for testing to obtain the translated oracle bone copy images.

[0076] The samples in the color plate test set are tested using the model-trained replica generation weights, and the color plate images are converted into replica images using the generator G to obtain the translated oracle bone replica images.

[0077] Step 5: Compare the model performance with existing methods and conduct a systematic evaluation to verify its effectiveness.

[0078] The present invention adopts the evaluation indicators SSIM, PSNR, FID and KID to evaluate the image generation quality. The method proposed in the present invention achieves significant improvement in these four quantitative evaluation indicators.

[0079] SSIM calculates the structural similarity between the generated image and the real image by comparing three indicators: brightness (L), contrast (C), and structure (S):

[0080] SSIM(x,y)=L(x,y)×C(x,y)×S(x,y)

[0081]

[0082] Among them, x represents the generated image, y represents the real image, μ x and μ y Represents the mean of the generated image x and the real image y, σ x and σ y Represents the variance of the generated image x and the real image y, σ xy represents the covariance of the generated image x and the real image y, b1, b2 and b3 are constants. L(x, y) is brightness, C(x, y) is contrast, and S(x, y) is structure.

[0083] PSNR evaluates image quality by comparing the peak signal-to-noise ratio between the original image and the reconstructed image; it is usually measured in decibels, with higher values ​​indicating better image quality; the formula is as follows:

[0084]

[0085] Where MSE is the mean square error, A(i, j) represents the pixel in the i-th row and j-th column of the original image; B(i, j) represents the pixel in the i-th row and j-th column of the original image; F(i, j) represents the pixel in the i-th row and j-th column of the reconstructed (generated) image; r represents the peak value of the fused image. The larger the peak signal-to-noise ratio, the closer the generated replica image is to the real replica image.

[0086] FID evaluates the quality of generated images by comparing the statistical characteristics of generated images and real images in the feature space of the pre-trained Inception network. The lower the FID value, the closer the distribution of the generated image is to the distribution of the real image, and the higher the quality of the generated image. The formula is as follows:

[0087]

[0088] Among them, μ real and μ fake Respectively represent the mean vector of the data distribution of the real replica image and the generated replica image, Tr represents the trace operation of the matrix, and C realand C fake Represent the covariance matrices of the real replica image and the generated replica image data respectively.

[0089] KID measures the difference between two groups of samples by calculating the square of the maximum mean difference between the features; the smaller the KID score, the stronger the visual similarity between the real image and the generated image; the formula is as follows:

[0090] KID(x, y) = || E x [Φ(x)]-E y [Φ(y)]|| 2

[0091] Where x and y represent the batches of real replica images and generated replica images, respectively, Φ represents the feature extraction function that maps the image to the embedding space, and E x and E y Represent the expected values ​​of images x and y respectively.

[0092] Step 6: Conduct ablation experiments to verify the effectiveness of the SCC module and the ESA module and their effect on improving the quality of the generated model image.

[0093] For details, see Fig.10 ,Further ,analyze the qualitative results and compare the model performance with ,existing methods. Fig.10It can be seen that compared with the existing methods, the model proposed in the present invention can not only maintain the complete topological structure of the characters, but also achieve high-fidelity reconstruction of detailed features. The edge clarity and continuity generated are closest to the standard copy, which is significantly better than other methods. The image evaluation results are shown in Table 1, where HED is the abbreviation of Holistically-nested edge detection and pix2pix is ​​the abbreviation of image-to-image translation with conditional adversarial networks. From the SSIM results in Table 1, it can be seen that the SSIM index of the present invention reaches 0.7149, which is 50.99% and 5.85% higher than the Canny and CycleGAN network methods, respectively. In addition, in terms of the PSNR index, the value of the method of the present invention is 19.7394, which is much higher than the Canny value of 1..8608 and the pix2pix value of 8.4437, which further confirms the advantages of the present invention in improving the quality of the generated copy image and suppressing the background noise interference of the oracle bone image. In terms of the two image quality evaluation indicators FID and KID×100, the method of the present invention reached the minimum values ​​of 42.61 and 3.54 respectively, indicating that the difference between the generated copy image and the real copy image is extremely small, and the image generation quality and attribute accuracy are significantly improved, demonstrating excellent oracle bone copy image generation capabilities.

[0094] Table 1 Objective index evaluation results of the present invention and other methods

[0095]

[0096]

[0097] For details, see Fig.11 Through a set of experiments, the effects of copy generation with and without the SCC module were compared. Fig.11 It can be seen that the local detail features of the generated image are more obvious after adding the SCC module, which improves the quality of the generated replica. The experimental results are shown in Table 2. After the introduction of the SCC module, the SSIM value increased from 0.6987 to 0.7149, an increase of about 2.3%, indicating that the structural fidelity of the generated replica image is better; the PSNR value increased from the baseline level to 19.7394dB, an increase of about 7.6%, indicating a significant improvement in the overall image quality; the FID value decreased by about 5%, from 44.87 to a lower level, reflecting that the generated replica image is closer to the real data distribution; the KID value decreased by about 6.3%, further confirming the advantages of the method in generating realism and detail fidelity. The experimental results consistently show the effectiveness of the SCC module and its excellent improvement in the quality of replica generation.

[0098] Table 2 SCC module experimental results

[0099]

[0100] For details, see Fig.12 Through a set of experiments, the effects of model generation with and without the ESA module were compared. Fig.12 It can be seen that after adding the ESA module, the generated image quality is higher, the key features of the oracle bones are more obvious, and the background noise interference of the oracle bone images is effectively suppressed. The experimental results are shown in Table 3. After the introduction of the ESA module, the SSIM value increased from 0.6859 to 0.7149, an increase of about 4.2%, indicating that the structural fidelity of the generated replica image is better; the PSNR value increased from the baseline level to 19.7394dB, an increase of about 17%, indicating a significant improvement in the overall image quality; the FID value decreased by about 6.7%, from 45.69 to a lower level, reflecting that the generated replica image is closer to the real data distribution; the KID value decreased by about 9.7%, further confirming the advantages of the method in generating realism and detail fidelity. The experimental results consistently show the effectiveness of the ESA module and its excellent effect on improving the quality of replica generation.

[0101] Table 3 ESA module experimental results

[0102]

[0103] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for automatically generating oracle bone copies based on an improved CycleGAN network, characterized in that: The steps are as follows: Step 1: Scan the books with a non-contact file and book scanner to obtain oracle bone inscription images, pre-process the oracle bone inscription images to obtain oracle bone images, obtain an oracle bone dataset, and divide the oracle bone dataset into a training set and a test set; Step 2: Build an improved CycleGAN network model: add a multi-scale feature extraction module and a style enhancement attention residual module to the generator network; Step 3: Use the oracle bone images in the training set to train the improved CycleGAN network model to obtain a trained oracle bone copy generation model; Step 4: Input the oracle bone images in the test set into the trained oracle bone copy generation model to obtain the translated oracle bone copy images.

2. The method for automatically generating oracle bone copies based on the improved CycleGAN network according to claim 1, characterized in that: The method for obtaining the oracle bone data set is: respectively intercepting two types of oracle bone images, the color version and the copy, from the oracle bone inscription images, and the color version and the copy form the oracle bone data set; The oracle bone dataset is divided into a training set and a test set in a ratio of 7:

3. The oracle bone images in the training set are then subjected to data enhancement methods such as geometric transformation, contrast enhancement, sharpness enhancement or Gaussian filtering to obtain an extended training set. The improved CycleGAN network model is trained using the extended training set. The evaluation indicators SSIM, PSNR, FID and KID are used to evaluate the image generation quality of the oracle bone model generation model.

3. The method for automatically generating oracle bone copies based on the improved CycleGAN network according to claim 1 or 2, characterized in that: The improved CycleGAN network model includes two sets of generator networks G and F and a discriminator network D. x and the discriminator network D y The two generator networks F are responsible for generating color oracle bone images from the replica oracle bone images, the generator network G is responsible for generating replica oracle bone images from the color oracle bone images, and the discriminator network D x The adversarial loss LD is used to compare the real sample images of the color plates in the oracle bone dataset with the color plate oracle bone images generated by the generator network F. X , discriminator network D y The adversarial loss LD is used to compare the real sample images of the replicas in the oracle bone dataset with the replica oracle bone images generated by the generator network G. Y .

4. The method for automatically generating oracle bone copies based on the improved CycleGAN network according to claim 3, characterized in that: The color version oracle bone image in the oracle bone data set is used as the real sample image real_A, and the pseudo sample image fake_B of the copy is obtained through the generator network G. The pseudo sample image fake_B obtains the bidirectional cycle consistency path image Cycle_A of the color version through the generator network F. The difference between the bidirectional cycle consistency path image Cycle_A and the real sample image real_A is measured by calculating the mean square error between the bidirectional cycle consistency path image Cycle_A and the real sample image real_A; the real sample image real_B of the copy in the oracle bone data set is obtained through the generator network F. The pseudo sample image fake_A obtains the bidirectional cycle consistency path image Cycle_B of the copy through the generator network G. The real sample image real_B and the bidirectional cycle consistency path image Cycle_B are compared to learn to distinguish between the real sample image and the generated image; The discriminator network D y The part receives the fake sample image fake_B and the real sample image real_B as input, extracts and distinguishes features through multiple convolutional layers, and finally outputs the true and false discrimination result; the discriminator network D x The part receives the fake sample image fake_A and the real sample image real_A as input, performs feature extraction and discrimination through multiple convolutional layers, and finally outputs the true and false discrimination result.

5. The method for automatically generating oracle bone copies based on the improved CycleGAN network according to claim 3, characterized in that: The generator network G and the generator network F both adopt an encoder-decoder architecture. The encoder converts the real sample image into feature representation through convolution layers and downsampling, and the decoder reconstructs the features into pseudo sample images using transposed convolution layers and upsampling. The generator adopts a parallel manner to add the designed multi-scale feature extraction module and style-enhanced attention residual module to achieve progressive feature conversion.

6. The method for automatically generating oracle bone copies based on the improved CycleGAN network according to claim 4 or 5, characterized in that: The style-enhanced attention residual module is set between the encoder and the decoder of the generator network G and the generator network F, and the multi-scale feature extraction module is set between the second convolution layer of the encoder and the first convolution layer of the decoder.

7. The method for automatically generating oracle bone copies based on the improved CycleGAN network according to claim 6, characterized in that: The input of the multi-scale feature extraction module is the feature map F after the first two convolutional layers. Conv2 ∈R B×C×H×W , the feature map F Conv2 It is divided into four convolution branches, which perform four parallel convolution operations with different convolution kernel sizes and different numbers of groups. The feature maps obtained by the four convolution branches are spliced ​​to obtain the feature map F Conv2 Output feature maps of the same size; where B is the batch size, C is the number of channels, and H and W are the height and width of the feature map.

8. The method for automatically generating oracle bone copies based on the improved CycleGAN network according to claim 6, characterized in that: The input of the style enhanced attention residual module is the feature representation output by the encoder. The feature representation output by the encoder is used as the input of the style enhanced attention residual module and then passed through two 3×3 convolutional layers for convolution operation. There is a batch normalization operation after each convolutional layer, and a ReLU activation function is processed before the second convolutional layer. The result obtained by the second convolutional layer is residually connected with the feature representation output by the input encoder as the input of the ESA module. The ESA module obtains the corresponding feature map by respectively passing the input feature map through global context encoding, local structure encoding, edge attention module, channel attention module and style statistical feature module. The corresponding feature maps are fused to obtain a fused feature map, which is residually connected with the feature representation output by the encoder again as the output of the ESARM module.

9. The method for automatically generating oracle bone copies based on the improved CycleGAN network according to claim 8, characterized in that: The input of the ESA module is the downsampled feature map F Downsampling ∈R B×C×H×W , where B is the batch size, C is the number of channels, H and W are the height and width of the feature map respectively; the feature map F Downsampling In the edge attention module, the feature map F1 is obtained by sequentially processing through a 3×3 convolution layer and a ReLU activation function, a 1×1 convolution layer and a Sigmoid activation function. The feature map F Downsampling In the channel attention module, the feature map F3 is obtained by adaptive flat pooling, 1×1 convolution layer and ReLU activation function, 1×1 convolution layer, and Sigmoid activation function. Downsampling In the style statistical feature module, convolution, average pooling, and standardization operations are performed in sequence to extract the feature map F. Downsampling The visual style features in the feature map F2 are obtained, and the feature map F Downsampling , feature map F2 and feature map F3 are multiplied element by element to obtain feature map F6, feature map F6 and feature map F1 are multiplied element by element to obtain feature map F7; feature map F Downsampling In the local structure coding, the feature map F4 is obtained by sequentially passing through a 3×3 convolution layer, a ReLU activation function, and a 3×3 convolution layer. The feature map F Downsampling In the global context coding, the feature map F5 is obtained by adaptive flat pooling, 1×1 convolution layer and ReLU activation function, and 1×1 convolution layer in sequence. The corresponding position elements of feature map F7, feature map F4 and feature map F5 are added to obtain a fused feature map.

10. The method for automatically generating oracle bone copies based on the improved CycleGAN network according to claims 4, 5, 7-8, characterized in that: In step 4, the color oracle bone images in the test set are tested using the replica generation weights trained in the oracle bone replica generation model, and the color oracle bone images are converted into replica oracle bone images using the generator network G to obtain translated oracle bone replica images; The loss function during training includes adversarial loss and cycle consistency loss, the generator network G:x→y and its discriminator network D Y The corresponding adversarial loss LD Y for: Generator network F:y→x and its discriminator network D X The corresponding adversarial loss LD X for: The cycle consistency loss L2 Loss is: Among them, S GAN (G,D Y ,X,Y) is the loss function of converting the color sample image x in the color image dataset to the replica sample image y in the replica image dataset. represents sampling from the replica image dataset, logD Y (y) represents the discriminator network D Y The logarithm of the predicted result for the true replica image; Denotes sampling from the color image dataset, D Y (G(x)) represents the discriminator network D Y The predicted probability value of the replica image generated by the generator network G, S GAN (G,D X ,X,Y) represents the loss function of converting the replica sample image y into the color sample image x in the replica image dataset, D X (x) represents the discriminator network D Y The predicted probability value of the real color plate image, D X (G(y)) represents the discriminator network D X The predicted probability value of the color image generated by the generator network F, S cyc (G, F) represents the sum of the losses of the sample images of the color plate image dataset, which are generated by the generator network G and then regenerated by the generator network F, and the real color plate image, and the color plate image generated by the generator network F and then regenerated by the generator network G. F(G(x)) represents the sample images in the color plate image dataset, which are generated by the generator network G and then regenerated by the generator network F; G(F(y)) represents the sample images in the replica image dataset, which are generated by the generator F and then regenerated by the generator G. The total loss function is: S(G,F,D X ,D Y )=S GAN (G,D Y ,x,y)+S GAN (F,D x ,y,x)+λ cyc S cyc (G,F) Among them, λ cyc is the cycle consistency loss weight.

Images