An oracle bone rubbing automatic generation method based on an improved CycleGAN network

Abstract

The application provides an oracle bone copy automatic generation method based on an improved CycleGAN network, and steps are as follows: an oracle bone recording image is obtained by scanning a book by using a non-contact volume book scanner, the oracle bone recording image is pretreated to obtain an oracle bone image, an oracle bone data set is obtained, and the oracle bone data set is divided into a training set and a test set; an improved CycleGAN network model is constructed: a multi-scale feature extraction module and a style strengthening attention residual module are added in a generator network; the oracle bone image in the training set is used to train the improved CycleGAN network model, and a trained oracle bone copy generation model is obtained; the oracle bone image in the test set is input into the trained oracle bone copy generation model, and a translated oracle bone copy image is obtained. The application can accurately capture and reproduce the structure and details of oracle bone script, generates a high-quality copy, provides an efficient image generation method for oracle bone script digital research, and provides a new idea for generating a high-quality oracle bone copy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence and computer vision, and particularly relates to a method for automatically generating oracle bone rubbings. BACKGROUND

[0002] As an important carrier for recording the shape and details of oracle bone characters, oracle bone rubbings have important academic value in the study of the origin and evolution of Chinese characters. Oracle bone inscriptions are a key part of the development of Chinese characters and have extremely important value in exploring the origin, development and evolution of ancient characters. Due to the long history of oracle bone inscriptions, the phenomenon of original pieces being lost is relatively common, and rubbings have become extremely important data in the study of oracle bone inscriptions. However, the existing oracle bone rubbings are rough and erroneous to some extent, which not only reduces the accuracy of the study of oracle bone inscriptions, but also may lead to deviation in the interpretation of ancient history and culture.

[0003] Traditional oracle bone rubbing production mainly adopts two technologies of line drawing extraction and image style conversion. Line drawing extraction focuses on the extraction of image edge contours, extracts lines and contours from oracle bone images to generate corresponding rubbings, while image style conversion focuses on applying the style of one image to another image by using machine learning algorithms to generate a new image, and the generated image combines the content of the original image and the style of the target image. The above methods have achieved good results, but the traditional method can only handle simple texture level conversion and highly relies on manual processing by oracle bone experts and scholars, and the recognition efficiency is low. Moreover, the traditional rubbing generation method has limitations in detail reproduction, and this problem is particularly evident when facing a large number of samples.

[0004] With the rapid development of deep learning technology, line drawing extraction and image style conversion technology have made significant progress, providing strong technical support for the digitization of oracle bone inscription research and significantly improving the work efficiency of rubbing generation, but there is still room for improvement in the stability of style transfer and semantic content preservation. One of the main methods of image style conversion is based on generative adversarial networks (GAN), however, during the training process, the GAN network is unstable, which may lead to the generated oracle bone rubbings showing discontinuity in shape and edge area, and the problem of insufficient detail reproduction. In addition, for the interference factors such as background noise and blur commonly existing in oracle bone images, cycle-consistent adversarial networks (CycleGAN) have obvious limitations in feature recognition and noise suppression. The traditional CycleGAN network has limited ability to capture fine-grained style features and structural information, and the traditional single attention module is difficult to balance the unity of global style and local details.

[0005] The invention patent with application number 202110888155.3 discloses a method for automatically extracting oracle bone piece outlines and character traces based on a generative adversarial network, which comprises the following steps: arranging pairs of oracle bone copy images and oracle bone rubbing images, constructing a copy generation dataset, designing and constructing a generative adversarial network model OBTGAN, training the generative model using the cycle training method of the CycleGAN model, and obtaining a generated copy image using the trained generative adversarial network. The generator in the OBTGAN uses a residual network module, inserts a SpatialAttention layer before the third downsampling, and uses a residual network embedded with a ChannelAttention structure to extract features again after downsampling. The upsampling module uses a Unet structure to perform skip connection between the low-level semantic features in the downsampling process and the generated feature matrix in the upsampling. The above method uses computer technology to replace the manual tracing step of copy making, and the generated copy has higher accuracy in character details, greatly reducing the oracle bone copy making time and providing the possibility of batch production of oracle bone copies. The model of the above patent has limited generalization ability and cannot accurately extract the outlines and character traces of different types of oracle bone pieces; the model has limited detail extraction capability and cannot completely restore all details on the rubbing, such as scratches and defects; the extraction effect may be affected for rubbings with poor image quality such as noise and blur. SUMMARY

[0006] In view of the technical problems of the traditional copy generation method having limitations in detail reproduction and low recognition efficiency, the present application provides an oracle bone copy automatic generation method based on an improved CycleGAN network, which can accurately capture and reproduce the structure and details of oracle bone script and generate more accurate and clear oracle bone copies.

[0007] To achieve the above purpose, the technical scheme of the present application is as follows: an oracle bone copy automatic generation method based on an improved CycleGAN network, which comprises the following steps:

[0008] Step 1: using a non-contact archive book scanner to scan the book to obtain an oracle bone recording image, pre-processing the oracle bone recording image to obtain an oracle bone image, obtaining an oracle bone dataset, and dividing the oracle bone dataset into a training set and a test set;

[0009] Step 2: constructing an improved CycleGAN network model: adding a multi-scale feature extraction module and a style enhancement attention residual module in the generator network;

[0010] Step 3: training the improved CycleGAN network model using the oracle bone images in the training set to obtain a trained oracle bone copy generation model;

[0011] Step 4: input the oracle bone image in the test set into the trained oracle bone transcript generation model to obtain the translated oracle bone transcript image.

[0012] Preferably, the method for obtaining the oracle bone data set is: respectively cutting the color edition and transcript two kinds of oracle bone images from the oracle bone catalog image, and the color edition and transcript two kinds of oracle bone images constitute the oracle bone data set.

[0013] Divide the oracle bone data set into a training set and a test set at a ratio of 7:3, and then perform data enhancement methods of geometric transformation, contrast enhancement, sharpness enhancement or Gaussian filtering on the oracle bone images in the training set to obtain an expanded training set; train the improved CycleGAN network model constructed by using the expanded training set;

[0014] Adopt evaluation indexes SSIM, PSNR, FID and KID to evaluate the image generation quality of the oracle bone transcript generation model.

[0015] Preferably, the improved CycleGAN network model comprises two sets of generator network G and generator network F and discriminator network D x and discriminator network D y The two generator networks F are responsible for generating color edition oracle bone images from transcript oracle bone images, the generator network G is responsible for generating transcript oracle bone images from color edition oracle bone images, and the discriminator network D x is used to compare the real sample images of the color edition in the oracle bone data set with the color edition oracle bone images generated by the generator network F to obtain an adversarial loss LD X , and the discriminator network D y is used to compare the real sample images of the transcript in the oracle bone data set with the transcript oracle bone images generated by the generator network G to obtain an adversarial loss LD Y .

[0016] Preferably, the color edition oracle bone images in the oracle bone data set are used as real sample images real_A to obtain transcript pseudo sample images fake_B through the generator network G, the pseudo sample images fake_B are used to obtain color edition bidirectional cycle consistency path images Cycle_A through the generator network F, and the difference between the bidirectional cycle consistency path images Cycle_A and the real sample images real_A is measured by calculating the mean square error between the bidirectional cycle consistency path images Cycle_A and the real sample images real_A; the real sample images real_B of the transcript in the oracle bone data set are used to obtain color edition pseudo sample images fake_A through the generator network F, the pseudo sample images fake_A are used to obtain transcript bidirectional cycle consistency path images Cycle_B through the generator network G, and the real sample images and generated images are learned to be distinguished by comparing the real sample images real_B and the bidirectional cycle consistency path images Cycle_B;

[0017] The discriminator network D y Part of the received fake sample image fake_B and real sample image real_B as input, through multiple convolution layers for feature extraction and discrimination, finally output true and false discrimination results; the discriminator network D x Part of the received fake sample image fake_A and real sample image real_A as input, through multiple convolution layers for feature extraction and discrimination, finally output true and false discrimination results.

[0018] Preferably, the generator network G and the generator network F both adopt the architecture of encoder-decoder, the encoder converts the real sample image into feature representation through convolution layer and down sampling, the decoder reconstructs the feature into the fake sample image by using transpose convolution layer and up sampling, and the generator adopts parallel mode to add designed multi-scale feature extraction module and style enhancement attention residual module to realize progressive feature conversion.

[0019] Preferably, the style enhancement attention residual module is arranged between the encoder and the decoder of the generator network G and the generator network F, and the multi-scale feature extraction module is arranged between the second convolution layer of the encoder and the first convolution layer of the decoder.

[0020] Preferably, the input of the multi-scale feature extraction module is the feature map F Conv2 ∈R B ×C×H×W The feature map F Conv2 is divided into four convolution branches to perform parallel convolution operation of four different convolution kernel sizes and different group numbers respectively, the feature maps obtained by the four convolution branches are spliced to obtain an output feature map with the same size as the feature map F Conv2 ; wherein B is batch size, C is channel number, H and W are height and width of the feature map.

[0021] Preferably, the input of the style enhancement attention residual module is the feature representation output by the encoder, and the feature representation output by the encoder is input into the style enhancement attention residual module, then the input feature representation is convoluted by two 3x3 convolution layers, each convolution layer is followed by a batch normalization operation, the second convolution layer is followed by a ReLU activation function, and the result of the second convolution layer is connected with the input feature representation output by the encoder to form the input of the ESA module. The ESA module respectively processes the input feature map by global context encoding, local structure encoding, edge attention module, channel attention module and style statistical feature module to obtain corresponding feature maps, fuses the corresponding feature maps to obtain a fused feature map, and then connects the fused feature map with the feature representation output by the encoder to form the output of the ESARM module.

[0022] Preferably, the input of the ESA module is the down-sampled feature map F Downsampling ∈R B×C×H×W , where B is the batch size, C is the number of channels, H and W are the height and width of the feature map respectively; the feature map F Downsampling In the edge attention module, the feature map F1 is obtained by sequentially processing through the 3x3 convolution layer and the ReLU activation function, the 1x1 convolution layer and the Sigmoid activation function; the feature map F Downsampling In the channel attention module, the feature map F3 is obtained by sequentially processing through the adaptive average pooling, the 1x1 convolution layer and the ReLU activation function, the 1x1 convolution layer and the Sigmoid activation function; the feature map F Downsampling In the style statistical feature module, the feature map F2 is obtained by sequentially processing through the convolution, the average pooling and the standardization operation, extracting the visual style feature in the feature map F Downsampling The feature map F2 is obtained by sequentially processing through the convolution, the average pooling and the standardization operation, extracting the visual style feature in the feature map F Downsampling The feature map F6 is obtained by element-wise multiplication of the feature map F2 and the feature map F3; the feature map F7 is obtained by element-wise multiplication of the feature map F6 and the feature map F1; the feature map F Downsampling In the local structure encoding, the feature map F4 is obtained by sequentially processing through the 3x3 convolution layer and the ReLU activation function, the 3x3 convolution layer; the feature map F Downsampling In the global context encoding, the feature map F5 is obtained by sequentially processing through the adaptive average pooling, the 1x1 convolution layer and the ReLU activation function, the 1x1 convolution layer; the fusion feature map is obtained by adding the corresponding position elements of the feature map F7, the feature map F4 and the feature map F5.

[0023] Preferably, in step 4, the trained copy generation weight in the oracle bone copy generation model is used to test the color plate oracle bone image in the test set, the generator network G is used to convert the color plate oracle bone image into a copy oracle bone image, and the translated oracle bone copy image is obtained;

[0024] The loss function in the training process includes the adversarial loss and the cycle consistency loss, the generator network G: x→y and its discriminator network D Y The corresponding adversarial loss LD Y is:

[0025]

[0026] The generator network F: y→x and its discriminator network D X The corresponding adversarial loss LD X is:

[0027]

[0028] The cycle consistency loss L2 Loss is:

[0029]

[0030] where S GAN (G,D Y ,X,Y) is a loss function of converting a color sample image x in the color image dataset into a copy sample image y in the copy image dataset, denotes sampling from the copy image dataset, log D Y (y) denotes the predicted probability value of the discriminator network D Y on the real copy image; denotes sampling from the color image dataset, D Y (G(x)) denotes the predicted probability value of the discriminator network D Y on the copy image generated by the generator network G, S GAN (G,D X ,X,Y) denotes a loss function of converting a copy sample image y in the copy image dataset into a color sample image x, D X (x) denotes the predicted probability value of the discriminator network D Y on the real color image, D X (G(y)) denotes the predicted probability value of the discriminator network D X on the color image generated by the generator network F, S cyc (G,F) denotes the sum of the loss of the copy image generated by the generator network G from the color sample image in the color image dataset and then regenerated by the generator network F from the copy image, and the loss of the copy image generated by the generator network F from the sample in the copy image dataset and then regenerated by the generator network G from the copy image, F(G(x)) denotes the copy image generated by the generator network G from the sample in the color image dataset and then regenerated by the generator network F from the copy image; G(F(y)) denotes the copy image generated by the generator F from the sample in the copy image dataset and then regenerated by the generator G from the copy image;

[0031] The total loss function is:

[0032] S(G,F,D X ,D Y )=S GAN (G,D Y ,x,y)+S GAN (F,D X ,y,x)+λ cyc S cyc (G,F)

[0033] where λ cyc is the cycle consistency loss weight.

[0034] Compared with the prior art, the present application has the beneficial effects: the present application is based on the idea of image style transfer, uses an improved CycleGAN network to construct a oracle bone rubbing style transfer model, and optimizes the network structure, by adding a multi-scale feature extraction module (Split Conv Concat, SCC) in the initial CycleGAN model, local detail features and global semantic information can be captured at different scales at the same time, realizing the synchronous capture of cross-scale local details and global semantic information, and then significantly improving the quality of the generated oracle bone rubbing. In addition, the present application also adds a style enhancement attention residual module (Enhanced Style Attention Residual Module, ESARM) in the generator of the network, realizes the selective enhancement of the key features of oracle bone and effectively suppresses the background noise interference of oracle bone images. The present application can accurately capture and reproduce the structure and details of oracle bone script, generate high-quality rubbing, provide an efficient image generation method for oracle bone script digital research, and provide a new idea for generating high-quality oracle bone rubbing.

[0035] The experimental results show that, compared with other methods, the present application has significant improvement in the four quantitative evaluation indexes of SSIM, PSNR, FID and KID. Further analysis of the qualitative results shows that the present application effectively maintains the topological integrity of the characters and realizes the highly accurate reconstruction of the details, and the edge definition and continuity are closest to the standard rubbing. The present application is suitable for high-standard generation of oracle bone rubbing, and provides strong support for ancient character research. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0037] Figure 1 The flowchart of the present application.

[0038] Figure 2 The example diagram of part of the high-definition catalog image.

[0039] Figure 3 The example diagram of part of the oracle bone image in the oracle bone data set.

[0040] Figure 4 The framework diagram of the improved CycleGAN network of the present application.

[0041] Figure 5 The single-branch framework diagram of the improved CycleGAN network of the present application.

[0042] Figure 6 Data processing detail diagram of the SCC module of the application.

[0043] Figure 7 Structure diagram of the ESARM module of the application.

[0044] Figure 8 Structure diagram of the ESA module of the application.

[0045] Figure 9 Data processing detail diagram of the ESA module of the application.

[0046] Figure 10 Comparison diagram of the copy generation result of the application and the copy generation result of other methods, wherein (a) is a color edition, (b) is a copy, (c) is a Canny algorithm, (d) is a HED algorithm, (e) is a pix2pix algorithm, (f) is a CycleGAN network, and (g) is the application.

[0047] Figure 11 Comparison diagram of the copy generation effect of experiments with and without the SCC module, wherein (a) is a color edition, (b) is a copy, (c) is without the SCC module, and (d) is the application.

[0048] Figure 12 Comparison diagram of the copy generation effect of experiments with and without the ESA module, wherein (a) is a color edition, (b) is a copy, (c) is without the ESA module, and (d) is the application. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the application.

[0050] As shown in Figure 1 An oracle bone copy automatic generation method based on an improved CycleGAN network specifically includes the following steps:

[0051] Step 1: Researchers use a professional non-contact archive book scanner to scan books to obtain high-definition oracle bone recording images, and obtain oracle bone images by preprocessing the obtained oracle bone recording images. The oracle bone images form an oracle bone data set, the oracle bone data set includes a test set and a training set, and the oracle bone images in the training set are subjected to data enhancement to obtain an expanded training set.

[0052] Specifically, refer to Figure 2Obtaining high-definition oracle inscription images, and respectively cutting out the color edition and the copy from the high-definition oracle inscription images, as shown in the color edition and the copy in the oracle bone image in Figure 3 The oracle bone data set is divided into a training set and a test set in a ratio of 7:3, and the oracle bone images in the training set are subjected to data enhancement methods such as geometric transformation, contrast enhancement, sharpness enhancement, and Gaussian filtering, to finally obtain an expanded training set. The above data enhancement methods can improve the accuracy and robustness of the model, reduce overfitting, and adapt to different application scenarios and task requirements.

[0053] Step 2: Constructing an improved CycleGAN network model: adding a multi-scale feature extraction module and a style-enhanced attention residual module in the generator network.

[0054] The multi-scale feature extraction module (SCC) in the improved CycleGAN network model simultaneously captures local detail features and global semantic information of the oracle bone color edition image at different scales. The style-enhanced attention residual module (ESARM) realizes selective enhancement of key features of the oracle bone and effectively suppresses background noise interference of the oracle bone color edition image, generating more accurate and clear oracle bone copies, which provides important clues for in-depth exploration of the origin and development of ancient Chinese characters.

[0055] As shown in Figure 4 The improved CycleGAN network model includes two groups of networks, a generator network G and a generator network F, and a discriminator network D x and a discriminator network D y The data in the improved CycleGAN network model includes real sample images (real_A, real_B) and generated pseudo sample images (fake_A, fake_B). The real sample images are oracle bone images of the oracle bone data set, and the pseudo samples are images generated by the generator. The real sample image real_A is converted into a pseudo sample image fake_B through the generator network G, the pseudo sample image fake_B is converted into a bidirectional cyclic consistency path image Cycle_A through the generator network F, and the difference between the bidirectional cyclic consistency path image Cycle_A and the real sample image real_A is measured by calculating the mean square error between them. The real sample image real_B is converted into a pseudo sample image fake_A through the generator network F, and the pseudo sample image fake_A is converted into a bidirectional cyclic consistency path image Cycle_B through the generator network G. The discriminator D y learns to distinguish between real sample images and generated images by comparing the real sample image real_B and the bidirectional cyclic consistency path image Cycle_B.

[0056] The real sample image real_B is used to generate the fake sample image fake_A by the generator network F, and the discriminator network D x The adversarial loss LD is obtained by comparing the real sample image real_A and the fake sample image fake_A X The real sample image real_A is used to generate the fake sample image fake_B by the generator network G, and the discriminator network D y The adversarial loss LD is obtained by comparing the real sample image real_B and the fake sample image fake_B Y The two generator networks F are responsible for generating the color plate image, and the generator network G is responsible for generating the copy image.

[0057] As shown in Figure 5 , the generator network G adopts an encoder-decoder architecture, the encoder converts the real sample image real_A into a feature representation through convolution layers and down-sampling, and the decoder reconstructs the features into the fake sample image fake_B using transpose convolution layers and up-sampling, and the generator adopts a parallel way to add the designed SCC module and ESARM module to realize the progressive feature conversion from the input image to the output image. The ESARM module is arranged between the encoder and the decoder, and the SCC module is arranged between the second convolution layer of the encoder and the first convolution layer of the decoder. The generator network F is used for the conversion of the copy image to the color plate image, and the generator network F generates the fake sample image fake_A according to the real sample image real_B, while the fake sample image fake_B can be converted into a color plate image by the generator network F, which is called a bidirectional cycle consistency path image Cycle_A. The generator network G is used for the conversion of the color plate image to the copy image, and the fake sample image fake_A can be converted into a copy image by the generator network G, which is called a bidirectional cycle consistency path image Cycle_B. The improved CycleGAN network model guarantees the reversibility of the conversion through the bidirectional cycle consistency path image (Cycle_A and Cycle_B), and combines the L2 reconstruction loss and the adversarial loss to guarantee the authenticity and accuracy of the generated image.

[0058] The discriminator network D y part receives the fake sample image fake_B and the real sample image real_B as input, extracts and discriminates features through multiple convolution layers, and finally outputs the true or false discrimination result. The discriminator network D x part receives the fake sample image fake_A and the real sample image real_A as input, extracts and discriminates features through multiple convolution layers, and finally outputs the true or false discrimination result.

[0059] As shown in Figure 6 , the input of the SCC module is the feature map F Conv2∈R B×C×H×W where B is the batch size, C is the number of channels, H and W are the height and width of the feature map, and Split refers to splitting the original feature map into four convolution branches for parallel convolution operations. Four kernel sizes (3x3, 5x5, 7x7, 9x9) and four different group numbers (1, 4, 8, 16) are used for parallel convolution operations. Since the convolution operation divides the original channel number C by the group number, the channel number of the feature map of the four branches is C / 4, but the spatial dimensions H and W of the feature map of the four branches do not change, so the size of the feature map calculated by the four convolution branches is C / 4xHxW. The feature maps calculated by the four convolution branches are connected to obtain a feature map with a size of CxHxW. The feature maps of the four convolution branches can be spliced to fuse multi-scale information, enhance the expression ability and generalization performance of the model, and improve the image generation effect.

[0060] Referring to Figures 7 to 9 , the ESA module in the ESARM module includes an ESA module, and the input of the ESA module is a feature map F Downsampling ∈R B×C×H×W , B is the batch size, C is the number of channels, H and W are the height and width of the feature map, and F Downsampling ∈R B×C×H×W goes through five core components: global context encoding, local structure encoding, edge attention module, channel attention module, and style statistical feature module to obtain corresponding features. Global context encoding is responsible for capturing global features in the feature map and extracting overall semantic information and structural information. Local structure encoding focuses on capturing local details and texture information in the feature map. The edge attention module emphasizes the edge information in the feature map, enhancing the clarity and structural features of the generated image. The channel attention module adaptively adjusts the weights of each channel, highlighting useful features and suppressing redundant features. The style statistical feature module extracts style features (such as color distribution, texture, etc.) and applies them to image generation, and describes the style through statistical information (such as mean and variance), which helps to generate more consistent style features. Finally, the five kinds of features obtained are combined through a feature fusion module and output, which can significantly improve the effect in image generation and style transfer tasks and improve the results from different dimensions. As shown in Figure 9 , the feature map F Downsampling goes through a 3x3 convolution layer and a ReLU activation function, a 1x1 convolution layer and a Sigmoid activation function in the edge attention module to obtain a feature map F1, and the feature map F Downsampling goes through adaptive average pooling, a 1x1 convolution layer and a ReLU activation function, a 1x1 convolution layer and a Sigmoid activation function in the channel attention module to obtain a feature map F3, and the feature map F DownsamplingIn the style statistical feature module, after convolution, average pooling, standardization and other operations, the feature map F is extracted. Downsampling The visual style features in the feature map F2 are obtained, and the feature map F Downsampling , feature map F2 is multiplied element-by-element with feature map F3 to obtain feature map F6, which enhances key features, integrates content and style, and establishes associations between features. Feature map F6 is multiplied element-by-element with feature map F1 to obtain feature map F7, which further combines the semantic information of the image (feature map F1) with the style information (feature map F6) to produce a feature representation that integrates content and style, allowing the model to better balance the expression of content and style when generating images. Feature map F Downsampling In the local structure coding, the feature map F4 is obtained by sequentially passing through the 3×3 convolution layer and the ReLU activation function, and the 3×3 convolution layer. The feature map F Downsampling In the global context encoding, the feature map F5 is obtained by adaptive flat pooling, 1×1 convolution layer and ReLU activation function, and 1×1 convolution layer. Since the number of channels of feature maps F7, feature maps F4 and feature maps F5 has not changed and is C, and the spatial size is also the same, the corresponding position elements of the three feature map matrices can be directly added. The number of channels and size of the output feature map are the same as those of feature map F. Downsampling of consistency.

[0061] The input to the ESARM module is the feature representation output by the encoder. This representation passes through two 3×3 convolutional layers, each followed by batch normalization. A ReLU activation function is applied before the second convolutional layer. Finally, it is residually connected with the feature representation output by the encoder and serves as the input to the ESA module. This enhances the nonlinearity of the feature representation: through continuous convolution and ReLU activation, more complex and abstract features can be extracted. The feature map obtained after the ESA module is residually connected with the feature representation output by the encoder and serves as the output of the ESARM module. This ensures that key information from the original encoder output is not lost and enables further interaction and integration of feature information at different levels.

[0062] Step 3: Use the oracle bone images in the extended training set to train the improved CycleGAN network model to obtain a trained oracle bone copy generation model.

[0063] The loss function in the training process mainly includes two kinds of losses, namely, an adversarial loss and a cycle consistency loss. The adversarial loss is used to match the distribution of the generated copy image with the data distribution of the real copy, so as to ensure that the style of the generated copy image is consistent with that of the real copy image. The adversarial loss can only ensure that the generated copy image generated by the generator is in the same distribution as the real copy image, but the generated copy cannot be distinguished from the real copy. The cycle consistency loss is needed to make the copies generated by the two generators consistent.

[0064] The improved CycleGAN network model includes two generators and discriminators, corresponding to two adversarial losses. Generator network G: x→y and its discriminator D Y The corresponding adversarial loss LD Y is:

[0065]

[0066] Generator network F: y→x and its discriminator D X The corresponding adversarial loss LD X is:

[0067]

[0068] The L2 loss of the cycle consistency loss is:

[0069]

[0070] Wherein, S GAN (G, D Y , X, Y) is the loss function of the sample x in the color image data set converted into the copy image y, indicates sampling from the copy image data set, and log D Y (y) represents the prediction result of the copy domain discriminator network D Y on the real copy image. indicates sampling from the color image data set, and D Y (G(x)) represents the prediction probability value of the copy domain discriminator network D Y on the copy image generated by the generator network G. S GAN (G, D X , X, Y) represents the loss function of the sample y in the copy image data set converted into the color image x. D X (x) represents the prediction probability value of the color domain discriminator network D Y on the real color image. D X (G(y)) represents the prediction probability value of the color domain discriminator D X on the color image generated by the generator F. S cyc(G, F) represents the sum of the loss of the sample of the color plate image data set after being generated by the generator G and then being regenerated by the generator F and the real color plate image and the loss of the sample of the copy image data set after being generated by the generator F and then being regenerated by the generator G and the real copy image. F(G(x)) represents the color plate image in the color plate image data set after being generated by the generator G and then being regenerated by the generator F. G(F(y)) represents the copy image in the copy image data set after being generated by the generator F and then being regenerated by the generator G. x represents a sample in the color plate image data set, and y represents a sample in the copy image data set.

[0071] L2 loss S of cycle consistency loss cyc (G, F) includes two parts: for the sample image x from the color plate image domain data set X, the learned mapping function should be able to make the sample image x return to the original image, that is: x→G(x)→F(G(x))≈x, which is called forward cycle consistency loss; for the sample image y from the copy image domain data set Y, the learned mapping function should be able to make the sample image y return to the original image, that is: y→F(y)→G(F(y))≈y, which is called backward cycle consistency loss.

[0072] The total loss function is represented as follows:

[0073] S(G, F, D X , D Y )=S GAN (G, D Y , x, y)+S GAN (F, D X , y, x)+λ cyc S cyc (G, F)

[0074] Wherein, λ cyc is the cycle consistency loss weight, generally taking a value between 1.0 and 10.0.

[0075] Step 4: input the oracle bone image in the test set into the trained oracle bone copy generation model for testing to obtain the translated oracle bone copy image.

[0076] Using the copy generation weight trained by the model, test the samples in the color plate test set, convert the color plate image into a copy image using the generator G, and obtain the translated oracle bone copy image.

[0077] Step 5: compare the performance of the model with existing methods, perform system evaluation, and verify its effectiveness.

[0078] The application adopts evaluation indexes SSIM, PSNR, FID and KID to evaluate the image generation quality, and the method provided in the application significantly improves the four quantitative evaluation indexes.

[0079] SSIM calculates the structural similarity of the generated image and the real image by comparing three indexes of brightness (L), contrast (C) and structure (S):

[0080] SSIM (x, y) = L (x, y) x C (x, y) x S (x, y)

[0081]

[0082] Wherein, x represents the generated image, y represents the real image, μ x and μ y respectively represent the mean of the generated image x and the real image y, σ x and σ y respectively represent the variance of the generated image x and the real image y, σ xy represents the covariance of the generated image x and the real image y, b1, b2 and b3 are constants. L (x, y) is the brightness, C (x, y) is the contrast, and S (x, y) is the structure.

[0083] PSNR evaluates the image quality by comparing the ratio of the peak value signal to noise between the original image and the reconstructed image; usually in decibels, the higher the value, the better the image quality; the formula is as follows:

[0084]

[0085] Wherein, MSE is the mean square error, A (i, j) represents the pixel of the i-th row and the j-th column in the original image; B (i, j) is the pixel of the i-th row and the j-th column in the original image, F (i, j) is the pixel of the i-th row and the j-th column in the reconstructed (generated) image; r represents the peak value of the fusion image, the greater the peak signal-to-noise ratio, the closer the generated copy image to the real copy image.

[0086] FID evaluates the quality of the generated image by comparing the statistical characteristics of the generated image and the real image in the pre-trained Inception network feature space; the lower the FID value, the closer the distribution of the generated image to the distribution of the real image, and the higher the quality of the generated image; the formula is as follows:

[0087]

[0088] Wherein, μ real and μ fake respectively represent the mean vector of the data distribution of the real copy image and the generated copy image, Tr represents the trace operation of the matrix, C realand C fake respectively represent the covariance matrix of real copy image and generated copy image data.

[0089] KID measures the difference between two groups of samples by calculating the square of the maximum mean difference between features; the smaller the KID score, the stronger the visual similarity between real images and generated images; the formula is as follows:

[0090] KID(x, y) = ||E x [Φ(x)]-E y [Φ(y)]|| 2

[0091] wherein x and y respectively represent batches of real copy images and generated copy images, Φ represents a feature extraction function for mapping images into an embedding space, E x and E y respectively represent the expected value of images x and y.

[0092] Step 6: Perform ablation experiments to verify the effectiveness of the SCC module and the ESA module and their role in improving the quality of copy image generation.

[0093] Specifically, referring to Figure 10 , further analyze the qualitative results, compare the model performance with existing methods, and through Figure 10It can be seen that compared with the existing method, the model proposed by the present application not only can maintain the complete topological structure of the character, but also realizes the high-fidelity reconstruction of the detail features, and the edge definition and continuity generated are most close to the standard copy, which is significantly better than other methods. The image evaluation results are shown in Table 1, wherein HED is the abbreviation of Holistically-nested edge detection, and pix2pix is the abbreviation of image-to-image translation with conditional adversarial networks. From the SSIM results in Table 1, it can be seen that the present application reaches 0.7149 in the SSIM index, which is 50.99% and 5.85% higher than the Canny and CycleGAN network methods respectively. In addition, in the PSNR index, the value of the present application method is 19.7394, which is much higher than the value of Canny 1.8608 and the value of pix2pix 8.4437, further confirming the advantages of the present application in improving the generated copy image quality and suppressing the background noise interference of oracle bone images. In the FID and KIDx100 two image quality evaluation indexes, the present application method respectively reaches the lowest values of 42.61 and 3.54, indicating that the difference between the generated copy image and the real copy image is very small, the image generation quality and attribute accuracy are obviously improved, and the excellent oracle bone copy image generation capability is exhibited.

[0094] Table 1 objective index evaluation results of the present application and other methods

[0095]

[0096]

[0097] Specifically, referring to Figure 11 , the copy generation effects with and without the SCC module are compared through a group of experiments, and it is found that Figure 11 It can be seen that the local detail features of the generated image are more obvious after adding the SCC module, which improves the quality of the generated copy. The experimental results are shown in Table 2, and after introducing the SCC module, the SSIM value increases from 0.6987 to 0.7149, with an increase of about 2.3%, indicating that the structure fidelity of the generated copy image is better; the PSNR value is improved from the baseline level to 19.7394 dB, with an increase of about 7.6%, indicating that the overall image quality is significantly improved; the FID value is reduced by about 5%, from 44.87 to a lower level, reflecting that the generated copy image is closer to the real data distribution; the KID value is reduced by about 6.3%, further confirming the advantages of the present application method in generating realistic and detail fidelity. The experimental results consistently show that the SCC module is effective and has excellent promotion effect on the quality of the generated copy.

[0098] Table 2 SCC module experimental results

[0099]

[0100] Specifically, referring to Figure 12 , the effect of generating a copy with or without an ESA module is compared through a set of experiments, and it is found that Figure 12 It can be seen that the image quality generated after adding the ESA module is higher, the key features of the oracle bone are more obvious, and the background noise interference of the oracle bone image is effectively suppressed. The experimental results are shown in Table 3. After introducing the ESA module, the SSIM value increases from 0.6859 to 0.7149, with an increase of about 4.2%, indicating that the structural fidelity of the generated copy image is better. The PSNR value is improved from the baseline level to 19.7394dB, with an increase of about 17%, indicating that the overall image quality is significantly improved. The FID value is reduced by about 6.7%, from 45.69 to a lower level, reflecting that the generated copy image is closer to the real data distribution. The KID value is reduced by about 9.7%, further confirming the advantages of the method in generating realistic and detail fidelity. The experimental results consistently show the effectiveness of the ESA module and its excellent role in improving the quality of the copy generation.

[0101] Table 3 ESA module experimental results

[0102]

[0103] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for automatically generating oracle bone copies based on an improved CycleGAN network, characterized in that: The steps are as follows: Step 1: Scan the books using a non-contact document and book scanner to obtain oracle bone inscription images, pre-process the oracle bone inscription images to obtain oracle bone images, and then divide the oracle bone dataset into a training set and a test set; Step 2: Build an improved CycleGAN network model: add a multi-scale feature extraction module and a style-enhanced attention residual module to the generator network; The improved CycleGAN network model includes two groups of generator networks G and F and a discriminator network D x and the discriminator network D y The two generator networks F are responsible for generating color oracle bone images from the replica oracle bone images, the generator network G is responsible for generating replica oracle bone images from the color oracle bone images, and the discriminator network D is responsible for generating the replica oracle bone images from the color oracle bone images. x The adversarial loss LD is used to compare the real sample images of the color version in the oracle bone dataset with the color version oracle bone images generated by the generator network F. X , discriminator network D y The adversarial loss LD is used to compare the real sample images of the replicas in the oracle bone dataset with the replica oracle bone images generated by the generator network G. Y ; The generator network G and the generator network F both adopt an encoder-decoder architecture, and the generator adopts a parallel approach to add a designed multi-scale feature extraction module and a style-enhanced attention residual module to achieve progressive feature conversion; Step 3: Use the oracle bone images in the training set to train the improved CycleGAN network model to obtain a trained oracle bone replica generation model; Step 4: Input the oracle bone images in the test set into the trained oracle bone copy generation model to obtain the translated oracle bone copy images.

2. The method for automatically generating oracle bone copies based on the improved CycleGAN network according to claim 1, characterized in that: The method for obtaining the oracle bone data set is as follows: two oracle bone images, a color version and a copy, are intercepted from the oracle bone inscription images, and the two oracle bone images, the color version and the copy, constitute the oracle bone data set; The oracle bone dataset was divided into a training set and a test set in a ratio of 7:

3. The oracle bone images in the training set were then subjected to data augmentation methods such as geometric transformation, contrast enhancement, sharpness enhancement, or Gaussian filtering to obtain an extended training set. The improved CycleGAN network model was trained using the extended training set. The evaluation indicators SSIM, PSNR, FID and KID are used to evaluate the image generation quality of the oracle bone model generation model.

3. The method for automatically generating oracle bone copies based on the improved CycleGAN network according to claim 1, characterized in that: The color version of the oracle bone image in the oracle bone dataset is used as the real sample image real_A, and the pseudo sample image fake_B of the replica is obtained through the generator network G. The pseudo sample image fake_B obtains the bidirectional cycle consistency path image Cycle_A of the color version through the generator network F. The difference between the bidirectional cycle consistency path image Cycle_A and the real sample image real_A is measured by calculating the mean square error between the bidirectional cycle consistency path image Cycle_A and the real sample image real_A; the real sample image real_B of the replica in the oracle bone dataset is used as the generator network F to obtain the pseudo sample image fake_A of the color version. The pseudo sample image fake_A obtains the bidirectional cycle consistency path image Cycle_B of the replica through the generator network G. The real sample image real_B is compared with the bidirectional cycle consistency path image Cycle_B to learn to distinguish between the real sample image and the generated image; The discriminator network D y The part receives the fake sample image fake_B and the real sample image real_B as input, performs feature extraction and discrimination through multiple convolutional layers, and finally outputs the true and false discrimination result; the discriminator network D x The part receives the fake sample image fake_A and the real sample image real_A as input, performs feature extraction and discrimination through multiple convolutional layers, and finally outputs the true and false discrimination result.

4. The method for automatically generating oracle bone copies based on the improved CycleGAN network according to claim 1, characterized in that: The encoder converts the real sample image into feature representation through convolution layers and downsampling, and the decoder reconstructs the features into pseudo sample images using transposed convolution layers and upsampling.

5. The method for automatically generating oracle bone copies based on the improved CycleGAN network according to claim 3 or 4, characterized in that: The style-enhanced attention residual module is set between the encoder and decoder of the generator network G and the generator network F, and the multi-scale feature extraction module is set between the second convolutional layer of the encoder and the first convolutional layer of the decoder.

6. The method for automatically generating oracle bone copies based on the improved CycleGAN network according to claim 5, characterized in that: The input of the multi-scale feature extraction module is the feature map F after the first two convolutional layers. Conv2 ∈R B×C×H×W , the feature map F Conv2 It is divided into four convolution branches, which perform four parallel convolution operations with different convolution kernel sizes and different numbers of groups. The feature maps obtained by the four convolution branches are spliced ​​to obtain the feature map F Conv2 Output feature maps of the same size; where B is the batch size, C is the number of channels, and H and W are the height and width of the feature map.

7. The method for automatically generating oracle bone copies based on the improved CycleGAN network according to claim 5, characterized in that: The input of the style enhanced attention residual module is the feature representation output by the encoder. The feature representation output by the encoder is used as the input of the style enhanced attention residual module and then passes through two 3×3 convolution layers for convolution operation. There is a batch normalization operation after each convolution layer, and a ReLU activation function is processed before the second convolution layer. The result obtained by the second convolution layer is residually connected with the feature representation output by the input encoder as the input of the ESA module. The ESA module obtains the corresponding feature map by global context encoding, local structure encoding, edge attention module, channel attention module and style statistical feature module respectively. The corresponding feature maps are fused to obtain a fused feature map, which is again residually connected with the feature representation output by the encoder as the output of the ESARM module.

8. The method for automatically generating oracle bone copies based on the improved CycleGAN network according to claim 7, characterized in that: The input of the ESA module is the downsampled feature map F Downsampling ∈R B×C×H×W , where B is the batch size, C is the number of channels, H and W are the height and width of the feature map respectively; the feature map F Downsampling In the edge attention module, the feature map F1 is obtained by sequentially processing the 3×3 convolution layer and ReLU activation function, the 1×1 convolution layer and the Sigmoid activation function. The feature map F Downsampling In the channel attention module, the feature map F3 is obtained by adaptive flat pooling, 1×1 convolution layer and ReLU activation function, 1×1 convolution layer, and Sigmoid activation function. Downsampling In the style statistical feature module, convolution, average pooling, and standardization operations are performed in sequence to extract the feature map F Downsampling The visual style features in the feature map F2 are obtained, and the feature map F Downsampling , feature map F2 and feature map F3 are multiplied element by element to obtain feature map F6, feature map F6 and feature map F1 are multiplied element by element to obtain feature map F7; feature map F Downsampling In the local structure coding, the feature map F4 is obtained by sequentially passing through the 3×3 convolution layer and the ReLU activation function, and the 3×3 convolution layer. The feature map F Downsampling In the global context encoding, the feature map F5 is obtained by adaptive flat pooling, 1×1 convolution layer and ReLU activation function, and 1×1 convolution layer. The corresponding position elements of feature map F7, feature map F4 and feature map F5 are added to obtain a fused feature map.

9. The method for automatically generating oracle bone copies based on the improved CycleGAN network according to any one of claims 3, 4, 6-8, characterized in that: In step 4, the color oracle bone images in the test set are tested using the replica generation weights trained in the oracle bone replica generation model, and the color oracle bone images are converted into replica oracle bone images using the generator network G to obtain translated oracle bone replica images; The loss function during training includes adversarial loss and cycle consistency loss, the generator network G:x→y and its discriminator network D Y The corresponding adversarial loss LD Y for: Generator network F:y→x and its discriminator network D X The corresponding adversarial loss LD X for: The cycle consistency loss L2 Loss is: Among them, S GAN (G,D Y ,X,Y) is the loss function of converting the color sample image x in the color image dataset to the replica sample image y in the replica image dataset, represents sampling from the replica image dataset, logD Y (y) represents the discriminator network D Y The logarithm of the predicted result for the true replica image; Indicates sampling from the color image dataset, D Y (G(x)) represents the discriminator network D Y The predicted probability value of the replica image generated by the generator network G, S GAN (G,D X ,X,Y) represents the loss function of converting the replica sample image y into the color sample image x in the replica image dataset, D X (x) represents the discriminator network D X The predicted probability value of the real color image, D X (F(y)) represents the discriminator network D X The predicted probability value of the color image generated by the generator network F, S cyc (G, F) represents the sum of the losses between the color plate sample images of the color plate image dataset, which are generated by the generator network G and then regenerated by the generator network F, and the real color plate images, and the color plate images generated by the samples of the replica image dataset, which are generated by the generator network F and then regenerated by the generator network G, and the real replica images. F(G(x)) represents the samples in the color plate image dataset, which are generated by the generator network G and then regenerated by the generator network F; G(F(y)) represents the samples in the replica image dataset, which are generated by the generator F and then regenerated by the generator G; The total loss function is: S(G,F,D X ,D Y )=S GAN (G,D Y ,x,y)+S GAN (F,D x ,y,x)+λ cyc S cyc (G,F) Among them, λ cyc is the cycle consistency loss weight.

Images