Image generation method, device, storage medium, equipment and program product

Abstract

The application discloses an image generation method and device, a storage medium, equipment and a program product, which are applied to scenes such as artificial intelligence, cultural heritage protection and reproduction, oracle bone script recognition and the like. The method comprises the following steps: acquiring an oracle bone script image to be processed, semantic prompt information and character structure information; acquiring original image features of the oracle bone script image to be processed; performing noise adding processing on the original image features to obtain noise-containing image features corresponding to the original image features; performing noise removing processing on the noise-containing image features based on the semantic prompt information and the character structure information to obtain target image features; and performing semantic image generation processing on the target image features to obtain a target semantic image. In the generation process of the semantic image, the noise removing process of the noise-containing image features is performed by using the semantic prompt information and the character structure information, so that the character structure accuracy and semantic richness of the generated semantic image are improved.

Description

Technical Field

[0001] This application relates to the field of information processing technology, specifically to an image generation method, apparatus, storage medium, device, and program product. Background Technology

[0002] Oracle bone script, inscribed on tortoise shells and animal bones during the Shang Dynasty, is one of the earliest mature Chinese writing systems discovered to date. It not only records the economy, culture, and social life of the Shang Dynasty but also provides invaluable data for studying the origin and evolution of Chinese characters. However, due to the complexity of the characters and the fact that some have not yet been fully deciphered, its research and interpretation still face many challenges. In recent years, with the development of artificial intelligence technology, especially advancements in image generation and natural language processing, deep learning-based semantic image generation technology for oracle bone script has gradually become a research hotspot.

[0003] Oracle bone script semantic image generation technology mainly relies on traditional image processing methods and deep learning-based generative models. Traditional methods typically rely on hand-designed feature extraction and image synthesis algorithms. While these methods can generate oracle bone script images to some extent, their effectiveness is limited by the accuracy of feature extraction and the complexity of the algorithms, making it difficult to handle complex character structures and semantic information. In recent years, deep learning-based generative models have made significant progress in image generation tasks. However, these models still suffer from the problem of unclear semantics in the generated deciphered images when processing oracle bone script images. Summary of the Invention

[0004] This application provides an image generation method, apparatus, storage medium, device, and program product that can effectively utilize the glyph structure information of oracle bone script images. By constraining the denoising process of noisy image features with glyph structure information, and at the same time utilizing the semantic prompt information of the oracle bone script image to be processed, the generated semantic image can effectively maintain the original glyph structure of the oracle bone script image and accurately generate a semantically rich semantic image.

[0005] On one hand, embodiments of this application provide an image generation method, the method comprising:

[0006] The process involves acquiring an oracle bone script image to be processed, semantic prompts for the oracle bone script image to be processed, and glyph structure information of the oracle bone script image to be processed. The semantic prompts are used to indicate the content expressed by the oracle bone script image to be processed, and the glyph structure information is used to indicate the structural features of the characters included in the oracle bone script image to be processed.

[0007] Obtain the original image features corresponding to the oracle bone script image to be processed;

[0008] The original image features are subjected to noise processing to obtain noisy image features corresponding to the original image features;

[0009] Based on the semantic prompt information and the character structure information, the noisy image features are denoised to obtain the target image features corresponding to the oracle bone script image to be processed.

[0010] The target image features are subjected to semantic image generation processing to obtain the target semantic image corresponding to the oracle bone script image to be processed.

[0011] On the other hand, embodiments of this application provide an image generation apparatus, the apparatus comprising:

[0012] The first acquisition unit is used to acquire the oracle bone image to be processed, the semantic prompt information of the oracle bone image to be processed, and the glyph structure information of the oracle bone image to be processed. The semantic prompt information is used to indicate the content expressed by the oracle bone image to be processed, and the glyph structure information is used to indicate the structural features of the characters included in the oracle bone image to be processed.

[0013] The second acquisition unit is used to acquire the original image features corresponding to the oracle bone script image to be processed.

[0014] A noise-adding processing unit is used to add noise to the original image features to obtain noisy image features corresponding to the image features;

[0015] The denoising unit is used to denoise the noisy image features based on the semantic prompt information and the glyph structure information to obtain the target image features corresponding to the oracle bone script image to be processed.

[0016] The image generation unit is used to perform semantic image generation processing on the features of the target image to obtain the target semantic image corresponding to the oracle bone script image to be processed.

[0017] In some embodiments, when the denoising unit is used to denoise the noisy image features based on the semantic prompting information and the glyph structure information to obtain the target image features corresponding to the oracle bone script image to be processed, it is specifically used for:

[0018] Obtain a fused text prompt that includes the semantic prompt information and the glyph structure information;

[0019] Calculate the cross-attention map between the fused text prompt and the noisy image features;

[0020] The noisy image features are denoised based on the cross-attention map to obtain the target image features corresponding to the oracle bone script image to be processed.

[0021] In some embodiments, when the image generation unit performs semantic image generation processing on the target image features to obtain the target semantic image corresponding to the oracle bone script image to be processed, it is specifically used for:

[0022] The target image features are processed to generate vector parameters, thereby obtaining the target vector parameters corresponding to the target image features;

[0023] The target vector parameters are subjected to semantic image generation processing to obtain the target semantic image corresponding to the oracle bone script image to be processed.

[0024] In some embodiments, the semantic image generation processing of the target image features to obtain the target semantic image corresponding to the oracle bone script image to be processed is performed by a semantic image generation model. The apparatus further includes a semantic image generation model training unit, which is specifically used for:

[0025] Obtain oracle bone script image samples and the corresponding vector parameter labels of the oracle bone script image samples;

[0026] Obtain the prediction vector parameters of the semantic image generation model to be trained for the oracle bone image sample;

[0027] The vector parameter constraint loss is determined based on the difference between the predicted vector parameters and the vector parameter labels;

[0028] Based on the vector parameter constraint loss, the parameters of the semantic image generation model to be trained are adjusted to obtain the semantic image generation model.

[0029] In some embodiments, the semantic image generation model training unit is further configured to:

[0030] The semantic image generation model to be trained generates the predicted semantic image corresponding to the predicted vector parameters.

[0031] The semantic image generation loss is determined based on the difference between the predicted semantic image and the oracle bone image sample;

[0032] When the semantic image generation model training unit is used to adjust the parameters of the semantic image generation model to be trained based on the vector parameter constraint loss to obtain the semantic image generation model, it is specifically used for:

[0033] Based on the vector parameter constraint loss and the semantic image generation loss, the parameters of the semantic image generation model to be trained are adjusted to obtain the semantic image generation model.

[0034] In some embodiments, when the semantic image generation model training unit determines the semantic image generation loss based on the difference between the predicted semantic image and the oracle bone image sample, it is specifically used for:

[0035] Data smoothing processing is performed on the predicted semantic image and the oracle bone image sample respectively;

[0036] The semantic image generation loss is determined based on the difference between the predicted semantic image after data smoothing and the oracle bone image sample after data smoothing.

[0037] In some embodiments, when the semantic image generation model training unit is used to obtain the prediction vector parameters of the semantic image generation model to be trained for the oracle bone image samples, it is specifically used for:

[0038] Obtain semantic prompt information samples corresponding to the oracle bone image samples and first glyph structure information samples corresponding to the oracle bone image samples. The first glyph structure information samples are text descriptions used to indicate the structural features of the characters included in the oracle bone image samples.

[0039] Data augmentation processing is performed on the oracle bone image samples;

[0040] The following processing is performed using the semantic image generation model to be trained:

[0041] The oracle bone script image samples after data augmentation are encoded to obtain the original image feature samples corresponding to the oracle bone script image samples.

[0042] The original image feature samples are subjected to noise processing to obtain noisy image feature samples corresponding to the original image feature samples;

[0043] Based on the semantic prompt information sample and the first character structure information sample, the noisy image feature sample is denoised to obtain the target image feature sample corresponding to the oracle bone image sample.

[0044] The target image feature samples are processed to generate vector parameters, thereby obtaining the predicted vector parameters corresponding to the oracle bone image samples.

[0045] In some embodiments, when the semantic image generation model training unit is used to perform denoising processing on the noisy image feature samples based on the semantic prompt information samples and the character structure information samples to obtain the target image feature samples corresponding to the oracle bone script image samples, it is specifically used for:

[0046] Obtain a fused text prompt sample containing the semantic prompt information sample and the first glyph structure information sample;

[0047] Calculate the cross-attention map samples between the fused text prompt samples and the noisy image feature samples;

[0048] Based on the cross-attention map samples, the noisy image feature samples are denoised to obtain the target image feature samples corresponding to the oracle bone image samples;

[0049] The semantic image generation model training unit is also used for;

[0050] Obtain a second glyph structure information sample, which is an image description used to indicate the structural features of the characters included in the oracle bone image sample;

[0051] Based on the second glyph structure information sample and the cross attention map sample, the glyph structure generation loss is calculated;

[0052] Based on the glyph structure generation loss, the parameters of the semantic image generation model to be trained are adjusted.

[0053] In some embodiments, the semantic image generation model training unit is further configured to:

[0054] The noise loss is determined based on the difference between the predicted noise and the noise label. The predicted noise is the noise predicted during the denoising process of the noisy image feature sample, and the noise label is the noise added when the oracle bone image sample is denoised.

[0055] Based on the noise loss, the parameters of the feature generation model to be trained are adjusted.

[0056] On the other hand, an embodiment of this application provides a computer-readable storage medium storing a computer program adapted for loading by a processor to execute the image generation method as described in any of the above embodiments.

[0057] On the other hand, an embodiment of this application provides a computer device, which includes a processor and a memory. The memory stores a computer program, and the processor executes the image generation method as described in any of the above embodiments by calling the computer program stored in the memory.

[0058] On the other hand, an embodiment of this application provides a computer program product, including computer instructions that, when executed by a processor, implement the image generation method as described in any of the above embodiments.

[0059] This application provides a scheme for acquiring an oracle bone script image to be processed, semantic prompt information of the oracle bone script image to be processed, and glyph structure information of the oracle bone script image to be processed. The semantic prompt information is used to indicate the content expressed by the oracle bone script image to be processed, and the glyph structure information is used to indicate the structural features of the characters included in the oracle bone script image to be processed. The scheme involves acquiring the original image features corresponding to the oracle bone script image to be processed; adding noise to the original image features to obtain noisy image features corresponding to the original image features; denoising the noisy image features based on the semantic prompt information and the glyph structure information to obtain target image features corresponding to the oracle bone script image to be processed; and performing semantic image generation processing on the target image features to obtain a target semantic image corresponding to the oracle bone script image to be processed. This application embodiment, by using semantic prompt information and glyph structure information for the denoising process of noisy image features, can ensure that the generated semantic image is semantically accurate while effectively maintaining the original glyph structure of the oracle bone script image. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a schematic diagram illustrating an application scenario of the image generation system provided in this application embodiment.

[0062] Figure 2 This is a schematic flowchart of the image generation method provided in an embodiment of this application.

[0063] Figure 3 An example image of an oracle bone script image to be processed is provided for an embodiment of this application.

[0064] Figure 4 This is a schematic diagram illustrating the principle of the image generation method provided in the embodiments of this application.

[0065] Figure 5 This is a schematic diagram illustrating a scenario for training a semantic image generation model and a feature generation model provided in an embodiment of this application.

[0066] Figure 6 This is a first experimental result diagram of the image generation method provided in the embodiments of this application.

[0067] Figure 7 This is a second experimental result diagram of the image generation method provided in the embodiments of this application.

[0068] Figure 8 This is a schematic diagram of the structure of the image generation apparatus provided in the embodiments of this application.

[0069] Figure 9 A schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0070] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0071] This application provides an image generation method, apparatus, storage medium, device, and program product. Exemplarily, the image generation method of this application can be executed by a computer device, which can be a terminal or server, etc. The terminal can be a smartphone, tablet, laptop, desktop computer, smart TV, smart speaker, wearable smart device, personal computer (PC), smart vehicle terminal, etc. The terminal may also include a client, which can be a video client, shopping application client, reading application client, browser client, or instant messaging client, etc. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms.

[0072] The embodiments of this application can be applied to scenarios such as artificial intelligence, cultural heritage protection and reproduction, and oracle bone script recognition.

[0073] First, some of the nouns or terms that appear in the description of the embodiments of this application are explained as follows:

[0074] Natural Language Processing (NLP) is an important field within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. NLP deals with natural language, the language people use in daily life, and is closely related to linguistics; it also involves computer science and mathematics. Pre-trained models, a crucial technique for model training in artificial intelligence, evolved from large language models in NLP. After fine-tuning, large language models can be widely applied to downstream tasks. NLP techniques typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0075] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instruction-based learning. Pre-trained models are the latest development in deep learning, integrating all of these techniques.

[0076] Diffusion models are a class of machine learning methods based on probabilistic generative models. Their core idea is to generate data by simulating physical diffusion processes (such as thermal or particle diffusion). Diffusion models achieve high-quality data generation by progressively adding noise to the data (forward diffusion process) and learning to recover the data from the noise (reverse denoising process).

[0077] Oracle bone script (OBS) is an ancient Chinese writing system inscribed on tortoise shells and animal bones during the late Shang Dynasty (approximately 14th to 11th centuries BC). It is the earliest known mature system of Chinese characters. Oracle bone script was primarily used to record significant events and is an important source of material evidence for the study of ancient Chinese history, culture, language, and social systems.

[0078] Oracle Glyph Vectorization (OGV) is a method for converting oracle bone script images into a vector format. The goal of this technology is to convert rasterized oracle bone script images into editable and scalable vector representations while preserving the structural integrity of the characters.

[0079] Scalable Vector Graphics (SVG) is a vector graphics format based on XML (Extensible Markup Language). It is used to describe two-dimensional graphics, including lines, curves, text, and complex shapes, and features lossless scaling, editability, and searchability.

[0080] DiffVG (Differentiable Vector Graphics) is a differentiable vector graphics rendering tool. It enables differentiable rendering of SVG, allowing the generation, optimization, and editing of vector graphics to be optimized end-to-end through deep learning.

[0081] Score Distillation Sampling (SDS) is a technique based on image diffusion models with wide applications in various fields, particularly text-to-3D generation, image synthesis, and editing, and is closely related to Oracle's semantic image generation scheme. The core idea of ​​SDS is to use a pre-trained 2D diffusion model to control the optimization problem. By optimizing the loss function, the pre-trained 2D diffusion model is "distilled" into a 3D representation, allowing the rendered samples to achieve high scores within the underlying 2D diffusion model. In text-to-3D generation, a pre-trained text-to-image model is used to calculate the loss and optimize the generation of the 3D model based on the degree of matching between the given text description and the corresponding 2D image of the rendered 3D model, making the generated 3D model more closely match the text description.

[0082] Latent space representation is crucial in machine learning, deep learning, and other fields. It's a representation that maps raw data to a low-dimensional abstract space. Through specific encoding methods, it transforms high-dimensional, complex raw data, such as images, text, and audio, into low-dimensional vectors. This process effectively extracts key features, achieves data compression and feature abstraction, and removes redundant information.

[0083] The solutions provided in this application relate to image generation technology using artificial intelligence, and are specifically illustrated through the following embodiments. Detailed descriptions are provided below. It should be noted that the order of description of the following embodiments is not intended to limit the priority of the embodiments.

[0084] Please see Figure 1 , Figure 1This is a schematic diagram illustrating an application scenario of the image generation system provided in this application embodiment. The image generation system can implement an image generation method. The system includes a client 10, a server 20, and a network 30. The client 10 and the server 20 can interact with each other via the network 30, which can be a wide area network (WAN), a local area network (LAN), or a combination of both.

[0085] In some embodiments, in the context of oracle bone script research, when researchers or cultural enthusiasts have a need to generate semantic images (or deciphered images) of specific oracle bone script images, users can input the oracle bone script image to be processed, the corresponding semantic prompts and glyph structure information through the client 10. The client 10 then generates and submits the corresponding semantic image generation request to the server 20.

[0086] Server 20, as the core generation and processing center of the system, receives a semantic image generation request from client 10, parses the request information, and obtains the oracle bone script image to be processed, the corresponding semantic prompts, and the character structure information. Server 20 encodes the oracle bone script image to be processed using an image encoder to obtain the original image features. Server 20 encodes the semantic prompts using a text encoder to obtain semantic text features. Server 20 adds noise to the original image features to obtain noisy image features corresponding to the original image features. Server 20 performs denoising processing on the noisy image features based on the semantic prompts and character structure information to obtain the target image features corresponding to the oracle bone script image to be processed. Server 20 performs semantic image generation processing on the target image features to generate the target semantic image corresponding to the oracle bone script image to be processed. Server 20 sends the generated target semantic image to client 10 for researchers or cultural enthusiasts to view and use.

[0087] This application provides an image generation method, which can be executed by a terminal or a server, or by both a terminal and a server. This application uses the example of the image generation method being executed by a server to illustrate the method.

[0088] Please see Figure 2 , Figure 2 This is a schematic flowchart illustrating an image generation method provided in an embodiment of this application. The method may include steps 110 to 150:

[0089] Step 110: Obtain the oracle bone image to be processed, the semantic prompt information of the oracle bone image to be processed, and the glyph structure information of the oracle bone image to be processed. The semantic prompt information is used to indicate the content expressed by the oracle bone image to be processed, and the glyph structure information is used to indicate the structural features of the characters included in the oracle bone image to be processed.

[0090] The server first needs to acquire the oracle bone inscription image to be processed, the semantic prompt information of the oracle bone inscription image to be processed, and the glyph structure information.

[0091] In some embodiments, the oracle bone inscription image to be processed can be acquired in a variety of ways, for example, scanning a physical oracle bone inscription with a high-resolution scanner to obtain a high-precision oracle bone inscription image to be processed, acquiring it from an existing oracle bone inscription image database such as the oracle bone inscription database of the National Library of China, or acquiring the oracle bone inscription image to be processed uploaded by a user through a terminal device.

[0092] The semantic prompt information of the oracle bone inscription image to be processed is used to indicate the content or meaning expressed by the oracle bone inscription image to be processed.

[0093] In some embodiments, the semantic prompt information of the oracle bone inscription image to be processed can be determined by relevant experts through annotating the oracle bone inscription image to be processed.

[0094] For example, please refer to Figure 3 , Figure 3 which is an example diagram of the oracle bone inscription image to be processed provided by the embodiment of the present application. The content expressed by the oracle bone inscription image to be processed is: mountain tops where the bird gather and inhabit, thus "mountain tops where the bird gather and inhabit" can be used as the semantic prompt information of this oracle bone inscription image to be processed.

[0095] The glyph structure information of the oracle bone inscription image to be processed is used to indicate the structural features of the characters included in the oracle bone inscription image to be processed.

[0096] The glyph structure information of the oracle bone inscription image to be processed is in text format. The function of the glyph structure information is to provide text-description-based glyph clues for semantic prompting and image generation, assist in understanding character meanings and structural relationships, thereby guiding the image generation process at the semantic level.

[0097] As content presented in text format, the glyph structure information is an abstract linguistic expression of the structural features of the characters included in the oracle bone inscription image to be processed. For example, in terms of composition, the glyph structure information includes text descriptions of the semantic components of characters. For the character "mountain", the corresponding glyph structure information can describe that its pictographic meaning represents the shape of mountain peaks, which is composed of three vertical strokes symbolizing continuous rolling mountains; for the character "bird", the corresponding glyph structure information can mention that it represents a bird, includes semantic components such as the head, body, wings and claws, and explains the ideographic function of each component in the glyph.

[0098] Step 120: acquiring the original image feature corresponding to the oracle bone inscription image to be processed.

[0099] Please see Figure 4 , Figure 4 This is a schematic diagram illustrating the principle of the image generation method provided in the embodiments of this application.

[0100] The server obtains the original image features z corresponding to the oracle bone script image to be processed.

[0101] In some embodiments, obtaining the original image features z corresponding to the oracle bone script image to be processed may include: encoding the oracle bone script image to be processed to obtain the original image features z corresponding to the oracle bone script image to be processed.

[0102] In some embodiments, encoding processing is performed on the oracle bone image to be processed, including: inputting the data-enhanced oracle bone image to be processed into a variational autoencoder (VAE) for encoding processing, and extracting the original image features corresponding to the oracle bone image to be processed.

[0103] In other embodiments, multi-layer convolutional neural networks can also be used to encode the oracle bone script images to be processed.

[0104] In some embodiments, the original image features corresponding to the oracle bone script image to be processed can be the latent space representation of the oracle bone script image to be processed. The latent space representation of the oracle bone script image to be processed is the representation obtained by mapping the oracle bone script image to be processed into a low-dimensional abstract space through a specific encoding method.

[0105] When using variational autoencoders (VAEs) or multilayer convolutional neural networks for encoding, the input oracle bone script image is transformed into a set of vectors in the latent space. These vectors contain key feature information of the original image, such as stroke structure and character shape, while also compressing and abstracting the data.

[0106] Step 130: Add noise to the original image features to obtain the noisy image features corresponding to the original image features.

[0107] In some embodiments, adding noise to the original image features to obtain noisy image features corresponding to the original image features includes: adding random Gaussian noise to the original image features to obtain noisy image features z corresponding to the original image features z. t .

[0108] Step 140: Based on semantic prompts and glyph structure information, denoising processing is performed on the noisy image features to obtain the target image features z corresponding to the oracle bone script image to be processed. ′ .

[0109] Based on the glyph structure information and semantic hints of the oracle bone script image to be processed, the server analyzes the noisy image features z. t Denoising is performed to obtain the target image features z corresponding to the oracle bone script image to be processed. ′ .

[0110] In some embodiments, denoising the noisy image features based on semantic prompting information and glyph structure information to obtain the target image features corresponding to the oracle bone script image to be processed may include the following steps 1401 to 1403:

[0111] Step 1401: Obtain a fused text prompt containing the semantic prompt information and the glyph structure information.

[0112] In some embodiments, obtaining a fused text prompt that includes semantic prompt information and glyph structure information includes: fusing the semantic prompt information and glyph structure information to obtain fused input information; and encoding the fused input information to obtain a fused text prompt.

[0113] In some embodiments, semantic prompt information and glyph structure information are fused to obtain fused input information, including: concatenating the vector representation corresponding to the semantic prompt information and the vector identifier corresponding to the glyph structure information to obtain fused input information.

[0114] In some embodiments, an attention mechanism can also be used to fuse semantic prompts and glyph structure information.

[0115] In some embodiments, encoding the fused input information to obtain a fused text prompt includes: inputting the fused input information into a text encoder for encoding processing to obtain the fused text prompt.

[0116] Step 1402: Calculate the cross-attention map between the fused text prompt and the noisy image features.

[0117] The server calculates the attention weights between the fused text prompts and the features of the noisy image, generating a cross-attention map.

[0118] Specifically, when calculating the cross-attention map between the fused text prompt and the noisy image features, the fused text prompt τ is calculated through a cross-attention mechanism. θ (y) and noisy image features z t The attention weights between them are used to generate a cross-attention map.

[0119] Specifically, it will integrate text prompts τ θ (y) and noisy image features z tThey are projected into a shared latent space to obtain two new feature representations: text feature projection Q and image feature projection K.

[0120] The formula for calculating the text feature projection Q is: Q = W q *τ θ (y), where W q The learnable weight matrix; the formula for calculating the image feature projection K is: K = W k *z t Among them, W k This is a learnable weight matrix.

[0121] Then merge text prompts τ θ (y) and noisy image features z t The formula for calculating the cross-attention matrix A between them is: Where d is the dimension of the feature vector, and the softmax function is used to normalize the attention weights to ensure that the attention distribution is a probability value.

[0122] The cross-attention map is obtained by weighting the features of the noisy image using a cross-attention matrix.

[0123] Step 1403: Denoise the noisy image features based on the cross-attention map to obtain the target image features corresponding to the oracle bone script image to be processed.

[0124] Specifically, the server uses cross-attention maps to perform weighted fusion of noisy image features to obtain enhanced image features (or weighted image features), and then performs feature recovery (i.e. denoising) on ​​the enhanced image features to obtain the target image features.

[0125] In some embodiments, enhanced image features can be input into a denoising network of a diffusion model for feature recovery.

[0126] Step 150: Perform semantic image generation processing on the target image features to obtain the target semantic image corresponding to the oracle bone script image to be processed.

[0127] The server performs semantic image generation processing on the features of the target image to obtain the target semantic image corresponding to the oracle bone script image to be processed.

[0128] In some embodiments, performing semantic image generation processing on the target image features to obtain the target semantic image corresponding to the oracle bone script image to be processed may include the following steps 1501 to 1502:

[0129] Step 1501: Perform vector parameter generation processing on the target image features to obtain the target vector parameters corresponding to the target image features.

[0130] In some embodiments, such as Figure 4 As shown, the target image features can be input into the vector parameter generation module for vector parameter generation processing to obtain the target vector parameters corresponding to the target image features.

[0131] In some embodiments, the vector parameter generation module can be constructed based on the OGV method, and the target vector parameters corresponding to the target image features can be SVG parameters.

[0132] Step 1502: Perform semantic image generation processing on the target vector parameters to obtain the target semantic image corresponding to the oracle bone script image to be processed.

[0133] In some embodiments, such as Figure 4 As shown, the target vector parameters can be input into the vector graphics rendering module for semantic image generation processing to obtain the target semantic image corresponding to the oracle bone script image to be processed.

[0134] In one specific embodiment provided in this application, the vector image rendering module can be a pre-trained DiffVG.

[0135] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating a scenario for training a semantic image generation model provided in an embodiment of this application. In some embodiments, the semantic image generation process performed on the features of the target image to obtain the target semantic image corresponding to the oracle bone script image to be processed is executed by the semantic image generation model. The training steps of the semantic image generation model may include the following steps 11 to 14:

[0136] Step 11, obtain the oracle bone script image samples and the corresponding vector parameter labels P1...P k .

[0137] In some embodiments, such as Figure 5 As shown, obtain the vector parameter labels P1...P corresponding to the oracle bone image samples. k This includes: using the OVG method to convert oracle bone image samples into vector parameter labels P1...P k .

[0138] Step 12: Obtain the prediction vector parameters of the semantic image generation model to be trained for the oracle bone image samples.

[0139] In some embodiments, the prediction vector parameters corresponding to the oracle bone image samples are obtained. This may include the following steps 121 to 126:

[0140] Step 121: Acquire a semantic prompt information sample corresponding to an oracle bone inscription image sample and a first glyph structure information sample corresponding to the oracle bone inscription image sample, wherein the first glyph structure information sample is a text description for indicating structural features of characters included in the oracle bone inscription image sample.

[0141] The data format of the first glyph structure information sample is a text format. The first glyph structure information functions to provide glyph clues based on text description for semantic prompting and image generation, assist the model in understanding character meanings and structural relationships, thereby guiding the image generation process at the semantic level.

[0142] As content presented in text format, the first glyph structure information sample is an abstract linguistic expression of structural features of characters included in the oracle bone inscription image sample. For example, from the perspective of composition, the first glyph structure information sample includes text descriptions of semantic components of characters. For the character "shan (mountain)", the corresponding first glyph structure information sample may describe that its pictographic meaning represents the shape of mountain peaks, which is composed of three vertical strokes, symbolizing continuous mountains; for the character "niao (bird)", the corresponding first glyph structure information sample may mention that it represents a bird, includes semantic components such as head, body, wings and claws, and explains the ideographic function of each component in the glyph.

[0143] Step 122: Perform data enhancement processing on the oracle bone inscription image sample.

[0144] In some embodiments, performing Augment data enhancement processing on the oracle bone inscription image sample includes performing at least one of rotation, scaling and translation on the oracle bone inscription image sample.

[0145] In some embodiments, the oracle bone inscription image sample can be input into a data enhancement processing module for data enhancement processing.

[0146] Performing data enhancement processing on oracle bone inscription image samples can improve the robustness of the model, prevent overfitting, and enhance the adaptive ability of the model to image changes.

[0147] Step 123: Perform encoding processing on the data-enhanced oracle bone inscription image sample by means of the to-be-trained semantic image generation model, to obtain an original image feature sample corresponding to the oracle bone inscription image sample.

[0148] In some embodiments, the data-enhanced oracle bone inscription image sample can be input into a variational autoencoder (Variational Autoencoder, VAE) in the semantic image generation model for encoding processing, to obtain an original image feature sample corresponding to the oracle bone inscription image sample.

[0149] Step 124: The original image feature samples are noise-added using the semantic image generation model to be trained, so as to obtain the noisy image feature samples corresponding to the original image feature samples.

[0150] In some embodiments, adding noise to the original image feature samples to obtain noisy image feature samples corresponding to the original image feature samples includes: adding random Gaussian noise to the original image feature samples to obtain noisy image feature samples corresponding to the original image feature samples.

[0151] Step 125: The semantic image generation model to be trained performs denoising processing on the noisy image feature samples based on the semantic prompt information samples and the first character structure information samples to obtain the target image feature samples corresponding to the oracle bone image samples.

[0152] The execution process of step 125 is completely consistent with the process of step 140 above, and will not be described in detail here.

[0153] Step 126: The target image feature samples are processed by generating vector parameters using the semantic image generation model to be trained, so as to obtain the predicted vector parameters corresponding to the oracle bone image samples.

[0154] In some embodiments, target image feature samples can be input into the vector parameter generation module of the semantic image generation model to be trained for vector parameter generation processing to obtain the predicted vector parameters corresponding to the target image feature samples.

[0155] Step 13: Determine the vector parameter constraint loss based on the difference between the predicted vector parameters and the vector parameter labels.

[0156] In some embodiments, based on the predicted vector parameters With vector parameter labels P1...P k The difference between them determines the vector parameter constraint loss L SKST This includes: for the predicted vector parameters With vector parameter labels P1...P k For each element in the vector parameter constraint loss L, calculate the absolute value of their differences, and sum all the absolute values ​​of the differences to obtain the vector parameter constraint loss L. SKST .

[0157] Step 14: Adjust the parameters of the semantic image generation model to be trained based on the vector parameter constraint loss to obtain the semantic image generation model.

[0158] In some embodiments, the loss based on vector parameter constraints L SKST Adjust the parameters of the semantic image generation model to be trained, including: calculating the vector parameter constraint loss L. SKSTThe gradient of the model parameters of the semantic image generation model to be trained is used to update the model parameters of the semantic image generation model to be trained. For example, optimizers such as Adam and SGD can be used to update the parameters.

[0159] Repeat steps 11 to 14 until the semantic image generation model converges or reaches the predetermined number of training rounds.

[0160] In some embodiments, the training steps of the semantic image generation model may further include the following steps 15 to 16:

[0161] Step 15: Generate the predicted semantic image corresponding to the predicted vector parameters using the semantic image generation model to be trained.

[0162] In some embodiments, such as Figure 5 As shown, the predicted vector parameters can be input into the vector parameter rendering module of the semantic image generation model to be trained, and the predicted semantic image corresponding to the predicted vector parameters can be obtained.

[0163] Step 16, based on the predicted semantic image The difference between the image and the oracle bone image sample R(P) determines the semantic image generation loss.

[0164] In some embodiments, such as Figure 5 As shown, the semantic image generation loss L is used to calculate the semantic image generation loss. tone The oracle bone image sample R(P) can be the original oracle bone image sample, or it can be the image obtained after rendering the DiffVG with fixed vector parameter labels as input.

[0165] In some embodiments, the semantic image generation loss L is determined based on the difference between the predicted semantic image and the oracle bone script image samples. tone ,include:

[0166] Data smoothing was performed on the predicted semantic images and oracle bone script image samples respectively;

[0167] Based on the difference between the predicted semantic image after data smoothing and the oracle bone image samples after data smoothing, the semantic image generation loss L is determined. tone .

[0168] In some embodiments, when performing data smoothing processing on predicted semantic images or oracle bone image samples, the predicted semantic images or oracle bone image samples can be input into the smoothing processing module for image smoothing processing.

[0169] For the predicted semantic image or oracle bone script image samples, the server inputs the predicted semantic image or oracle bone script image samples into the smoothing processing module for smoothing processing to eliminate irregular edges, which can improve image quality.

[0170] In one specific embodiment provided in this application, the smoothing module can be a low-pass filter (LPF). An LPF allows low-frequency signals to pass through while suppressing high-frequency noise, such as edge irregularities and artifacts.

[0171] The semantic image generation loss is determined by comparing the predicted semantic image after data smoothing with the oracle bone image samples after data smoothing. This allows the semantic image generation model to generate semantic images that both conform to the character structure constraints and have good visual quality.

[0172] In some embodiments, the loss based on vector parameter constraints L SKST Adjusting the parameters of the semantic image generation model to be trained yields the semantic image generation model, including:

[0173] Based on vector parameter constraint loss L SKST and semantic image generation loss L tone Adjust the parameters of the semantic image generation model to be trained to obtain the semantic image generation model.

[0174] In some embodiments, the loss based on vector parameter constraints L SKST and semantic image generation loss L tone Adjusting the parameters of the semantic image generation model to be trained yields the semantic image generation model, including: constraining the loss L based on vector parameters. SKST and semantic image generation loss L tone Determine the combined loss, and adjust the parameters of the semantic image generation model to be trained based on the combined loss to obtain the semantic image generation model.

[0175] In some embodiments, denoising the noisy image feature samples based on semantic prompt information samples and glyph structure information samples to obtain target image feature samples corresponding to oracle bone script image samples includes: obtaining a fused text prompt sample containing semantic prompt information samples and first glyph structure information samples; calculating a cross-attention map sample between the fused text prompt sample and the noisy image feature samples; and denoising the noisy image feature samples based on the cross-attention map sample to obtain target image feature samples corresponding to oracle bone script image samples.

[0176] In some embodiments, the training steps of the semantic image generation model may further include steps 17 to 19:

[0177] Step 17, obtaining a second glyph structure information sample, wherein the second glyph structure information sample is an image description used for indicating the structural features of characters included in the oracle bone inscription image sample.

[0178] The second glyph structure information sample exists in an image format, and is an accurate spatial positioning and quantitative expression of the character structure in the oracle bone inscription image sample.

[0179] In some embodiments, the second glyph structure information sample may include bounding box information of characters, spatial position coordinate information of strokes, spatial relationship data between various parts of a character, and the like.

[0180] Wherein, the bounding box information of a character can specifically determine the position range of each character in the image through coordinate values. For example, for the oracle bone inscription character "wood" (mu), its bounding box may be determined by the top-left coordinate (x1,y1) and the bottom-right coordinate (x2,y2), which accurately defines the spatial area of the character "mu" in the image; the spatial position coordinate information of strokes is used to indicate the coordinates of the start point, turning points and end point of each stroke. For the curved stroke of the character "water" (shui), the coordinates of each key node on the curve will be accurately marked to determine the shape and direction of the stroke; the spatial relationship data between various parts of a character is used to indicate the spatial features of each sub-part of the character. For example, for the character "forest" (lin), the relative position and spacing information between the two "mu" characters will be recorded.

[0181] Step 18, calculating glyph structure generation loss L based on the second glyph structure information sample and cross-attention map samples GSDS .

[0182] In some embodiments, the glyph structure generation loss L GSDS can be calculated using the Score Distillation Sampling method.

[0183] Specifically, define intra-region constraint

[0184] extra-region constraint

[0185] wherein represents the j-th cross-attention map corresponding to the j-th character token b in the cross-attention map sample at each time step t j , Topk( , P) represents selecting the top P positions with the highest response in , K is the character set corresponding to the second glyph structure information sample, including M character tokens, M j is a binary mask corresponding to the character token b j , which is used to determine the region corresponding to the second glyph structure information sample. For example, if M jIf an element corresponds to a position within the region corresponding to the second character structure information sample, then the value of that element is 1; otherwise, it is 0.

[0186] Glyph structure generation loss L GSDS =L IR +L OR .

[0187] In this way, the cross-attention map samples are constrained from both the key region (the region corresponding to the second character structure information sample) and the outside of the key region, so that the image features generated by the semantic image generation model are more consistent with the character structure of the oracle bone image samples.

[0188] Step 19, generate loss L based on glyph structure GSdS Adjust the parameters of the semantic image generation model to be trained.

[0189] In some embodiments, the loss L is generated based on the glyph structure. GSDS Adjust the parameters of the feature generation model to be trained, including: vector parameter constraint loss L. SKST Semantic image generation loss L tone The generation loss of the character structure L GSDS Determine the combined loss, and adjust the parameters of the semantic image generation model to be trained based on the combined loss to obtain the semantic image generation model.

[0190] In some embodiments, gradient descent can be used to adjust the parameters of the semantic image generation model to be trained.

[0191] Specifically, calculate the vector parameter constraint loss L. SKST Semantic image generation loss L tone The generation loss of the character structure L GSDS The gradient of the model parameters of the semantic image generation model to be trained is used to update the model parameters of the semantic image generation model to be trained based on the gradient using an optimizer (such as Adam, SGD, etc.).

[0192] In some embodiments, the training steps of the semantic image generation model may further include the following steps 20 to 21:

[0193] Step 20: Determine the noise loss based on the difference between the predicted noise and the noise label. The predicted noise is the noise predicted during the denoising process of the noisy image feature samples, and the noise label is the noise added when the oracle bone image samples are denoised.

[0194] Step 21: Adjust the parameters of the semantic image generation model to be trained based on noise loss.

[0195] Specifically, in the denoising process of noisy image feature samples, the feature generation model predicts the noise, compares the predicted noise with the noise added when the second oracle bone image sample is denoised (i.e., the noise label), calculates the difference between the two, and thus determines the noise loss.

[0196] In some embodiments, mean squared error (MSE) can be used to measure the difference between the predicted noise and the noise label. Specifically, assuming the predicted noise is... If the noise label is N, then the noise loss is L. noise The calculation formula is:

[0197] In some embodiments, based on noise loss L noise Adjust the parameters of the feature generation model to be trained, including: based on noise loss L noise The combined loss is determined, and the parameters of the semantic image generation model to be trained are adjusted using gradient descent.

[0198] Specifically, calculate the noise loss L. noise The gradient of the model parameters of the semantic image generation model to be trained is used to update the model parameters of the semantic image generation model to be trained based on the gradient using an optimizer (such as Adam, SGD, etc.).

[0199] In some embodiments, based on noise loss L noise Adjust the parameters of the feature generation model to be trained, including: vector parameter constraint loss L. SKST Semantic image generation loss L tone The loss in character structure generation L GSDS and noise loss L noise Determine the combined loss, and adjust the parameters of the semantic image generation model to be trained based on the combined loss to obtain the semantic image generation model.

[0200] In the solution provided in the embodiments of this application, the character structure generation loss L GSDS Operating on the character structure dimension, by imposing constraints on the character structure information based on oracle bone script images on the cross-attention map, the model is prompted to learn the character structure features of oracle bone script images in depth. This helps to preserve the structural characteristics of oracle bone script images, avoid semantic elements deviating from the stroke positions of the original characters, and ensure the accuracy of the character structure of the generated semantic image; vector parameter constraint loss L SKSTOperating on the geometric dimension (or glyph dimension), it constrains the consistency between the target image features generated by the model and the real vector parameter labels, ensuring that the generated image has a high similarity to the real image at the glyph level. This guarantees that the generated semantic image can accurately reflect the glyph features of the oracle bone script image, and is closer to the real oracle bone script image in terms of geometric attributes such as shape, size, and proportion, thereby improving the accuracy of the generated image in geometric representation; Image generation loss L tone It operates on the overall visual and semantic expression dimensions. By smoothing the generated predicted semantic image and comparing it with the semantic image label, it guides the model to improve the quality of the generated semantic image, making it not only more natural and beautiful in appearance, but also more in line with human expectations for semantic understanding of oracle bone images, thus enhancing the visual and semantic accuracy and effectiveness of the generated semantic image.

[0201] Glyph structure generation loss L GSDS Vector parameter constraint loss L SKST and image generation loss L tone The three losses work together to train the model, achieving three major goals: faithful glyph structure, geometric accuracy, and semantic representation that conforms to human semantic understanding.

[0202] Please see Figures 6 to 7 , Figures 6 to 7 The experimental results provided in this application embodiment demonstrate that the semantic image deciphered by the method provided in this application embodiment (i.e., the semantic image corresponding to "ours" in the figure) better preserves the objective integrity of the glyph structure compared to the ClipDraw model and Word As Image model in related technologies. It provides visually enhanced expressive forms and offers more interpretable inspiration, helping experts interpret OBS. The method provided in this application embodiment outperforms state-of-the-art baseline models in terms of semantics, visual appeal, and glyph maintenance on the OBS dataset, significantly improving readability and aesthetics. Furthermore, it can provide expert-level insights into unseen Oracle characters, making it a valuable tool for advancing OBS deciphering.

[0203] The embodiments of this application provide a scheme for obtaining an oracle bone script image to be processed, semantic prompt information of the oracle bone script image to be processed, and glyph structure information of the oracle bone script image to be processed. The semantic prompt information is used to indicate the content expressed by the oracle bone script image to be processed, and the glyph structure information is used to indicate the structural features of the characters included in the oracle bone script image to be processed. The scheme involves obtaining the original image features corresponding to the oracle bone script image to be processed; adding noise to the original image features to obtain noisy image features corresponding to the original image features; denoising the noisy image features based on the semantic prompt information and glyph structure information to obtain the target image features corresponding to the oracle bone script image to be processed; and performing semantic image generation processing on the target image features to obtain the target semantic image corresponding to the oracle bone script image to be processed. By using the semantic prompt information and glyph structure information to perform the denoising process on the noisy image features, the generated semantic image can be semantically accurate while effectively maintaining the original glyph structure of the oracle bone script image, thus accurately generating a semantically rich semantic image.

[0204] To facilitate better implementation of the image generation method of this application, this application also provides an image generation apparatus. Please refer to... Figure 8 , Figure 8 This is a schematic diagram of the structure of an image generation apparatus provided in an embodiment of this application. The image generation apparatus 800 may include:

[0205] The first acquisition unit 810 is used to acquire the oracle bone image to be processed, the semantic prompt information of the oracle bone image to be processed, and the character structure information of the oracle bone image to be processed. The semantic prompt information is used to indicate the content expressed by the oracle bone image to be processed, and the character structure information is used to indicate the structural features of the characters included in the oracle bone image to be processed.

[0206] The second acquisition unit 820 is used to acquire the original image features corresponding to the oracle bone script image to be processed.

[0207] The noise processing unit 830 is used to add noise to the original image features to obtain the noisy image features corresponding to the image features;

[0208] The denoising processing unit 840 is used to denoise the features of the noisy image based on semantic prompt information and character structure information to obtain the target image features corresponding to the oracle bone script image to be processed.

[0209] The image generation unit 850 is used to perform semantic image generation processing on the features of the target image to obtain the target semantic image corresponding to the oracle bone script image to be processed.

[0210] In some embodiments, when the denoising processing unit 840 performs denoising processing on the noisy image features based on semantic prompting information and glyph structure information to obtain the target image features corresponding to the oracle bone script image to be processed, it is specifically used for:

[0211] Obtain fused text prompts that include semantic and glyph structure information;

[0212] Calculate the cross-attention map between the fused text prompt and noisy image features;

[0213] The denoising process of the noisy image features is performed based on the cross-attention map to obtain the target image features corresponding to the oracle bone script image to be processed.

[0214] In some embodiments, when the image generation unit 850 performs semantic image generation processing on the target image features to obtain the target semantic image corresponding to the oracle bone script image to be processed, it is specifically used for:

[0215] The target image features are processed to generate vector parameters, resulting in target vector parameters corresponding to the target image features.

[0216] Semantic image generation processing is performed on the target vector parameters to obtain the target semantic image corresponding to the oracle bone script image to be processed.

[0217] In some embodiments, the semantic image generation process of the target image features to obtain the target semantic image corresponding to the oracle bone script image to be processed is performed by a semantic image generation model. The apparatus further includes a semantic image generation model training unit, which is specifically used for:

[0218] Obtain oracle bone script image samples and their corresponding vector parameter labels;

[0219] Obtain the predicted vector parameters of the semantic image generation model to be trained for the oracle bone image samples;

[0220] The vector parameter constraint loss is determined based on the difference between the predicted vector parameters and the vector parameter labels;

[0221] Based on vector parameter constraint loss, the parameters of the semantic image generation model to be trained are adjusted to obtain the semantic image generation model.

[0222] In some embodiments, the semantic image generation model training unit is further configured to:

[0223] The semantic image generation model to be trained generates a predicted semantic image corresponding to the predicted vector parameters.

[0224] The semantic image generation loss is determined based on the difference between the predicted semantic image and the oracle bone image samples;

[0225] The semantic image generation model training unit, when used to adjust the parameters of the semantic image generation model to be trained based on vector parameter constraint loss to obtain the semantic image generation model, is specifically used for:

[0226] Based on vector parameter constraint loss and semantic image generation loss, the parameters of the semantic image generation model to be trained are adjusted to obtain the semantic image generation model.

[0227] In some embodiments, when the semantic image generation model training unit determines the semantic image generation loss based on the difference between the predicted semantic image and the oracle bone script image sample, it is specifically used for:

[0228] Data smoothing was performed on the predicted semantic images and oracle bone script image samples respectively;

[0229] The semantic image generation loss is determined based on the difference between the predicted semantic image after data smoothing and the oracle bone image sample after data smoothing.

[0230] In some embodiments, when the semantic image generation model training unit is used to obtain the prediction vector parameters of the semantic image generation model to be trained for oracle bone image samples, it is specifically used for:

[0231] Obtain semantic prompt information samples and first glyph structure information samples corresponding to oracle bone image samples. The first glyph structure information sample is a text description used to indicate the structural features of the characters included in the oracle bone image sample.

[0232] Data augmentation processing was performed on oracle bone script image samples;

[0233] The following processing is performed using the semantic image generation model to be trained:

[0234] Encode the oracle bone script image samples after data augmentation to obtain the original image feature samples corresponding to the oracle bone script image samples.

[0235] Noise is added to the original image feature samples to obtain noisy image feature samples corresponding to the original image feature samples;

[0236] Denoising is performed on the noisy image feature samples based on semantic prompt information samples and first character structure information samples to obtain the target image feature samples corresponding to the oracle bone script image samples.

[0237] Vector parameter generation processing is performed on the feature samples of the target image to obtain the predicted vector parameters corresponding to the oracle bone image samples.

[0238] In some embodiments, when the semantic image generation model training unit is used to denoise noisy image feature samples based on semantic prompt information samples and glyph structure information samples to obtain target image feature samples corresponding to oracle bone script image samples, it is specifically used for:

[0239] Obtain a fused text prompt sample that includes semantic prompt information samples and first glyph structure information samples;

[0240] Calculate the cross-attention map samples between the fused text prompt samples and the noisy image feature samples;

[0241] Based on the cross-attention map samples, the noisy image feature samples are denoised to obtain the target image feature samples corresponding to the oracle bone image samples.

[0242] The semantic image generation model training unit is also used for;

[0243] Obtain a second glyph structure information sample, which is an image description used to indicate the structural features of the characters included in the oracle bone image sample;

[0244] Based on the second character structure information sample and the cross attention map sample, the character structure generation loss is calculated;

[0245] Based on the character structure generation loss, the parameters of the feature generation model to be trained are adjusted.

[0246] In some embodiments, the semantic image generation model training unit is further configured to:

[0247] The noise loss is determined based on the difference between the predicted noise and the noise label. The predicted noise is the noise predicted during the denoising process of the noisy image feature samples, and the noise label is the noise added when the oracle bone image samples are denoised.

[0248] Based on noise loss, the parameters of the feature generation model to be trained are adjusted.

[0249] It should be noted that the functions of each module in the image generation device 800 in this application embodiment can be referred to the specific implementation of any embodiment in the above method embodiments, and will not be repeated here.

[0250] Each unit in the above-described device can be implemented entirely or partially through software, hardware, or a combination thereof. Each unit can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each unit.

[0251] For example, the image generating apparatus 800 can be integrated into a terminal or server that has storage and a processor and thus computing power, or the image generating apparatus 800 can be the terminal or server.

[0252] In some embodiments, this application also provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0253] Figure 9 A schematic structural diagram of the computer device provided in the embodiments of this application, such as Figure 9 As shown, the computer device 900 may include: a communication interface 901, a memory 902, a processor 903, and a communication bus 904. The communication interface 901, memory 902, and processor 903 communicate with each other via the communication bus 904. The communication interface 901 is used for data communication between the device 900 and external devices. The memory 902 can be used to store software programs and modules, and the processor 903 executes the software programs and modules stored in the memory 902, such as the software programs for the corresponding operations in the aforementioned method embodiments.

[0254] In some embodiments, the processor 903 may invoke software programs and modules stored in the memory 902 to perform the following operations: acquiring an oracle bone script image to be processed, semantic prompt information of the oracle bone script image to be processed, and glyph structure information of the oracle bone script image to be processed, wherein the semantic prompt information is used to indicate the content expressed by the oracle bone script image to be processed, and the glyph structure information is used to indicate the structural features of the characters included in the oracle bone script image to be processed; acquiring the original image features corresponding to the oracle bone script image to be processed; adding noise to the original image features to obtain noisy image features corresponding to the original image features; performing denoising processing on the noisy image features based on the semantic prompt information and glyph structure information to obtain target image features corresponding to the oracle bone script image to be processed; and performing semantic image generation processing on the target image features to obtain a target semantic image corresponding to the oracle bone script image to be processed.

[0255] In some embodiments, computer device 900 may be integrated into a terminal or server that has storage and a processor, thus possessing computing capabilities; alternatively, computer device 300 may be the terminal or server. The terminal may be a smartphone, tablet, laptop, smart TV, smart speaker, wearable smart device, personal computer, or similar device. The server may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0256] This application also provides a computer-readable storage medium for storing a computer program. This computer-readable storage medium can be applied to a computer device, and the computer program causes the computer device to execute the corresponding processes in the methods described above in the embodiments of this application; for brevity, further details are omitted here.

[0257] This application also provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the corresponding processes in the methods described above in the embodiments of this application. For brevity, these details will not be elaborated further here.

[0258] This application also provides a computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the corresponding processes in the methods described above in the embodiments of this application. For brevity, these details will not be elaborated further here.

[0259] It should be understood that the processor in the embodiments of this application may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0260] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0261] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0262] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0263] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0264] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0265] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0266] In addition, the functional units in the embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0267] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer or a server) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0268] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image generation method, characterized in that, The method includes: The process involves acquiring an oracle bone script image to be processed, semantic prompts for the oracle bone script image to be processed, and glyph structure information of the oracle bone script image to be processed. The semantic prompts are used to indicate the content expressed by the oracle bone script image to be processed, and the glyph structure information is used to indicate the structural features of the characters included in the oracle bone script image to be processed. Obtain the original image features corresponding to the oracle bone script image to be processed; The original image features are subjected to noise processing to obtain noisy image features corresponding to the original image features; Based on the semantic prompt information and the character structure information, the noisy image features are denoised to obtain the target image features corresponding to the oracle bone script image to be processed. The target image features are subjected to semantic image generation processing to obtain the target semantic image corresponding to the oracle bone script image to be processed.

2. The image generation method as described in claim 1, characterized in that, The denoising process for the noisy image features based on the semantic prompt information and the glyph structure information includes: Obtain a fused text prompt that includes the semantic prompt information and the glyph structure information; Calculate the cross-attention map between the fused text prompt and the noisy image features; The noisy image features are denoised based on the cross-attention map to obtain the target image features corresponding to the oracle bone script image to be processed.

3. The image generation method as described in claim 1, characterized in that, The step of performing semantic image generation processing on the target image features to obtain the target semantic image corresponding to the oracle bone script image to be processed includes: The target image features are processed to generate vector parameters, thereby obtaining the target vector parameters corresponding to the target image features; The target vector parameters are subjected to semantic image generation processing to obtain the target semantic image corresponding to the oracle bone script image to be processed.

4. The image generation method as described in claim 3, characterized in that, The semantic image generation process, which performs semantic image generation on the features of the target image to obtain the target semantic image corresponding to the oracle bone script image to be processed, is performed by a semantic image generation model. The training steps of the semantic image generation model include: Obtain oracle bone script image samples and the corresponding vector parameter labels of the oracle bone script image samples; Obtain the prediction vector parameters of the semantic image generation model to be trained for the oracle bone image sample; The vector parameter constraint loss is determined based on the difference between the predicted vector parameters and the vector parameter labels; Based on the vector parameter constraint loss, the parameters of the semantic image generation model to be trained are adjusted to obtain the semantic image generation model.

5. The image generation method as described in claim 4, characterized in that, The training steps of the semantic image generation model also include: The semantic image generation model to be trained generates the predicted semantic image corresponding to the predicted vector parameters. The semantic image generation loss is determined based on the difference between the predicted semantic image and the oracle bone image sample; The step of adjusting the parameters of the semantic image generation model to be trained based on the vector parameter constraint loss to obtain the semantic image generation model includes: Based on the vector parameter constraint loss and the semantic image generation loss, the parameters of the semantic image generation model to be trained are adjusted to obtain the semantic image generation model.

6. The image generation method as described in claim 5, characterized in that, Based on the difference between the predicted semantic image and the oracle bone image sample, the semantic image generation loss is determined, including: Data smoothing processing is performed on the predicted semantic image and the oracle bone image sample respectively; The semantic image generation loss is determined based on the difference between the predicted semantic image after data smoothing and the oracle bone image sample after data smoothing.

7. The image generation method as described in claim 4, characterized in that, The step of obtaining the prediction vector parameters of the semantic image generation model to be trained for the oracle bone image samples includes: Obtain semantic prompt information samples corresponding to the oracle bone image samples and first glyph structure information samples corresponding to the oracle bone image samples. The first glyph structure information samples are text descriptions used to indicate the structural features of the characters included in the oracle bone image samples. Data augmentation processing is performed on the oracle bone image samples; The following processing is performed using the semantic image generation model to be trained: The oracle bone script image samples after data augmentation are encoded to obtain the original image feature samples corresponding to the oracle bone script image samples. The original image feature samples are subjected to noise processing to obtain noisy image feature samples corresponding to the original image feature samples; Based on the semantic prompt information sample and the first character structure information sample, the noisy image feature sample is denoised to obtain the target image feature sample corresponding to the oracle bone image sample. The target image feature samples are processed to generate vector parameters, thereby obtaining the predicted vector parameters corresponding to the oracle bone image samples.

8. The image generation method as described in claim 7, characterized in that, The step of denoising the noisy image feature samples based on the semantic prompt information samples and the character structure information samples to obtain the target image feature samples corresponding to the oracle bone script image samples includes: Obtain a fused text prompt sample containing the semantic prompt information sample and the first glyph structure information sample; Calculate the cross-attention map samples between the fused text prompt samples and the noisy image feature samples; Based on the cross-attention map samples, the noisy image feature samples are denoised to obtain the target image feature samples corresponding to the oracle bone image samples; The method further includes: Obtain a second glyph structure information sample, which is an image description used to indicate the structural features of the characters included in the oracle bone image sample; Based on the second glyph structure information sample and the cross attention map sample, the glyph structure generation loss is calculated; Based on the glyph structure generation loss, the parameters of the semantic image generation model to be trained are adjusted.

9. The image generation method as described in claim 8, characterized in that, The method further includes: The noise loss is determined based on the difference between the predicted noise and the noise label. The predicted noise is the noise predicted during the denoising process of the noisy image feature sample, and the noise label is the noise added when the oracle bone image sample is denoised. Based on the noise loss, the parameters of the semantic image generation model to be trained are adjusted.

10. An image generation apparatus, characterized in that, The device includes: The first acquisition unit is used to acquire the oracle bone image to be processed, the semantic prompt information of the oracle bone image to be processed, and the glyph structure information of the oracle bone image to be processed. The semantic prompt information is used to indicate the content expressed by the oracle bone image to be processed, and the glyph structure information is used to indicate the structural features of the characters included in the oracle bone image to be processed. The second acquisition unit is used to acquire the original image features corresponding to the oracle bone script image to be processed. A noise-adding processing unit is used to add noise to the original image features to obtain noisy image features corresponding to the image features; The denoising unit is used to denoise the noisy image features based on the semantic prompt information and the glyph structure information to obtain the target image features corresponding to the oracle bone script image to be processed. The image generation unit is used to perform semantic image generation processing on the features of the target image to obtain the target semantic image corresponding to the oracle bone script image to be processed.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted for loading by a processor to perform the image generation method as described in any one of claims 1-9.

12. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program, and the processor executing the image generation method as described in any one of claims 1-9 by calling the computer program stored in the memory.

13. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the image generation method according to any one of claims 1-9.

Images