Target image acquisition and model acquisition method, device, equipment and storage medium

Abstract

The present disclosure provides a method, device, equipment and storage medium for acquiring a target image and a model, which involves artificial intelligence fields such as computer vision, deep learning and large models, and can be applied to scenes such as artificial intelligence content generation. The method may include: acquiring a first image to be processed and a first text content; acquiring image features of the first image and acquiring text features of the first text content; generating target image features based on the acquired image features and text features; generating a target image based on the target image features, wherein the target image is an image obtained by adding the first text content to the first image.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to target image acquisition and model acquisition methods, devices, equipment and storage media in the fields of computer vision, deep learning and large models. Background Art

[0002] At present, the technology of using optical character recognition (OCR) models to recognize text content in images has been widely used in various industries, such as education, medical care, and finance. In order to train the OCR model, a large number of training samples need to be constructed. To this end, a text image editing method can be used to generate a target image, and then the training samples of the OCR model can be constructed based on the obtained target image. Summary of the invention

[0003] The present invention provides a method, device, equipment and storage medium for acquiring a target image and a model.

[0004] A method for acquiring a target image, comprising:

[0005] Obtaining a first image and first text content to be processed;

[0006] Acquire image features of the first image, and acquire text features of the first text content;

[0007] Generate target image features according to the image features and the text features;

[0008] A target image is generated according to the target image features, where the target image is an image obtained by adding the first text content to the first image.

[0009] A model acquisition method, comprising:

[0010] Obtain training samples, each of which includes a training picture;

[0011] Using the training samples to train a vector quantization generative adversarial network model;

[0012] Among them, the vector quantization generative adversarial network model consists of two parts: an image encoder and an image decoder; after training is completed, the image encoder is used to generate image features of a first image to be processed, and the image decoder is used to generate a target image based on the target image features, and the target image features are generated based on the image features of the first image and the text features of the first text content to be processed, and the target image is the image obtained by adding the first text content to the first image.

[0013] A target image acquisition device, comprising: an information acquisition module, a feature acquisition module, a feature generation module and an image generation module;

[0014] The information acquisition module is used to acquire the first image and the first text content to be processed;

[0015] The feature acquisition module is used to acquire the image features of the first image and acquire the text features of the first text content;

[0016] The feature generation module is used to generate target image features according to the image features and the text features;

[0017] The image generation module is used to generate a target image according to the target image features, and the target image is an image obtained by adding the first text content to the first image.

[0018] A model acquisition device, comprising: a sample acquisition module and a model training module;

[0019] The sample acquisition module is used to acquire training samples, each training sample includes a training picture;

[0020] The model training module is used to train the vector quantization generative adversarial network model using the training samples; wherein the vector quantization generative adversarial network model consists of two parts: an image encoder and an image decoder; after the training is completed, the image encoder is used to generate image features of a first image to be processed, and the image decoder is used to generate a target image according to the target image features, wherein the target image features are generated according to the image features of the first image and the text features of the first text content to be processed, and the target image is the image obtained by adding the first text content to the first image.

[0021] An electronic device, comprising:

[0022] at least one processor; and

[0023] a memory communicatively connected to the at least one processor; wherein,

[0024] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described above.

[0025] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method as described above.

[0026] A computer program product comprises a computer program / instruction, wherein the computer program / instruction implements the method described above when executed by a processor.

[0027] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0029] Figure 1 This is a flow chart of an embodiment of the target image acquisition method disclosed in the present invention;

[0030] Figure 2 is a schematic diagram of the first picture of the present disclosure;

[0031] Figure 3 A schematic diagram of the target image described in the present disclosure;

[0032] Figure 4 This is a schematic diagram of the overall implementation process of the target image acquisition method disclosed in the present invention;

[0033] Figure 5 A flowchart of an embodiment of the model acquisition method disclosed in the present invention;

[0034] Figure 6 Schematic diagram of the training method of the VQGAN model described in the present disclosure;

[0035] Figure 7 Schematic diagram of the structure of the target image acquisition device embodiment 700 of the present disclosure;

[0036] Figure 8 It is a schematic diagram of the structure of the model acquisition device embodiment 800 of the present disclosure;

[0037] Fig. 9 A schematic block diagram of an electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0038] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0039] In addition, it should be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0040] Figure 1 Flow chart of an embodiment of the target image acquisition method disclosed in the present invention. Figure 1 As shown, the following specific implementation methods are included.

[0041] In step 101, a first image and first text content to be processed are obtained.

[0042] In step 102, image features of a first image are obtained, and text features of a first text content are obtained.

[0043] In step 103, target image features are generated according to the acquired image features and text features.

[0044] In step 104, a target image is generated according to the target image features, where the target image is an image obtained by adding the first text content to the first image.

[0045] In practical applications, the required target images can be edited and generated based on text content and images. Traditional implementation methods are usually complicated, such as requiring editing and splitting of images and converting text content into text images, which is inefficient.

[0046] By adopting the scheme described in the above method embodiment, based on the text image editing method of the Wensheng graph, for the first image to be processed and the first text content, some relatively simple operations such as feature acquisition and feature processing can be performed to obtain the required target image, thereby improving the image generation efficiency and ensuring the generated image effect.

[0047] Preferably, the first picture may be a picture including the second text content. Accordingly, adding the first text content to the first picture may include: replacing the second text content in the first picture with the first text content.

[0048] And / or, preferably, the first picture may be a picture that does not include text content, and accordingly, adding the first text content to the first picture may include: adding the first text content to the first picture.

[0049] Take the first picture including the second text content as an example, Figure 2 is a schematic diagram of the first picture of the present disclosure, such as Figure 2As shown, the second text content is "LEAVES", and assuming that the first text content is "RESULT", then after replacement, the following can be obtained: Figure 3 The target image shown, Figure 3 A schematic diagram of the target image described in the present disclosure.

[0050] Assumptions Figure 2 The first picture shown does not include the second text content, such as only including the background content shown (the second text content is regarded as the foreground), then the first text content can also be added to the first picture to obtain the desired target picture.

[0051] In practical applications, which of the above methods to adopt can be determined according to actual needs, which is very flexible and convenient.

[0052] Preferably, the acquired target image and the first text content can also be used to form a training sample, the first text content is the label corresponding to the target image, and the training sample is used to train the OCR model. The obtained training sample can be used as the training sample of the OCR model, and the OCR model is trained using the training sample.

[0053] For the target image, its corresponding label is known, that is, the first text content. Accordingly, the target image and the first text content can be used to form a training sample, and the training sample can be used to train the OCR model, that is, the training samples required for the OCR model training can be automatically constructed, thereby reducing the implementation cost and improving the training effect of the OCR model. Subsequently, the trained OCR model can be used to perform actual optical character recognition.

[0054] Preferably, for the first image, a picture encoder (tokenizer) in a pre-trained vector quantization generative adversarial network (VQGAN) model can be used to encode the first image, so as to obtain the required picture features of the first image.

[0055] In addition, preferably, the image decoder (detokenizer) in the VQGAN model can be used to decode the target image features to obtain the target image. The VQGAN model consists of two parts: an image encoder and an image decoder.

[0056] Among them, using the image encoder, the first image can be encoded into a discrete feature representation, and the image features of the first image can be obtained, thereby realizing the image-to-feature conversion. Using the image decoder, the target image features can be decoded into the target image, thereby realizing the feature-to-image conversion.

[0057] In addition, preferably, a pre-trained text encoder may be used to encode the first text content, thereby obtaining text features of the first text content.

[0058] That is, the first text content can be encoded using a text encoder, so that the required text features can be obtained efficiently and accurately. In practical applications, a trained language model can be used as a text encoder.

[0059] Preferably, a pre-trained generative decoder may be used to decode the acquired image features and text features, thereby obtaining target image features including both image features and text features.

[0060] That is, the image features and text features can be used as the input of the generative decoder, so as to efficiently and accurately obtain the required target image features. In practical applications, a transformer decoder can be used as a generative decoder.

[0061] Combined with the above introduction, Figure 4 Schematic diagram of the overall implementation process of the target image acquisition method disclosed in the present invention. Figure 4 As shown, assuming that the first picture includes the second text content, first, the picture encoder in the VQGAN model can be used to encode the first picture to obtain the picture features of the first picture, and the text encoder can be used to encode the first text content to obtain the text features of the first text. After that, the generative decoder can be used to decode the picture features and the text features to obtain the target picture features including both the picture features and the text features. The generative decoder can be an autoregressive model. Further, the picture decoder in the VQGAN model can be used to decode the target picture features to obtain the desired target picture.

[0062] It can be seen that the VQGAN model is a main model required for the target image acquisition method described in the present disclosure. The following is an explanation of how to obtain the model.

[0063] Figure 5 Flow chart of an embodiment of the model acquisition method described in the present disclosure. Figure 5 As shown, the following specific implementation methods are included.

[0064] In step 501, training samples are obtained, each of which includes a training picture.

[0065] In step 502, the VQGAN model is trained using training samples; wherein the VQGAN model consists of two parts: an image encoder and an image decoder; after the training is completed, the image encoder is used to generate image features of a first image to be processed, and the image decoder is used to generate a target image based on the target image features, the target image features are generated based on the image features of the first image and the text features of the first text content to be processed, and the target image is the image obtained by adding the first text content to the first image.

[0066] By adopting the scheme described in the above method embodiment, based on the text image editing method of the Wensheng graph, for the first image to be processed and the first text content, some relatively simple operations such as feature acquisition and feature processing implemented by the VQGAN model can be used to obtain the required target image, thereby improving the image generation efficiency and ensuring the generated image effect.

[0067] Preferably, the training picture may be a training picture including text content, and the method of using training samples to train the VQGAN model may include: during the training process, using a picture encoder to generate picture features of the training picture, and obtaining a picture generated by a picture decoder according to the picture features, obtaining the difference between the generated picture and the training picture, and updating the model parameters of the picture encoder and the picture decoder according to the difference.

[0068] Figure 6 Schematic diagram of the training method of the VQGAN model described in this disclosure. Figure 6 As shown, for the input training picture, the picture encoder can be used to encode it into a discrete feature representation, that is, to obtain the picture features of the training picture, and then the picture decoder can be used to generate a picture according to the picture features, that is, the discrete feature representation is decoded into the training picture using the picture decoder.

[0069] Theoretically, any image can be used as a training image, but the training image is usually a training image that includes text content. The image encoder and image decoder in the VQGAN model are trained together. The image encoder is first used to encode the input training image into image features, and then the image decoder is used to generate an image based on the image features. From the perspective of the image decoder, it is hoped that the generated image is consistent with the input training image. Accordingly, the difference between the generated image and the training image can be compared, and the model parameters can be updated accordingly based on the difference.

[0070] Once the VQGAN model is trained to convergence, it can be put into practical use, such as applying it to Figure 1In the method corresponding to the illustrated embodiment, a first picture may be encoded using a picture encoder to obtain picture features of the first picture, and a target picture feature may be decoded using a picture decoder to obtain a target picture.

[0071] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the described order of actions, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present disclosure. In addition, for parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0072] In summary, the scheme described in the embodiment of the method disclosed herein can improve the efficiency of image generation and ensure the effect of the generated images. Moreover, it can be applied to any images and text content, that is, it has wide applicability. In addition, the generated images can be used to construct training samples of the OCR model, thereby improving the training effect of the OCR model, etc.

[0073] The above is an introduction to the method embodiment. The following is a further explanation of the scheme disclosed in the present invention through an apparatus embodiment.

[0074] Figure 7 FIG. 7 is a schematic diagram of the structure of the target image acquisition device embodiment 700 described in the present disclosure. Figure 7 As shown, it includes: an information acquisition module 701, a feature acquisition module 702, a feature generation module 703 and a picture generation module 704.

[0075] The information acquisition module 701 is used to acquire the first image and the first text content to be processed.

[0076] The feature acquisition module 702 is used to acquire the image features of the first image and acquire the text features of the first text content.

[0077] The feature generation module 703 is used to generate target image features according to the acquired image features and text features.

[0078] The image generation module 704 is used to generate a target image according to the target image features, where the target image is an image obtained by adding the first text content to the first image.

[0079] By adopting the scheme described in the above-mentioned device embodiment, based on the text image editing method of the Wensheng graph, for the first image to be processed and the first text content, some relatively simple operations such as feature acquisition and feature processing can be performed to obtain the required target image, thereby improving the image generation efficiency and ensuring the generated image effect.

[0080] Preferably, the first picture may be a picture including the second text content. Accordingly, adding the first text content to the first picture may include: replacing the second text content in the first picture with the first text content.

[0081] And / or, preferably, the first picture may be a picture that does not include text content, and accordingly, adding the first text content to the first picture may include: adding the first text content to the first picture.

[0082] In addition, preferably, the image generation module 704 can also use the acquired target image and the first text content to form a training sample, where the first text content is a label corresponding to the target image, and the training sample is used to train the OCR model.

[0083] Preferably, for the first picture, the feature acquisition module 702 can use the picture encoder in the pre-trained VQGAN model to encode the first picture, so as to obtain the picture features of the first picture, and the picture generation module 704 can use the picture decoder in the VQGAN model to decode the target picture features, so as to obtain the target picture. The VQGAN model consists of two parts: a picture encoder and a picture decoder.

[0084] Among them, using the image encoder, the first image can be encoded into a discrete feature representation, and the image features of the first image can be obtained, thereby realizing the image-to-feature conversion. Using the image decoder, the target image features can be decoded into the target image, thereby realizing the feature-to-image conversion.

[0085] In addition, preferably, the feature acquisition module 702 may use a pre-trained text encoder to encode the first text content, thereby obtaining text features of the first text content.

[0086] Preferably, the feature generation module 703 may also use a pre-trained generative decoder to decode the acquired image features and text features, thereby obtaining target image features including both image features and text features.

[0087] Figure 8 FIG. 8 is a schematic diagram of the structure of the model acquisition device embodiment 800 described in the present disclosure. Figure 8 As shown, it includes: a sample acquisition module 801 and a model training module 802.

[0088] The sample acquisition module 801 is used to acquire training samples, each of which includes a training picture.

[0089] The model training module 802 is used to train the VQGAN model using training samples; wherein the VQGAN model consists of two parts: an image encoder and an image decoder; after the training is completed, the image encoder is used to generate image features of the first image to be processed, and the image decoder is used to generate a target image based on the target image features, and the target image features are generated based on the image features of the first image and the text features of the first text content to be processed, and the target image is the image obtained by adding the first text content to the first image.

[0090] By adopting the scheme described in the above-mentioned device embodiment, based on the text image editing method of the Wensheng graph, for the first image to be processed and the first text content, some relatively simple operations such as feature acquisition and feature processing implemented by the VQGAN model can be used to obtain the required target image, thereby improving the image generation efficiency and ensuring the generated image effect.

[0091] Preferably, the training picture may be a training picture including text content, and the model training module 802 may use the training samples to train the VQGAN model in a manner that includes: during the training process, using the picture encoder to generate picture features of the training picture, and obtaining the picture generated by the picture decoder according to the picture features, obtaining the difference between the generated picture and the training picture, and updating the model parameters of the picture encoder and the picture decoder according to the difference.

[0092] Theoretically, any image can be used as a training image, but the training image is usually a training image that includes text content. The image encoder and image decoder in the VQGAN model are trained together. The image encoder is first used to encode the input training image into image features, and then the image decoder is used to generate an image based on the image features. From the perspective of the image decoder, it is hoped that the generated image is consistent with the input training image. Accordingly, the difference between the generated image and the training image can be compared, and the model parameters can be updated accordingly based on the difference.

[0093] Figure 7 and Figure 8 The specific working process of the illustrated device embodiment can refer to the relevant description in the aforementioned method embodiment and will not be described in detail.

[0094] In summary, the scheme described in the embodiment of the device disclosed herein can improve the efficiency of image generation and ensure the effect of the generated images. Moreover, it can be applied to any images and text content, that is, it has wide applicability. In addition, the generated images can be used to construct training samples of the OCR model, thereby improving the training effect of the OCR model, etc.

[0095] The scheme disclosed in the present invention can be applied to the field of artificial intelligence, especially computer vision, deep learning, and large models. Artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It has both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing. Artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0096] The pictures and text contents in the embodiments described in this disclosure are not targeted at a specific user and do not reflect the personal information of a specific user. In the technical solution of this disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0097] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0098] Fig. 9 A schematic block diagram of an electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0099] like Fig. 9 As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0100] A number of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0101] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as the methods described in the present disclosure. For example, in some embodiments, the methods described in the present disclosure may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the method described in the present disclosure may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the method described in the present disclosure in any other appropriate manner (e.g., by means of firmware).

[0102] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0103] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0104] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0105] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0106] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0107] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0108] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0109] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for acquiring a target image, comprising: Obtaining a first image and first text content to be processed; Acquiring the image features of the first image, including: encoding the first image using an image encoder in a pre-trained vector quantization generative adversarial network model to obtain the image features; and acquiring text features of the first text content; Generate target image features according to the image features and the text features; Generate a target picture according to the target picture features, including: using the picture decoder in the vector quantization generative adversarial network model to decode the target picture features to obtain the target picture, the vector quantization generative adversarial network model is composed of the picture encoder and the picture decoder, and the target picture is a picture obtained by adding the first text content to the first picture.

2. The method according to claim 1, wherein: The first picture is a picture including second text content; Adding the first text content to the first picture includes: replacing the second text content in the first picture with the first text content.

3. The method according to claim 1, wherein: The first picture is a picture that does not include text content; Adding the first text content to the first picture includes: adding the first text content to the first picture.

4. The method according to claim 2 or 3, further comprising: The target image and the first text content are used to form a training sample, the first text content is a label corresponding to the target image, and the training sample is used to train an optical character recognition model.

5. The method according to any one of claims 1 to 3, wherein: The acquiring of the text feature of the first text content includes: encoding the first text content using a pre-trained text encoder to obtain the text feature.

6. The method according to any one of claims 1 to 3, wherein: The generating target image features according to the image features and the text features comprises: decoding the image features and the text features by using a pre-trained generative decoder to obtain the target image features including both the image features and the text features.

7. A model acquisition method, comprising: Obtain training samples, each of which includes a training picture; Using the training samples to train a vector quantization generative adversarial network model; Among them, the vector quantization generative adversarial network model consists of two parts: an image encoder and an image decoder; after training, the image encoder is used to encode the first image to be processed to obtain image features, and the image decoder is used to decode the target image features to obtain the target image, and the target image features are generated according to the image features of the first image and the text features of the first text content to be processed, and the target image is the image obtained after the first text content is added to the first image.

8. The method according to claim 7, wherein: The training picture is a training picture including text content; The method of training the vector quantization generative adversarial network model using the training samples includes: during the training process, using the picture encoder to generate picture features of the training picture, obtaining the picture generated by the picture decoder according to the picture features, obtaining the difference between the generated picture and the training picture, and updating the model parameters of the picture encoder and the picture decoder according to the difference.

9. A target image acquisition device, comprising: Information acquisition module, feature acquisition module, feature generation module and image generation module; The information acquisition module is used to acquire the first image and the first text content to be processed; The feature acquisition module is used to acquire the picture features of the first picture, including: encoding the first picture using a picture encoder in a pre-trained vector quantization generative adversarial network model to obtain the picture features; and acquiring text features of the first text content; The feature generation module is used to generate target image features according to the image features and the text features; The image generation module is used to generate a target image according to the target image features, including: using the image decoder in the vector quantization generative adversarial network model to decode the target image features to obtain the target image, the vector quantization generative adversarial network model is composed of the image encoder and the image decoder, and the target image is the image obtained by adding the first text content to the first image.

10. The device according to claim 9, wherein: The first picture is a picture including second text content; Adding the first text content to the first picture includes: replacing the second text content in the first picture with the first text content.

11. The device according to claim 9, wherein: The first picture is a picture that does not include text content; Adding the first text content to the first picture includes: adding the first text content to the first picture.

12. The device according to claim 10 or 11, wherein: The image generation module is further used to use the target image and the first text content to form a training sample, the first text content is a label corresponding to the target image, and the training sample is used to train the optical character recognition model.

13. The device according to any one of claims 9 to 11, wherein: The feature acquisition module encodes the first text content using a pre-trained text encoder to obtain the text feature.

14. The device according to any one of claims 9 to 11, wherein: The feature generation module decodes the image features and the text features using a pre-trained generative decoder to obtain the target image features that include both the image features and the text features.

15. A model acquisition device, comprising: Sample acquisition module and model training module; The sample acquisition module is used to acquire training samples, each training sample includes a training picture; The model training module is used to train the vector quantization generative adversarial network model using the training samples; wherein the vector quantization generative adversarial network model consists of two parts: an image encoder and an image decoder; after the training is completed, the image encoder is used to encode the first image to be processed to obtain image features, and the image decoder is used to decode the target image features to obtain the target image, wherein the target image features are generated based on the image features of the first image and the text features of the first text content to be processed, and the target image is the image obtained by adding the first text content to the first image.

16. The device according to claim 15, wherein: The training picture is a training picture including text content; During the training process, the model training module uses the picture encoder to generate picture features of the training picture, obtains the picture generated by the picture decoder based on the picture features, obtains the difference between the generated picture and the training picture, and updates the model parameters of the picture encoder and the picture decoder based on the difference.

17. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to make a computer execute the method according to any one of claims 1 to 8.

19. A computer program product, comprising a computer program / instruction, wherein when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images