Image generation method and device, electronic equipment and readable storage medium

Abstract

The present disclosure provides an image generation method and device, electronic equipment and readable storage medium, relates to the technical field of artificial intelligence, specifically to the technical field of deep learning and computer vision, and can be applied to scenarios such as character recognition. The image generation method comprises: obtaining a first image, obtaining text content in the first image; determining a target text library according to the text content; and generating a second image according to the first image and target text in the target text library. The present disclosure can simplify the steps during image generation and improve the efficiency during image generation.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, in particular to the technical field of deep learning and computer vision, and can be applied to scenarios such as character recognition. Specifically, an image generation method and device, an electronic device, and a readable storage medium are provided. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, many problems in real life can be solved based on artificial intelligence technology, for example, text content in an image is recognized based on a text recognition model.

[0003] However, the prior art needs to use a large number of images to train a text recognition model, and the images used for training need to be obtained at a huge human cost, which has the technical problems of high cost and low efficiency of image acquisition. SUMMARY

[0004] According to a first aspect of the present disclosure, an image generation method is provided, comprising: obtaining a first image to obtain text content in the first image; determining a target text library according to the text content; and generating a second image according to the first image and target text in the target text library.

[0005] According to a second aspect of the present disclosure, an image generation device is provided, comprising: an obtaining unit configured to obtain a first image to obtain text content in the first image; a determining unit configured to determine a target text library according to the text content; and a generating unit configured to generate a second image according to the first image and target text in the target text library.

[0006] According to a third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described above.

[0007] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to perform the method described above.

[0008] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method described above.

[0009] It can be seen from the above technical solution that the second image is generated according to the first image and the target text library determined by the text content in the first image, so that the steps during image generation can be simplified, and the efficiency during image generation can be improved.

[0010] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0011] The accompanying drawings are used to better understand the present scheme and do not constitute a limitation on the present disclosure. Among them:

[0012] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0013] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure;

[0014] Figure 3 is a schematic diagram according to a third embodiment of the present disclosure;

[0015] Figure 4 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0016] Figure 5 is a block diagram of an electronic device for implementing the image generation method according to the embodiments of the present disclosure. DETAILED DESCRIPTION

[0017] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to help understanding, and should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, in order to be clear and concise, the description below omits the description of well-known functions and mechanisms.

[0018] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure. As Figure 1 shown, the image generation method of the present embodiment specifically includes the following steps:

[0019] S101, acquiring a first image to obtain text content in the first image;

[0020] S102, determining a target text library according to the text content;

[0021] S103, generating a second image according to the first image and the target text in the target text library.

[0022] The image generation method of the embodiment first acquires a first image and obtains text content in the first image, then determines a target text library according to the obtained text content, and finally generates a second image according to the acquired first image and the target text in the determined target text library. The embodiment can simplify the steps during image generation and improve the efficiency during image generation.

[0023] When the embodiment performs S101 to acquire the first image, the image input by the input end can be used as the first image, or the image selected by the input end on the network can be used as the first image.

[0024] The second image generated by the embodiment can be used to train the preset text recognition model to obtain a target text recognition model, so as to achieve the purpose of recognizing the text content in the image by using the target text recognition model.

[0025] Therefore, in order to ensure that the target text recognition model obtained by training has good recognition performance in some specific scenarios, the optional implementation manner that can be adopted by the embodiment when performing S101 to acquire the first image is to determine a target scenario and acquire an image corresponding to the determined target scenario as the first image.

[0026] In the embodiment, when S101 is performed to determine the target scenario, the scenario input by the input end or the scenario selected by the input end can be used as the target scenario. The determined target scenario can be at least one of an industrial scenario, a traffic scenario, a financial scenario, a financial scenario, etc. After S101 is performed to determine the target scenario, the determined target scenario can be used as a keyword to acquire an image corresponding to the determined target scenario by network searching.

[0027] It can be understood that the number of the first images acquired by the embodiment in S101 can be one or multiple. The number of the acquired first images is not limited by the embodiment.

[0028] After S101 is performed to acquire the first image, the embodiment can use model recognition or manual annotation to obtain the text content in the first image. If multiple first images are acquired, the text content in each first image is obtained by the embodiment in S101.

[0029] After S101 is performed to acquire the first image and obtain the text content in the first image, S102 is performed to determine the target text library according to the text content. The number of the target text libraries determined by the embodiment in S102 is one or multiple, and each target text library contains multiple different target texts.

[0030] Specifically, when determining the target text library according to the obtained text content in S102, the embodiment can adopt an optional implementation manner as follows: obtaining a text type of the text content; and taking a text library corresponding to the obtained text type as the target text library.

[0031] That is, the embodiment determines the target text library according to the text type of the text content in the first image, and different target text libraries contain target texts corresponding to different text types, so that the embodiment can ensure that the text content in the generated second image has the same text type as the text content in the first image, thereby improving the accuracy of the generated second image.

[0032] When performing S102, the embodiment can use a plurality of pre-trained text classifiers to obtain the text type of the text content; wherein each text classifier corresponds to one or more preset text types, and after inputting the text content into the text classifier, the text classifier can output the confidence that the text content belongs to one or more preset text types.

[0033] When performing S102, the embodiment can take the preset text type corresponding to the maximum confidence output by each text classifier as the text type of the text content; or can sort the preset text types in descending order of confidence, and take the preset text types ranked in the top N positions as the text type of the text content, N being a positive integer greater than or equal to 1.

[0034] For example, if the text classifiers used by the embodiment when performing S102 are text classifier 1 and text classifier 2, if the preset text types corresponding to the text classifier 1 are English type, Chinese type, Korean type and Japanese type, and the preset text types corresponding to the text classifier 2 are address type, company type, name type and title type, the text classifier 1 is used to output the confidence that the text content belongs to the English type, the confidence that the text content belongs to the Chinese type, the confidence that the text content belongs to the Korean type and the confidence that the text content belongs to the Japanese type, and the text classifier 2 is used to output the confidence that the text content belongs to the address type, the confidence that the text content belongs to the company type, the confidence that the text content belongs to the name type and the confidence that the text content belongs to the title type.

[0035] When performing S102, the embodiment can take the text library corresponding to the obtained text type as the target text library according to the preset correspondence between the text type and the text library; the embodiment can also display the text library corresponding to the obtained text type in the input end, and take the text library selected by the input end as the target text library.

[0036] After performing S102 to determine the target text library according to the text content, the embodiment performs S103 to generate a second image according to the first image and the target text in the target text library.

[0037] Specifically, when performing S103 to generate the second image according to the first image and the target text in the target text library, the optional implementation manner that can be adopted by the embodiment is: obtaining a style vector of the first image, the style vector obtained by the embodiment is a vector for representing the style of the text in the first image, such as color, style, size, etc.; obtaining a semantic vector of the target text in the target text library, the semantic vector obtained by the embodiment is a vector for representing the semantic of the text content of the target text; and generating the second image using the obtained style vector and semantic vector.

[0038] That is, the embodiment generates the second image through the style vector of the obtained first image and the semantic vector of the target text, which can make the generated second image have the same image style as the first image and also have the same text content as the target text, thereby simplifying the steps when generating the second image according to the first image and improving the efficiency when generating the second image according to the first image.

[0039] In the embodiment, when performing S103 to obtain the style vector of the first image, the first image can be input into a pre-trained style extractor, so that the output result of the style extractor is taken as the style vector of the first image.

[0040] In the embodiment, when performing S103 to obtain the semantic vector of the target text, the target text can be input into a pre-trained semantic extractor, so that the output result of the semantic extractor is taken as the semantic vector of the target text.

[0041] In the embodiment, when performing S103 to generate the second image using the obtained style vector and semantic vector, the style vector and the semantic vector can be input into a pre-trained image synthesis model, so that the output result of the image synthesis model is taken as the second image.

[0042] It can be understood that if the embodiment obtains multiple first images and multiple target texts, the corresponding second image can be generated for each first image and each target text.

[0043] In order to make the generated second image have more rich and varied image styles on the basis of the image style of the first image, when performing S103 to generate the second image using the obtained style vector and semantic vector, the optional implementation manner that can be adopted by the embodiment is: obtaining a degree of freedom parameter, the embodiment can arbitrarily select one or more degree of freedom parameters in a pre-set degree of freedom parameter interval, and the selected degree of freedom parameter is used to adjust the style vector to obtain a style adjustment result; and generating the second image using the style vector, the degree of freedom parameter and the semantic vector.

[0044] In this embodiment, when generating a second image using style vectors, degrees of freedom parameters, and semantic vectors in step S103, the result of multiplying the style vectors and degrees of freedom parameters can be used as the style adjustment result. The style adjustment result and the semantic vector are then input into a pre-trained image synthesis model, and the output of the image synthesis model is used as the second image.

[0045] In other words, this embodiment can use the degree of freedom parameter to adjust the style vector to obtain the style adjustment result, so that the image style of the second image generated based on the image style of the first image is not too monotonous, thus having a richer diversity.

[0046] In addition, after generating the second image in S103, this embodiment may also include the following: using the generated second image and the target text used when generating the second image as training samples for a preset text recognition model.

[0047] In this embodiment, after using the generated second image and the target text used when generating the second image as training samples for the preset text recognition model, the preset text recognition model can be trained using the training samples to obtain the target text recognition model.

[0048] Specifically, in this embodiment, when training a preset text recognition model using training samples, the optional implementation method is as follows: the target text used when generating the second image is used as the text annotation result of the second image; the second image is input into the preset text recognition model to obtain the text prediction result output by the preset text recognition model for the second image; the loss function value is calculated based on the text annotation result and the text prediction result of the second image; the parameters of the preset text recognition model are adjusted based on the calculated loss function value until the preset text recognition model converges to obtain the target text recognition model.

[0049] In other words, after generating the second image based on the first image, this embodiment can also use the generated second image and the corresponding target text as training samples for the preset text recognition model, thereby training the preset text recognition model to obtain the target text recognition model. This can reduce the cost of obtaining training samples and obtain a large number of training samples adapted to the target scene in a short time, further improving the training efficiency of the preset text recognition model.

[0050] Figure 2 This is a schematic diagram according to the second embodiment of this disclosure. (See diagram below.) Figure 2As shown in the figure, this embodiment illustrates a flowchart of the process of generating the second image: a semantic extractor is used to extract semantic vectors from the target text, and a style extractor is used to extract style vectors from the first image; the extracted style vectors are adjusted using degree-of-freedom parameters; the semantic vectors and the adjusted style vectors are input into an image synthesis model to obtain a second image corresponding to a different image style.

[0051] Figure 3 This is a schematic diagram according to the third embodiment of this disclosure. (See diagram below.) Figure 3 As shown in the figure, this embodiment illustrates the training flowchart of the text recognition model based on the image generation method of the above embodiment: After acquiring the first image corresponding to the financial scenario, the text content and style vector of each first image are obtained respectively; the target corpus is determined according to the text content of the first image, and the semantic vector of each target text in the target corpus is obtained; after adjusting the style vector using the corresponding degree of freedom parameters, the adjusted style vector and semantic vector are input into the image synthesis model to obtain multiple second images; the multiple second images obtained are used as training samples to train the preset text recognition model to obtain a target text recognition model adapted to the financial scenario.

[0052] Figure 4 This is a schematic diagram according to the fourth embodiment of this disclosure. (See diagram below.) Figure 4 As shown, the image generation apparatus 400 of this embodiment includes:

[0053] Acquisition unit 401 is used to acquire a first image and obtain the text content in the first image;

[0054] Determining unit 402 is used to determine the target text library based on the text content;

[0055] The generation unit 403 is used to generate a second image based on the first image and the target text in the target text library.

[0056] When acquiring the first image, the acquisition unit 401 can use the image input by the input terminal as the first image, or it can use the image selected by the input terminal on the network as the first image.

[0057] Since the second image generated in this embodiment can be used to train a preset text recognition model to obtain a target text recognition model, so as to achieve the purpose of using the target text recognition model to recognize the text content in the image.

[0058] Therefore, in order to ensure that the trained text recognition model has good recognition performance in some specific scenarios, the acquisition unit 401 can adopt the following optional implementation method when acquiring the first image: determine the target scene; acquire the image corresponding to the determined target scene as the first image.

[0059] When determining the target scene, the acquisition unit 401 can use the scene input by the input terminal or the scene selected by the input terminal as the target scene. The determined target scene can be at least one of industrial scene, transportation scene, financial scene, etc. After determining the target scene, the acquisition unit 401 can use the determined target scene as a keyword to obtain the image corresponding to the determined target scene through network search.

[0060] It is understood that the number of first images acquired by the acquisition unit 401 can be one or more, and this embodiment does not limit the number of first images acquired.

[0061] After acquiring the first image, the acquisition unit 401 can use model recognition or manual annotation to obtain the text content in the first image; if multiple first images are acquired, the acquisition unit 401 will obtain the text content in each first image respectively.

[0062] In this embodiment, after the acquisition unit 401 acquires the first image and obtains the text content in the first image, the determination unit 402 determines the target text library based on the text content; wherein, the number of target text libraries determined by the determination unit 402 is one or more, and each target text library contains multiple different target texts.

[0063] Specifically, when determining the target text library based on the obtained text content, the determining unit 402 may adopt the following optional implementation methods: obtaining the text type of the text content; and using the text library corresponding to the obtained text type as the target text library.

[0064] In other words, the determining unit 402 determines the target text library based on the text type of the text content in the first image. Different target text libraries contain target texts with different text types. Therefore, the determining unit 402 can ensure that the text content in the generated second image has the same text type as the text content in the first image, thereby improving the accuracy of the generated second image.

[0065] The determining unit 402 can use multiple pre-trained text classifiers to obtain the text type of the text content; wherein, each text classifier corresponds to one or more preset text types, and after the text content is input into the text classifier, the text classifier can output the confidence level of the text content belonging to one or more preset text types.

[0066] The determining unit 402 can use the preset text type corresponding to the maximum confidence score output by each text classifier as the text type of the text content; or it can sort the preset text types in descending order of confidence score and use the top N preset text types as the text type of the text content, where N is a positive integer greater than or equal to 1.

[0067] The determining unit 402 can use the text library corresponding to the acquired text type as the target text library according to the preset correspondence between text types and text libraries; the determining unit 402 can also display the text library corresponding to the acquired text type on the input terminal and use the text library selected by the input terminal as the target text library.

[0068] In this embodiment, after the determining unit 402 determines the target text library, the generating unit 403 generates a second image based on the first image and the target text in the target text library.

[0069] Specifically, when generating a second image based on the first image and the target text in the target text library, the generation unit 403 may adopt the following optional implementation methods: obtaining the style vector of the first image; obtaining the semantic vector of the target text in the target text library; and using the obtained style vector and semantic vector to generate the second image.

[0070] In other words, the generation unit 403 generates a second image by using the style vector of the first image and the semantic vector of the target text. This not only ensures that the generated second image has the same image style as the first image, but also has the same text content as the target text, thereby simplifying the steps in generating a second image from the first image and improving the efficiency of generating a second image from the first image.

[0071] When the generation unit 403 obtains the style vector of the first image, it can input the first image into a pre-trained style extractor, and then use the output of the style extractor as the style vector of the first image.

[0072] When generating unit 403 obtains the semantic vector of the target text, it can input the target text into a pre-trained semantic extractor and use the output of the semantic extractor as the semantic vector of the target text.

[0073] When generating a second image using the acquired style vector and semantic vector, the generation unit 403 can input the style vector and semantic vector into a pre-trained image synthesis model, and then use the output of the image synthesis model as the second image.

[0074] In order to make the generated second image have a richer image style based on the image style of the first image, when generating the second image using the acquired style vector and semantic vector, the generation unit 403 can adopt the following optional implementation methods: obtain the degree of freedom parameters; and generate the second image using the style vector, degree of freedom parameters and semantic vector.

[0075] When generating a second image using style vectors, degrees of freedom parameters, and semantic vectors, the generation unit 403 can use the multiplication result between the style vectors and degrees of freedom parameters as the style adjustment result, input the style adjustment result and semantic vectors into the pre-trained image synthesis model, and then use the output of the image synthesis model as the second image.

[0076] In other words, the generation unit 403 can use the degree of freedom parameter to adjust the style vector to obtain the style adjustment result, so that the image style of the second image generated based on the image style of the first image will not be too monotonous, thus having a richer diversity.

[0077] In addition, the image generation apparatus 400 of this embodiment also includes a processing unit 404, which performs the following after the generation unit 403 generates the second image: using the generated second image and the target text used when generating the second image as training samples for a text recognition model.

[0078] The processing unit 404 uses the generated second image and the target text used when generating the second image as training samples for a preset text recognition model, and then uses the training samples to train the preset text recognition model to obtain the target text recognition model.

[0079] Specifically, when the processing unit 404 trains the preset text recognition model using training samples, the optional implementation methods are as follows: the target text used when generating the second image is used as the text annotation result of the second image; the second image is input into the preset text recognition model to obtain the text prediction result output by the preset text recognition model for the second image; the loss function value is calculated based on the text annotation result and the text prediction result of the second image; the parameters of the preset text recognition model are adjusted based on the calculated loss function value until the preset text recognition model converges to obtain the target text recognition model.

[0080] In other words, after generating the second image based on the first image, the processing unit 404 can use the generated second image and the corresponding target text as training samples for the preset text recognition model, thereby training the preset text recognition model to obtain the target text recognition model. This reduces the cost of obtaining training samples and allows for the acquisition of a large number of training samples adapted to the target scene in a short time, further improving the training efficiency of the preset text recognition model.

[0081] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0082] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0083] like Figure 5 The diagram shown is a block diagram of an electronic device for an image generation method according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0084] like Figure 5 As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 502 or a computer program loaded from storage unit 508 into random access memory (RAM) 503. RAM 503 may also store various programs and data required for the operation of device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0085] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0086] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as image generation methods. For example, in some embodiments, the image generation method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508.

[0087] In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by computing unit 501, one or more steps of the image generation method described above may be performed. Alternatively, in other embodiments, computing unit 501 may be configured to perform the image generation method by any other suitable means (e.g., by means of firmware).

[0088] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0089] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0090] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0091] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0092] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0093] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the management difficulties and weak business scalability inherent in traditional physical hosts and VPS (Virtual Private Server) services. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0094] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0095] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An image generation method, comprising: Acquire the first image and obtain the text content of the first image; Based on the text content, determine the target text library; A second image is generated based on the first image and the target text in the target text library, wherein the target text is all the text in the target text library; The step of determining the target text library based on the text content includes: Obtain the text type of the text content; The text library corresponding to the text type shall be used as the target text library; The step of generating a second image based on the first image and the target text in the target text library includes: Obtain the style vector of the first image; Obtain the semantic vector of the target text in the target text library; The style vector and the semantic vector are input into the image synthesis model, and the second image is obtained based on the output of the image synthesis model.

2. The method according to claim 1, wherein, The acquisition of the first image includes: Identify the target scenario; Obtain the image corresponding to the target scene and use it as the first image.

3. The method according to claim 1, wherein, The step of generating the second image using the style vector and the semantic vector includes: Obtain the degrees of freedom parameters, which are used to adjust the style vector; The second image is generated using the style vector, the degree-of-freedom parameter, and the semantic vector.

4. The method according to any one of claims 1-3, further comprising: After generating the second image, the second image and the target text used when generating the second image are used as training samples for a preset text recognition model.

5. An image generation apparatus, comprising: An acquisition unit is used to acquire a first image and obtain the text content in the first image; The determining unit is used to determine the target text library based on the text content; The generation unit is configured to generate a second image based on the first image and the target text in the target text library, wherein the target text is all the text in the target text library; Specifically, when determining the target text library based on the text content, the determining unit performs the following: Obtain the text type of the text content; The text library corresponding to the text type shall be used as the target text library; When the generation unit generates the second image based on the first image and the target text in the target text library, it specifically performs the following: Obtain the style vector of the first image; Obtain the semantic vector of the target text in the target text library; The style vector and the semantic vector are input into the image synthesis model, and the second image is obtained based on the output of the image synthesis model.

6. The apparatus according to claim 5, wherein, When acquiring the first image, the acquisition unit specifically performs the following: Identify the target scenario; Obtain the image corresponding to the target scene and use it as the first image.

7. The apparatus according to claim 5, wherein, When the generation unit generates the second image using the style vector and the semantic vector, it specifically performs the following: Obtain the degrees of freedom parameters, which are used to adjust the style vector; The second image is generated using the style vector, the degree-of-freedom parameter, and the semantic vector.

8. The apparatus according to any one of claims 5-7, further comprising a processing unit, After the generation unit generates the second image, the second image and the target text used when generating the second image are used as training samples for a preset text recognition model.

9. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-4.

11. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-4.

Images