Image synthesis method, device, electronic device and storage medium

Abstract

The present disclosure provides an image synthesis method, apparatus, electronic device, and storage medium, relating to the field of image processing technology, and more particularly, to the field of image synthesis technology. A specific implementation scheme comprises: obtaining at least one character image, wherein the characters in the character image are characters of a first type; generating a first image using the at least one character image; and synthesizing the first image with a background image such that the first image covers a first region of the background image, where the first region is an area of ​​the background image where no characters of a second type exist. The present disclosure can automatically synthesize images containing characters of different types.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to the field of image synthesis technology. Background Art

[0002] Currently, the demand for synthetic images is growing in related technologies. For example, in many scenarios, neural network models are needed to recognize specific types of characters in images. Training such models requires a large number of training samples, which must include both specific characters and other types of characters. Therefore, automatically generating images containing different types of characters has become a technical problem that needs to be solved. Summary of the Invention

[0003] The present disclosure provides a method, apparatus, electronic device, and storage medium for image synthesis.

[0004] According to one aspect of the present disclosure, there is provided an image synthesis method, comprising:

[0005] Acquire at least one character image, where the characters in the character image are characters of a first type;

[0006] generating a first image using the at least one character image;

[0007] The first image is synthesized with a background image so that the first image covers a first area of ​​the background image, where the first area is an area of ​​the background image where no second type characters exist.

[0008] According to another aspect of the present disclosure, there is provided an image synthesis device, comprising:

[0009] An acquisition module, configured to acquire at least one character image, wherein the characters in the character image are characters of a first type;

[0010] A generating module, configured to generate a first image using the at least one character image;

[0011] The synthesis module is used to synthesize the first image with the background image so that the first image covers a first area of ​​the background image, where the first area is an area in the background image where no second type of characters exist.

[0012] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method of any embodiment of the present disclosure.

[0016] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method according to any embodiment of the present disclosure.

[0017] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, which implements the method according to any embodiment of the present disclosure when executed by a processor.

[0018] The image synthesis method and device proposed in the embodiments of the present disclosure avoid obstruction of the second type of characters in the background image during synthesis by synthesizing a first image containing first type characters with a background image, and covering the first image over an area of ​​the background image where second type characters do not exist, thereby being able to automatically synthesize images containing different types of characters.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0021] Figure 1 is a schematic diagram of an application scenario according to an embodiment of the present disclosure;

[0022] Figure 2 is a flowchart of an implementation of an image synthesis method 200 according to an embodiment of the present disclosure;

[0023] Figure 3A This is a schematic diagram of the display effect of an image synthesized using an image synthesis method according to an embodiment of the present disclosure. Figure 1 ;

[0024] Figure 3B This is a schematic diagram of the display effect of an image synthesized using an image synthesis method according to an embodiment of the present disclosure. Figure 2 ;

[0025] Figure 4 is a flowchart of an implementation of an image synthesis process according to an embodiment of the present disclosure;

[0026] Figure 5 is a structural diagram of an image synthesis device according to an embodiment of the present disclosure;

[0027] Figure 6 is a structural diagram of an image synthesis device according to another embodiment of the present disclosure;

[0028] Figure 7 It is a block diagram of an electronic device used to implement the image synthesis method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0030] In related technologies, the demand for synthetic images is growing. For example, neural network models used to identify a specific type of character in an image have a wide range of applications. The training process of such models requires a large number of training samples, and these training samples need to include the specific type of character as well as other types of characters. Taking the recognition model of handwriting in document scenarios as an example, in the case of handwriting in printed documents, the extraction, recognition, and erasure of these handwritings all rely on the recognition model of handwriting in document scenarios. The training of such models requires a large number of training samples. Currently, training samples are generally generated manually, which requires a large amount of time and manpower costs.

[0031] The embodiment of the present disclosure proposes an image synthesis method, which can be applied to a data processing device, for example, the device can be deployed in a terminal or server or other processing device to realize image synthesis. Figure 1 The application scenarios shown are as follows: Figure 1 As shown, the application scenario may include a simulation server 110 and a model training server 120. Taking the device applying the method as an example, which is deployed in the simulation server 110, the simulation server 110 can execute the image synthesis method, automatically synthesize images containing different types of characters, and send the synthesized images as model training samples to the model training server 120 for use by the model training server 120 to improve the efficiency of model training.

[0032] The simulation server 110 and the model training server 120 can be independent servers, or server clusters or distributed systems, or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, and big data and artificial intelligence platforms.

[0033] It should be noted that Figure 1 This is only an example of an application scenario of the present disclosure. The image synthesis method proposed in the present disclosure can not only be used to generate training samples for a recognition model (for example, the recognition model can be used to recognize handwriting), but can also be applied to other fields. The present disclosure does not limit the application scenarios of the synthesized images.

[0034] Figure 2 FIG. 2 is a flowchart of an implementation of an image synthesis method 200 according to an embodiment of the present disclosure, comprising:

[0035] S210: Acquire at least one character image, where the characters in the character image are characters of the first type;

[0036] S220: Generate a first image using at least one character image;

[0037] S230: Synthesize the first image and the background image so that the first image covers a first area of ​​the background image, where the first area is an area of ​​the background image where no second type of characters exist.

[0038] In some embodiments, the first type of characters may include handwritten characters, and the second type of characters may include printed characters. In the following embodiments, for ease of description, the first type of characters is specifically handwritten characters, and the second type of characters is specifically printed characters. However, the embodiments of the present disclosure are not limited to the first type of characters and the second type of characters.

[0039] It can be seen from the above implementation process that when synthesizing an image, the embodiment of the present disclosure covers the first image containing the first type of characters on the area of ​​the background image that does not contain the second type of characters, thereby avoiding the obstruction of the original second type of characters in the background image during image synthesis, and enabling the synthesized image to completely retain the original second type of characters in the background image and contain the first type of characters in the first image.

[0040] The image synthesis method proposed in the embodiments of the present disclosure includes at least the following two implementation methods. The background images in the two methods are different, and the specific implementation methods during synthesis are also different.

[0041] In the first one, the background image only contains the second type of characters.

[0042] In this case, when synthesizing the image, the first image is used to cover the first area in the background image, and the first area is the area in the background image that does not contain the second type of characters; since there are only second type characters in the background image, it can be seen that the first area is the blank area in the background image.

[0043] Taking the example of the first image containing handwritten characters and the background image containing printed characters, the first image and the background image are synthesized, that is, the handwritten characters in the first image are synthesized into the blank area of ​​the background image.

[0044] like Figure 3A This is a schematic diagram of the display effect of an image synthesized using an image synthesis method according to an embodiment of the present disclosure. Figure 1 .like Figure 3A As shown, the display effect of the synthesized image is: handwritten characters are added to the blank area of ​​the background image.

[0045] In the second type, the background image includes the first type of characters and the second type of characters.

[0046] In this case, when synthesizing the image, the first image is used to cover the first area of ​​the background image. The first area is the area of ​​the background image that does not contain the second type of characters. Since the background image contains both the first type of characters and the second type of characters, the first area can be the area of ​​the background image where the first type of characters are located, or it can be a blank area of ​​the background image. In one embodiment, the area of ​​the background image where the first type of characters are located is used as the first area.

[0047] Taking the example that the first image contains handwritten characters and the background image contains printed characters and handwritten characters, the first image and the background image are synthesized, that is, the handwritten characters in the first image are used to cover the original handwritten characters in the background image.

[0048] like Figure 3B This is a schematic diagram of the display effect of an image synthesized using an image synthesis method according to an embodiment of the present disclosure. Figure 2 .like Figure 3B As shown, the display effect of the synthesized image is: the original handwritten characters in the background image are replaced by the handwritten characters in the first image.

[0049] It can be seen that there is a certain connection between the above two methods. For example, the image synthesized by the first method can be used as the background image in the second method.

[0050] The above briefly describes the principles of the two specific implementations of the image synthesis method in the disclosed embodiments. Determining the first region to ensure the display quality of the synthesized image is a common problem faced by both methods, and the disclosed embodiments employ different methods for determining the first region. The following details the two synthesis methods in the disclosed embodiments.

[0051] Method 1: Handwriting synthesis based on printed text detection

[0052] This method includes at least the following steps:

[0053] Step 1.1, background image acquisition:

[0054] This step can collect a small amount of scene data, such as collecting clean background images without handwriting, such as books, documents, notebooks, test papers, etc.

[0055] Step 1.2, determining the second area in the background image where the second type of characters are located:

[0056] In this embodiment, the second type of characters in the background image are printed characters. In this disclosed embodiment, the background image can be input into a pre-trained text detection model, which outputs the position information of each line of text. This position information is the position information of the second area where the second type of characters are located.

[0057] The text detection model can be specifically a convolutional neural network model. The model can be composed of convolutional layers, pooling layers, etc., wherein the convolutional layers can mainly include 3*3 convolutions and 1*1 convolutions. The model can use network structures such as the Differentiable Binarization Network (DBNet) and the Progressive Scale Expansion Network (PSENet). The embodiments of this disclosure do not limit the specific structure and form of the text detection model.

[0058] The location information of the single second area determined in this step can be expressed in the following form:

[0059] [x, y, w, h];

[0060] Wherein, x and y represent the horizontal coordinate and the vertical coordinate of the upper left corner of the second area respectively, and the adopted coordinate system may be a two-dimensional coordinate system with the lower left corner of the background image as the origin;

[0061] w and h represent the width and height of the second region respectively;

[0062] The units of x, y, w, and h can be pixels, or length units such as millimeters and centimeters.

[0063] The above representation of the second region location information is only an example, and the present disclosure does not limit this. The present disclosure may adopt other representations to describe the second region location information, which are not exhaustive here.

[0064] One piece of position information can correspond to a row (or column) of text in the background image (such as printed text in this example). If there are multiple rows (or columns) of text in the background image, the position information corresponding to each row (or column) of text can be obtained through the text detection model, that is, a set of position information can be obtained. The set of position information is shown below:

[0065] P = {[x1, y1, w1, h1];

[0066] [x2, y2, w2, h2];

[0067] …

[0068] [xn,yn,wn,hn]

[0069] }

[0070] In the above example, position information of the second areas where n second-type characters are located is determined, and each piece of position information corresponds to a second area in the background image.

[0071] Step 1.3, generate the first image:

[0072] In one embodiment, the embodiment of the present disclosure can select at least one character image, splice the at least one selected character image to obtain a spliced ​​image; then adjust at least one of the color, grayscale and size of the spliced ​​image, and use the adjusted image as the first image.

[0073] Take the use of handwritten Chinese characters to generate the first image as an example. A handwritten Chinese character library is pre-set, and each handwritten Chinese character in the library is a grayscale image; N character images are randomly selected from the handwritten Chinese character library, and the height of each character image is H; the N selected character images are horizontally spliced ​​to generate a row text image, denoted as L, and the row text image contains the handwritten Chinese characters in the aforementioned N character images. It should be noted that the aforementioned splicing method is only an example, and the embodiment of the present disclosure can also adopt other splicing methods, for example, the selected N character images are vertically spliced ​​to obtain a column text image; and so on.

[0074] Afterwards, at least one of the color, grayscale, and size of the stitched image is adjusted.

[0075] Taking color adjustment as an example, the adjustment method may include the following steps:

[0076] (1) Binarize each pixel value of the stitched image, set the pixel value greater than 0 to 1, and vice versa; after binarization, convert the stitched image into a black and white image.

[0077] (2) Convert the black and white image obtained in the previous step into a color image. Taking the conversion into an RGB image as an example, the corresponding color channels (red (R), green (G), and blue (B)) can be multiplied according to the RGB colors. For example, for red (255, 0, 0), the following formula can be used for calculation:

[0078] L R=L R *255 L G =L G *0 L B =L B *0

[0079] Among them, L R , L G , L B Corresponding to the red, green and blue channels of image L respectively;

[0080] L' represents the pixel value of the image.

[0081] The above is an example of adjusting the color. After adjustment in the manner of this example, the color of the spliced ​​image is adjusted to red. The embodiment of the present disclosure may also adopt other color adjustment methods, and / or convert the spliced ​​image into other forms of color images. The purpose of adjusting the color in this step is to simulate handwriting of different colors, so that a composite image with richer forms can be obtained; if such composite images are used as training samples for text recognition models, it can have a positive effect on improving the training effect and efficiency of the text recognition model. In addition, in the above example, a unified rule is used to adjust each pixel in the image; in other examples of the present disclosure, different rules can also be used to adjust pixels at different positions.

[0082] The above is an example of color adjustment. Embodiments of the present disclosure can also adjust the grayscale and size of the stitched image. For example, the grayscale values ​​of each pixel in the stitched image can be randomly adjusted to adjust the grayscale of the entire image. Another example is scaling the stitched image and / or adjusting the aspect ratio of the stitched image.

[0083] Through the above three steps, the background image is acquired, the second area where the second type of characters (such as printed characters) are located in the background image is determined, and the first image containing the first type of characters (such as handwritten characters) is generated. After that, the synthesis process of the first image and the background image can be performed. It should be noted that, among the above three steps, except that step 1.2 needs to be executed after step 1.1, there is no restriction on the execution order of step 1.3. For example, step 1.3 can be executed before or after step 1.1 or step 1.2, or it can be executed synchronously with step 1.1 or step 1.2.

[0084] Step 1.4, image synthesis:

[0085] Figure 4 is a flowchart of an image synthesis process according to an embodiment of the present disclosure, such as Figure 4As shown, in a possible implementation, synthesizing the first image with the background image so that the first image covers the first area of ​​the background image may include:

[0086] S410: Randomly select a blind selection area in the background image, where the size of the blind selection area is the same as that of the first image;

[0087] S420: If the blind selection area meets the first condition, determine the blind selection area as the first area, and synthesize the first image with the background image so that the first image covers the first area of ​​the background image;

[0088] The first condition includes: an overlap rate between the blind selection area and a second area where any second type of character is located in the background image is less than or equal to a preset threshold.

[0089] As can be seen from the above process, this example uses a method of first random selection and then verification to determine the first area. That is, a blind selection area is randomly selected, and then it is verified whether the second type of characters in the background image will be blocked if the first image is placed in the blind selection area; if there is no occlusion (such as the overlap rate between the blind selection area and the second area where any second type of characters in the background image are located is less than or equal to a preset threshold), it means that the first image can be placed in the blind selection area, that is, the blind selection area can be used as the first area; if there is occlusion (such as the overlap rate between the blind selection area and the second area where any second type of characters in the background image are located is greater than a preset threshold), it means that it is not appropriate to place the first image in the blind selection area, that is, the blind selection area is not suitable as the first area. In this case, a new blind selection area can be selected and verified.

[0090] like Figure 4 As shown, in some embodiments, the process further includes randomly reselecting a blind selection area (i.e., returning to step S420) when the blind selection area does not meet the first condition, and determining whether the reselected blind selection area meets the first condition, until the number of times the first condition is not met reaches a preset threshold, and then terminating the current process.

[0091] For example, take the following steps to merge images:

[0092] (1) After selecting a background image G and determining the first image L containing handwritten characters, a blind selection area is randomly determined in the background image G. The position information of the blind selection area is expressed in the following form:

[0093] [X, Y, W, H];

[0094] Wherein, X and Y are the horizontal coordinate and vertical coordinate of the upper left corner of the blind selection area, respectively. The coordinate system used may be a two-dimensional coordinate system with the lower left corner of the background image as the origin;

[0095] W and H represent the width and height of the blind selection area respectively;

[0096] The units of X, Y, W, and H can be pixels, or length units such as millimeters and centimeters.

[0097] X, Y, W, H can be determined in the following ways:

[0098] X=Random(0,G w -L w -1);

[0099] Y=Random(0,G h -L h -1);

[0100] W=L w ;

[0101] H=L h ;

[0102] Among them, Random(a,b) means randomly generating a number from [a,b], G w , G h Indicates the width and height of the background image G, L w , L h represents the width and height of the first image L;

[0103] (2) Calculate whether the overlap rate (IOU, Intersection Of Union) between the blind selection area determined in step (1) and each second area in the background image is greater than a threshold value T. For example, the position information of the blind selection area is respectively calculated with the position information of each second area in the position information set determined in step 1.2 above to obtain the IOU of the blind selection area and each second area. That is, calculate the IOU of [X, Y, W, H] and each element in the set P (each element represents a second area). Wherein,

[0104] P = {[x1, y1, w1, h1];

[0105] [x2, y2, w2, h2];

[0106] …

[0107] [xn,yn,wn,hn]

[0108] }

[0109] In some examples, considering that the overlap rate between handwritten characters and printed characters in a document scene is relatively small, the threshold T may be set to between 0.02 and 0.1.

[0110] For example, if the IOU of the blind selection area and any second area is not greater than T, the blind selection area is considered reasonable, that is, the blind selection area is determined to be the first area, and the first image L can be synthesized with the background image G, and the first image L is placed in the first area of ​​the background image G during synthesis.

[0111] After the image is synthesized, the first region (that is, the reasonably blind selected region) can be determined as a second region in the background image. For example, the position information of the first region is added to the above set P, and the updated set P is:

[0112] P = {[x1, y1, w1, h1];

[0113] [x2, y2, w2, h2];

[0114] …

[0115] [xn,yn,wn,hn]

[0116] [X, Y, W, H]

[0117] }

[0118] In this way, when a new image is synthesized using the background image later, the first image can avoid the position where the first image was placed during the previous image synthesis; in this way, for the same background image, the position where the first image is placed is different each time it is synthesized, ensuring the diversity and richness of the synthesized image.

[0119] If the IOU between the blind selected region and one or more second regions is greater than T, a new blind selected region can be re-determined, i.e., the above step (1) is returned to be executed until the number of returns exceeds a predetermined threshold R (e.g., R=3), in which case the image synthesis is considered to have failed. After that, a background image can be randomly selected again and / or the first image can be generated, and the image synthesis can be repeated.

[0120] In addition, the present method can use an image fusion method to perform image synthesis, such as alpha fusion or Poisson fusion method, etc. The mask image used in image fusion can be obtained by binarizing the character image.

[0121] The above step-by-step description describes how to synthesize an image using Method 1. In the disclosed embodiment, the above steps can be repeated to batch synthesize images containing a first type of characters (e.g., handwritten characters) and a second type of characters (e.g., printed characters). Because determining whether the blindly selected area meets the first condition is relatively simple and time-efficient, this multiple-trial approach can quickly and efficiently complete image synthesis when synthesizing a large number of images.

[0122] In addition, by limiting the threshold value of the number of failures (the above-mentioned predetermined threshold R), repeated attempts and failures using a background image with less blank areas can be avoided, thereby increasing the image synthesis speed.

[0123] Method 2: Based on handwritten text detection and handwritten handwriting replacement synthesis

[0124] This method includes at least the following steps:

[0125] Step 2.1, background image acquisition:

[0126] This step can collect background images including first-type characters and second-type characters, for example, collecting background images containing both printed characters (second-type characters) and handwritten characters (first-type characters), for example, images with handwritten characters in blank spaces of printed books, documents, test papers, etc.

[0127] In addition, this step may also use the image synthesized in method 1 as the background image.

[0128] Step 2.2, determine the first region from the background image:

[0129] In some embodiments, the method of determining the first area includes: determining an area in the background image where the first type of characters are located, and determining the area where the first type of characters are located as the first area.

[0130] Since the area where the first type of characters are located does not include the second type of characters, the first area determined in this way can ensure that the newly added first type of characters in the composite image will not affect the second type of characters in the original background image.

[0131] The method of determining the area where the first type of characters are located in the background image may include the following process:

[0132] (1) inputting a background image into a pre-trained first-type character recognition model, and determining a mask image of the first-type character in the background image by the first-type character recognition model;

[0133] (2) Remove noise from the mask image;

[0134] (3) Performing connected domain detection on the mask image after noise removal to obtain multiple contour points;

[0135] (4) Generate at least one minimum bounding rectangle using multiple contour points, and use the minimum bounding rectangle as the area where the first type of characters are located in the background image.

[0136] For example, in the above step (2), the mask image may be first eroded and then dilated to remove noise from the mask image.

[0137] For example, in the above step (3), a 4-neighborhood connectivity test or an 8-neighborhood connectivity test may be used on the mask image after the noise is removed to implement connected domain detection on the mask image after the noise is removed.

[0138] The first type character recognition model used in this step can be trained using the image synthesized in Method 1. The first type character recognition model can quickly identify the first type of characters in the background image. Using this model can increase the speed of determining the first area, thereby increasing the overall image synthesis speed.

[0139] Step 2.3, generate the first image:

[0140] This step is similar to the specific method of generating the first image in the above-mentioned method 1, and the relevant description in the above-mentioned step 1.3 can be referred to. The difference is that in this example, the size of the first image is determined according to the size of the first area determined in step 2.2. For example, after selecting multiple character images for splicing and adjusting the color or grayscale, the image can be scaled to obtain the first image; the size of the first image is equal to the size of the first area. In addition, since the size of the first image is equal to the size of the first area, when splicing to generate the first image, the number N of spliced ​​character images can be determined based on the aspect ratio of the first area.

[0141] In addition, this method can use image fusion method to synthesize images. Since the first area itself has content before synthesis, the alpha fusion method can achieve better synthesis effect. The mask image used in image fusion can be obtained by binarizing the character image.

[0142] The above step-by-step introduction to the method of synthesizing an image using method 2, the embodiment of the present disclosure can repeat the above steps to batch synthesize images containing the first type of characters (such as handwritten characters) and the second type of characters (such as printed characters).

[0143] In summary, the image synthesis method proposed in the embodiment of the present disclosure can automatically generate a composite image containing different types of characters by adding a first type of character (such as a handwritten character) to a blank area of ​​an image containing a second type of character (such as a printed character), or replacing the original first type of character in an image containing a second type of character (such as a printed character) and a first type of character (such as a handwritten character) with a new first type of character. The composite image has a rich variety of forms and types. This type of composite image is used as a training sample for a character recognition model, which can improve the efficiency and effectiveness of training the character recognition model.

[0144] Figure 5 FIG. 5 is a schematic structural diagram of an image synthesis device 500 according to an embodiment of the present disclosure, comprising:

[0145] An acquisition module 510 is configured to acquire at least one character image, wherein the characters in the character image are characters of a first type;

[0146] A generating module 520, configured to generate a first image using the at least one character image;

[0147] The synthesis module 530 is configured to synthesize the first image with the background image so that the first image covers a first area of ​​the background image, where the first area is an area of ​​the background image where no second type of characters exist.

[0148] In one embodiment, the first type of characters includes handwritten characters, and the second type of characters includes printed characters.

[0149] Figure 6 is a structural diagram of an image synthesis device 600 according to an embodiment of the present disclosure, such as Figure 6 As shown, in one embodiment, the synthesis module 530 includes:

[0150] A random selection submodule 531 is configured to randomly select a blind selection area in the background image, where the size of the blind selection area is the same as that of the first image;

[0151] a synthesis submodule 532, configured to, if the blind selection area satisfies a first condition, determine the blind selection area as the first area, and synthesize the first image with the background image so that the first image covers the first area of ​​the background image;

[0152] The first condition includes: an overlap rate between the blind selection area and a second area where any second type of characters in the background image are located is less than or equal to a preset threshold.

[0153] In one embodiment, the synthesis submodule 532 is further configured to determine the first region as one of the second regions in the background image.

[0154] In one embodiment, the synthesis submodule 532 is further used to randomly reselect a blind selection area when the blind selection area does not meet the first condition, and determine whether the reselected blind selection area meets the first condition until the number of times the first condition is not met reaches a preset threshold.

[0155] In one embodiment, the background image includes the first type of characters and the second type of characters;

[0156] The synthesis module 530 includes:

[0157] The first region determining submodule 533 is configured to determine a region in the background image where the first type of characters are located, and determine the region where the first type of characters are located as the first region.

[0158] In one embodiment, the first area determination submodule 533 is used to input the background image into a pre-trained first type character recognition model, and determine the mask image of the first type of characters in the background image by the first type of character recognition model; remove noise from the mask image; perform connected domain detection on the mask image after noise removal to obtain multiple contour points; use the multiple contour points to generate at least one minimum bounding rectangle, and use the minimum bounding rectangle as the area where the first type of characters in the background image are located.

[0159] In one embodiment, the generation module 520 is configured to splice the at least one character image to obtain a spliced ​​image; adjust at least one of the color, grayscale, and size of the spliced ​​image, and use the adjusted image as the first image.

[0160] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, please refer to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0161] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0162] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0163] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0164] like Figure 7As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. Various programs and data required for the operation of the device 700 can also be stored in the RAM 703. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0165] Various components in device 700 are connected to I / O interface 705, including an input unit 706, such as a keyboard, mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, optical disk, etc.; and a communication unit 709, such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows device 700 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0166] The computing unit 701 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as the image synthesis method. For example, in some embodiments, the image synthesis method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the image synthesis method described above can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the image synthesis method by any other suitable means (e.g., via firmware).

[0167] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0168] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0169] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0170] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0171] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0172] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0173] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0174] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. An image synthesis method, comprising: Acquire at least one character image, where the characters in the character image are characters of a first type; splicing the at least one character image to obtain a spliced ​​image, adjusting the spliced ​​image, and using the adjusted image as the first image; A blind selection area having the same size as the first image is randomly selected in the background image, and if the blind selection area satisfies a first condition, the blind selection area is determined as the first area, and the first image is synthesized with the background image so that the first image covers the first area of ​​the background image; wherein the first condition includes: an overlap rate between the blind selection area and a second area in the background image where any second-type characters are located is less than or equal to a preset threshold; the first area is an area in the background image where no second-type characters exist; and the first-type characters and the second-type characters are two different types of characters.

2. The method according to claim 1, further comprising: The first area is determined as one of the second areas in the background image.

3. The method according to claim 1 or 2, further comprising: If the blind selection area does not meet the first condition, a new blind selection area is randomly selected, and it is determined whether the reselected blind selection area meets the first condition. When the number of times the first condition is not met reaches a preset threshold, the current process is terminated.

4. The method according to claim 1, wherein The background image includes the first type of characters and the second type of characters; The method further includes: determining an area in the background image where the first type of characters are located, and determining the area where the first type of characters are located as the first area.

5. The method according to claim 4, wherein The determining of the area in the background image where the first type of characters are located includes: Inputting the background image into a pre-trained first-type character recognition model, and determining a mask image of the first-type character in the background image by the first-type character recognition model; removing noise from the mask image; Perform connected domain detection on the mask image after noise removal to obtain multiple contour points; At least one minimum bounding rectangle is generated using the multiple contour points, and the minimum bounding rectangle is used as the area where the first type of characters are located in the background image.

6. The method according to claim 1, wherein The step of splicing the at least one character image to obtain a spliced ​​image, adjusting the spliced ​​image, and using the adjusted image as the first image comprises: splicing the at least one character image to obtain a spliced ​​image; At least one of the color, grayscale, and size of the spliced ​​image is adjusted, and the adjusted image is used as the first image.

7. The method according to claim 1, wherein The first type of characters includes handwritten characters, and the second type of characters includes printed characters.

8. An image synthesis device comprising: An acquisition module, configured to acquire at least one character image, wherein the characters in the character image are characters of a first type; a generating module, configured to splice the at least one character image to obtain a spliced ​​image, adjust the spliced ​​image, and use the adjusted image as the first image; A synthesis module is used to randomly select a blind selection area of ​​the same size as the first image in the background image, so that when the blind selection area meets a first condition, the blind selection area is determined as the first area, and the first image is synthesized with the background image so that the first image covers the first area of ​​the background image; wherein the first condition includes: the overlap rate between the blind selection area and the second area where any second type of character is located in the background image is less than or equal to a preset threshold; the first area is an area in the background image where no second type of character exists; and the first type of character and the second type of character are two different types of characters.

9. The device according to claim 8, wherein The synthesis module is further configured to determine the first area as one of the second areas in the background image.

10. The device according to claim 8 or 9, wherein The synthesis module is further configured to, when the blind selection area does not satisfy the first condition, randomly reselect a blind selection area and determine whether the reselected blind selection area satisfies the first condition until the number of times the first condition is not satisfied reaches a preset threshold.

11. The device according to claim 8, wherein The background image includes the first type of characters and the second type of characters; The synthesis module includes: The first region determining submodule is configured to determine a region in the background image where the first type of characters are located, and determine the region where the first type of characters are located as the first region.

12. The device according to claim 11, wherein The first region determination submodule is configured to input the background image into a pre-trained first type character recognition model, and determine a mask image of the first type of character in the background image by the first type character recognition model; removing noise from the mask image; Connected domain detection is performed on the mask image after noise removal to obtain multiple contour points; at least one minimum bounding rectangle is generated using the multiple contour points, and the minimum bounding rectangle is used as the area where the first type of character is located in the background image.

13. The device according to claim 8, wherein The generating module is configured to splice the at least one character image to obtain a spliced ​​image; adjust at least one of the color, grayscale, and size of the spliced ​​image, and use the adjusted image as the first image.

14. The device according to claim 8, wherein The first type of characters includes handwritten characters, and the second type of characters includes printed characters.

15. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

17. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images