Image generation method, device, electronic device and storage medium

Abstract

The embodiments of the present application disclose an image generation method, device, electronic device, and storage medium. The image generation method first obtains a first edge image by performing edge detection on a reference scene image, and then quickly obtains multiple second edge images through image transformation processing. Accordingly, the image type of the second edge image can be adjusted based on a scene image generation model to obtain multiple target scene images, and then multiple target sample images can be generated. Therefore, only a small number of reference scene images and benchmark images need to be collected to generate a large number of target sample images. Compared with the method of collecting a large number of benchmark images, under the premise of generating the same number of target sample images, the image generation method provided by the embodiments of the present application can effectively reduce the number of image collections, reduce the generation cost of target sample images, and improve the generation efficiency of target sample images. It can be widely used in technical fields such as artificial intelligence, image processing, and cloud computing.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an image generation method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, the application of various image processing models is becoming increasingly widespread. Before applying a model, it must be trained to achieve optimal performance. To improve model training effectiveness, a large number of training images are generally required. In addition to the aforementioned scenarios, other scenarios may also require a large number of images, such as when testing image processing software.

[0003] Currently, target sample images (such as training images or test images) can be generated based on real reference images. This can reduce image acquisition costs compared to manual collection. However, in related technologies, there is generally a one-to-one correspondence between reference images and target sample images. Generating target sample images still requires collecting a large number of reference images, and the efficiency of generating target sample images still needs to be improved. Summary of the Invention

[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0005] The embodiments of the present application provide an image generation method, device, electronic device, and storage medium, which can improve the generation efficiency of target sample images.

[0006] In one aspect, an embodiment of the present application provides an image generation method, comprising:

[0007] Acquire a reference scene image, perform edge detection on the reference scene image, and obtain a first edge image corresponding to the reference scene image;

[0008] performing image transformation processing on the first edge image to obtain a plurality of second edge images corresponding to the first edge image;

[0009] Inputting the second edge image into a preset scene image generation model, adjusting the image type of the second edge image based on the scene image generation model, and obtaining a target scene image corresponding to the second edge image;

[0010] A reference image is acquired, and the reference image is synthesized with each of the target scene images to generate a plurality of target sample images.

[0011] On the other hand, an embodiment of the present application further provides an image generating device, comprising:

[0012] an edge detection module, configured to acquire a reference scene image, perform edge detection on the reference scene image, and obtain a first edge image corresponding to the reference scene image;

[0013] a transformation module, configured to perform image transformation processing on the first edge image to obtain a plurality of second edge images corresponding to the first edge image;

[0014] a model processing module, configured to input the second edge image into a preset scene image generation model, adjust the image type of the second edge image based on the scene image generation model, and obtain a target scene image corresponding to the second edge image;

[0015] The synthesis module is used to obtain a reference image, synthesize the reference image with each of the target scene images, and generate a plurality of target sample images.

[0016] Furthermore, the scene image generation model includes a plurality of sequentially connected convolutional layers and a plurality of sequentially connected deconvolutional layers, and the above-mentioned model processing module is specifically used to:

[0017] Performing convolution processing on the second edge image based on the multiple convolution layers to obtain a target convolution image;

[0018] Deconvolution processing is performed on the target convolution image based on the multiple deconvolution layers to obtain a target scene image corresponding to the second edge image.

[0019] Furthermore, the number of the convolutional layers is the same as the number of the deconvolutional layers, and the above-mentioned model processing module is specifically used to:

[0020] Acquire a first transition image to be input to a target deconvolution layer; wherein the target deconvolution layer is a deconvolution layer currently to be deconvolved among the multiple deconvolution layers;

[0021] Determining a target convolution layer corresponding to the target deconvolution layer from the plurality of convolution layers, and obtaining a second transition image outputted after the target convolution layer performs convolution processing;

[0022] Splicing the first transition image and the second transition image to obtain a spliced ​​image;

[0023] Deconvolution processing is performed on the spliced ​​image based on the target deconvolution layer until a target scene image corresponding to the second edge image is obtained.

[0024] Furthermore, the image generation device further includes a model training module, which is used to:

[0025] Acquire a training edge image set and a label scene image set; wherein the training edge image set includes a plurality of training edge images, the label scene image set includes a plurality of label scene images, and the image type of the training edge image set is different from the image type of the label scene image set;

[0026] Adjusting the image type of a target training edge image based on the scene image generation model to obtain a training scene image corresponding to the target training edge image; wherein the target training edge image is one of the plurality of training edge images;

[0027] Inputting the training scene image into a first judgment model to obtain a first judgment result, inputting the label scene image into the first judgment model to obtain a second judgment result, and calculating a first loss value corresponding to the scene image generation model based on the first judgment result and the second judgment result;

[0028] Adjust parameters of the scene image generation model according to the first loss value.

[0029] Furthermore, the above model training module is specifically used to:

[0030] adjusting the image type of the training scene image based on the edge image generation model to obtain a restored edge image corresponding to the training scene image;

[0031] Calculating a norm between any training edge image other than the target training edge image in the training edge image set and the restored edge image, and calculating a restoration loss value according to the norm;

[0032] Calculating a second loss value corresponding to the edge image generation model based on the second judgment model;

[0033] A target loss value is obtained according to the sum of the first loss value, the second loss value and the restoration loss value, and parameters of the scene image generation model are adjusted according to the target loss value.

[0034] Furthermore, the edge detection module is specifically used for:

[0035] Performing grayscale processing on the reference scene image to obtain a grayscale image corresponding to the reference scene image;

[0036] Performing Gaussian filtering on the grayscale image to obtain a filtered image;

[0037] Calculating a pixel gradient intensity matrix of the filtered image, traversing each image pixel in the pixel gradient intensity matrix, comparing the gradient intensity of a currently traversed target pixel with the gradient intensity of each adjacent pixel, and determining edge pixels from the image pixels based on the comparison results;

[0038] A first edge image corresponding to the reference scene image is obtained according to the edge pixel points.

[0039] Furthermore, the edge detection module is specifically used for:

[0040] If the gradient intensity of the target pixel is greater than the gradient intensity of each of the adjacent pixels, comparing the gradient intensity of the target pixel with a preset first intensity threshold and a second intensity threshold; wherein the first intensity threshold is greater than the second intensity threshold;

[0041] If the gradient intensity of the target pixel point is greater than or equal to the first intensity threshold, the target pixel point is determined as an edge pixel point;

[0042] If the gradient intensity of the target pixel point is less than the first intensity threshold and greater than or equal to the second intensity threshold, the neighborhood pixel point corresponding to the target pixel point is determined; when the gradient intensity of the neighborhood pixel point is greater than or equal to the first intensity threshold, the target pixel point is determined as an edge pixel point.

[0043] Furthermore, the above transformation module is specifically used for:

[0044] performing rotation processing on the first edge image to obtain a plurality of second edge images corresponding to the first edge image;

[0045] performing cropping processing on the first edge image to obtain a plurality of second edge images corresponding to the first edge image;

[0046] performing a flipping process on the first edge image to obtain a plurality of second edge images corresponding to the first edge image;

[0047] Gaussian filtering is performed on the first edge image to obtain a plurality of second edge images corresponding to the first edge image.

[0048] Furthermore, the reference image is a document image, and the edge detection module is specifically used to:

[0049] Acquire a reference paper image, and crop at least one of a paper texture image, a paper shadow image, or a paper light image from the reference paper image;

[0050] resizing at least one of the paper texture image, the paper shadow image, or the paper light image;

[0051] At least one of the paper texture image, the paper shadow image, or the paper light image after size adjustment is used as a reference scene image.

[0052] Furthermore, the plurality of target scene images include a target material image corresponding to the paper material image, a target shadow image corresponding to the paper shadow image, and a target light image corresponding to the paper light image, and the synthesis module is specifically configured to:

[0053] Separating the document image to obtain a content image and an original background image corresponding to the document image;

[0054] Combining the original background image with the target material image, the target shadow image, and the target light image to obtain multiple effect background images;

[0055] The content image is synthesized with each of the effect background images to generate a plurality of target sample images.

[0056] Furthermore, the above synthesis module is specifically used for:

[0057] Obtaining a first image matrix of the original background image and a second image matrix of each of the target scene images; wherein the first image matrix and the second image matrix have the same size;

[0058] multiplying the first image matrix and corresponding matrix elements in each of the second image matrices to obtain a plurality of target sample image matrices;

[0059] Generate corresponding effect background images according to the plurality of target sample image matrices.

[0060] On the other hand, an embodiment of the present application further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned image generation method when executing the computer program.

[0061] On the other hand, an embodiment of the present application further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the above-mentioned image generation method.

[0062] In another aspect, embodiments of the present application further provide a computer program product, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to implement the above-described image generation method.

[0063] The embodiments of the present application include at least the following beneficial effects: by performing edge detection on a reference scene image to obtain a first edge image, performing image transformation processing on the first edge image to obtain multiple second edge images corresponding to the first edge image, and adjusting the image type of the second edge image based on a preset scene image generation model to obtain a target scene image corresponding to the second edge image. It can be seen that the embodiments of the present application do not directly adjust the display style of the reference image to obtain the target sample image, but first perform edge detection on the reference scene image to obtain the first edge image. Due to the simple structure of the first edge image, multiple second edge images can be quickly obtained through image transformation processing. Accordingly, the image type of the second edge image can be adjusted based on the scene image generation model to obtain multiple target scene images. Subsequently, the reference image is synthesized with each target scene image to generate multiple target sample images. Therefore, the image generation method provided by the embodiments of the present application only needs to collect a small number of reference scene images and reference images to generate a large number of target sample images. Compared with the method of collecting a large number of reference images, under the premise of generating the same number of target sample images, the image generation method provided by the embodiments of the present application can effectively reduce the number of image collections, reduce the generation cost of target sample images, and improve the generation efficiency of target sample images.

[0064] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present application. The purposes and other advantages of the present application can be achieved and obtained through the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The accompanying drawings are used to provide a further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0066] Figure 1 A schematic diagram of generating a training image in the related art provided in an embodiment of the present application;

[0067] Figure 2 A schematic diagram of an implementation environment provided for an embodiment of the present application;

[0068] Figure 3A schematic diagram of another implementation environment provided for an embodiment of the present application;

[0069] Figure 4 A flowchart of an image generation method provided in an embodiment of the present application;

[0070] Figure 5 A schematic diagram of the overall process of edge detection on a reference scene image provided in an embodiment of the present application;

[0071] Figure 6 A schematic diagram of the structure of a scene image generation model provided in an embodiment of the present application;

[0072] Figure 7 A schematic diagram of a processing flow of a scene image generation model provided in an embodiment of the present application;

[0073] Figure 8 A schematic diagram of the overall process of an example of an image generation method provided in an embodiment of the present application;

[0074] Figure 9 A schematic diagram of a training architecture for a scene image generation model provided in an embodiment of the present application;

[0075] Figure 10 A schematic diagram of another training architecture for the scene image generation model provided in an embodiment of the present application;

[0076] Figure 11 Another structural diagram of the scene image generation model provided in an embodiment of the present application;

[0077] Figure 12 A schematic diagram of adding random noise provided in an embodiment of the present application;

[0078] Figure 13 A schematic diagram of the document image synthesis process provided in an embodiment of the present application;

[0079] Figure 14 A schematic diagram of the structure of an image generating device provided in an embodiment of the present application;

[0080] Figure 15 A partial structural block diagram of a terminal provided in an embodiment of the present application;

[0081] Figure 16 A partial structural block diagram of the server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0082] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0083] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:

[0084] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also involves studying the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0085] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and smart transportation.

[0086] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning through demonstration.

[0087] Computer vision (CV) is the science of making machines "see." Specifically, it refers to the use of cameras and computers to replace the human eye in identifying and measuring objects, and further processing the images to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0088] With the rapid development of artificial intelligence technology, the application of various image processing models has become more and more widespread. Before the model is applied, the image processing model needs to be trained to optimize the performance of the image processing model. In order to improve the training effect of the model, a large number of training images are generally required to train the image processing model. For example, the image processing model can be a model related to OCR (Optical Character Recognition), or various image special effects adding models, image recognition models, etc., which are not limited in the embodiments of the present application.

[0089] In addition to the above scenarios, other scenarios may also have a large number of image requirements. For example, a large number of test images are also needed when testing image processing software.

[0090] Currently, target sample images (such as training images or test images) can be generated based on real reference images. This can reduce image acquisition costs compared to manual collection. However, in related technologies, there is generally a one-to-one correspondence between reference images and target sample images. Generating target sample images still requires collecting a large number of reference images, and the efficiency of generating target sample images still needs to be improved.

[0091] Take the target sample image as the training image and the image processing model as the OCR-related model as an example to illustrate. Figure 1 , Figure 1A schematic diagram of generating training images in the related technology provided in an embodiment of the present application, wherein A and A' are two image data sets with the same distribution (the same distribution means the same image type, but different display styles), image data set A includes multiple real-captured reference images (for example, captured using a terminal with a camera function, or downloaded from the Internet, etc.), image data set A' includes multiple training images obtained by processing the reference images based on an image generation model, the reference images in image data set A and the training images in image data set A' are one-to-one corresponding, that is, the number of reference images in image data set A is the same as the number of training images in image data set A'. Therefore, assuming that 100 training images need to be generated, then 100 reference images need to be collected and input into the image generation model. The cost of collecting reference images is still high, and the efficiency of generating training images still needs to be improved.

[0092] Based on this, the embodiments of the present application provide an image generation method, device, electronic device, and storage medium, which can improve the generation efficiency of target sample images.

[0093] Reference Figure 2 , Figure 2 A schematic diagram of an implementation environment provided for an embodiment of the present application, the implementation environment including a first server 201. Exemplarily, when the server 201 is used to train an OCR model, it can first obtain a reference scene image, perform edge detection on the reference scene image to obtain a first edge image corresponding to the reference scene image, perform image transformation processing on the first edge image to obtain multiple second edge images corresponding to the first edge image, input the second edge images into a preset scene image generation model, adjust the image type of the second edge images based on the scene image generation model, obtain target scene images corresponding to the second edge images, obtain a baseline image, synthesize the baseline image with each target scene image to generate multiple target sample images, and then use the multiple target sample images to train the OCR model.

[0094] Reference Figure 3 , Figure 3A schematic diagram of another implementation environment provided for an embodiment of the present application, the implementation environment includes a second server 301 and multiple terminals 302. The multiple terminals 302 and the second server 301 constitute a distributed architecture. Specifically, when the server 201 is used to train the OCR model, each terminal 302 obtains a reference scene image, performs edge detection on the reference scene image, obtains a first edge image corresponding to the reference scene image, performs image transformation processing on the first edge image, obtains multiple second edge images corresponding to the first edge image, inputs the second edge image into a preset scene image generation model, adjusts the image type of the second edge image based on the scene image generation model, obtains a target scene image corresponding to the second edge image, obtains a reference image, synthesizes the reference image with each target scene image, generates multiple target sample images, and then sends the target sample images to the second server 301. The second server 301 then uses the target sample images sent by each terminal 302 to train the OCR model.

[0095] The above-mentioned first server 201 and second server 301 can be independent physical servers, or a server cluster or distributed system composed of multiple physical servers. They can also be cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0096] In addition, the first server 201 and the second server 301 can also be a node server in the blockchain network.

[0097] The terminal 302 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a vehicle-mounted terminal, etc., but is not limited thereto. The terminal 302 and the second server 301 can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment of the present application.

[0098] The method provided in the embodiments of the present application can be applied to various technical fields, including but not limited to cloud technology, artificial intelligence, image processing and other technical fields.

[0099] Reference Figure 4 , Figure 4 This is a flow chart of an image generation method provided in an embodiment of the present application. The image generation method can be executed by a server or by a terminal and a server in cooperation. The image generation method includes but is not limited to the following steps 401 to 404.

[0100] Step 401: Acquire a reference scene image, perform edge detection on the reference scene image, and obtain a first edge image corresponding to the reference scene image.

[0101] In one possible implementation, the reference scene image is a real captured image, and the reference scene image can be determined according to the type of target sample image to be generated. For example, if the target sample image to be generated is a document image, the reference scene image can be a shadow image under indoor lighting, a shadow image under outdoor lighting, a shadow image under outdoor sunlight, an indoor lighting image, an outdoor lighting image, a paper texture image, a paper material image, etc.; if the target sample image to be generated is a facial image, the reference scene image can be a shadow image under indoor lighting, a shadow image under outdoor lighting, a shadow image under outdoor sunlight, an indoor lighting image, an outdoor lighting image, a facial background image, a facial texture image, etc. It is understandable that the embodiment of the present application does not limit the specific type of the reference scene image. Among them, the reference scene image can be obtained by shooting a real scene through the terminal camera, or downloading it from the Internet, etc., which is not limited in the embodiment of the present application.

[0102] In one possible implementation, edge detection is performed on the reference scene image, that is, the image edge of the reference scene image is extracted. The image edge is the position where the grayscale value changes more dramatically. By performing edge detection on the reference scene image, the first edge image obtained only contains the edge information of the reference scene image, and thus has the advantage of a simple structure, which is conducive to improving the feasibility and efficiency of subsequent image transformation processing.

[0103] It is understandable that after edge detection is performed on a reference scene image, a first edge image corresponding to the reference scene image can be obtained. When there are multiple reference scene images, the first edge images corresponding to each reference scene image are obtained.

[0104] Step 402: performing image transformation processing on the first edge image to obtain a plurality of second edge images corresponding to the first edge image.

[0105] The number of second edge images is greater than the number of first edge images. That is, the image transformation process is primarily used to generate a large number of second edge images. Performing image transformation on one first edge image can generate multiple second edge images. In other words, one reference scene image can correspond to multiple second edge images. In one possible implementation, different second edge images can be obtained by changing transformation parameters of the image transformation process.

[0106] In one possible implementation, image transformation processing is performed on the first edge image to obtain multiple second edge images corresponding to the first edge image. Specifically, there are the following methods:

[0107] One method is to rotate the first edge image to obtain multiple second edge images corresponding to the first edge image. The edge information in the second edge images obtained after the rotation process is different from the edge information in the first edge image. In this case, the transformation parameters can be rotation parameters, such as rotation direction, rotation angle, etc. Different rotation parameters can produce different second edge images. The rotation process can be clockwise or counterclockwise, and the rotation angle can be 90 degrees, 180 degrees, etc., which are not limited in the embodiments of the present application.

[0108] Another way is to crop the first edge image to obtain multiple second edge images corresponding to the first edge image; wherein, the edge information in the second edge image obtained after the cropping process is different from the edge information of the first edge image. At this time, the transformation parameter can be a cropping parameter, such as a cropping position, a cropping size, etc. Different cropping parameters can obtain different second edge images. The cropping standard can be determined according to actual conditions. For example, the center point of the first edge image can be used as a reference, and the first edge image can be cropped according to a preset cropping size; or the vertex of the first edge image can be used as a reference, and the first edge image can be cropped according to a preset cropping size. The specific cropping size can be determined according to actual needs, and the embodiments of the present application are not limited thereto.

[0109] Another method is to flip the first edge image to obtain multiple second edge images corresponding to the first edge image; wherein the edge information in the second edge images obtained after the flipping process is different from the edge information of the first edge image, in which case the transformation parameter can be a flipping parameter, such as a flipping direction, etc. Different flipping parameters can produce different second edge images, and after the flipping process, the first edge image and the second edge image are symmetrical about the flipping axis. The flipping process can be horizontal flipping or vertical flipping, which is not limited in this embodiment of the application.

[0110] Another method is to perform Gaussian filtering on the first edge image to obtain multiple second edge images corresponding to the first edge image, wherein the edge information in the second edge image obtained after Gaussian filtering is different from the edge information of the first edge image. At this time, the transformation parameter can be a Gaussian filtering parameter, such as the Gaussian kernel size, etc. Different Gaussian filtering parameters can obtain different second edge images. Specifically, the first edge image can be convolved with a preset Gaussian kernel to obtain the second edge image. The specific structure of the Gaussian kernel can be determined according to actual needs, and the embodiment of the present application is not limited thereto.

[0111] The second edge image obtained after the image transformation processing contains edge information that is different from the edge information of the first edge image, thereby facilitating the subsequent generation of multiple different target sample images. In addition, considering the size matching problem of the multiple target sample images generated subsequently, the second edge image after the image transformation processing can be resized so that the size of the second edge image reaches the target value. For example, the second edge image can be the same size as the target sample image. If the size of the target sample image to be generated subsequently is 640*640, the size of the second edge image must also be adjusted to 640*640. It is understood that the adjustment method can be enlargement or reduction.

[0112] It can be understood that the above four methods of performing image transformation processing on the first edge image can be executed according to any combination, that is, one method can be selected to execute, or multiple methods can be selected to execute. When multiple methods are selected to execute, it is beneficial to improve the diversity of the obtained second edge image, so that more types of target sample images are generated subsequently.

[0113] Step 403: Input the second edge image into a preset scene image generation model, adjust the image type of the second edge image based on the scene image generation model, and obtain a target scene image corresponding to the second edge image.

[0114] Among them, different image types contain different types of image feature information. In an embodiment of the present application, the scene image generation model is used to convert the edge image into a scene image, that is, to establish a mapping relationship between the edge image and the scene image. The scene image generation model is a pre-trained model, the second edge image is the input of the scene image generation model, and the target scene image is the output of the scene image generation model. The second edge image and the target scene image can be in a one-to-one correspondence. Therefore, when multiple second edge images are input into the scene image generation model, multiple target scene images can be obtained, that is, one reference scene image can correspond to multiple target scene images.

[0115] In one possible implementation, the target scene image may be one or more types of images such as a texture image, a lighting image, a shadow image, and a material image, wherein the image type of the target scene image is determined by the reference scene image. For example, when the reference scene image is a texture image, the corresponding target scene image is also a texture image, but the display style of the target scene image is different from that of the reference scene image.

[0116] It can be seen that the embodiment of the present application does not directly adjust the display style of the reference image to obtain the target sample image, but first performs edge detection on the reference scene image to obtain a first edge image. Since the first edge image has a simple structure, multiple second edge images can be quickly obtained through image transformation processing. Accordingly, the image type of the second edge image can be adjusted based on the scene image generation model to obtain multiple target scene images.

[0117] Step 404: Acquire a reference image, and synthesize the reference image with each target scene image to generate a plurality of target sample images.

[0118] The reference image is a real captured image. The type of the reference image can be determined according to actual needs. For example, if you want to train an OCR model, the reference image can be a document image; if you want to train a face recognition model, the reference image can be a face image; if you want to test beauty software, the reference image can be a face image, and so on. By synthesizing the reference image with each target scene image, you can generate a target sample image with a different display style from the reference image. For example, the display style of the target sample image can be similar to Figure 1 The display styles of images in the image dataset A' are shown.

[0119] In one possible implementation, the synthesis processing may be to use the pixel value matrix of the reference image and the pixel value matrix of the target scene image to perform operations to obtain the pixel value matrix of the target sample image, and then generate the target sample image. The specific operation method may be addition, multiplication, etc., which is not limited in the embodiments of the present application.

[0120] Since there are multiple target scene images, one reference image can generate multiple target sample images. In summary, the image generation method provided in the embodiment of the present application only needs to collect a small number of reference scene images and reference images to generate a large number of target sample images. Compared with the method of collecting a large number of reference images, under the premise of generating the same number of target sample images, the image generation method provided in the embodiment of the present application can effectively reduce the number of image collections, reduce the generation cost of target sample images, and improve the generation efficiency of target sample images.

[0121] In addition, since the target scene image is a texture image, lighting image, shadow image, material image and other types of images, the target sample image obtained after synthesis processing of the reference image has a higher authenticity, making the target sample image closer to the artificially collected data, which is conducive to improving the use effect in subsequent model training, software testing and other application scenarios.

[0122] In one possible implementation, in step 401, edge detection is performed on the reference scene image to obtain a first edge image corresponding to the reference scene image. Specifically, the reference scene image can be grayscaled to obtain a grayscale image corresponding to the reference scene image; the grayscale image can be Gaussian filtered to obtain a filtered image; the pixel gradient intensity matrix of the filtered image is calculated, each image pixel in the pixel gradient intensity matrix is ​​traversed, the gradient intensity of the currently traversed target pixel is compared with the gradient intensity of each adjacent pixel, and the edge pixel is determined from the image pixel based on the comparison result; and the first edge image corresponding to the reference scene image is obtained based on the edge pixel.

[0123] Specifically, refer to Figure 5 , Figure 5 The following is a schematic diagram of the overall process for edge detection on a reference scene image provided in an embodiment of the present application. The reference scene image can be an image of a material such as shadow, lighting, or texture. Since the edge of an image is a location where the grayscale value changes dramatically, the reference scene image is first grayscaled, and the resulting grayscale image is used as the basis for edge detection. In one possible implementation, the grayscale processing of the reference scene image can be performed by first extracting the sampling values ​​of the RGB channels of the reference scene image, and then performing a weighted average based on the sampling values ​​of the RGB channels of the reference scene image to obtain a grayscale matrix of the reference scene image, thereby obtaining a grayscale image.

[0124] Next, the grayscale image is subjected to Gaussian filtering to achieve the effect of denoising, which is beneficial to improving the accuracy of subsequent edge detection. Among them, a two-dimensional Gaussian kernel of a preset size can be used to perform a convolution operation with the grayscale image. The Gaussian kernel is a discrete approximation of a continuous Gaussian function and can be obtained by discrete sampling and normalization of the Gaussian surface. The Gaussian kernel used in the Gaussian filtering process can be a Gaussian function with two dimensions, x and y, and the standard deviation in the two dimensions is generally the same, which can be specifically expressed as:

[0125]

[0126] Where G(x,y) represents the Gaussian kernel, (x,y) represents the pixel value of the pixel point, and σ is the standard deviation of the Gaussian distribution.

[0127] Next, after obtaining the filtered image, the pixel gradient intensity matrix of the filtered image is further calculated. In one possible implementation, the target operator can be used to calculate the pixel gradient intensity matrix of the filtered image. The target operator can be a Sobel operator, and the Sobel operator can be two 3*3 matrices, namely matrix Sx and matrix Sy. Matrix Sx is used to calculate the pixel gradient matrix Gx in the x direction of the filtered image, and matrix Sy is used to calculate the pixel gradient matrix Gy in the y direction of the filtered image. Specifically, it can be expressed as:

[0128]

[0129] Wherein, I is the gray value matrix of the filtered image, the origin of the coordinate system of the pixel intensity matrix of the filtered image can be at the upper left corner of the filtered image, the positive x direction is from left to right, and the positive y direction is from top to bottom.

[0130] The pixel gradient intensity matrix of the filtered image can be obtained based on the pixel gradient matrix Gx and the pixel gradient matrix Gy. Specifically, the gradient amplitude in the x direction can be calculated based on the pixel gradient matrix Gx, and the gradient amplitude in the y direction can be calculated based on the pixel gradient matrix Gy to obtain the pixel gradient intensity matrix of the filtered image.

[0131] Next, non-maximum pixel gradient suppression processing is performed based on the pixel gradient intensity matrix. Specifically, the gradient intensity of the target pixel point currently traversed in the pixel gradient intensity matrix can be compared with the gradient intensity of each adjacent pixel point. If the gradient intensity of the target pixel point is the maximum value, the target pixel point is retained as the edge pixel point. If the gradient intensity of the target pixel point is not the maximum value, the target pixel point is suppressed, and the suppression processing is to set the pixel value of the target pixel point to zero. Among them, the adjacent pixel points of the target pixel point are the pixel points adjacent to the target pixel point along the positive and negative directions of x and y. In short, they are the pixel points adjacent to the target pixel point above, below, left and right. The number of adjacent pixel points of the target pixel point is four. The gradient intensity of the target pixel point currently traversed is compared with the gradient intensity of each adjacent pixel point. The edge pixel point is determined from the image pixel points based on the comparison result. This can eliminate the stray response brought by edge detection and play a "slimming" role on the edge.

[0132] After traversing each image pixel point in the pixel gradient intensity matrix and comparing the gradient intensity, a plurality of edge pixel points can be obtained, and a first edge image corresponding to the reference scene image can be obtained based on these edge pixel points.

[0133] On this basis, in one possible implementation, when edge pixels are determined from image pixels based on the gradient intensity comparison results, threshold hysteresis processing may be further performed. Specifically, if the gradient intensity of the target pixel is greater than the gradient intensity of each adjacent pixel, the gradient intensity of the target pixel may be further compared with a preset first intensity threshold and a second intensity threshold. If the gradient intensity of the target pixel is greater than or equal to the first intensity threshold, the target pixel may be considered a strong edge pixel. In this case, the target pixel may be determined as the final edge pixel. The first intensity threshold is greater than the second intensity threshold. The values ​​of the first intensity threshold and the second intensity threshold may be determined according to actual needs and are not limited in the embodiments of the present application. In addition, the ratio between the first intensity threshold and the second intensity threshold may be between 2:1 and 3:1, which is conducive to improving the rationality of the first intensity threshold and the second intensity threshold.

[0134] If the gradient intensity of the target pixel is less than the first intensity threshold and greater than or equal to the second intensity threshold, the target pixel can be considered to be a weak edge pixel. At this time, further isolated weak edge suppression processing can be performed. Specifically, the neighborhood pixels corresponding to the target pixel can be determined. When the gradient intensity of the neighborhood pixels is greater than or equal to the first intensity threshold, the target pixel is determined to be an edge pixel. Among them, for the nine-square grid centered on the target pixel, the neighborhood pixels corresponding to the target pixel are the remaining eight pixels in the nine-square grid except the target pixel. It can be seen that the neighborhood pixels corresponding to the target pixel include the adjacent pixels corresponding to the target pixel. Generally speaking, if the target pixel is a weak edge pixel caused by a real edge, the target pixel will be connected to a strong edge pixel. If the target pixel is a noise response point, the target pixel will not be connected to a strong edge pixel. Therefore, by judging whether the gradient intensity of the neighborhood pixels of the target pixel is greater than or equal to the first intensity threshold, it can be determined whether the target pixel is connected to a strong edge pixel. As long as the target pixel is connected to a strong edge pixel, the target pixel can be retained as the final edge pixel.

[0135] If the gradient intensity of the target pixel point is less than the second intensity threshold, the target pixel point may be suppressed.

[0136] By further comparing the gradient intensity of the target pixel with the preset first intensity threshold and the second intensity threshold, the accuracy and rationality of determining the edge pixel points can be improved. Moreover, when comparing the gradient intensity of the target pixel with the preset first intensity threshold and the second intensity threshold, a more refined comparison is performed by introducing the neighborhood pixels of the target pixel point, which is conducive to improving the hierarchical nature of the comparison of the gradient intensity of the target pixel point with the preset first intensity threshold and the second intensity threshold, thereby improving the overall refinement of determining the edge pixel points.

[0137] Finally, the material edge information of the material image can be obtained based on the edge pixels retained after the non-maximum pixel gradient suppression processing, threshold hysteresis processing and isolated weak edge suppression processing.

[0138] The following describes in detail a processing process of the scene image generation model provided by an embodiment of the present application.

[0139] Reference Figure 6 , Figure 6 A structural schematic diagram of a scene image generation model provided in an embodiment of the present application. In one possible implementation, the scene image generation model includes multiple convolutional layers connected in sequence and multiple deconvolution layers connected in sequence. The convolutional layer is used to perform convolution processing on the image input to the scene image generation model. In the embodiment of the present application, the convolutional layer is used to reduce the size of the second edge image, and the deconvolution layer is used to perform deconvolution processing on the image input to the scene image generation model. Deconvolution processing is the inverse process of convolution processing. In the embodiment of the present application, the deconvolution layer is used to increase the size of the second edge image.

[0140] based on Figure 6 The scene image generation model shown in the above step 403, the second edge image is input into the preset scene image generation model, and the image type of the second edge image is adjusted based on the scene image generation model to obtain a target scene image corresponding to the second edge image. Specifically, the second edge image can be convolved based on multiple convolution layers to obtain a target convolution image; and the target convolution image can be deconvolved based on multiple deconvolution layers to obtain a target scene image corresponding to the second edge image.

[0141] When the second edge image is convolved based on multiple convolutional layers, the input of the current convolutional layer is the output of the previous convolutional layer, and the input of the current deconvolutional layer is the output of the previous deconvolutional layer. For example, assuming Figure 6The scene image generation model shown is provided with three convolution layers and three deconvolution layers. The input of the first convolution layer is the second edge image, the input of the second convolution layer is the transition image output by the first convolution layer, the input of the third convolution layer is the transition image output by the first convolution layer, the input of the first deconvolution layer is the transition image output by the third convolution layer, the input of the second deconvolution layer is the transition image output by the first deconvolution layer, and the input of the third deconvolution layer is the transition image output by the second deconvolution layer.

[0142] By convolving the second edge image with multiple convolutional layers and then deconvolving it with multiple deconvolutional layers, the target scene image can have a larger field of view, improving the quality of the generated target scene image. Furthermore, by reducing the size of the second edge image through multiple convolutional layers, the computational effort of subsequent deconvolutional layers can be reduced, improving the efficiency of the deconvolution process.

[0143] It can be understood that the number of convolutional layers and deconvolutional layers in the scene image generation model can be determined according to actual needs, and the number of convolutional layers and deconvolutional layers can be the same or different. In the embodiment of the present application, only the number of convolutional layers and deconvolutional layers is three for illustration. In fact, the embodiment of the present application does not limit the number of convolutional layers and deconvolution layers.

[0144] In one possible implementation, when the number of convolutional layers and deconvolutional layers is the same (e.g. Figure 6 In the scene image generation model shown, the number of convolution layers and deconvolution layers is three each). In this case, when deconvolution processing is performed on the target convolution image based on multiple deconvolution layers to obtain a target scene image corresponding to the second edge image, a first transition image to be input to the target deconvolution layer can be obtained; a target convolution layer corresponding to the target deconvolution layer is determined from the multiple convolution layers, and a second transition image output after the convolution processing of the target convolution layer is obtained; the first transition image and the second transition image are spliced ​​to obtain a spliced ​​image; and the spliced ​​image is deconvolution processed based on the target deconvolution layer until the target scene image corresponding to the second edge image is obtained.

[0145] Specifically, based on Figure 6 The scene image generation model shown in Figure 7 , Figure 7A schematic diagram of a processing flow for a scene image generation model provided in an embodiment of the present application, wherein the target deconvolution layer is the deconvolution layer currently to be deconvolved among multiple deconvolution layers, and the transition image is an intermediate image generated during the convolution or deconvolution process, that is, the transition image is an image other than the image input to the scene image generation model and the image output by the scene image generation model. When the target deconvolution layer is deconvolution layer one, the first transition image is the transition image output by convolution layer three; when the target deconvolution layer is deconvolution layer two, the first transition image is the transition image output by deconvolution layer one; when the target deconvolution layer is deconvolution layer three, the first transition image is the transition image output by deconvolution layer two.

[0146] Since the number of convolutional layers and deconvolutional layers is the same, there is a one-to-one correspondence between convolutional layers and deconvolutional layers. For example, when the target deconvolutional layer is deconvolutional layer one, the target convolutional layer is convolutional layer three, and accordingly, the second transition image is the transition image output by convolutional layer three; when the target deconvolutional layer is deconvolutional layer two, the target convolutional layer is convolutional layer two, and the second transition image is the transition image output by convolutional layer two; when the target deconvolutional layer is deconvolutional layer three, the target convolutional layer is convolutional layer one, and the second transition image is the transition image output by convolutional layer one.

[0147] When the target deconvolution layer is deconvolution layer 1, the first transition image and the second transition image are actually the same image. This is a special case and will not be further explained here. When the target deconvolution layer is deconvolution layer 2, the stitched image is obtained by stitching the transition image output by deconvolution layer 1 with the transition image output by convolution layer 2. When the target deconvolution layer is deconvolution layer 3, the stitched image is obtained by stitching the transition image output by deconvolution layer 2 with the transition image output by convolution layer 1. Each target deconvolution layer performs deconvolution processing on the corresponding stitched image in this way, and finally, deconvolution layer 3 outputs the target scene image corresponding to the second edge image.

[0148] In one possible implementation, the first transition image and the second transition image have the same size, and the first transition image and the second transition image are spliced ​​together. The grayscale value matrix corresponding to the first transition image and the grayscale value matrix corresponding to the second transition image can be residually connected, or the grayscale value matrix corresponding to the first transition image and the grayscale value matrix corresponding to the second transition image can be directly added together. This embodiment of the present application does not limit this.

[0149] When performing deconvolution operation using the deconvolution layer, by splicing the first transition image with the second transition image output by the corresponding convolution layer, the image output to the deconvolution layer can carry more image information in the second edge image, which is beneficial to improving the image type adjustment effect of the second edge image and making the subsequently generated target scene image more accurate.

[0150] The following uses a document image as an example to illustrate the overall process of the image generation method provided in the embodiment of the present application.

[0151] Reference Figure 8 , Figure 8 The following is a schematic diagram of the overall process of an example of an image generation method provided by an embodiment of the present application, wherein image B is an actually captured real scene image. First, edge detection is performed on image B to obtain image C, which is the edge information of the real scene image. Next, image C is transformed to obtain a large number of images C'. Next, image C' is input into a scene image generation model to obtain a large number of images B'. Finally, image A and image B' are synthesized to obtain a large number of images A'. It can be seen that the embodiment of the present application can greatly increase the number of edge images by performing edge detection on the real scene image and transforming the obtained real edge information. A large number of scene images can then be obtained by synthesizing these scene images with reference document images to obtain a large number of sample document images. Therefore, only a small number of real scene images and a small number of reference document images need to be collected to generate a large number of sample document images. Compared with the method of collecting a large number of reference document images, while generating the same number of sample document images, the number of images collected can be effectively reduced, the generation cost of sample document images can be reduced, and the generation efficiency of sample document images can be improved.

[0152] It is understandable that, due to the high acquisition cost of image A, the image generation method provided in the embodiment of the present application actually converts the task of acquiring image A into the task of acquiring image B, and obtains the final image A' by generating image B'. In order to further reduce the difficulty of collecting image B, image B is converted into an image C with a simple structure. Image C can be copied in large quantities through simple transformation processing to obtain image C', and then image B' is obtained through the scene image generation model. Finally, image B' is synthesized with a small amount of image A to obtain a large number of images A'. It can be seen that the image generation method provided in the embodiment of the present application converts the task of converting a complex image A into image A' into several subtasks, and converts the task of acquiring a large number of images A into the task of acquiring a small number of images A and a small number of images B, effectively reducing the number of images acquired, reducing the generation cost of image A', and improving the generation efficiency of image A'.

[0153] The following describes in detail the training principle of the above scene image generation model.

[0154] In one possible implementation, referring to Figure 9 , Figure 9A schematic diagram of a training architecture for a scene image generation model provided in an embodiment of the present application, wherein the image data set X is a training edge image set, the image data set Y is a label scene image set, G is a scene image generation model, and D is a training edge image set. Y For the first judgment model, when training the scene image generation model, a training edge image set and a label scene image set can be obtained; based on the scene image generation model, the image type of the target training edge image is adjusted to obtain the training scene image corresponding to the target training edge image; the training scene image is input into the first judgment model to obtain a first judgment result, and the label scene image is input into the first judgment model to obtain a second judgment result, and the first loss value corresponding to the scene image generation model is calculated according to the first judgment result and the second judgment result; the parameters of the scene image generation model are adjusted according to the first loss value.

[0155] Among them, the training edge image set includes multiple training edge images, and the training edge images serve as training samples of the scene image generation model; the label scene image set includes multiple label scene images, and the label scene images serve as labels for training the scene image generation model; and the image type of the training edge image set is different from the image type of the label scene image set.

[0156] In one possible implementation, the training edge images in the training edge image set and the label scene images in the label scene image set can be one-to-one corresponding. In this case, the label scene image input into the first judgment model is the label scene image corresponding to the training edge image.

[0157] In one possible implementation, the training edge images in the training edge image set and the labeled scene images in the labeled scene image set do not need to correspond one to one. It is only necessary that the training edge images in the training edge image set belong to one image type and the labeled scene images in the labeled scene image set belong to another image type. This is a model training of image style transfer, which is conducive to reducing training costs and improving training efficiency.

[0158] The purpose of training the scene image generation model is to make the training scene images generated by the scene image generation model as close as possible to the labeled scene images, that is, the first judgment model identifies the training scene images as real scene images. The first judgment result and the second judgment result can be used to obtain a first loss value generated by the scene image generation model, and then the parameters of the scene image generation model can be adjusted based on the first loss value. Adjusting the parameters of the scene image generation model can include adjusting the convolution parameters of a convolutional layer or a deconvolution layer in the scene image generation model.

[0159] Among them, the first loss value can be expressed as:

[0160] L(G,D Y ,X,Y)=E y～pdata(y) [logD Y (y)]+E x～pdata(x) [log(1-D Y (G(x)))]

[0161] Among them, L(G,D Y ,X,Y) represents the first loss value, G represents the scene image generation model, D Y Represents the first judgment model, X represents the training edge image set, x represents the training edge image, Y represents the label scene image set, y represents the training edge image, E represents the expected value of the distribution function, and pdata represents the image distribution.

[0162] The target training edge image is one of the multiple training edge images. That is, during the training of the scene image generation model, the training scene images corresponding to each training edge image are generated based on the training scene image generation model. During the training process, the parameters of the first judgment model can also be adjusted to make the training scene images generated by the scene image generation model increasingly realistic, thereby increasing the discriminative ability of the first judgment model and improving training effectiveness.

[0163] In one possible implementation, the first judgment model may be provided with a convolutional layer and a fully connected layer. After the training scene image is input into the first judgment model, the first judgment model first convolves the training scene image through the convolutional layer and then passes the convolved image through the fully connected layer to obtain a first judgment result. The principle of deriving the second judgment model from the first judgment model is similar and will not be repeated here.

[0164] In one possible implementation, referring to Figure 10 , Figure 10 Schematic diagram of another training architecture of the scene image generation model provided in an embodiment of the present application. On the basis of using the first judgment model to train the scene image generation model, an edge image generation model and a second judgment model can be further introduced to train the scene image generation model, wherein the image data set X is a training edge image set, the image data set Y is a label scene image set, G is a scene image generation model, and D Y is the first judgment model, D X is the second judgment model, F is the edge image generation model, G and F are the mappings from X to Y and Y to X respectively, and the first judgment model D Y and the second judgment model D XThe mapped images can be judged and processed respectively. Among them, the image x in the image data set X is mapped by the scene image generation model G to obtain the image Y′, and the first judgment model D Y The image Y′ is also mapped back to the distribution of the image data set X through the edge image generation model F to obtain the image x′; and the image y in the image data set Y is mapped back to the distribution of the image data set X through the edge image generation model F to obtain the image X′, and is mapped back to the distribution of the image data set X through the second judgment model D X The judgment process is performed, and the image X′ is also mapped back to the distribution of the image data set Y through the scene image generation model G to obtain the image y′.

[0165] Specifically, when adjusting the parameters of the scene image generation model according to the first loss value, the image type of the training scene image can be adjusted based on the edge image generation model to obtain a restored edge image corresponding to the training scene image; the norm between any training edge image other than the target training edge image in the training edge image set and the restored edge image is calculated, and the restoration loss value is calculated based on the norm; the second loss value corresponding to the edge image generation model is calculated based on the second judgment model; the target loss value is obtained according to the sum of the first loss value, the second loss value and the restoration loss value, and the parameters of the scene image generation model are adjusted according to the target loss value.

[0166] Specifically, the edge image generation model functions in the opposite manner to the scene image generation model. That is, the input of the edge image generation model is a scene image, and the output is an edge image. The processing of the edge image generation model is actually the inverse of the processing of the scene image generation model. For details, please refer to the aforementioned description of the working principle of the edge image generation model, and the working principle of the edge image generation model will not be repeated here. Furthermore, the working principle of the second judgment model is similar to that of the first judgment model and will not be repeated here.

[0167] When the model network capacity is large enough, the model can map the same image set to a random arrangement of any image in the target domain. Therefore, if the model training is carried out solely by the first loss value, the final model performance needs to be improved. By mapping the training scene image generated by the scene image generation model to a restored edge image through the edge image generation model, it can be judged whether the training scene image generated by the scene image generation model can be restored to the original training edge image more accurately, so as to judge the model performance of the scene image generation model. Moreover, in the embodiment of the present application, any training edge image other than the target training edge image is used to calculate the restoration loss value. Compared with using the target training edge image itself to calculate the restoration loss value, it can avoid the situation of data overfitting, achieve the effect of data enhancement, improve the training effect of the scene image generation model, and make the scene image generation model have better robustness in different application scenarios.

[0168] Similarly, in order to improve the performance of the edge image generation model, the scene image generation model can also be used to map the edge image generated by the edge image generation model into a scene image. Based on this, the restoration loss value can be expressed as:

[0169] L(G,F)=E x～pdata(x) [‖F(G(x))-x‖1]+E y～pdata(y) [‖G(F(y))-y‖1]

[0170] Among them, L(G,F) represents the restoration loss value, and F represents the edge image generation model.

[0171] In addition, a second loss value corresponding to the edge image generation model can also be calculated based on the second judgment model. The calculation method of the second loss value is similar to the calculation method of the first loss value. The second loss value can be expressed as:

[0172] L(F,D x ,Y,X)=E x～pdata(x) [logD x (x)]+E y～pdata(y) [log(1-D x (F(y)))]

[0173] Among them, D x Represents the second judgment model.

[0174] In summary, the target loss value can be expressed as:

[0175] L(G,F,D x ,D Y )=L(G,D Y ,X,Y)+L(F,D x ,Y,X)+L(G,F)

[0176] Among them, L(G,F,D x ,D Y ) represents the target loss value.

[0177] On the basis of using the first judgment model to train the scene image generation model, by further introducing the edge image generation model and the second judgment model, and then obtaining the first loss value, the second loss value and the restoration loss value, the final target loss value can contain more loss information, which is conducive to improving the training effect of the scene image generation model and improving the performance of the scene image generation model.

[0178] When applying the aforementioned scene image generation model, the second edge image and the target scene image can have a one-to-one correspondence. That is, when a second edge image is input to the scene image generation model, the output is a target scene image corresponding to the second edge image. Furthermore, to further increase the number of generated target scene images, the scene image generation model can also add a coding layer to generate random noise.

[0179] Based on this, another processing process of the scene image generation model provided by an embodiment of the present application is described in detail below.

[0180] Reference Figure 11 , Figure 11 Another structural diagram of the scene image generation model provided in the embodiment of the present application, wherein the scene image generation model includes a plurality of sequentially connected convolutional layers, a plurality of sequentially connected deconvolutional layers and a coding layer, wherein the processing principle of the plurality of sequentially connected convolutional layers and the plurality of sequentially connected deconvolutional layers can be referred to the embodiment of the present application. Figure 6 The explanation of the model structure shown is not repeated here. On this basis, a coding layer is further added, wherein the coding layer is used to generate random noise. When the scene image generation model is used to generate the target scene image, the coding layer repeatedly generates different random noises, and these different random noises are added to the second edge image respectively. Then, the scene image generation model is used to generate the target scene image. In this way, a second edge image is input to the scene image generation model. By adding different random noises, the scene image generation model can generate different target scene images, which is conducive to increasing the number of target scene images generated. Accordingly, under the premise that the number of target scene images is required to be certain, the number of reference scene images collected can be reduced accordingly, the generation cost of the target sample images can be reduced, and the generation efficiency of the target sample images can be improved.

[0181] Specifically, refer to Figure 12 , Figure 12 The schematic diagram of adding random noise provided in the embodiment of the present application is that after the coding layer generates random noise, the random noise can be added to the second edge image before the second edge image is input into the scene image generation model. Therefore, a second edge image with different random noises added can be obtained. Then, the second edge image with different random noises added can be input into the scene image generation model to obtain multiple different target scene images. In addition, random noise can be added to the transition images output by each convolution layer separately. For example, random noise can be added only to the transition image output by convolution layer 1, or random noise can be added only to the transition images output by convolution layer 1 and convolution layer 2, or random noise can be added to the transition images output by convolution layer 1, convolution layer 2, and convolution layer 3.

[0182] The image generation method provided in the embodiment of the present application can be applied to different scenarios, and is described below using a reference image as an example of a document image.

[0183] In one possible implementation, when acquiring a reference scene image, a reference paper image can be acquired, and at least one of a paper material image, a paper shadow image, or a paper light image can be cropped from the reference paper image; at least one of the paper material image, the paper shadow image, or the paper light image can be resized; and at least one of the resized paper material image, the paper shadow image, or the paper light image can be used as a reference scene image.

[0184] Since the reference image is a document image, obtaining the reference scene image based on the reference paper image can make the ultimately generated target sample image closer to the real document image, thereby improving the image quality of the target sample image.

[0185] Among them, there can be multiple different cropping combinations when cropping the reference paper image, that is, one or more combinations of paper material images, paper shadow images or paper light images can be cropped from the reference paper image, for example, the paper material image can be cropped from the reference paper image, or the paper material image and paper shadow image can be cropped from the reference paper image, or the paper material image, paper shadow image and paper light image can be cropped from the reference paper image, and so on.

[0186] In one possible implementation, by cropping three different types of images, namely, paper material image, paper shadow image, and paper light image, from a reference paper image, and then resizing the paper material image, paper shadow image, and paper light image, only one reference paper image needs to be collected to obtain a variety of different types of scene images through cropping and resizing, which is conducive to further reducing the number of image acquisitions, reducing the generation cost of target sample images, and improving the generation efficiency of target sample images.

[0187] On this basis, the target scene image generated by the scene image generation model includes a target material image corresponding to the paper material image, a target shadow image corresponding to the paper shadow image, and a target light image corresponding to the paper light image. In the above step 404, the reference image is synthesized with each target scene image to generate multiple target sample images. Specifically, the document image can be separated to obtain the content image and the original background image corresponding to the document image; the original background image is synthesized with the target material image, the target shadow image, and the target light image respectively to obtain multiple effect background images; the content image is synthesized with each effect background image to generate multiple target sample images.

[0188] Among them, the content image can be a text image, an icon image, etc. The image information contained in the content image depends on the actual content of the document image, and is not limited in the embodiment of this application.

[0189] Specifically, refer to Figure 13 , Figure 13 A schematic diagram of the document image synthesis process provided in an embodiment of the present application, by first separating the document image into the corresponding content image and the original background image, and then synthesizing the original background image with the target material image, the target shadow image, and the target light image respectively, can reduce the interference with the content in the document image during the synthesis process. Subsequently, the content image is synthesized with each effect background image, which can greatly improve the display quality of the content in the document image, so that the quality of the generated target sample image is closer to the baseline image.

[0190] In one possible implementation, the document image separation process may be performed using a pre-trained image segmentation model.

[0191] In one possible implementation, when the original background image is synthesized with each target scene image to obtain multiple effect background images, a first image matrix of the original background image and a second image matrix of each target scene image can be obtained; corresponding matrix elements in the first image matrix and each second image matrix are multiplied to obtain multiple target sample image matrices; and corresponding effect background images are generated based on the multiple target sample image matrices.

[0192] Among them, when adjusting the size of the paper material image, paper shadow image and paper light image, the size of the paper material image, paper shadow image and paper light image is adjusted to be the same as the document image, so that the size of the first image matrix and the second image matrix is ​​the same, which facilitates the operation of the first image matrix and the second image matrix.

[0193] In one possible implementation, the reference image may be an animal image in addition to a document image. Accordingly, the obtained target sample image may be used to train an image recognition model. For example, if the function of the image recognition model is to identify the species of animals in the image, the image generation method provided in the embodiment of the present application may be used to generate a large number of sample animal images to improve the recognition accuracy of the image recognition model under different lighting, shadows, backgrounds, and other styles.

[0194] In one possible implementation, the reference image may be a facial image in addition to a document image. Accordingly, the obtained target sample image may be used to train a facial recognition model. For example, if the function of the facial recognition model is to perform facial recognition unlocking, facial recognition payment, facial recognition vehicle start, etc., the image generation method provided in the embodiment of the present application may be used to generate a large number of sample facial images to improve the recognition accuracy of the facial recognition model under different lighting, shadows, backgrounds, and other styles.

[0195] It will be appreciated that, although the various steps in the above-mentioned various flow charts are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless clearly stated in the present embodiment, the execution of these steps does not have strict order restrictions, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the above-mentioned flow charts can include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of the steps or stages in other steps or other steps.

[0196] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the characteristics of the target object, such as the target object attribute information or attribute information set, the permission or consent of the target object will be obtained first, and the collection, use and processing of such data will comply with the relevant laws, regulations and standards of the relevant countries and regions. In addition, when the embodiment of the present application needs to obtain the attribute information of the target object, the target object's separate permission or separate consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the target object's separate permission or separate consent, the necessary target object-related data for the normal operation of the embodiment of the present application will be obtained.

[0197] Reference Figure 14 , Figure 14 This is a schematic diagram of the structure of an image generation device provided in an embodiment of the present application. The image generation device 1400 includes:

[0198] The edge detection module 1401 is configured to obtain a reference scene image, perform edge detection on the reference scene image, and obtain a first edge image corresponding to the reference scene image;

[0199] A transformation module 1402 is configured to perform image transformation processing on the first edge image to obtain a plurality of second edge images corresponding to the first edge image;

[0200] The model processing module 1403 is configured to input the second edge image into a preset scene image generation model, adjust the image type of the second edge image based on the scene image generation model, and obtain a target scene image corresponding to the second edge image;

[0201] The synthesis module 1404 is used to obtain a reference image, and synthesize the reference image with each target scene image to generate multiple target sample images.

[0202] Furthermore, the scene image generation model includes a plurality of sequentially connected convolutional layers and a plurality of sequentially connected deconvolutional layers. The model processing module 1403 is specifically used to:

[0203] Performing convolution processing on the second edge image based on multiple convolution layers to obtain a target convolution image;

[0204] Deconvolution processing is performed on the target convolution image based on multiple deconvolution layers to obtain a target scene image corresponding to the second edge image.

[0205] Furthermore, the number of convolutional layers and the number of deconvolutional layers are the same, and the above-mentioned model processing module 1403 is specifically used to:

[0206] Acquire a first transition image to be input to a target deconvolution layer; wherein the target deconvolution layer is a deconvolution layer currently to be deconvolved among the multiple deconvolution layers;

[0207] Determining a target convolution layer corresponding to the target deconvolution layer from the multiple convolution layers, and obtaining a second transition image output by the target convolution layer after convolution processing;

[0208] Splicing the first transition image and the second transition image to obtain a spliced ​​image;

[0209] The spliced ​​image is deconvolved based on the target deconvolution layer until a target scene image corresponding to the second edge image is obtained.

[0210] Furthermore, the image generation device further includes a model training module, which is used to:

[0211] Acquire a training edge image set and a label scene image set; wherein the training edge image set includes a plurality of training edge images, the label scene image set includes a plurality of label scene images, and the image type of the training edge image set is different from the image type of the label scene image set;

[0212] Adjusting the image type of the target training edge image based on the scene image generation model to obtain a training scene image corresponding to the target training edge image; wherein the target training edge image is one of the multiple training edge images;

[0213] Inputting the training scene image into the first judgment model to obtain a first judgment result, inputting the label scene image into the first judgment model to obtain a second judgment result, and calculating a first loss value corresponding to the scene image generation model based on the first judgment result and the second judgment result;

[0214] Parameters of the scene image generation model are adjusted according to the first loss value.

[0215] Furthermore, the above model training module is specifically used to:

[0216] Adjust the image type of the training scene image based on the edge image generation model to obtain a restored edge image corresponding to the training scene image;

[0217] Calculate the norm between any training edge image other than the target training edge image in the training edge image set and the restored edge image, and calculate the restoration loss value based on the norm;

[0218] Calculating a second loss value corresponding to the edge image generation model based on the second judgment model;

[0219] A target loss value is obtained according to the sum of the first loss value, the second loss value and the restoration loss value, and the parameters of the scene image generation model are adjusted according to the target loss value.

[0220] Furthermore, the edge detection module 1401 is specifically configured to:

[0221] Grayscale processing is performed on the reference scene image to obtain a grayscale image corresponding to the reference scene image;

[0222] Perform Gaussian filtering on the grayscale image to obtain a filtered image;

[0223] Calculate the pixel gradient intensity matrix of the filtered image, traverse each image pixel in the pixel gradient intensity matrix, compare the gradient intensity of the currently traversed target pixel with the gradient intensity of each adjacent pixel, and determine the edge pixel from the image pixels based on the comparison results;

[0224] A first edge image corresponding to the reference scene image is obtained according to the edge pixel points.

[0225] Furthermore, the edge detection module 1401 is specifically configured to:

[0226] If the gradient intensity of the target pixel is greater than the gradient intensity of each adjacent pixel, the gradient intensity of the target pixel is compared with a preset first intensity threshold and a second intensity threshold; wherein the first intensity threshold is greater than the second intensity threshold;

[0227] If the gradient intensity of the target pixel is greater than or equal to the first intensity threshold, the target pixel is determined as an edge pixel;

[0228] If the gradient intensity of the target pixel point is less than the first intensity threshold and greater than or equal to the second intensity threshold, the neighboring pixel points corresponding to the target pixel point are determined; when the gradient intensity of the neighboring pixel points is greater than or equal to the first intensity threshold, the target pixel point is determined as an edge pixel point.

[0229] Furthermore, the transformation module 1402 is specifically configured to:

[0230] performing a rotation process on the first edge image to obtain a plurality of second edge images corresponding to the first edge image;

[0231] Performing cropping on the first edge image to obtain a plurality of second edge images corresponding to the first edge image;

[0232] Flipping the first edge image to obtain a plurality of second edge images corresponding to the first edge image;

[0233] Gaussian filtering is performed on the first edge image to obtain a plurality of second edge images corresponding to the first edge image.

[0234] Furthermore, the reference image is a document image, and the edge detection module 1401 is specifically used to:

[0235] Acquire a reference paper image, and crop at least one of a paper texture image, a paper shadow image, or a paper light image from the reference paper image;

[0236] resizing at least one of the paper texture image, the paper shadow image, or the paper light image;

[0237] At least one of the paper texture image, the paper shadow image, or the paper light image after size adjustment is used as a reference scene image.

[0238] Furthermore, the multiple target scene images include a target material image corresponding to the paper material image, a target shadow image corresponding to the paper shadow image, and a target light image corresponding to the paper light image. The synthesis module 1404 is specifically configured to:

[0239] Separating the document image to obtain a content image and an original background image corresponding to the document image;

[0240] The original background image is synthesized with the target material image, the target shadow image, and the target light image to obtain multiple effect background images;

[0241] The content image is synthesized with each effect background image to generate multiple target sample images.

[0242] Furthermore, the synthesis module 1404 is specifically configured to:

[0243] Obtaining a first image matrix of the original background image and a second image matrix of each target scene image; wherein the first image matrix and the second image matrix have the same size;

[0244] Multiplying the first image matrix and corresponding matrix elements in each second image matrix to obtain a plurality of target sample image matrices;

[0245] Generate corresponding effect background images according to multiple target sample image matrices.

[0246] The image generation device 1400 and the image generation method provided in the embodiment of the present application are based on the same inventive concept. Therefore, based on the above-mentioned image generation device 1400, only a small number of reference scene images and benchmark images need to be collected to generate a large number of target sample images. Compared with the method of collecting a large number of benchmark images, under the premise of generating the same number of target sample images, the image generation method provided in the embodiment of the present application can effectively reduce the number of image collections, reduce the generation cost of target sample images, and improve the generation efficiency of target sample images.

[0247] The electronic device for executing the above-mentioned image generation method provided in the embodiment of the present application may be a terminal, referring to Figure 15 , Figure 15 This is a partial structural block diagram of a terminal provided in an embodiment of the present application. The terminal includes: a radio frequency (RF) circuit 1510, a memory 1520, an input unit 1530, a display unit 1540, a sensor 1550, an audio circuit 1560, a wireless fidelity (WiFi) module 1570, a processor 1580, and a power supply 1590. Those skilled in the art will understand that Figure 15 The terminal structure shown in the figure does not constitute a limitation to the terminal, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0248] The RF circuit 1510 may be used for receiving and sending signals during information transmission or calls. In particular, after receiving downlink information from the base station, it is sent to the processor 1580 for processing. In addition, the designed uplink data is sent to the base station.

[0249] The memory 1520 may be used to store software programs and modules. The processor 1580 executes various functional applications and data processing of the terminal by running the software programs and modules stored in the memory 1520 .

[0250] The input unit 1530 may be configured to receive input digital or character information and generate key signal input related to the terminal's settings and function control. Specifically, the input unit 1530 may include a touch panel 1531 and other input devices 1532 .

[0251] The display unit 1540 may be configured to display input information or provided information and various menus of the terminal. The display unit 1540 may include a display panel 1541 .

[0252] The audio circuit 1560 , the speaker 1561 , and the microphone 1562 may provide an audio interface.

[0253] In this embodiment, the processor 1580 included in the terminal can execute the image generating method of the previous embodiment.

[0254] The electronic device for executing the above-mentioned image generation method provided in the embodiment of the present application may also be a server, referring to Figure 16 , Figure 16 This is a partial structural block diagram of a server provided in an embodiment of the present application. The server 1600 may vary greatly due to different configurations or performances, and may include one or more central processing units (CPUs) 1622 (for example, one or more processors) and a memory 1632, and one or more storage media 1630 (for example, one or more mass storage devices) storing application programs 1642 or data 1644. Among them, the memory 1632 and the storage medium 1630 may be temporary storage or permanent storage. The program stored in the storage medium 1630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server 1600. Furthermore, the central processing unit 1622 may be configured to communicate with the storage medium 1630 to execute a series of instruction operations in the storage medium 1630 on the server 1600.

[0255] The server 1600 may also include one or more power supplies 1626, one or more wired or wireless network interfaces 1650, one or more input and output interfaces 1658, and / or one or more operating systems 1641, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0256] The processor in the server 1600 may be configured to execute the image generating method.

[0257] An embodiment of the present application further provides a computer-readable storage medium, which is used to store program code, and the program code is used to execute the image generation method of each of the aforementioned embodiments.

[0258] The present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to implement the above-mentioned image generation method.

[0259] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0260] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0261] It should be understood that in the description of the embodiments of the present application, multiple (or multiple items) means more than two, greater than, less than, exceed, etc. are understood to exclude the number itself, and above, below, within, etc. are understood to include the number itself.

[0262] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0263] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0264] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0265] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0266] It should also be understood that the various implementation methods provided in the embodiments of the present application can be combined arbitrarily to achieve different technical effects.

[0267] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the above implementation mode. Technical personnel familiar with the art can also make various equivalent modifications or substitutions under the shared conditions that do not violate the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. An image generation method, characterized in that: include: Acquire a reference scene image, perform edge detection on the reference scene image, and obtain a first edge image corresponding to the reference scene image, wherein the reference scene image is determined according to the type of the target sample image to be generated; performing image transformation processing on the first edge image to obtain a plurality of second edge images corresponding to the first edge image; inputting the second edge image into a preset scene image generation model, converting the second edge image into a scene image based on the scene image generation model, and obtaining a target scene image corresponding to the second edge image, wherein the image type of the target scene image is determined by the reference scene image, and the target scene image and the reference scene image have different display styles; A reference image is acquired, and the reference image is synthesized with each of the target scene images to generate a plurality of target sample images. The target sample images have the same distribution as the reference image but different display styles.

2. The image generation method according to claim 1, wherein: The scene image generation model includes a plurality of sequentially connected convolutional layers and a plurality of sequentially connected deconvolutional layers, and adjusting the image type of the second edge image based on the scene image generation model to obtain a target scene image corresponding to the second edge image includes: Performing convolution processing on the second edge image based on the multiple convolution layers to obtain a target convolution image; Deconvolution processing is performed on the target convolution image based on the multiple deconvolution layers to obtain a target scene image corresponding to the second edge image.

3. The image generation method according to claim 2, wherein: The number of the convolution layers is the same as the number of the deconvolution layers, and the deconvolution processing is performed on the target convolution image based on the multiple deconvolution layers to obtain the target scene image corresponding to the second edge image, including: Acquire a first transition image to be input to a target deconvolution layer; wherein the target deconvolution layer is a deconvolution layer currently to be deconvolved among the multiple deconvolution layers; Determining a target convolution layer corresponding to the target deconvolution layer from the plurality of convolution layers, and obtaining a second transition image outputted after the target convolution layer performs convolution processing; Splicing the first transition image and the second transition image to obtain a spliced ​​image; Deconvolution processing is performed on the spliced ​​image based on the target deconvolution layer until a target scene image corresponding to the second edge image is obtained.

4. The image generation method according to any one of claims 1 to 3, characterized in that: The scene image generation model is trained according to the following steps: Acquire a training edge image set and a label scene image set; wherein the training edge image set includes a plurality of training edge images, the label scene image set includes a plurality of label scene images, and the image type of the training edge image set is different from the image type of the label scene image set; Adjusting the image type of a target training edge image based on the scene image generation model to obtain a training scene image corresponding to the target training edge image; wherein the target training edge image is one of the plurality of training edge images; Inputting the training scene image into a first judgment model to obtain a first judgment result, inputting the label scene image into the first judgment model to obtain a second judgment result, and calculating a first loss value corresponding to the scene image generation model based on the first judgment result and the second judgment result; Adjust parameters of the scene image generation model according to the first loss value.

5. The image generation method according to claim 4, characterized in that The adjusting the parameters of the scene image generation model according to the first loss value includes: adjusting the image type of the training scene image based on the edge image generation model to obtain a restored edge image corresponding to the training scene image; Calculating a norm between any training edge image other than the target training edge image in the training edge image set and the restored edge image, and calculating a restoration loss value according to the norm; Calculating a second loss value corresponding to the edge image generation model based on the second judgment model; A target loss value is obtained according to the sum of the first loss value, the second loss value and the restoration loss value, and parameters of the scene image generation model are adjusted according to the target loss value.

6. The image generation method according to claim 1, wherein: The performing edge detection on the reference scene image to obtain a first edge image corresponding to the reference scene image includes: Performing grayscale processing on the reference scene image to obtain a grayscale image corresponding to the reference scene image; Performing Gaussian filtering on the grayscale image to obtain a filtered image; Calculating a pixel gradient intensity matrix of the filtered image, traversing each image pixel in the pixel gradient intensity matrix, comparing the gradient intensity of a currently traversed target pixel with the gradient intensity of each adjacent pixel, and determining edge pixels from the image pixels based on the comparison results; A first edge image corresponding to the reference scene image is obtained according to the edge pixel points.

7. The image generation method according to claim 6, wherein: Determining edge pixels from the image pixels according to the comparison result includes: If the gradient intensity of the target pixel is greater than the gradient intensity of each of the adjacent pixels, comparing the gradient intensity of the target pixel with a preset first intensity threshold and a second intensity threshold; wherein the first intensity threshold is greater than the second intensity threshold; If the gradient intensity of the target pixel point is greater than or equal to the first intensity threshold, the target pixel point is determined as an edge pixel point; If the gradient intensity of the target pixel point is less than the first intensity threshold and greater than or equal to the second intensity threshold, the neighborhood pixel point corresponding to the target pixel point is determined; when the gradient intensity of the neighborhood pixel point is greater than or equal to the first intensity threshold, the target pixel point is determined as an edge pixel point.

8. The image generation method according to claim 1, wherein: The performing image transformation processing on the first edge image to obtain a plurality of second edge images corresponding to the first edge image includes at least one of the following: performing rotation processing on the first edge image to obtain a plurality of second edge images corresponding to the first edge image; performing cropping processing on the first edge image to obtain a plurality of second edge images corresponding to the first edge image; performing a flipping process on the first edge image to obtain a plurality of second edge images corresponding to the first edge image; Gaussian filtering is performed on the first edge image to obtain a plurality of second edge images corresponding to the first edge image.

9. The image generation method according to claim 1, wherein: The reference image is a document image, and obtaining a reference scene image includes: Acquire a reference paper image, and crop at least one of a paper texture image, a paper shadow image, or a paper light image from the reference paper image; resizing at least one of the paper texture image, the paper shadow image, or the paper light image; At least one of the paper texture image, the paper shadow image, or the paper light image after size adjustment is used as a reference scene image.

10. The image generation method according to claim 9, wherein: The plurality of target scene images include a target material image corresponding to the paper material image, a target shadow image corresponding to the paper shadow image, and a target light image corresponding to the paper light image. The synthesizing the reference image with each of the target scene images to generate the plurality of target sample images includes: Separating the document image to obtain a content image and an original background image corresponding to the document image; Combining the original background image with the target material image, the target shadow image, and the target light image to obtain multiple effect background images; The content image is synthesized with each of the effect background images to generate a plurality of target sample images.

11. The image generation method according to claim 10, wherein: The synthesizing process of the original background image and each of the target scene images to obtain a plurality of effect background images includes: Obtaining a first image matrix of the original background image and a second image matrix of each of the target scene images; wherein the first image matrix and the second image matrix have the same size; multiplying the first image matrix and corresponding matrix elements in each of the second image matrices to obtain a plurality of target sample image matrices; Generate corresponding effect background images according to the plurality of target sample image matrices.

12. An image generating device, characterized in that: include: An edge detection module is configured to acquire a reference scene image, perform edge detection on the reference scene image, and obtain a first edge image corresponding to the reference scene image, wherein the reference scene image is determined according to the type of the target sample image to be generated; a transformation module, configured to perform image transformation processing on the first edge image to obtain a plurality of second edge images corresponding to the first edge image; a model processing module, configured to input the second edge image into a preset scene image generation model, convert the second edge image into a scene image based on the scene image generation model, and obtain a target scene image corresponding to the second edge image, wherein the image type of the target scene image is determined by the reference scene image, and the target scene image and the reference scene image have different display styles; The synthesis module is used to obtain a reference image, synthesize the reference image with each of the target scene images, and generate a plurality of target sample images. The target sample images have the same distribution as the reference image but different display styles.

13. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the image generating method according to any one of claims 1 to 11 is implemented.

14. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the image generation method according to any one of claims 1 to 11 is implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the image generating method according to any one of claims 1 to 11 is implemented.

Images