Background image generation method, electronic device and computer storage medium

Abstract

The embodiments of the present application provide a background image generation method, an electronic device and a storage medium. The background image generation method comprises: acquiring a real image generated on the basis of a physical file; performing detection for multiple different types of target content on the real image to obtain a plurality of corresponding target content image areas and a plurality of corresponding target content masks; fusing the plurality of target content masks to obtain a fusion mask; and erasing from the real image an image area corresponding to the fusion mask, and, on the basis of the erasure result, obtaining a background image corresponding to the real image.

Description

Background image generation method, electronic device and computer storage medium

[0001] This application claims priority to the Chinese invention patent application with application date of October 18, 2023, application number "202311355147.8", and patent name "Background image generation method, electronic device and computer storage medium", all contents of which are hereby incorporated by reference. Technical Field

[0002] The embodiments of the present application relate to the field of computer technology, and in particular to a background image generation method, an electronic device, and a computer storage medium. Background Art

[0003] With the development of computer technology, people often need to electronically scan documents in their daily lives and work, such as electronic books, invoice reimbursement, work document scanning and printing, document scanning, and other application scenarios.

[0004] Because professional scanners are expensive, scanning software came into being. Through scanning software, users can realize convenient and low-cost document scanning anytime and anywhere. With the widespread use of scanning software, scanning scenarios are becoming more and more diverse, which requires scanning software to be able to iterate and update continuously to adapt to the needs of new scenarios. However, the iterative update of scanning software requires a large number of training samples in various scanning scenarios. To this end, in one existing method, professional rendering software (such as unity3D, etc.) is used to build a virtual lighting environment, which can include different light sources, different obstructions, different texture backgrounds, etc. By changing the position, strength, color temperature, etc. of the light source and then matching different obstructions and paper textures to simulate the scene of the user scanning the document, the background of the generated scanned image is made more realistic, thereby making the scanned image more realistic. However, in this method, firstly, the threshold for using this type of professional rendering software is high, and it takes a long time to debug and learn to achieve a certain simulation effect, which is time-consuming and results in a high overall implementation cost; secondly, the image background simulated by this type of professional rendering software has a relatively large deviation in data distribution from the actual sample distribution, which makes the background effect of the generated scanned image poor. As a result, although the machine learning model that implements software scanning works well on simulated training samples, it works poorly on real data.

[0005] Summary of the Invention

[0006] In view of this, an embodiment of the present application provides a background image generation solution to at least partially solve the above-mentioned problem.

[0007] According to a first aspect of an embodiment of the present application, a background image generation method is provided, including: obtaining a real image generated based on a physical file; performing multiple different types of target content detection on the real image to obtain corresponding multiple target content image areas and corresponding multiple target content masks; fusing the multiple target content masks to obtain a fused mask; erasing the image area corresponding to the fused mask from the real image, and obtaining a background image corresponding to the real image based on the erasure result.

[0008] According to the second aspect of an embodiment of the present application, an electronic device is provided, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method described in the first aspect.

[0009] According to a third aspect of the embodiments of the present application, a computer storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described in the first aspect is implemented.

[0010] According to another aspect of the embodiments of the present application, a computer program product is provided, including computer instructions, which instruct a computing device to perform operations corresponding to the method described in the first aspect.

[0011] According to the solution provided by the embodiments of the present application, it is possible to perform target content detection and mask processing based on a real image containing file information, generate a fusion mask based on multiple target content masks, and erase image regions based on the fusion mask to obtain a background image corresponding to the real image. Thus, on the one hand, there is no need for image processing personnel to perform complex parameter learning and debugging; the corresponding background image can be generated based on the real image, reducing the cost of background image generation. On the other hand, because the background image is generated based on the real image, its data distribution is consistent with the real sample distribution, which not only improves the background effect of the generated scanned image, but also, when the scanned image is subsequently synthesized based on the background image as a training sample, it can also improve the training effect of the machine learning model that implements software scanning. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0013] FIG1 is a schematic diagram of an exemplary system applicable to the embodiment of the present application.

[0014] FIG2 is a flowchart of the steps of a background image generating method according to an embodiment of the present application.

[0015] FIG3 is a flowchart of an optional sub-step of step S104 of the present application.

[0016] FIG4 is a flowchart of an optional sub-step of step S106 of the present application.

[0017] FIG5 is a flowchart of an optional sub-step of step S108 of the present application.

[0018] FIG6 is a schematic diagram of an example scenario in an embodiment of the present application.

[0019] FIG7A is a schematic diagram of an example scenario of synthesizing a sample image of a scanned document in an embodiment of the present application.

[0020] FIG7B is a schematic diagram of another example scenario of synthesizing a sample image of a scanned document in an embodiment of the present application.

[0021] FIG8 is a schematic structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field should fall within the scope of protection of the embodiments of the present application.

[0023] The specific implementation of the embodiment of the present application is further explained below in conjunction with the accompanying drawings of the embodiment of the present application.

[0024] Figure 1 illustrates an exemplary system applicable to the embodiments of the present application. As shown in Figure 1 , the system 100 may include a cloud service 102, a communication network 104, and / or one or more user devices 106, with Figure 1 illustrating multiple user devices. It should be noted that the embodiments of the present application may be implemented collaboratively by the cloud service 102 and the user devices 106, or independently by the user devices 106.

[0025] The cloud server 102 can be any suitable device for storing information, data, programs, and / or any other suitable type of content, including but not limited to a distributed storage system, a server cluster, a computing cloud server cluster, and the like. In some embodiments, the cloud server 102 can perform any suitable function. For example, when the cloud server 102 and the user device 106 collaborate to implement the solutions of the embodiments of the present application, in some embodiments, the cloud server 102 can receive a real image generated based on a physical file and sent by the user device 106, and generate a background image corresponding to the real image. As an alternative example, in some embodiments, the cloud server 102 can first perform multiple different types of target content detection on the real image to obtain multiple corresponding target content image regions and multiple corresponding target content masks; then, the cloud server 102 can fuse the multiple target content masks to obtain a fused mask; then, the cloud server 102 can erase the image region corresponding to the fused mask from the real image, and obtain a background image corresponding to the real image based on the erasure results. As another example, in some alternative embodiments, the cloud server 102 can send the generated background image to the user device 106. In some optional embodiments, the cloud service end 102 can synthesize a scanned document sample image based on the background image (for example, synthesizing a scanned document sample image based on the background image and a preset file content image; or synthesizing a scanned document sample image based on the background image and a preset file content text), and train a machine learning model for generating scanned files based on the scanned document sample image.

[0026] In some embodiments, the communication network 104 can be any suitable combination of one or more wired and / or wireless networks. For example, the communication network 104 can include any one or more of the following: the Internet, an intranet, a wide area network (WAN), a local area network (LAN), a wireless network, a digital subscriber line (DSL) network, a frame relay network, an asynchronous transfer mode (ATM) network, a virtual private network (VPN), and / or any other suitable communication network. The user device 106 can be connected to the communication network 104 via one or more communication links (e.g., communication link 112), and the communication network 104 can be linked to the cloud service end 102 via one or more communication links (e.g., communication link 114). The communication link can be any communication link suitable for transmitting data between the user device 106 and the cloud service end 102, such as a network link, a dial-up link, a wireless link, a hard-wired link, any other suitable communication link, or any suitable combination of such links.

[0027] User device 106 may include any one or more user devices suitable for presenting images, interacting with users, and the like. When the user device 106 independently implements the solution of the embodiments of the present application, the user device 106 may be used to generate a background image corresponding to the real image based on a real image generated based on a physical document. As an optional example, in some embodiments, the user device 106 may first perform multiple different types of target content detection on the real image to obtain multiple corresponding target content image regions and multiple corresponding target content masks; then, the multiple target content masks may be fused to obtain a fused mask; then, the image region corresponding to the fused mask may be erased from the real image, and the background image corresponding to the real image may be obtained based on the erasure result. In some optional embodiments, the user device 106 may synthesize a sample scanned document image based on the background image (e.g., synthesizing a sample scanned document image based on the background image and a preset document content image; or synthesizing a sample scanned document image based on the background image and preset document content text). Furthermore, the user device 106 may send the sample scanned document image to the cloud server 102 to train the machine learning model used to generate scanned documents in the cloud server 102. In some embodiments, the user device 106 may include any suitable type of device. For example, in some embodiments, user device 106 may include a mobile device, a tablet computer, a laptop computer, a desktop computer, and / or any other suitable type of user device.

[0028] Based on the above system, an embodiment of the present application provides a background image generation solution, which is described below through multiple embodiments.

[0029] FIG2 is a flowchart of the steps of a background image generation method according to an embodiment of the present application. According to the first aspect of the present application, a background image generation method is provided. As shown in FIG2 , the method includes steps S102, S104, S106 and S108, specifically:

[0030] S102: Acquire a real image generated based on the physical file.

[0031] In this application, a physical document may refer to a physical document for scanning. A physical document may be of any type, for example, including but not limited to books, invoices, contracts, business licenses, work documents, credentials, ID cards, driver's licenses, etc.

[0032] The real image in this application has the characteristics of a real sample, and its data distribution is consistent with that of the real sample, which makes it possible to generate a background image based on the real image, and the data distribution of the obtained background image is consistent with that of the real sample.

[0033] Optionally, the real image generated based on the physical document in step S102 of the present application can be an image obtained by pre-capturing the physical document using a camera. For example, taking an invoice as an example, the real image generated based on the invoice can be obtained as follows: the user places the invoice to be scanned on a support (such as a desktop) or holds it in their hand, and then uses the camera on the mobile terminal to capture the invoice from the direction of the invoice, thereby obtaining the corresponding real image. It should be understood that this is merely an example and does not constitute any limitation of the present application.

[0034] It should be understood that the real image generated based on the physical file in step S102 can be obtained by photographing the real image when needed and then directly obtaining it, or by photographing the real image in advance and then storing the real image in a predetermined storage space (including but not limited to storage media such as disks, hard disks, memories, or databases, etc.), and then directly obtaining it from the storage space when needed. This application does not impose any restrictions on this.

[0035] Optionally, the real image may also include images with defects after being processed by the model (for example, these images may be images with defects that are accumulated after being processed by the machine learning model used to implement software scanning by users (including online users and / or offline users) in the history of using the scanning software). The required real images can be manually selected from the images with defects after being processed by the model and stored in the storage space, or the required real images can be retrieved from the images with defects after being processed by the model according to the rules by the algorithm and stored in the storage space. For example, defects may be objects in the image, including but not limited to shadows, creases, transparent characters, fingers, clips, etc. These objects can be used as part of the background of the real image and belong to the characteristics of the real sample, and their data distribution is consistent with that of the real sample. By generating a background image through such a real image, and subsequently using the background image to synthesize a sample image of a scanned document to train the machine learning model used to generate the scanned document, the capability boundary of the machine learning model can be expanded.

[0036] S104: Performing multiple different types of target content detection on the real image to obtain corresponding multiple target content image regions and corresponding multiple target content masks.

[0037] The real image generated based on the physical file obtained in step S102 includes multiple different types of target content, which are usually information content with practical significance. Therefore, in step S104 of the present application, multiple different types of target content detection can be performed on the real image to obtain corresponding multiple target content image areas and corresponding multiple target content masks, so that the subsequent steps can use multiple target content masks to perform data processing to obtain a background image that better meets the needs.

[0038] In this application, the target content can be determined based on the corresponding real image. In some optional embodiments, the target content in this application includes at least one of the following types: text type content, chart type content, color block pattern type content, and seal type content. Optionally, referring to the flowchart shown in Figure 3, step S104 includes sub-steps S1041, S1042, and S1043, specifically:

[0039] S1041: Perform at least two of the following target content detections on the real image: detection of text, detection of charts, detection of color blocks and patterns, and detection of seals.

[0040] In this application, any feasible target detection algorithm can be used to detect text, graphics, color block patterns and seals.

[0041] In this application, by detecting at least two different types of target content in real images, such as text, graphics, color block patterns, and seals, the background image generation method in this application can better adapt to the needs of document scanning in real life.

[0042] Furthermore, this application does not specifically limit the use of text, graphics, color patterns, or seals. For example, the following description provides a schematic understanding: text may include, but is not limited to, Chinese characters, Arabic numerals, and English text; graphics may include, but is not limited to, bar charts, pie charts, line charts, and tables; color patterns may include any regular or irregular color pattern; and seals may include, but are not limited to, regular shapes such as circular and rectangular seals, or other shapes.

[0043] It should also be noted that while this application detects at least two different types of target content in real images, including text, graphics, color block patterns, and seals, this does not necessarily mean that multiple different types of target content can be detected from real images. This depends on the actual conditions of the target image, and this application does not impose any restrictions on this. For example, if the real image contains only text-type content and no graphics, color block patterns, seals, or other types of content, although multiple different types of target content will be detected, the resulting detection results may only contain results corresponding to text-type content. Of course, this is merely an example and does not constitute any limitation on this application.

[0044] In addition, it should be noted that, in this application, “multiple”, “plurality” and other numbers related to “multiple” mean two or more.

[0045] S1042: According to the detection result, a plurality of image regions corresponding to the detected plurality of types of target contents are obtained.

[0046] Specifically, based on the detection results of the target content detection obtained in sub-step S1041, multiple image areas corresponding to the detected multiple types of target content can be determined from the real image, and these image areas include at least part of the content in the real image except the background.

[0047] Since sub-step S1041 detects at least two different types of target content in a real image, including text, graphics, color patterns, and seals, after the corresponding content is detected, it can be appropriately processed to obtain the corresponding image area. For example, in some optional embodiments, sub-step S1042 includes at least one of the following sub-steps S1042A, S1042B, S1042C, and S1042D, specifically:

[0048] S1042A: If text-type content is detected according to the detection result, text recognition is performed on the text-type content to determine the image areas corresponding to the respective characters according to the text recognition result.

[0049] Optionally, any suitable text recognition algorithm may be used to recognize text content, and the image regions corresponding to the respective characters may be determined based on the text recognition results. For example, text recognition algorithms include but are not limited to OCR (Optical Character Recognition) algorithms.

[0050] S1042B: If it is determined based on the detection result that chart-type content is detected, image segmentation is performed on the chart-type content to obtain an image region corresponding to the chart-type content.

[0051] Optionally, any suitable image segmentation algorithm may be used to segment the chart type content to obtain the image region corresponding to the chart type content. For example, image segmentation algorithms include but are not limited to semantic segmentation algorithms, instance segmentation algorithms, and the like.

[0052] S1042C: If it is determined based on the detection result that a color block pattern type content is detected, image segmentation is performed on the color block pattern type content to obtain an image area corresponding to the color block pattern content.

[0053] Optionally, any suitable image segmentation algorithm may be used to segment the color block pattern type content to obtain the image region corresponding to the color block pattern type content. For example, image segmentation algorithms include but are not limited to semantic segmentation algorithms, instance segmentation algorithms, and the like.

[0054] S1042D: If it is determined that seal type content is detected according to the detection result, text recognition and frame recognition are performed on the seal type content to determine the image area corresponding to the seal according to the recognition result.

[0055] Optionally, any suitable text recognition algorithm and any suitable frame recognition algorithm may be used to recognize the seal type content to obtain the image area corresponding to the seal type content. For example, the text recognition algorithm includes but is not limited to an OCR (Optical Character Recognition) algorithm.

[0056] Based on this, in this application, through the optional implementation of sub-steps S1042A to S1042D, different methods are used to adaptively determine the image area corresponding to the target content for different target contents determined according to the detection results, so that the determination result of the image area can be obtained accurately and effectively.

[0057] S1043: Masking the multiple image regions to obtain corresponding multiple target content masks.

[0058] After obtaining multiple target regions in sub-step S1042, masking can be performed on the multiple image regions to obtain corresponding multiple target content masks. The specific method for obtaining the masks can be implemented by those skilled in the art using any appropriate method, including but not limited to obtaining the masks using a machine learning model for masking target objects in an image. Furthermore, optionally, when outputting the masks corresponding to the image regions, the machine learning model can also simultaneously output the confidence levels corresponding to the respective masks.

[0059] Based on this, in this application, through the optional implementation of the above-mentioned sub-steps S1041 to S1043, it is possible to accurately and effectively implement various types of target content detection on real images to obtain corresponding multiple target content image areas and corresponding multiple target content masks, thereby facilitating subsequent data processing to obtain a background image that better meets the needs.

[0060] S106: Fusing multiple target content masks to obtain a fused mask.

[0061] After obtaining multiple target content masks, the multiple target content masks are fused (such as by merging, etc.) to obtain a fused mask that better meets the data processing requirements. By fusing the masks, on the one hand, the effect of generating the background image in the subsequent steps can be better improved, and on the other hand, it is also convenient to reduce the amount of calculation when generating the background image in the subsequent steps.

[0062] The specific implementation of step S106 is not limited in this application, and can be satisfied as needed. In some optional embodiments, referring to the flowchart shown in FIG4 , step S106 includes sub-steps S1061 and S1062, specifically:

[0063] S1061: Filter the multiple target content masks according to the confidence level of each target content mask in the multiple target content masks.

[0064] For example, as mentioned above, when the machine learning model outputs the mask corresponding to the image area, it may also output the confidence corresponding to the mask. In this step, the target content masks with lower confidence can be filtered out according to the confidence of each target content mask among multiple target content masks to ensure the accuracy of the target content mask.

[0065] The specific implementation of sub-step S1061 is not limited in this application. In some optional embodiments, filtering can be performed by setting a confidence threshold. For example, step S1061 can specifically determine whether the confidence level of each target content mask among multiple target content masks is greater than the confidence threshold. If so, the target content mask is retained; if not, the target content mask is not retained, thereby achieving the purpose of filtering multiple target content masks. The confidence threshold can be set as needed, for example, it can be set to 70%, 80%, etc.

[0066] Alternatively, sub-step S1061 may also set other conditions for filtering as needed, and this application does not impose any specific restrictions here.

[0067] S1062: Fusing the filtered target content masks to obtain a fused mask.

[0068] Exemplarily, the target content masks may be fused using methods such as mask merging.

[0069] Based on this, in this application, through the optional implementation of the above-mentioned sub-steps S1061 to S1062, multiple target content masks can be effectively fused to obtain a fused mask. On the one hand, fusing multiple target content masks into a whole can be more convenient for subsequent processing, such as erasing processing, to improve processing speed; on the other hand, it can also better avoid erasing excessive background areas in the real image when subsequently erasing the image area corresponding to the fused mask, thereby better improving the effect of generating the background image in the subsequent steps and reducing the amount of calculation when generating the background image subsequently.

[0070] S108: Erasing the image area corresponding to the fusion mask from the real image, and obtaining a background image corresponding to the real image according to the erasing result.

[0071] After obtaining the fusion mask in step S106, the image area corresponding to the fusion mask can be erased from the real image so that only the background remains in the erased real image, thereby obtaining the background image corresponding to the real image based on the erasing result.

[0072] Based on this, the background image generation method of the above steps S102 to S108 in this application can detect target content and perform mask processing based on a real image containing file information, generate a fusion mask based on multiple target content masks, and erase the image area based on the fusion mask, thereby obtaining a background image corresponding to the real image. Therefore, on the one hand, there is no need for image processing personnel to perform complex parameter learning and debugging, and the corresponding background image can be generated based on the real image, which reduces the cost of background image generation; on the other hand, the background image is generated based on the real image, so the data distribution is consistent with the real sample distribution, which not only makes the background effect of the generated scanned image better; but also, when the scanned image is synthesized based on the background image as a training sample, it can also make the training effect of the machine learning model for software scanning better.

[0073] In this application, the specific implementation of step S108 is not limited, as long as the corresponding requirements can be met. For example, when implementing step S108 and its optional implementations, the content of the image area corresponding to the fusion mask can be erased from the real image to obtain an erased result, and the erased area can be filled with the content of the adjacent area to obtain a background image corresponding to the real image based on the erased result. For example, erasure can be implemented using, including but not limited to, traditional image filling technology, image_inpainting technology based on artificial intelligence (AI), or a comparison method, and this application does not impose any restrictions on this.

[0074] In some optional embodiments, after obtaining the fusion mask in step S106, the background image generation method further includes: performing morphological processing on the fusion mask according to morphological processing parameters; and "erasing the image area corresponding to the fusion mask from the real image" in step S108 includes: erasing the image area corresponding to the fusion mask that has been subjected to morphological processing from the real image.

[0075] Optionally, by performing morphological processing on the fusion mask and erasing the morphologically processed fusion mask from the real image, a better effect of obtaining a background image corresponding to the real image according to such an erasing result is achieved.

[0076] Optionally, the morphological processing performed on the fusion mask may include at least one of an erosion process and a dilation process. Different morphological parameters may be used to perform different types of morphological processing according to the needs of different real images, and this application does not specifically limit this.

[0077] In some optional embodiments, referring to the flowchart shown in FIG5 , step S108 includes sub-steps S1081 and S1082. Specifically:

[0078] S1081: Erasing the image area corresponding to the fusion mask from the real image to obtain an erased image;

[0079] S1082: Calculate the information entropy of the erased image. If the information entropy meets a preset information entropy threshold, use the erased image as the background image corresponding to the real image.

[0080] Specifically, in sub-step S1082, information entropy can be used to represent the richness of the image content. A higher information entropy value indicates a higher amount of information and richer content, while a lower entropy value indicates less content. Therefore, by calculating the information entropy of the erased image, it is possible to determine whether the image area corresponding to the fusion mask in the real image has been completely erased.

[0081] In this application, a preset information entropy threshold can be used to determine whether the information entropy of the erased image meets the requirements. If the information entropy of the erased image meets the preset information entropy threshold, it can be considered that the image area corresponding to the fusion mask in the real image has been erased cleanly and can be used as the background image corresponding to the real image.

[0082] Optionally, if the information entropy of the erased image is lower than a preset information entropy threshold, the information entropy of the erased image satisfies the preset information entropy threshold. Conversely, if the information entropy of the erased image is higher than the preset information entropy threshold, the information entropy of the erased image does not satisfy the preset information entropy threshold. If the information entropy of the erased image satisfies the preset information entropy threshold, the erased image can be used as the background image corresponding to the real image.

[0083] Based on this, in this application, through the optional implementation of sub-steps S1081 to S1082, it can be ensured that the image area corresponding to the fusion mask in the real image is erased cleanly, and the erased image obtained after erasing is used as the background image of the real image, which can make the background effect of the generated scanned image better, and when the scanned image is subsequently synthesized based on the background image as a training sample, it can also make the training effect of the machine learning model that realizes software scanning better.

[0084] In this application, the preset information entropy threshold can be set as needed. In some optional embodiments, the preset information entropy threshold is determined by determining a preset number of historical images similar to the currently processed real image and determining the preset information entropy threshold based on the minimum information entropy value among the background images corresponding to the historical images. This approach allows for a more reasonable determination of the preset information entropy threshold, resulting in a more reasonable result in determining whether the information entropy of the erased image meets the preset information entropy threshold.

[0085] Specifically, a historical image, or a historical real image, differs from the real image currently being processed only in time. Optionally, the similarity between a historical image and a real image currently being processed in this application may refer to a similarity in at least one of the historical image's type and content layout. For example, if the historical image is a real image generated based on an invoice, and the real image currently being processed is also a real image generated based on an invoice, then the two are considered similar. Another example is if the historical image is a real image generated based on a business license, and the real image currently being processed is also a real image generated based on a business license, then the two are considered similar. Another example is if the historical image is a real image generated based on a page of a book with text and graphic content, and the real image currently being processed is a real image generated based on a page of a contract with text and graphic content, then the two are also considered similar. It should be understood that these examples are merely illustrative and do not constitute any limitation to this application. Furthermore, the number of presets can be selected as needed to meet the needs.

[0086] For ease of understanding, let's take an example: assuming the preset number is 100, we can determine 100 historical images similar to the real image currently being processed, erase all content except the background from each of the 100 historical images, and calculate the corresponding information entropy values ​​to obtain 100 information entropy values. The minimum information entropy value among the 100 information entropy values ​​is determined and used as the preset information entropy threshold. It should be understood that this example is merely for ease of understanding and does not constitute any limitation to this application.

[0087] In some optional embodiments, if the information entropy of the erased image does not meet the preset information entropy threshold, the morphological processing parameters are adjusted to perform morphological processing on the fusion mask, and the fusion mask after the morphological processing is completed is used to return to step S108 and re-execute the step of "erasing the image area corresponding to the fusion mask from the real image".

[0088] Specifically, when the information entropy of the erased image does not meet the preset information entropy threshold, the morphological processing parameters are adjusted to perform morphological processing on the fusion mask, and the fusion mask after the morphological processing is completed is used to return to the real image, erase the image area corresponding to the fusion mask and re-execute, so as to facilitate the adjustment of the information entropy of the erased image that does not meet the preset information entropy threshold to meet the preset information entropy threshold, so as to ensure that the image area corresponding to the fusion mask in the real image is erased cleanly, and the erased image obtained by erasing is used as the background image of the real image, which can make the background effect of the generated scanned image better, and when the scanned image is subsequently synthesized based on the background image as a training sample, it can also make the training effect of the machine learning model for software scanning better.

[0089] FIG6 is a schematic diagram of a scene example in an embodiment of the present application. Referring to FIG6 , the background image generation method of the present application is generally understood. FIG6 shows a real image generated based on a physical file (as shown in the left interface of FIG6 ). The real image includes text type content, icon type content, color block pattern type content, and seal type content. After the background image generation method in the present application, the background image obtained by output (as shown in the right interface of FIG6 ) is erased from the above-mentioned text type content, icon type content, color block pattern type content, and seal type content. It should be understood that the example in FIG6 does not serve as any limitation to the present application.

[0090] In some optional embodiments, the background image generation method further includes: synthesizing a scanned document sample image based on the background image and a preset document content image. In this way, the present application can effectively synthesize a scanned document sample image for easy use.

[0091] Specifically, the preset file content image can be an existing image that can represent the file content. The preset file content image can also include an image representing the file content that is segmented from an image (the image mentioned here can include but is not limited to a real image). For example, the file content can include at least one of text content, graphic content, color block pattern content, seal content, etc. The preset file content image can be stored in a suitable storage space after being segmented so that it can be obtained when needed. Optionally, the preset file content image can be a solid color background or a transparent background, which can be more convenient to synthesize with the background image generated in step S108 to obtain a scanned file sample image. In an optional manner, the above-mentioned solid color background can be a white background, which is closer to the actual scanned file background and is also easier to synthesize.

[0092] Refer to the scene example shown in Figure 7A for understanding, which shows an exemplary process of synthesizing a scanned document sample image based on a background image and a preset document content image. It should be understood that the example shown in Figure 7A does not serve as any limitation to the embodiments of the present application.

[0093] Optionally, a scanned document sample image may be synthesized by overlaying or pasting a preset document content image onto a suitable area of ​​the background image.

[0094] Alternatively, in some other optional embodiments, the background image generation method further comprises: synthesizing a scanned document sample image based on the background image and a preset document content text. In this way, the present application can also effectively synthesize a scanned document sample image for easy use.

[0095] Specifically, the preset file content text can be text content generated as needed. It should be noted that the difference between the file content text and the above-mentioned file content image is that the file content image is an existing image, while the file content text is generated as needed. For example, if the user wants the middle area of ​​the scanned file sample image to include the word "invoice", the user can input the text content of the word "invoice" through the software, and adjust the appropriate text format (such as font, font size, etc.), and adjust such "invoice" text content to the middle area of ​​the background image manually or automatically, thereby realizing the synthesis of a scanned file sample image that meets the needs based on the background image and the preset file content text; it should be understood that this example does not serve as any limitation to this application.

[0096] Refer to the scene example shown in Figure 7B for understanding, which shows an exemplary process of synthesizing a scanned document sample image based on a background image and preset document content text. It should be understood that the example shown in Figure 7B does not serve as any limitation to the embodiments of the present application.

[0097] In some optional embodiments, the background image generation method further includes: training a machine learning model for generating scanned documents based on sample images of the scanned documents.

[0098] The scanned document sample images obtained by the technical solution of the present application are used as training samples. The background images of the training samples are generated based on real images, and therefore the data distribution is consistent with the real sample distribution. Therefore, the background effect of the training samples is better, and it can make the training effect of the machine learning model used to generate scanned documents better. In addition, since the technical solution in the present application can efficiently generate background images and scanned document sample images (i.e., training samples), it is also convenient to help the machine learning model to iterate quickly and specifically.

[0099] In summary, the solution provided in the embodiments of the present application can detect and mask target content based on a real image containing file information, generate a fusion mask based on multiple target content masks, and erase the image area based on the fusion mask, thereby obtaining a background image corresponding to the real image. Therefore, on the one hand, there is no need for image processing personnel to perform complex parameter learning and debugging, and the corresponding background image can be generated based on the real image, which reduces the cost of background image generation. On the other hand, the background image is generated based on the real image, so the data distribution is consistent with the real sample distribution, which not only makes the background effect of the generated scanned image better, but also, when the scanned image is synthesized based on the background image as a training sample, it can also make the training effect of the machine learning model that implements software scanning better.

[0100] It is understandable that the above description of the background image generation method is merely an exemplary description of the present application and does not constitute any limitation to the present application.

[0101] According to a second aspect of an embodiment of the present application, an electronic device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; the memory is configured to store at least one executable instruction, wherein the executable instruction causes the processor to perform an operation corresponding to the method described in the first aspect. Referring to FIG8 , a schematic diagram of the structure of an electronic device according to an embodiment of the present application is shown, and the specific embodiments of the present application do not limit the specific implementation of the electronic device.

[0102] As shown in FIG8 , the electronic device 800 may include: a processor 802, a communications interface 804, a memory 806, and a communication bus 808.

[0103] The processor 802 , the communication interface 804 , and the memory 806 communicate with each other via a communication bus 808 .

[0104] The communication interface 804 is used to communicate with other electronic devices or servers.

[0105] The processor 802 is configured to execute the program 810 , and specifically to execute the relevant steps in the above-mentioned background image generation method embodiment.

[0106] Specifically, the program 810 may include program codes, which include computer operation instructions.

[0107] Processor 802 may be a CPU, a Graphics Processing Unit (GPU), an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs, or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0108] The memory 806 is used to store the program 810. The memory 806 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0109] The program 810 may include multiple computer instructions. Specifically, the program 810 may enable the processor 802 to execute operations corresponding to the background image generation method described in any of the aforementioned method embodiments through the multiple computer instructions.

[0110] The specific implementation of each step in program 810 can refer to the corresponding description of the corresponding steps and units in the above-mentioned method embodiment, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working process of the above-mentioned devices and modules can refer to the corresponding process description in the above-mentioned method embodiment, and will not be repeated here.

[0111] According to a third aspect of the embodiments of the present application, the embodiments of the present application further provide a computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any of the aforementioned method embodiments. The computer storage medium includes, but is not limited to, a compact disc read-only memory (CD-ROM), random access memory (RAM), a floppy disk, a hard disk, or a magneto-optical disk.

[0112] An embodiment of the present application further provides a computer program product, including computer instructions, which instruct a computing device to execute operations corresponding to any background image generation method in the above-mentioned multiple method embodiments.

[0113] The electronic device 800 / computer storage medium / computer program product embodiment in the embodiment of the present application has been described in detail in the aforementioned background image generation method embodiment, so its related content and beneficial effects can be understood with reference to the above-mentioned method embodiment and will not be repeated here.

[0114] In addition, it should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to sample data used to train the model, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0115] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0116] The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk or magneto-optical disk), or as computer code that is originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded via a network and will be stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor or programmable or dedicated hardware (such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA)). It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., random access memory (RAM), read-only memory (ROM), flash memory, etc.) that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown here, the execution of the code converts the general-purpose computer into a dedicated computer for executing the method shown here.

[0117] Those skilled in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for specific applications, but such implementation should not be considered to be beyond the scope of the embodiments of this application.

[0118] The above implementation methods are only used to illustrate the embodiments of the present application, and are not intended to limit the embodiments of the present application. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of the present application, and the scope of patent protection of the embodiments of the present application should be defined by the claims.

Claims

1. A background image generation method, comprising: Get real images generated based on physical files; Performing a plurality of different types of target content detection on the real image to obtain a plurality of corresponding target content image regions and a plurality of corresponding target content masks; Fusing the multiple target content masks to obtain a fused mask; From the real image, the image area corresponding to the fusion mask is erased, and a background image corresponding to the real image is obtained according to the erasing result.

2. The method according to claim 1, wherein: The step of fusing the multiple target content masks to obtain a fused mask includes: filtering the multiple target content masks according to the confidence of each target content mask in the multiple target content masks; The filtered target content masks are fused to obtain a fused mask.

3. The method according to claim 2, wherein: After obtaining the fusion mask, the method further comprises: performing morphological processing on the fusion mask according to morphological processing parameters; Erasing the image region corresponding to the fusion mask from the real image includes: erasing the image region corresponding to the fusion mask that has been subjected to morphological processing from the real image.

4. The method according to any one of claims 1 to 3, wherein: Erasing the image area corresponding to the fusion mask from the real image, and obtaining a background image corresponding to the real image according to the erasing result, includes: Erasing the image area corresponding to the fusion mask from the real image to obtain an erased image; The information entropy of the erased image is calculated, and if the information entropy meets a preset information entropy threshold, the erased image is used as a background image corresponding to the real image.

5. The method according to claim 4, wherein: The method further comprises: If the information entropy does not meet the preset information entropy threshold, the morphological processing parameters are adjusted to perform morphological processing on the fusion mask, and the fusion mask after the morphological processing is completed is used to return to the step of erasing the image area corresponding to the fusion mask from the real image and re-execute it.

6. The method according to claim 4, wherein: The preset information entropy threshold is determined in the following way: A preset number of historical images similar to the real image currently being processed are determined, and the preset information entropy threshold is determined according to a minimum value among the information entropies of the background images corresponding to the historical images.

7. The method according to claim 1, wherein: The target content includes at least one of the following types: text type content, chart type content, color block pattern type content, and seal type content; The performing a plurality of different types of target content detection on the real image to obtain a plurality of corresponding target content image regions and a plurality of corresponding target content masks includes: Performing at least two of the following target content detections on the real image: detection on text, detection on charts, detection on color block patterns, and detection on seals; According to the detection results, a plurality of image regions corresponding to the detected plurality of types of target contents are obtained; The multiple image regions are masked to obtain corresponding multiple target content masks.

8. The method according to claim 7, wherein: The obtaining, according to the detection result, a plurality of image regions corresponding to the detected plurality of types of target contents comprises at least one of the following: If it is determined according to the detection result that text type content is detected, then text recognition is performed on the text type content to determine the image areas corresponding to the respective characters according to the text recognition result; If it is determined according to the detection result that a chart type content is detected, performing image segmentation on the chart type content to obtain an image area corresponding to the chart type content; If it is determined according to the detection result that a color block pattern type content is detected, image segmentation is performed on the color block pattern type content to obtain an image area corresponding to the color block pattern content; If it is determined according to the detection result that seal type content is detected, text recognition and frame recognition are performed on the seal type content to determine the image area corresponding to the seal according to the recognition result.

9. The method according to any one of claims 1 to 3, wherein: The method further comprises: Synthesize a scanned document sample image based on the background image and a preset document content image; or, A scanned document sample image is synthesized based on the background image and the preset document content text.

10. The method according to claim 9, wherein: The method further comprises: Based on the scanned document sample images, a machine learning model for generating scanned documents is trained.

11. An electronic device, comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method according to any one of claims 1 to 10.

12. A computer storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

13. A computer program product, comprising computer instructions, wherein the computer instructions instruct a computing device to execute operations corresponding to the method according to any one of claims 1 to 10.

Images