A method, apparatus, device, and storage medium for amplifying image data.

Abstract

This invention discloses a method, apparatus, device, and storage medium for image data augmentation. The method includes: acquiring a target background image and a target subject image to be fused; fusing the target subject image and the target background image to obtain a preliminary fused image; binarizing the preliminary fused image to obtain a subject mask image corresponding to the target subject image and a background mask image corresponding to the target background image; determining a subject enhancement image and a target background image based on the preliminary fused image, the subject mask image, and the background mask image, respectively; and generating a target augmented image based on the subject enhancement image and the background enhancement image. This enables large-scale data augmentation in data expansion scenarios, increasing data richness.

Description

Technical Field

[0001] This invention relates to the field of data processing, and more particularly to a method, apparatus, device, and storage medium for amplifying image data. Background Technology

[0002] Data plays an indispensable and crucial role in the development of artificial intelligence. The amount of data often affects the efficiency and capabilities of artificial intelligence. At present, artificial intelligence requires a large amount of data. Therefore, data augmentation is an urgent problem to be solved in data processing.

[0003] Existing data augmentation technologies often rely heavily on existing datasets. When existing datasets are scarce or lack richness, the resulting data augmentation is less rich, less flexible, or even impossible, making effective large-scale data augmentation impossible. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and storage medium for amplifying image data to achieve large-scale and rich amplified data.

[0005] According to one aspect of the present invention, an image data amplification method is provided, comprising:

[0006] Obtain the target background image and the target subject image to be fused, and fuse the target subject image with the target background image to obtain a preliminary fused image;

[0007] The preliminary fused image is binarized to obtain a subject mask image corresponding to the target subject image and a background mask image corresponding to the target background image;

[0008] The enhanced subject image and the target background image are determined based on the preliminary fused image, the subject mask image, and the background mask image, respectively.

[0009] A target augmented image is generated based on the subject augmented image and the background augmented image.

[0010] According to another aspect of the present invention, an image data amplification apparatus is provided, comprising:

[0011] The image fusion module is used to acquire the target background image and the target subject image to be fused, and to fuse the target subject image with the target background image to obtain a preliminary fused image;

[0012] The image processing module is used to perform binarization processing on the preliminary fused image to obtain a subject mask image corresponding to the target subject image and a background mask image corresponding to the target background image;

[0013] An image determination module is used to determine a subject enhancement image and a target background image based on the preliminary fused image, the subject mask image, and the background mask image, respectively.

[0014] The target augmented image generation module is used to generate a target augmented image based on the subject augmented image and the background augmented image.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] At least one processor; and

[0017] A memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the image data augmentation method according to any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the image data augmentation method according to any embodiment of the present invention.

[0020] The technical solution of this invention involves acquiring a target background image and a target subject image to be fused, fusing the target subject image with the target background image to obtain a preliminary fused image, and performing binarization processing on the preliminary fused image to obtain a subject mask image corresponding to the target subject image and a background mask image corresponding to the target background image. This binarization process enhances the contrast between the subject and the background. Furthermore, a subject enhancement image and a background enhancement image are determined based on the preliminary fused image, the subject mask image, and the background mask image, respectively. Finally, a target augmented image is generated based on the subject enhancement image and the background enhancement image. By processing and fusing the target background image and the target subject image multiple times, this invention solves the technical problem of poor flexibility in augmentation in existing technologies, enabling the generation of millions of target augmented images in scenarios with millions of data points, thus improving the richness of large-scale data augmentation.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of an image data amplification method provided in Embodiment 1 of the present invention;

[0024] Figure 2 This is a flowchart of another image data amplification method provided in Embodiment 2 of the present invention;

[0025] Figure 3 This is a structural diagram of an image data amplification device provided in Embodiment 3 of the present invention;

[0026] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the image data amplification method of this invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] Example 1

[0030] Figure 1This is a flowchart illustrating an image data amplification method according to Embodiment 1 of the present invention. This embodiment is applicable to situations where image data is amplified. The method can be executed by an image data amplification device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0031] S110. Obtain the target background image and the target subject image to be fused, and fuse the target subject image with the target background image to obtain a preliminary fused image.

[0032] The target background image can be the image information to be fused, serving as a background. The target subject image can be the image to be fused, serving as the subject. For example, the target subject image can be an image containing subjects such as text, pictures, people, objects, and buildings. The preliminary fused image can be the image obtained by fusing the target subject image and the target image information.

[0033] Specifically, the target background image and the target subject image are obtained. The pixel values ​​of the target background information and the target subject image are obtained respectively. The pixel values ​​of the target background information and the target subject image are then fused to generate a preliminary fused image.

[0034] For example, the pixel values ​​of each pixel in the target background image and the pixel values ​​of each pixel in the target subject image can be added, multiplied, weighted and then added, or weighted and then multiplied to generate a preliminary fused image; or, the target pixel in the target background image corresponding to the subject information can be determined based on the subject information in the target subject image, and the pixel value of the target pixel in the target background image can be replaced with the pixel value of the subject information to obtain a preliminary fused image.

[0035] Optionally, in another embodiment of the present invention, obtaining the target background image to be fused includes: obtaining sample background images from a pre-built background database, and using the obtained sample background images as the target background images to be fused, wherein the sample background images are obtained by extracting background information from sample image data.

[0036] The background database can be a pre-built database that stores multiple sample background images.

[0037] Optionally, multiple sample image data are acquired in advance, and background images are extracted from the multiple sample image data through image feature extraction as sample background images. These sample background images are then stored in a database as a background database.

[0038] Considering that the background image extracted from the sample image data may have some missing information, the specific approach can be to extract the background from the sample image to obtain an initial background image, and then perform image inpainting on the initial background image to obtain the sample background image. There are various methods for image inpainting; for example, information can be filled in based on a standard image, or an image inpainting model can be pre-trained based on the desired background image, and then the initial background image can be inpainted using the image inpainting model.

[0039] Specifically, among the multiple sample background images stored in the pre-built background database, one sample background image is randomly selected and used as the target background image to be fused.

[0040] Optionally, in another embodiment of the present invention, obtaining the target background image to be fused includes: obtaining a target subject image from a pre-built subject database, and generating a subject enhancement image based on the target subject image; or, receiving subject information input by a user for the target background image, and generating a subject enhancement image from the received subject information.

[0041] The subject database can be a pre-built database that stores multiple sample subject images. Taking an address where the subject information is in text form as an example, the subject database can be an address database that stores text related to the address.

[0042] Optionally, multiple sample image data are acquired in advance, and the subject information is extracted from the multiple sample image data through image feature extraction to serve as the subject images. These subject images are then stored in a database as the subject database.

[0043] Specifically, among the multiple sample subject images stored in the pre-built subject database, any sample subject image is randomly selected and used as the subject enhancement image.

[0044] Optionally, the system receives subject information input by the user in a preset subject information input area, obtains the target background image corresponding to the user-input subject information, acquires a blank image of the same size as the target background image, and performs image fusion between the received subject information and the blank image to generate a subject-enhanced image. The blank image can be an image of the same size as the target background image with a pixel value of 0.

[0045] S120. The preliminary fused image is binarized to obtain a subject mask image corresponding to the target subject image and a background mask image corresponding to the target background image.

[0046] The subject mask image can be a binarized image that includes the target subject image and the weakened target background image in the preliminary fused image; the background mask image can be a binarized image that includes the target background image and the weakened target subject image in the preliminary fused image.

[0047] Specifically, the subject information belonging to the target subject image in the preliminary fused image is identified, the pixel value of the pixel corresponding to the subject information in the preliminary fused image is set to 255 or 1, and the pixel value of the other pixel in the preliminary fused image except for the pixel corresponding to the subject information is set to 0, so as to weaken the background information of the target background image in the preliminary fused image, and obtain a binarized image. This binarized image is used as the subject mask image corresponding to the target subject image.

[0048] Similarly, background information belonging to the target background image is identified in the preliminary fused image. The pixel values ​​of the pixels corresponding to the background information in the preliminary fused image are set to 255 or 1, and the pixel values ​​of the pixels that are removed from the preliminary fused image are set to 0, so as to weaken the background information of the target background image in the preliminary fused image and obtain a binarized image. This binarized image is used as the background mask image corresponding to the target background image.

[0049] S130. Determine the subject enhancement image and the background enhancement image based on the preliminary fused image, the subject mask image, and the background mask image, respectively.

[0050] The subject enhancement image can be understood as the image in which the subject information of the target subject image is more prominent after processing. Optionally, the subject enhancement image can be the target subject image jointly generated by the preliminary fused image, the subject mask image, and the background mask image. The background enhancement image can be understood as the image in which the subject information of the target background image is more prominent after processing. Optionally, the background enhancement image can be the target background image jointly generated by the preliminary fused image, the subject mask image, and the background mask image.

[0051] Specifically, a preliminary fused image, a subject mask image, and a background mask image are acquired. The target subject image in the preliminary fused image is enhanced using the preliminary fused image, the subject mask image, and the background mask image to generate a preliminary fused image with enhanced target subject image. This preliminary fused image with enhanced target subject image is used as the subject enhanced image. The target background image in the preliminary fused image is enhanced using the preliminary fused image, the subject mask image, and the background mask image to generate a preliminary fused image with enhanced target background image. This preliminary fused image with enhanced target background image is used as the background enhanced image.

[0052] S140. Generate a target augmented image based on the subject augmented image and the background augmented image.

[0053] The target augmented image can be the target image obtained by fusing the subject augmented image and the background augmented image.

[0054] Optionally, in another embodiment of the present invention, generating the target augmented image based on the subject enhancement image and the background enhancement image includes:

[0055] The enhanced image of the subject and the enhanced image of the background are added together to obtain the target augmented image.

[0056] Specifically, after obtaining the subject enhancement image and the background enhancement image, the two images are added together to obtain the target augmented image. The specific formula for adding the subject enhancement image and the background enhancement image to obtain the target augmented image is as follows:

[0057] R4 = R3 + R'3

[0058] R4 is the target augmented image, R3 is the subject augmented image, and R′3 is the background augmented image.

[0059] The technical solution of this invention involves acquiring a target background image and a target subject image to be fused, fusing the target subject image with the target background image to obtain a preliminary fused image, and performing binarization processing on the preliminary fused image to obtain a subject mask image corresponding to the target subject image and a background mask image corresponding to the target background image. This binarization process enhances the contrast between the subject and the background. Furthermore, a subject enhancement image and a background enhancement image are determined based on the preliminary fused image, the subject mask image, and the background mask image, respectively. Finally, a target augmented image is generated based on the subject enhancement image and the background enhancement image. By processing and fusing the target background image and the target subject image multiple times, this invention solves the technical problem of poor flexibility in augmentation in existing technologies, enabling the generation of millions of target augmented images in scenarios with millions of data points, thus improving the richness of large-scale data augmentation.

[0060] Example 2

[0061] Figure 2 This document provides a flowchart of another image data augmentation method for Embodiment 2 of the present invention. This embodiment primarily describes the methods for obtaining the enhanced subject image and the enhanced background image, such as... Figure 2 As shown, the method includes:

[0062] S210. Obtain the target background image and the target subject image to be fused, and fuse the target subject image with the target background image to obtain a preliminary fused image.

[0063] S220. The preliminary fused image is binarized to obtain a subject mask image corresponding to the target subject image and a background mask image corresponding to the target background image.

[0064] S230. Generate a subject feature map based on the preliminary fused image and the subject mask image, and generate a background feature map based on the preliminary fused image and the background mask image.

[0065] The main feature map can be a feature map that only includes the target main image; the background feature map can be a feature map that only includes the target background image.

[0066] Specifically, in the subject mask image, the pixel value of the pixel corresponding to the subject information is 255 or 1, and the pixel value of the remaining pixels is 0. The subject mask image is multiplied with the preliminary fused image, retaining the subject information in the preliminary fused image and removing the remaining information in the preliminary fused image to generate a subject feature map that only includes the subject information. In the background mask image, the pixel value of the pixel corresponding to the background information is 255 or 1, and the pixel value of the remaining pixels is 0. The background mask image is multiplied with the preliminary fused image, retaining the background information in the preliminary fused image and removing the remaining information in the preliminary fused image to generate a subject feature map that only includes the background information.

[0067] S240. Perform a weighted summation process on the subject feature map and the background feature map to obtain a subject fusion image and a background fusion image.

[0068] Among them, the subject fusion image can be a fusion image that enhances the target subject image; the background fusion image can be a fusion image that enhances the target background image.

[0069] Specifically, the main feature map and the background feature map are weighted and summed separately to generate a main fusion image that enhances the target main image and a background fusion image that enhances the target background image.

[0070] Optionally, the step of performing a weighted summation process on the subject feature map and the background feature map to obtain the subject fused image includes:

[0071] The subject feature map is weighted based on a preset first weight, and the background feature map is weighted based on a preset second weight. The sum of the first weight and the second weight is equal to 1, both the first weight and the second weight are greater than or equal to 0 and less than or equal to 1, and the first weight is greater than the second weight. The weighted subject feature map and the weighted background feature map are summed to obtain the subject fused image.

[0072] Wherein, the first weight can be a weight value for adjusting the degree of enhancement of the target subject image; the first weight can be a weight value for adjusting the degree of enhancement of the target background image; the sum of the first weight and the second weight is equal to 1, and both the first weight and the second weight are greater than 0 and less than 1.

[0073] Specifically, when calculating the fused subject image by weighted summation of the subject feature map and the background feature map, the first weight value corresponding to the target subject image is greater than the second weight value corresponding to the target background image. Therefore, the weighted subject feature map and the weighted background feature map are summed to calculate the fused subject image. The formula for summing the weighted subject feature map and the weighted background feature map is shown below:

[0074] R1=λ1*P x1 +λ2*P x2

[0075] Where λ1 is the first weight value, λ2 is the second weight value, λ1+λ2=1, 0<λ1<1, 0<λ2<1, λ1>λ2; The main feature map; R1 is the background feature map; R2 is the main fused image.

[0076] Optionally, when calculating the fused subject image by weighted summation of the subject feature map and the background feature map, the third weight value corresponding to the target subject image is less than the fourth weight value corresponding to the target background image. Therefore, the weighted subject feature map and the weighted background feature map are summed to calculate the fused background image. The sum of the third weight and the fourth weight equals 1, and both the third weight and the fourth weight are greater than 0 and less than 1. The formula for summing the weighted subject feature map and the weighted background feature map is shown below:

[0077]

[0078] Where λ3 is the third weight value, λ4 is the fourth weight value, λ3+λ4=1, 0≤λ3≤1, 0≤λ4≤1, λ3<λ4; λ5 and λ6 can be equal or unequal; The main feature map; R1 is the background feature map; R2 is the background fusion image.

[0079] S250. Determine the enhanced subject image based on the fused subject image, the subject mask image, the background feature map, and the background mask image.

[0080] Optionally, in another embodiment of the present invention, determining the subject enhancement image based on the subject fusion image, the subject mask image, the background mask image, and the background feature map includes: multiplying the subject fusion image and the subject mask image to obtain a first fusion image; multiplying the background feature map and the background mask image to obtain a second fusion image; and performing a weighted summation of the first fusion image and the second fusion image to obtain the subject enhancement image.

[0081] Specifically, the subject fusion image and the subject mask image are multiplied to obtain the first fusion image. The background feature map and the background mask image are weighted to obtain the second fusion image. The first fusion image (multiplied by the subject fusion image and the subject mask image using a fifth weight value) and the second fusion image (multiplied by the background feature map and the background mask image using a sixth weight value) are then weighted and summed to obtain the enhanced subject image. Here, the sum of the fifth and sixth weights equals 1, both the fifth and sixth weights are greater than 0 and less than 1, and the fifth weight is greater than the sixth weight. The calculation formula for the weighted summation of the first fusion image (multiplied by the subject fusion image and the subject mask image using the fifth weight value) and the second fusion image (multiplied by the background feature map and the background mask image using the sixth weight value) is as follows:

[0082]

[0083] Wherein, R3 is the subject enhancement image; λ5 is the fifth weight value, λ6 is the sixth weight value, λ5+λ6=1, 0<λ6<1; λ5>λ6, λ5 and λ6 can be equal or not equal; The main image is a mask image. Background mask image; R1 is the background feature map; R2 is the main fused image.

[0084] S260. Determine a background enhancement image based on the background fusion image, the background mask image, the subject feature map, and the subject mask image.

[0085] Specifically, the background feature map and the background mask image are weighted to obtain a first fused image. The subject fused image and the subject mask image are multiplied to obtain a second fused image. The second fused image (multiplied by the subject fused image and the subject mask image using an eighth weight value) and the first fused image (multiplied by the background feature map and the background mask image using a seventh weight value) are then weighted and summed to obtain a background enhancement image. Here, the sum of the seventh and eighth weights equals 1, both the seventh and eighth weights are greater than 0 and less than 1, and the seventh weight is greater than the eighth weight. The formula for the weighted summation of the second fused image (multiplied by the subject fused image and the subject mask image using the eighth weight value) and the first fused image (multiplied by the background feature map and the background mask image using the seventh weight value) is shown below:

[0086]

[0087] Where R′3 is the background enhancement image; λ7 is the seventh weight value, λ8 is the eighth weight value; λ7+λ8=1, 0<λ7<1, 0<λ8<1, λ7>λ8, λ7 and λ8 can be equal or not; The main mask image; Background mask image; R1 is the background feature map; R2 is the background fusion image.

[0088] S270. Generate a target augmented image based on the subject augmented image and the background augmented image.

[0089] The technical solution of this invention generates a subject feature map and a background feature map based on a preliminary fused image, a subject mask image, and an image mask image. It extracts features from the subject image and the background image, making the contrast between the subject image and the background image greater and more clearly displaying the features of the subject image and the background image. Then, it performs weighted summation processing to obtain a subject fused image and a background fused image, further extracting the target subject image and the target background image, making the target subject image and the target background image closer to the real image. Based on the preliminary fused image, the subject mask image, and the background mask image, the subject enhancement image and the background enhancement image are determined respectively. The subject image and the background image are enhanced by the preliminary fused image, the subject mask image, and the background mask image. By calculating the subject enhancement image and the background enhancement image separately, the background part in the subject enhancement image and the subject part in the background enhancement image are weakened respectively, which further improves the contrast and clarity of the subject image and the background image, and improves the display effect of the subject image and the background image. The target augmented image is generated based on the subject enhancement image and the background enhancement image. By processing and fusing the target background image and the target subject image multiple times, even in the case of millions of data information, millions of target augmented images can be generated, which is beneficial for large-scale data augmentation and increases the richness of the augmented images.

[0090] Example 3

[0091] Figure 3 This is a schematic diagram of an image data amplification device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device specifically includes: an image fusion module 310, an image processing module 320, an image determination module 330, and a music control module 340. Wherein:

[0092] Image fusion module 310 is used to acquire a target background image and a target subject image to be fused, and to fuse the target subject image with the target background image to obtain a preliminary fused image;

[0093] Image processing module 320 is used to perform binarization processing on the preliminary fused image to obtain a subject mask image corresponding to the target subject image and a background mask image corresponding to the target background image;

[0094] The image determination module 330 is used to determine the subject enhancement image and the background enhancement image based on the preliminary fused image, the subject mask image, and the background mask image, respectively.

[0095] The target augmented image generation module 340 is used to generate a target augmented image based on the subject augmented image and the background augmented image.

[0096] The technical solution of this invention involves acquiring a target background image and a target subject image to be fused, fusing the target subject image with the target background image to obtain a preliminary fused image, and performing binarization processing on the preliminary fused image to obtain a subject mask image corresponding to the target subject image and a background mask image corresponding to the target background image. This binarization process enhances the contrast between the subject and the background. Furthermore, a subject enhancement image and a background enhancement image are determined based on the preliminary fused image, the subject mask image, and the background mask image, respectively. Finally, a target augmented image is generated based on the subject enhancement image and the background enhancement image. By processing and fusing the target background image and the target subject image multiple times, this invention solves the technical problem of poor flexibility in augmentation in existing technologies, enabling the generation of millions of target augmented images in scenarios with millions of data points, thus improving the richness of large-scale data augmentation.

[0097] Optionally, the image fusion module 310 is specifically used for:

[0098] Sample background images are obtained from a pre-built background database and used as target background images for fusion. The sample background images are obtained by extracting background information from sample image data.

[0099] Optionally, the image fusion module 310 is further configured to:

[0100] Obtain target subject images from a pre-built subject database, and generate target subject images to be fused based on the target subject images; or,

[0101] The system receives subject information input by the user for the target background image and generates a target subject image to be fused from the received subject information.

[0102] Optionally, the image determination module 330 is specifically used for:

[0103] A subject feature map is generated based on the preliminary fused image and the subject mask image, and a background feature map is generated based on the preliminary fused image and the background mask image;

[0104] The subject feature map and the background feature map are weighted and summed to obtain the subject fused image and the background fused image;

[0105] Based on the fused subject image, the subject mask image, the background feature map, and the background mask image, a subject enhancement image is determined;

[0106] A background enhancement image is determined based on the background fusion image, the background mask image, the subject feature map, and the subject mask image.

[0107] Optionally, the image determination module 330 is further configured to:

[0108] The main feature map is weighted based on a preset first weight, and the background feature map is weighted based on a preset second weight, wherein the sum of the first weight and the second weight is equal to 1, both the first weight and the second weight are greater than or equal to 0 and less than or equal to 1, and the first weight is greater than the second weight.

[0109] The weighted subject feature map and the weighted background feature map are summed to obtain the fused subject image.

[0110] Optionally, the image determination module 330 is further configured to:

[0111] The first fused image is obtained by multiplying the fused image of the subject with the fused image of the subject.

[0112] The background feature map and the background mask image are multiplied together to obtain the second fused image;

[0113] The first fused image and the second fused image are weighted and summed to obtain the enhanced main image.

[0114] Optionally, the target amplified image generation module 340 is specifically used for:

[0115] The enhanced image of the subject and the enhanced image of the background are added together to obtain the target augmented image.

[0116] The image data amplification device provided in the embodiments of the present invention can execute the image data amplification method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the image data amplification method.

[0117] It is worth noting that in the embodiments of the above-mentioned image data amplification device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0118] Example 4

[0119] Figure 4A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0120] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0121] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0122] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as image data augmentation methods.

[0123] In some embodiments, the image data augmentation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the image data augmentation method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the image data augmentation method by any other suitable means (e.g., by means of firmware).

[0124] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0125] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0126] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0127] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0128] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0129] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0130] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0131] Example 5

[0132] This embodiment provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the program implements the steps of the image data augmentation method provided in any embodiment of the present invention, the method comprising:

[0133] Obtain the target background image and the target subject image to be fused, and fuse the target subject image with the target background image to obtain a preliminary fused image;

[0134] The preliminary fused image is binarized to obtain a subject mask image corresponding to the target subject image and a background mask image corresponding to the target background image;

[0135] The subject enhancement image and the background enhancement image are determined based on the preliminary fused image, the subject mask image, and the background mask image, respectively.

[0136] A target augmented image is generated based on the subject augmented image and the background augmented image.

[0137] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0138] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0139] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0140] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0141] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0142] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0143] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for amplifying image data, characterized in that, include: Obtain the target background image and the target subject image to be fused, and fuse the target subject image with the target background image to obtain a preliminary fused image; The preliminary fused image is binarized to obtain a subject mask image corresponding to the target subject image and a background mask image corresponding to the target background image; The enhanced subject image and enhanced background image are determined based on the preliminary fused image, the subject mask image, and the background mask image, respectively. Generate a target augmented image based on the subject augmented image and the background augmented image; The step of determining the subject enhancement image and the background enhancement image based on the preliminary fused image, the subject mask image, and the background mask image respectively includes: A subject feature map is generated based on the preliminary fused image and the subject mask image, and a background feature map is generated based on the preliminary fused image and the background mask image; The subject feature map and the background feature map are weighted and summed to obtain a subject fusion image and a background fusion image; wherein, the subject fusion image is a fusion image that enhances the target subject image; and the background fusion image is a fusion image that enhances the target background image. Based on the fused subject image, the subject mask image, the background feature map, and the background mask image, a subject enhancement image is determined; A background enhancement image is determined based on the background fusion image, the background mask image, the subject feature map, and the subject mask image.

2. The method according to claim 1, characterized in that, The acquisition of the target background image to be fused includes: Sample background images are obtained from a pre-built background database and used as target background images for fusion. The sample background images are obtained by extracting background information from sample image data.

3. The method according to claim 1, characterized in that, The acquisition of the target background image to be fused includes: Obtain a target subject image from a pre-built subject database, and generate a subject-enhanced image based on the target subject image; or, Receive subject information input by the user for the target background image, and generate a subject enhancement image from the received subject information.

4. The method according to claim 1, characterized in that, The step of performing a weighted summation process on the subject feature map and the background feature map to obtain a fused subject image includes: The main feature map is weighted based on a preset first weight, and the background feature map is weighted based on a preset second weight, wherein the sum of the first weight and the second weight is equal to 1, both the first weight and the second weight are greater than or equal to 0 and less than or equal to 1, and the first weight is greater than the second weight. The weighted subject feature map and the weighted background feature map are summed to obtain the fused subject image.

5. The method according to claim 1, characterized in that, Determining the enhanced subject image based on the fused subject image, the subject mask image, the background mask image, and the background feature map includes: The first fused image is obtained by multiplying the fused image of the subject with the fused image of the subject. The background feature map and the background mask image are multiplied together to obtain the second fused image; The first fused image and the second fused image are weighted and summed to obtain the enhanced main image.

6. The method according to claim 1, characterized in that, The step of generating a target augmented image based on the subject augmented image and the background augmented image includes: The enhanced image of the subject and the enhanced image of the background are added together to obtain the target augmented image.

7. An image data amplification device, characterized in that, include: The image fusion module is used to acquire the target background image and the target subject image to be fused, and to fuse the target subject image with the target background image to obtain a preliminary fused image; The image processing module is used to perform binarization processing on the preliminary fused image to obtain a subject mask image corresponding to the target subject image and a background mask image corresponding to the target background image; An image determination module is used to determine a subject enhancement image and a background enhancement image based on the preliminary fused image, the subject mask image, and the background mask image, respectively. A target augmented image generation module is used to generate a target augmented image based on the subject augmented image and the background augmented image; The image determination module is specifically used for: A subject feature map is generated based on the preliminary fused image and the subject mask image, and a background feature map is generated based on the preliminary fused image and the background mask image; The subject feature map and the background feature map are weighted and summed to obtain a subject fusion image and a background fusion image; wherein, the subject fusion image is a fusion image that enhances the target subject image; and the background fusion image is a fusion image that enhances the target background image. Based on the fused subject image, the subject mask image, the background feature map, and the background mask image, a subject enhancement image is determined; A background enhancement image is determined based on the background fusion image, the background mask image, the subject feature map, and the subject mask image.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, which enables the at least one processor to perform the image data amplification method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the method for amplifying image data according to any one of claims 1-6.

Images