Image data augmentation method, electronic device, storage medium, and program

Abstract

The embodiment of the invention discloses an image data augmentation method, electronic equipment, a storage medium and a program, and the method comprises the steps: obtaining a target to-be-processed image, and extracting a target object in the target to-be-processed image; generating a newly added background image of the target to-be-processed image according to the associated processing image of the target to-be-processed image; and synthesizing a target object in the target to-be-processed image and the newly-added background image of the target to-be-processed image to obtain a target newly-added image corresponding to the target to-be-processed image. According to the technical scheme of the embodiment of the invention, the diversity and characterization capability of the image data can be enhanced.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image processing, and particularly relate to an image data augmentation method, an electronic device, a storage medium and a program. BACKGROUND

[0002] Target detection, as a core task in the field of computer vision, aims to automatically identify and accurately locate target objects of a specific category in images or videos. The richness of features contained in image data is one of the key factors determining the accuracy of a detection model. Currently, common image data augmentation methods mainly rely on geometric transformations such as rotation and scaling, or extend the data set by adjusting visual attributes such as illumination and contrast.

[0003] The inventors have found in the process of implementing the present application that the prior art has the following defects: on the one hand, if the richness of features of image data is insufficient, the detection model will be insufficient in distinguishing ability, resulting in missed detection or positioning deviation; on the other hand, in real scenes, some image samples containing specific features are difficult to form a large-scale data set due to factors such as difficulty in acquisition and high cost of scene reproduction, thereby seriously affecting the detection performance of the detection model in complex similar scenes. SUMMARY

[0004] Embodiments of the present application provide an image data augmentation method, an electronic device, a storage medium and a program, which can enhance the diversity and representation ability of image data.

[0005] According to an aspect of the present application, an image data augmentation method is provided, comprising:

[0006] acquiring a target image to be processed, and extracting a target object in the target image to be processed;

[0007] generating a new background image of the target image to be processed according to an associated processing image of the target image to be processed;

[0008] synthesizing the target object in the target image to be processed and the new background image of the target image to be processed to obtain a target new image corresponding to the target image to be processed.

[0009] According to another aspect of the present application, an image data augmentation device is provided, comprising:

[0010] a target object extraction module configured to acquire a target image to be processed, and extract a target object in the target image to be processed;

[0011] a new background image generation module configured to generate a new background image of the target image to be processed according to an associated processing image of the target image to be processed;

[0012] The target new image acquisition module is configured to synthesize the target object in the target to-be-processed image and a new background image of the target to-be-processed image to obtain a target new image corresponding to the target to-be-processed image.

[0013] According to another aspect of the present application, an electronic device is provided, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory connected to the at least one processor in communication; wherein

[0016] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the image data augmentation method according to any one of the embodiments of the present application.

[0017] According to another aspect of the present application, a computer readable storage medium is provided, the computer readable storage medium stores computer instructions for enabling a processor to implement the image data augmentation method according to any one of the embodiments of the present application when executed by the processor.

[0018] According to another aspect of the present application, a computer program product is also provided, comprising a computer program for implementing the image data augmentation method according to any one of the embodiments of the present application when executed by a processor.

[0019] The embodiments of the present application acquire a target to-be-processed image and extract a target object in the target to-be-processed image. Further, a new background image of the target to-be-processed image is generated according to an associated processing image of the target to-be-processed image, so as to synthesize the target object in the target to-be-processed image and the new background image of the target to-be-processed image, and obtain a target new image corresponding to the target to-be-processed image. The above-mentioned scheme can effectively simulate the background change in a real scene by synthesizing the target object and the new background image to obtain the target new image, realize flexible combination of image features, solve the limitation problem of the existing image data augmentation method in image feature fusion, and enhance the diversity and representation ability of image data.

[0020] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description only show some embodiments of the present application, and other drawings can be obtained by those of ordinary skill in the art without any creative effort based on these drawings.

[0022] Figure 1 is a flow chart of an image data augmentation method provided by an embodiment of the present application;

[0023] Figure 2 is a flow chart of an image data augmentation method provided by an embodiment of the present application;

[0024] Figure 3 is a schematic diagram of an image data augmentation device provided by an embodiment of the present application;

[0025] Figure 4 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the person skilled in the art better understand the present application, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without any creative effort should be within the scope of protection of the present application.

[0027] It should be noted that the terms "target", "association" and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0028] Embodiment one

[0029] Figure 1is a flowchart of an image data augmentation method provided by Embodiment One of the present application. The present embodiment can be applied to the generation of new images based on target objects in a target image and background images. The method can be executed by an image data augmentation device, which can be implemented in software and / or hardware and can be integrated into an electronic device, which can be a terminal device or a server device, as long as it can execute the image data augmentation method. The present application does not limit the specific type of electronic device. Accordingly, as shown in Figure 1 the method includes the following operations:

[0030] S110, a target image to be processed is obtained, and a target object in the target image to be processed is extracted.

[0031] The target image to be processed can be any image that needs to be augmented. The target object can be any object in the target image to be processed. For example, the target object can include, but is not limited to, natural objects such as animals and plants, man-made objects such as cars and tables, and human beings and related elements, and the present application does not limit the specific type of target object.

[0032] It can be understood that the target image to be processed can include multiple objects, and any one or more of the objects can be selected as the target object, and the remaining part of the target image to be processed other than the target object can be selected as the remaining part of the target image to be processed, as the background to be processed of the target image to be processed. Therefore, when data augmentation is needed based on an image, the image can be selected as the target image to be processed, and the object in the target image to be processed can be extracted as the target object. In a specific example, assuming that a human landscape image is selected as the target image to be processed, the human in the image can be selected as the target object in the target image to be processed, so that the image data diversity can be significantly improved through background changes.

[0033] S120, generating a new background image of the target image to be processed based on an associated processing image of the target image to be processed.

[0034] The associated processing image can be an image associated with the target image to be processed. The new background image can be a new image generated based on the associated processing image of the target image to be processed.

[0035] Specifically, the background image of the target image to be processed, or any image in the same data set as the target image to be processed can be selected as the associated processing image of the target image to be processed. Further, the associated processing image of the target image to be processed can be processed, so that the new background image of the target image to be processed can be generated based on the associated processing image of the target image to be processed.

[0036] S130, synthesizing the target object in the target to-be-processed image and the newly added background image of the target to-be-processed image to obtain a target newly added image corresponding to the target to-be-processed image.

[0037] The target newly added image can be an image obtained by synthesizing the target object and the newly added background image.

[0038] Correspondingly, after obtaining the newly added background image of the target to-be-processed image, the target object in the target to-be-processed image and the newly added background image of the target to-be-processed image can be synthesized, so that the target newly added image corresponding to the target to-be-processed image can be obtained. The above scheme realizes the diversity of image data by only changing the background while keeping the target object unchanged, and can guarantee the high similarity of image data.

[0039] The embodiment of the application obtains a target to-be-processed image, and extracts a target object in the target to-be-processed image. Further, a newly added background image of the target to-be-processed image is generated according to an associated processing image of the target to-be-processed image, so that the target object in the target to-be-processed image and the newly added background image of the target to-be-processed image are synthesized to obtain a target newly added image corresponding to the target to-be-processed image. The above scheme can effectively simulate the background change in a real scene by synthesizing the target object and the newly added background image to obtain the target newly added image, realizes flexible combination of image features, solves the limitation problem of existing image data augmentation methods in image feature fusion, and can enhance the diversity and representation ability of image data.

[0040] Embodiment two

[0041] Figure 2 is a flowchart of an image data augmentation method provided by the second embodiment of the application. The present embodiment is based on the above-mentioned embodiments and is specific. In the present embodiment, a plurality of specific and optional implementation manners of generating a newly added background image of a target to-be-processed image according to an associated processing image of the target to-be-processed image, and synthesizing a target object in the target to-be-processed image and the newly added background image of the target to-be-processed image to obtain a target newly added image corresponding to the target to-be-processed image are given. Meanwhile, optional implementation operations after extracting the target object in the target to-be-processed image and before generating the newly added background image of the target to-be-processed image according to the associated processing image of the target to-be-processed image are also given. Correspondingly, as shown in Figure 2 The method of the present embodiment can include:

[0042] S210, obtaining a target to-be-processed image, and extracting a target object in the target to-be-processed image.

[0043] S220, extract foreground color features of the target object in the target image to be processed.

[0044] The foreground color features can be color features of pixels in a region where the target object is located in the target image to be processed. For example, the foreground color features can include, but are not limited to, RGB (Red-Green-Blue) values of pixels in the region where the target object is located, color space features, color texture features, and the like. The specific type of the foreground color features is not limited in the embodiments of the present application.

[0045] The existing data augmentation method mainly relies on geometric transformation and illumination adjustment to enhance the robustness of the target detection model. Although this method can improve the adaptability of the target detection model to the changes in the view angle and illumination, it cannot effectively solve the detection problem caused by color similarity. More importantly, the sample collection cost is high and the annotation difficulty is great for the target object with high similarity to the background color in the real scene, which leads to a serious shortage of data available for training.

[0046] In the embodiments of the present application, after the target object in the target image to be processed is extracted, the background color migration can be used to generate new images with high similarity between the target object and the background. Specifically, the foreground color features of the region where the target object is located in the target image to be processed can be extracted first.

[0047] In a specific example, assuming that the foreground color features of the target object are the RGB values of the pixels in the region where the target object is located, the foreground mask annotation technology can be used to mark the pixel region of the target object in the target image to be processed as foreground with a mask value of 1, and mark the pixel region other than the target object in the target image to be processed as background to be processed with a mask value of 0. Further, the foreground color features of the target object extracted from the target image to be processed can be represented by the following formula:

[0048]

[0049] wherein x∈[0,H-1] and x is the row index of the pixel, y∈[0,W-1] and y is the column index of the pixel, H is the height of the target image to be processed, W is the width of the target image to be processed, I(x,y) is the RGB value of the pixel, M(x,y) is a binary mask function, M(x,y) represents that the pixel belongs to the foreground region where the target object is located in the target image to be processed, and P is the RGB value of the pixel in the region where the target object is located.

[0050] S230, determine an associated candidate image of the target image to be processed.

[0051] The associated candidate image can be a plurality of images associated with the target image to be processed.

[0052] Specifically, after extracting the foreground color feature of the target object in the target to-be-processed image, the associated candidate image of the target to-be-processed image can be determined. For example, the background image of the target to-be-processed image or the image in the same data set as the target to-be-processed image can be taken as the associated candidate image of the target to-be-processed image.

[0053] S240, randomly selecting an image from the associated candidate images as the associated processing image of the target to-be-processed image.

[0054] Correspondingly, after determining the associated candidate image of the target to-be-processed image, an image can be randomly selected from the associated candidate image as the associated processing image of the target to-be-processed image.

[0055] S250, adjusting the image size of the associated processing image of the target to-be-processed image according to the image size of the target to-be-processed image.

[0056] The image size of the target to-be-processed image can be the size of the target to-be-processed image. The image size of the associated processing image can be the size of the associated processing image.

[0057] Correspondingly, after determining the associated processing image, the image size of the associated processing image of the target to-be-processed image can be enlarged or reduced according to the image size of the target to-be-processed image.

[0058] In a specific example, the image size of the associated processing image of the target to-be-processed image can be adjusted to the image size of the target to-be-processed image, or the image size of the associated processing image of the target to-be-processed image can be adjusted to a size that has a certain proportional relationship with the image size of the target to-be-processed image.

[0059] S260, determining a target reference foreground color according to the foreground color feature of the target object in the target to-be-processed image.

[0060] The target reference foreground color can be any one of the foreground color features.

[0061] Correspondingly, after obtaining the foreground color feature of the target object, any one of the foreground color features can be selected as the target reference foreground color, or a color can be selected from the foreground color features as the target reference foreground color according to a preset rule.

[0062] In an optional embodiment of the present application, the determining the target reference foreground color according to the foreground color feature of the target object in the target image to be processed can include: determining pixel color data of a target object region in the target image to be processed according to the foreground color feature of the target object in the target image to be processed; performing statistics on the pixel color data of the target object region to obtain a statistical result of the pixel color data of the target object region; performing screening on the statistical result of the pixel color data of the target object region to obtain a candidate color set of the target object region; and randomly selecting a color from the candidate color set of the target object region as the target reference foreground color.

[0063] The pixel color data can be pixel color values of the target object region. The statistical result of the pixel color data can be a result obtained by performing statistics on the pixel color data. The candidate color set can be a set of one or more colors obtained by performing screening on the statistical result of the pixel color data.

[0064] In the embodiment of the present application, to determine the target reference foreground color, first, the pixel color data of the target object region in the target image to be processed can be determined according to the foreground color feature of the target object in the target image to be processed, and the pixel color data of the target object can be statistically analyzed. Further, the statistical result of the pixel color data of the target object region can be screened according to a preset rule, and one or more colors thereof can be taken as a candidate color set of the target object region. After obtaining the candidate color set of the target object region, a color can be randomly selected as the target reference foreground color, so as to ensure the diversity and rationality when adjusting the image color of the associated processing image.

[0065] In a specific example, a color histogram of the pixel color data of the target object region can be calculated, and the top N colors with the highest frequencies in the color histogram can be selected as the candidate color set. Further, a color can be randomly selected from the N colors as the target reference foreground color. For example, N can be 3, and the embodiment of the present application does not limit the specific value of N.

[0066] S270, adjusting the image color of the associated processing image according to the target reference foreground color to obtain the new background image.

[0067] Correspondingly, after determining the target reference foreground color, the image color of the associated processing image can be adjusted according to the target reference foreground color to obtain the new background image, so as to ensure the consistency in color between the new background image and the target object.

[0068] In an optional embodiment of the present application, the adjusting the image color of the associated processing image according to the target reference foreground color to obtain the new background image can include: calculating an image color average value of the image color of the associated processing image; calculating a color offset between the target reference foreground color and the image color average value of the image color of the associated processing image; and performing offset processing on the image color of the associated processing image according to the color offset to obtain the new background image.

[0069] The image color average value can be an average value of all pixel colors in the associated processing image. The color offset can be a difference value between the target reference foreground color and the image color average value of the image color of the associated processing image.

[0070] In the embodiment of the present application, to adjust the image color of the associated processing image, the image color average value of the image color of the associated processing image can be calculated based on the following formula:

[0071]

[0072] The image color average value is μ bg H is the height of the associated processing image, W is the height of the associated processing image, and B(i, j) is the pixel value of the associated processing image at position (i, j). i is the row index of the associated processing image, and j is the column index of the associated processing image.

[0073] Further, the color offset between the target reference foreground color and the image color average value of the image color of the associated processing image can be calculated based on the following formula:

[0074] Δ = μ bg -c t

[0075] The color offset between the target reference foreground color and the image color average value of the image color of the associated processing image is Δ bg The image color average value is μ t The target reference foreground color is c

[0076] After the color offset between the target reference foreground color and the image color average value of the image color of the associated processing image is calculated, offset processing can be performed on the image color of the associated processing image according to the color offset, so that the new background image can be obtained.

[0077] In an optional embodiment of the present application, the step of offsetting the image color of the associated processing image according to the color offset to obtain the new background image can include: calculating a difference value between the image color of the associated processing image and the color offset to obtain a current background image color; and performing truncation processing on the numerical value of the current background image color in a case where it is determined that the numerical value of the current background image color exceeds a preset color threshold value.

[0078] The current background image color can be a difference value between the image color of the associated processing image and the color offset. The preset color threshold value can be a preset numerical value of color.

[0079] It can be understood that, for an image with an 8-bit color depth, the value range of the color numerical value is 0-255, and thus the preset color numerical threshold value can be 0-255. In the process of offsetting the image color of the associated processing image according to the color offset, first, a difference value between the image color of the associated processing image and the color offset can be calculated, and the difference value can be taken as the current background image color. Further, the numerical value of the current background image color can be judged. If the numerical value of the current background image color exceeds the preset color threshold value, the numerical value of the current background image color can be truncated, and the truncated current background image color can be taken as the image color of the new background image. If the numerical value of the current background image color does not exceed the preset color threshold value, the current background image color can be directly taken as the image color of the new background image.

[0080] In a specific example, if the numerical value of the current background image color is less than 0, the numerical value of the current background image color can be modified to 0. If the numerical value of the current background image color is greater than 255, the numerical value of the current background image color can be modified to 255.

[0081] Exemplarily, the above scheme can be implemented based on the following formula:

[0082] B adjusted (i,j)=Clip(B(i,j)-Δ,0,255)

[0083] B adjusted (i,j) is the new background image, B(i,j) is the associated processing image, Δ is the color offset between the target reference foreground color and the image color average value of the image color of the associated processing image, and Clip() is a truncation function.

[0084] S280, determining a pixel correspondence relationship between a target object in the target to-be-processed image and a pixel value in the new background image of the target to-be-processed image.

[0085] The pixel correspondence relationship can be a one-to-one correspondence or a proportional correspondence, and the specific type of the pixel correspondence relationship is not limited in the embodiment of the application.

[0086] Specifically, when synthesizing the target object and the new background image, the pixel correspondence relationship between the pixel values of the target object in the target image to be processed and the new background image of the target image to be processed can be determined first. Specifically, each pixel point in the target object in the target image to be processed can be mapped to the coordinate system of the new background image through coordinate mapping transformation, so as to realize the fusion of the target object and the new background image.

[0087] S290, according to the pixel correspondence relationship, replacing the pixel value of the background to be processed in the target image to be processed except the target object with the corresponding pixel value of the new background image, to obtain the target new image corresponding to the target image to be processed.

[0088] The pixel value of the background to be processed can be the pixel value of the background to be processed in the target image to be processed except the target object.

[0089] Specifically, after obtaining the pixel correspondence relationship between the pixel values of the target object in the target image to be processed and the new background image of the target image to be processed, the pixel value of the background to be processed in the target image to be processed except the target object can be replaced with the corresponding pixel value of the new background image, so as to realize the synthesis of the target object and the new background image. The above scheme can simulate the scene where the target object is similar to the background color, without the need for manual collection or annotation of massive real data, and can significantly reduce the data acquisition cost.

[0090] Specifically, if the image size of the target image to be processed is the same as the image size of the new background image, the pixel value of the new background image can be directly used to replace the pixel value of the background to be processed in the target image to be processed except the target object; if there is a proportional relationship between the image size of the new background image and the image size of the target image to be processed, the pixel value of the new background image can be scaled first to match the image size of the target image to be processed, and then the pixel value of the background to be processed in the target image to be processed except the target object can be replaced.

[0091] In one specific example, the target new image corresponding to the target image to be processed can be generated based on the following formula:

[0092] I synthetic = I foreground ☉M+B adjusted ☉(1-M)

[0093] wherein, I synthetic is a pixel value corresponding to the target new image, I foreground is a pixel value corresponding to the target object, M is a mask, B adjusted is a pixel value corresponding to the new background image.

[0094] In an optional embodiment of the present application, after synthesizing the target object in the target to-be-processed image and the new background image of the target to-be-processed image to obtain the target new image corresponding to the target to-be-processed image, the method can further include: generating image sample data corresponding to the target to-be-processed image according to the target new image corresponding to the target to-be-processed image; training a target detection model according to the image sample data; wherein the target detection model is used for target detection according to an input image.

[0095] wherein, the image sample data can be sample data used for training the target detection model. The target detection model can be a computer vision model used for identifying and locating one or more target objects in an image. The input image can be an image to be subjected to target detection,

[0096] In an embodiment of the present application, after obtaining the target new image corresponding to the target to-be-processed image, the target new image corresponding to the target to-be-processed image can be subjected to format conversion and labeling, so as to generate image sample data corresponding to the target to-be-processed image. Further, the target detection model can be trained according to the image sample data, so that the target detection model can perform target detection according to an input image. In the above scheme, the image sample data corresponding to the target to-be-processed image includes images with similar target objects and background colors. Training the target detection model according to the image sample data can significantly improve the target detection capability of the target detection model in a complex environment with high similarity between target objects and background, and enhance the robustness of the target detection model.

[0097] It can be understood that the image sample data can include but is not limited to image data with high similarity between target object color and background color, and the embodiment of the present application does not limit the specific type of image sample data.

[0098] The embodiment of the present application obtains a target image to be processed, extracts a target object in the target image to be processed, and extracts a foreground color feature of the target object in the target image to be processed. Further, a related candidate image of the target image to be processed is determined, and a random image is selected from the related candidate image as a related processing image of the target image to be processed, so as to adjust the image size of the related processing image of the target image to be processed according to the image size of the target image to be processed. Further, a target reference foreground color is determined according to the foreground color feature of the target object in the target image to be processed, so as to adjust the image color of the related processing image according to the target reference foreground color, and obtain a new background image. After obtaining the new background image, a pixel correspondence relationship between the target object in the target image to be processed and the pixel value in the new background image of the target image to be processed is determined, so as to replace the background pixel value to be processed in the target object in the target image to be processed with the corresponding pixel value of the new background image according to the pixel correspondence relationship, and obtain a target new image corresponding to the target image to be processed. The above scheme can effectively simulate the background change in the real scene by synthesizing the target object and the new background image to obtain the target new image, realize flexible combination of image features, solve the limitation problem of the existing image data augmentation method in image feature fusion, and can enhance the diversity and representation ability of image data.

[0099] In the technical scheme of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information (such as face information or image data) comply with relevant laws and regulations and do not violate public order and good customs.

[0100] It should be noted that the related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, analyzed data, etc.) involved in the present disclosure are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data comply with relevant laws, regulations and standards in the relevant region.

[0101] It should be noted that any arrangement and combination of the technical features among the above embodiments also belong to the protection scope of the present application.

[0102] Embodiment three

[0103] Figure 3 is a schematic diagram of an image data augmentation device provided by the third embodiment of the present application, as shown in Figure 3 The device comprises a target object extraction module 310, a new background image generation module 320 and a target new image acquisition module 330, wherein:

[0104] The target object extraction module 310 is configured to acquire a target image to be processed and extract a target object in the target image to be processed.

[0105] The new background image generation module 320 is configured to generate a new background image of the target image to be processed according to an associated processing image of the target image to be processed.

[0106] The target new image acquisition module 330 is configured to synthesize the target object in the target image to be processed and the new background image of the target image to be processed to obtain a target new image corresponding to the target image to be processed.

[0107] The embodiment of the present application acquires a target image to be processed and extracts a target object in the target image to be processed. Further, a new background image of the target image to be processed is generated according to an associated processing image of the target image to be processed, so as to synthesize the target object in the target image to be processed and the new background image of the target image to be processed to obtain a target new image corresponding to the target image to be processed. The above scheme synthesizes the target object and the new background image to obtain the target new image, which can effectively simulate the background change in a real scene, realizes flexible combination of image features, solves the limitation problem of the existing image data augmentation method in image feature fusion, and can enhance the diversity and representation ability of image data.

[0108] Optionally, the device can further include a foreground color feature extraction module configured to extract a foreground color feature of the target object in the target image to be processed.

[0109] Optionally, the new background image generation module 320 is specifically configured to determine a target reference foreground color according to the foreground color feature of the target object in the target image to be processed, and adjust the image color of the associated processing image according to the target reference foreground color to obtain the new background image.

[0110] Optionally, the device can further include an associated processing image adjustment module configured to determine an associated candidate image of the target image to be processed, randomly select an image from the associated candidate image as the associated processing image of the target image to be processed, and adjust the image size of the associated processing image of the target image to be processed according to the image size of the target image to be processed.

[0111] Optionally, the new background image generation module 320 is further configured to: determine pixel color data of a target object region in the target to-be-processed image according to a foreground color feature of a target object in the target to-be-processed image; perform statistics on the pixel color data of the target object region to obtain a statistical result of the pixel color data of the target object region; perform screening on the statistical result of the pixel color data of the target object region to obtain a candidate color set of the target object region; and randomly select a color from the candidate color set of the target object region as the target reference foreground color.

[0112] Optionally, the new background image generation module 320 is further configured to: calculate an image color average value of an image color of the associated processing image; calculate a color offset between the target reference foreground color and the image color average value of the image color of the associated processing image; and perform offset processing on the image color of the associated processing image according to the color offset to obtain the new background image.

[0113] Optionally, the new background image generation module 320 is further configured to: calculate a difference value between the image color of the associated processing image and the color offset to obtain a current background image color; and perform truncation processing on a numerical value of the current background image color in a case where the numerical value of the current background image color exceeds a preset color threshold.

[0114] Optionally, the target new image acquisition module 330 is specifically configured to: determine a pixel correspondence relationship between a target object in the target to-be-processed image and pixel values in the new background image of the target to-be-processed image; and replace pixel values of a to-be-processed background in the target to-be-processed image except the target object with corresponding pixel values of the new background image according to the pixel correspondence relationship to obtain a target new image corresponding to the target to-be-processed image.

[0115] The image data augmentation apparatus described above can execute the image data augmentation method provided by any embodiment of the present application, and has the corresponding function modules and beneficial effects of executing the method. Technical details not described in detail in the present embodiment can be referred to the image data augmentation method provided by any embodiment of the present application.

[0116] Since the image data augmentation device described above is a device that can execute the image data augmentation method in the embodiments of the present application, based on the image data augmentation method described in the embodiments of the present application, those skilled in the art can understand the specific implementation of the image data augmentation device of the present embodiment and its various forms, so here the image data augmentation device how to implement the image data augmentation method in the embodiments of the present application will not be described in detail. As long as those skilled in the art implement the device used in the image data augmentation method in the embodiments of the present application, it belongs to the scope of protection of the present application.

[0117] Embodiment Four

[0118] Figure 4 A structural diagram of an electronic device 10 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent a variety of forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0119] As Figure 4 shown, the electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0120] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunications networks.

[0121] The processor 11 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the image data augmentation method.

[0122] In some embodiments, the image data augmentation method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the image data augmentation method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the image data augmentation method by any other appropriate means, such as by means of firmware.

[0123] Optionally, the image data augmentation method can comprise: obtaining a target image to be processed, and extracting a target object in the target image to be processed; generating a new background image of the target image to be processed according to an associated processing image of the target image to be processed; and synthesizing the target object in the target image to be processed and the new background image of the target image to be processed to obtain a target new image corresponding to the target image to be processed.

[0124] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0125] Computer programs for implementing the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer program, when executed, can cause instructions defined in the flow charts and / or block diagrams to be implemented. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, and partially on a remote machine or entirely on a remote machine or server.

[0126] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0127] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0128] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), blockchain network, and the Internet.

[0129] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0130] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be executed in parallel, in series, or in a different order, as long as the desired results of the technical solutions of the present disclosure are achieved, and the present disclosure is not limited herein.

[0131] The specific implementation described above does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An image data augmentation method, characterized in that, include: Acquire the target image to be processed, and extract the target object from the target image to be processed; A new background image is generated for the target image to be processed based on the associated processed image. The target object in the target image to be processed and the newly added background image of the target image to be processed are combined to obtain the target newly added image corresponding to the target image to be processed.

2. The method according to claim 1, characterized in that, After extracting the target object from the target image to be processed, the method further includes: Extract the foreground color features of the target object in the target image to be processed; The step of generating a new background image for the target image based on the associated processed image of the target image to be processed includes: The target reference foreground color is determined based on the foreground color features of the target object in the target image to be processed; The image color of the associated processed image is adjusted according to the target reference foreground color to obtain the newly added background image.

3. The method according to claim 1, characterized in that, Before generating the new background image of the target image based on the associated processed image of the target image, the method further includes: Determine the associated candidate images of the target image to be processed; Randomly select one image from the associated candidate images as the associated processing image of the target image to be processed; Adjust the image size of the associated processing image based on the image size of the target image to be processed.

4. The method according to claim 2, characterized in that, Determining the target reference foreground color based on the foreground color features of the target object in the target image to be processed includes: The pixel color data of the target object region in the target image to be processed is determined based on the foreground color features of the target object in the target image to be processed. The pixel color data of the target object region is statistically analyzed to obtain the statistical results of the pixel color data of the target object region. The statistical results of the pixel color data of the target object region are filtered to obtain a candidate color set for the target object region; Randomly select a color from the candidate color set of the target object region as the target reference foreground color.

5. The method according to claim 2, characterized in that, The step of adjusting the image color of the associated processed image based on the target reference foreground color to obtain the newly added background image includes: Calculate the average image color of the associated processed image; Calculate the color offset between the target reference foreground color and the average image color of the associated processed image; The image color of the associated image is shifted according to the color offset to obtain the new background image.

6. The method according to claim 5, characterized in that, The step of shifting the image color of the associated image according to the color shift amount to obtain the new background image includes: The difference between the image color of the associated processed image and the color offset is calculated to obtain the current background image color; If the value of the current background image color exceeds a preset color threshold, the value of the current background image color is truncated.

7. The method according to claim 1, characterized in that, The step of synthesizing the target object in the target image to be processed and the newly added background image of the target image to obtain the target newly added image corresponding to the target image to be processed includes: Determine the pixel correspondence between the target object in the target image to be processed and the pixel values ​​in the newly added background image of the target image to be processed; Based on the pixel correspondence, the pixel values ​​of the background to be processed in the target image to be processed, excluding the target object, are replaced with the pixel values ​​corresponding to the newly added background image, thereby obtaining the target newly added image corresponding to the target image to be processed.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that is executed by the at least one processor to enable the at least one processor to perform the image data augmentation method according to any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the image data augmentation method according to any one of claims 1-7.

10. A computer program product comprising a computer program / instructions, wherein, When the computer program / instructions are executed by the processor, they implement the image data augmentation method according to any one of claims 1-7.

Images