Image amplification device and method

Abstract

An image augmentation apparatus includes a storage module and a processing module electrically connected to the storage module. The storage module stores at least one image generation model for obtaining an augmented image of the image to be processed based on an image to be processed. The processing module obtains a labeled feature vector based on annotation information for distinguishing a foreground and a background in the image to be processed. The processing module uses the image generation model to obtain a target augmented image corresponding to the image to be processed, based on the labeled feature vector, the image to be processed, and a target modification string for adjusting the foreground or background in the image to be processed.

Description

[Technical Field]

[0001] This invention relates to an image augmentation apparatus and method, and more particularly to an image augmentation apparatus and method for generating an augmented image of an image. [Previous Technology]

[0002] In the financial industry and other fields requiring high security, verifying user identity is a crucial procedure. With technological advancements, artificial intelligence (AI) technology is increasingly being used to identify users and confirm whether they are the individuals the system identifies them as. These AI identification systems can significantly improve the accuracy and efficiency of identity verification while reducing human error in traditional manual verification processes. However, to build accurate and reliable identity recognition models, AI systems need to collect a large amount of user image data as training material for the model.

[0003] However, in practice, users often find it difficult or are unwilling to provide sufficient image data themselves, which leads to a bottleneck in the model training process due to a lack of diverse data. Therefore, how to effectively augment image data based on a small number of user images has become a key technical challenge in training efficient and accurate AI recognition models.

[0004] To address the aforementioned issues, there is an urgent need to propose a method for efficient image augmentation based on a small amount of user image data. This method can generate diverse and highly realistic image data. This not only reduces data collection costs but also significantly improves the training effect of AI recognition models, thus enabling more accurate application in high-requirement identity verification scenarios such as the financial industry. [Summary of the Invention]

[0005] Therefore, the object of the present invention is to provide an image augmentation method for generating an augmented image of an image.

[0006] Thus, the image augmentation method of the present invention is implemented by an image augmentation device, which stores at least an image generation model for obtaining an augmented image of the image to be processed based on an image to be processed. The image augmentation method includes a step (A) and a step (B).

[0007] Step (A) is to obtain a labeled feature vector by means of the image augmentation device based on a labeling information used to distinguish one foreground and one background in the image to be processed.

[0008] Step (B) is to use the image augmentation device to obtain a target augmented image corresponding to the image to be processed based on the labeled feature vector, the image to be processed, and a target modification string used to adjust the foreground or background in the image to be processed, using the image generation model.

[0009] Another object of the present invention is to provide an image augmentation apparatus for generating an augmented image of an image.

[0010] Therefore, the image augmentation apparatus of the present invention includes a storage module and a processing module electrically connected to the storage module.

[0011] The storage module stores at least one image generation model for obtaining an augmented image of the image to be processed based on an image to be processed.

[0012] The processing module uses the image generation model to obtain a target augmented image corresponding to the image to be processed, based on the labeled feature vector, the image to be processed, and a target modification string used to adjust the foreground or background in the image to be processed.

[0013] The advantage of the present invention is that: the image augmentation device generates a labeled feature vector for distinguishing the foreground or the background in the image to be processed; then, the image generation model is used to generate a target augmented image that corresponds to the image to be processed and conforms to the description of the target modifier string based on the labeled feature vector, the target modifier string for adjusting the foreground or the background in the image to be processed, and the image to be processed; and the target augmented image is used as training data.

Implementation Method

[0014] Before the present invention is described in detail, it should be noted that similar elements are represented by the same numbers in the following description.

[0015] Referring to FIG1, an embodiment of the image augmentation device 1 of the present invention includes a storage module 11 and a processing module 12 electrically connected to the storage module 11.

[0016] The storage module 11 stores an encoder, a multimodal large language model, and an image generation model (img2img model). The encoder converts information to be processed into a feature vector of a fixed dimension to be analyzed. The multimodal large language model generates a modifier string for adjusting the image to be analyzed, based at least on an image to be analyzed and a prompt string for generating a string for adjusting the image to be analyzed. The image generation model obtains an augmented image of the image to be processed, based at least on the image to be processed.

[0017] It is worth noting that, in this embodiment, the encoder includes an autoencoder or a clip image encoder, but is not limited thereto. The multimodal large language model includes Lava, GPT-4 Vision, mPLUG-Owl, or VisualGLM, but is not limited thereto. The image generation model includes stable diffusion img2img, but is not limited thereto.

[0018] The image augmentation device 1 may be a tablet computer, a laptop computer, a smartphone or a personal computer, but is not limited thereto.

[0019] Referring to Figures 2 and 3, the following will describe the operation details of each component in the image amplification device 1 by performing an image amplification method in conjunction with this embodiment of the image amplification device 1. The image amplification method includes steps S01 to S03.

[0020] In step S01, the processing module 12 obtains a labeled feature vector using the encoder based on annotation information used to distinguish between a foreground and a background in an image to be processed. Specifically, the annotation information includes mask information about the image to be processed, as shown in Figure 3. The mask information is used to indicate the pixel positions of m pixels belonging to the foreground in the image to be processed, and to indicate the pixel positions of n pixels belonging to the background in the image to be processed. The number of pixels in the image to be processed is m+n, where m>0 and n>0. Therefore, different sizes of the image to be processed (i.e., different numbers of pixels) correspond to the mask information about their own number of pixels. To further explain, the encoder is used to reduce the dimensionality of the annotation information corresponding to the image to be processed of different sizes, so as to convert it into a fixed-dimensional labeled feature vector. In this way, the image augmentation method can be applied to images to be processed of various sizes. Those skilled in the art can obtain the encoder by preparing multiple training annotation information in advance and using an autoencoder. Furthermore, in this embodiment, the foreground includes a person, and the background includes the area in the image to be processed other than the person, but is not limited thereto.

[0021] In step S02, the processing module 12, based on the labeled feature vector, the image to be processed, and the prompt string used to command the multimodal large language model to generate a string that adjusts the foreground or background in the image to be processed and conforms to a specified format, obtains the target modification string using the multimodal large language model. The specified format is defined by the designer. Specifically, the prompt string is "Please generate a modification string that adjusts the facial expressions, age, clothing, hairstyle, background, background light source, or background effects of the input image and conforms to the specified format," but is not limited to this.

[0022] In step S03, the processing module 12 uses the image generation model to obtain a target augmented image corresponding to the image to be processed, based on the labeled feature vector, the image to be processed, and the target modification string used to adjust the foreground or background in the image to be processed. In summary, each time the processing module 12 performs the image augmentation method on the image to be processed in step S02, it generates different and diverse target modification strings. Then, in step S03, the image generation model can generate the corresponding target augmented image based on the current target modification string. In this way, after performing the image augmentation method multiple times, the processing module 12 can obtain target augmented images that correspond to the image to be processed and are different from each other.

[0023] In summary, the image augmentation device 1 of the present invention generates a labeled feature vector for distinguishing the foreground or the background in the image to be processed, then uses the multimodal large language model to generate a target modifier string to adjust the foreground or the background in the image to be processed, and finally uses the image generation model to generate a target augmented image that corresponds to the image to be processed and conforms to the description of the target modifier string based on the labeled feature vector, the target modifier string for adjusting the foreground or the background in the image to be processed, and the image to be processed, and uses the target augmented image as training data, thus effectively achieving the purpose of the present invention.

[0024] However, the above description is only an embodiment of the present invention and should not be construed as limiting the scope of the present invention. Any simple equivalent changes and modifications made in accordance with the scope of the patent application and the contents of the patent specification of the present invention shall still fall within the scope of the patent of the present invention. [Simplified Explanation of the Diagram]

[0025] Other features and effects of the present invention will be clearly presented in the embodiments with reference to the drawings, wherein: FIG1 is a block diagram illustrating an embodiment of the image augmentation apparatus of the present invention; FIG2 is a flowchart illustrating an image augmentation method performed by the embodiment of the image augmentation apparatus of the present invention; and FIG3 is a schematic diagram illustrating an image to be processed and a labeling information corresponding to the image to be processed in the image augmentation method performed by the embodiment.

Claims

1. An image augmentation method, implemented by an image augmentation device storing an image generation model, a multimodal large language model, and an encoder, wherein the image generation model is used to obtain an augmented image of the image to be processed based on at least one image to be processed, the multimodal large language model is used to generate a modifier string for adjusting the image to be analyzed based on at least one image to be analyzed and a prompt string for generating a string for adjusting the image to be analyzed, and the encoder is used to convert information to be processed into a feature vector of a fixed dimension, the image augmentation method comprising the following steps: (A) using the image augmentation device, obtaining a labeled feature vector based on labeled information for distinguishing a foreground and a background in an image to be processed using the encoder; (B) Using the image augmentation device, based on the labeled feature vector, the image to be processed, and the prompt string used to command the multimodal large language model to generate a string that adjusts the foreground or background in the image to be processed, a target modification string for adjusting the foreground or background in the image to be processed is obtained using the multimodal large language model; and (C) Using the image augmentation device, based on the labeled feature vector, the image to be processed, and the target modification string, a target augmented image corresponding to the image to be processed is obtained using the image generation model.

2. The image augmentation method as described in claim 1, wherein, In step (A), the annotation information includes masking information about the image to be processed. The masking information is used to indicate the pixel positions of m pixels belonging to the foreground in the image to be processed, and to indicate the pixel positions of n pixels belonging to the background in the image to be processed. The number of pixels in the image to be processed is m+n, where m>0 and n>0.

3. An image augmentation apparatus, comprising: a storage module storing an image generation model, a multimodal large language model, and an encoder, wherein the image generation model is used to obtain an augmented image of the image to be processed based on at least one image to be processed; the multimodal large language model is used to generate a modifier string for adjusting the image to be analyzed based on at least one image to be analyzed and a prompt string for generating a string for adjusting the image to be analyzed; and the encoder is used to convert information to be processed into a feature vector of a fixed dimension; and a processing module electrically connected to the storage module; wherein... The processing module obtains a labeled feature vector using the encoder based on annotation information used to distinguish between a foreground and a background in an image to be processed. Based on the labeled feature vector, the image to be processed, and a prompt string used to instruct the multimodal large language model to generate a string that adjusts the foreground or background in the image to be processed, the processing module obtains a target modification string used to adjust the foreground or background in the image to be processed using the multimodal large language model. Based on the labeled feature vector, the image to be processed, and the target modification string, the processing module uses the image generation model to obtain a target augmented image corresponding to the image to be processed.

4. The image augmentation apparatus as claimed in claim 3, wherein, The annotation information includes masking information about the image to be processed. The masking information is used to indicate the pixel positions of m pixels belonging to the foreground in the image to be processed, and to indicate the pixel positions of n pixels belonging to the background in the image to be processed. The number of pixels in the image to be processed is m+n, where m>0 and n>0.

Images