Systems and methods for training machine models using augmented data

Abstract

Systems and methods for training machine models using augmented data. One example method includes identifying a set of images captured simultaneously by a set of cameras fixed to one or more image collection systems. For each image in the set, a training output for that image is identified. For one or more images in the set, an augmented set of augmented images is generated. Generating the augmented images includes modifying the images using image manipulation functions that preserve camera attributes. The augmented training images are associated with the training output of the images. A set of parameters for a predictive computer model is trained based on a training set of images including the images and the set of augmented images to predict the training output.

Description

[0001] This application is a divisional application of the invention patent application with international application number PCT / US2019 / 055683, international application date of October 10, 2019, entered the Chinese national phase on May 18, 2021, Chinese national application number 201980075941.2, and invention title "System and method for training machine models using augmented data".

[0002] Cross-references to related applications

[0003] This application claims priority to U.S. Provisional Application No. 62 / 744,534, filed October 11, 2018, entitled “TRAINING MACHINE MODELS WITH DATA AUGMENTATION THAT RETAINS SENSOR CHARACTERISTICS”. The entire contents of U.S. Provisional Application No. 62 / 744,534 are incorporated herein by reference. Background Technology

[0004] Embodiments of the present invention generally relate to systems and methods for training data in a machine learning environment, and more specifically to augmenting training data by including additional data such as sensor characteristics in the training dataset.

[0005] In typical machine learning applications, data can be augmented in various ways to avoid overfitting the model to the characteristics of the capture devices used to acquire the training data. For example, in a typical collection of images used to train a computer model, the images may represent objects captured using many different capture environments with varying sensor characteristics relative to the captured objects. For instance, such images may be captured using a variety of sensor characteristics, such as different scales (e.g., significantly different distances within the image), using various focal lengths, using various lens types, using different pre- or post-processing software environments, sensor array hardware, etc. These sensors may also differ in various external parameters, such as the position and orientation of the imaging sensor relative to the environment when capturing the image. All these different types of sensor characteristics can cause the captured images to appear differently and in diverse ways throughout the image collection, making it more difficult to train the computer model correctly.

[0006] Many applications of neural networks learn from data captured under a variety of conditions and are deployed on a wide range of sensor configurations (e.g., applications running on various types of mobile phones). To address the differences between sensors used to capture images, developers can use modifications such as flipping, rotating, or cropping images to augment the image training data, thereby generalizing the developed model to camera properties such as focal length, axis skew, position, and rotation.

[0007] To address these variations and deploy trained networks across a variety of sources, training data can be augmented or manipulated to increase the robustness of the trained model. However, these methods often prevent the model from learning effectively for any particular camera configuration by applying transformations that modify camera attributes in the augmented images. Summary of the Invention

[0008] One embodiment is a method for training a set of parameters for a predictive computer model. This embodiment may include: identifying a set of images captured by a set of cameras while fixed to one or more image collection systems; for each image in the set of images, identifying the training output of the image; for one or more images in the set of images, generating a set of augmented images of augmented images by modifying the images using image manipulation functions that preserve the camera properties of the images, and associating the augmented training images with the training output of the images; and training the set of parameters of the predictive computer model to predict the training output based on an image training set including the images and the set of augmented images.

[0009] Another embodiment may include a system having one or more processors and a non-transitory computer storage medium storing instructions that, when executed by the one or more processors, cause the processors to perform operations including: identifying a set of images captured by a set of cameras while fixed to one or more image acquisition systems; for each image in the set of images, identifying a training output of the image; for one or more images in the set of images, generating an augmented set of augmented images of an augmented set of images by modifying the images using image manipulation functions that preserve the camera properties of the images, and associating the augmented training images with the training output of the images; and training the set of parameters of a predictive computer model based on the image training set including the images and the set of augmented images to predict the training output.

[0010] Another embodiment may include a non-transitory computer-readable medium having instructions for execution by a processor, which, when executed by the processor, cause the processor to: identify a set of images captured by a set of cameras while fixed to one or more image acquisition systems; for each image in the set of images, identify a training output of the image; for one or more images in the set of images, generate an augmented image of an augmented image of an augmented image by modifying the image using image manipulation functions that preserve the camera properties of the image, and associate the augmented training image with the training output of the image; and train a computer model to learn to predict the training output based on an image training set including the image and the set of augmented images. Attached Figure Description

[0011] Figure 1 This is a block diagram of an environment for training and deploying computer models according to one embodiment.

[0012] Figure 2 Example images captured using the same camera features are shown.

[0013] Figure 3 This is a block diagram of the components of a model training system according to one embodiment.

[0014] Figure 4 This is a data flow diagram illustrating an example of generating an augmented image based on labeled training images according to one embodiment.

[0015] The accompanying drawings illustrate various embodiments of the invention for illustrative purposes only. Those skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods shown herein can be employed without departing from the principles of the invention described herein. Detailed Implementation

[0016] One embodiment is a system that trains a computer model using an image that has been augmented to preserve the camera properties of the original captured image. These camera properties can include intrinsic or extrinsic attributes of the camera. Such intrinsic attributes can include characteristics of the sensor itself, such as dynamic range, field of view, focal length, and lens distortion. Extrinsic attributes can describe the camera's configuration relative to the capture environment, such as the camera's angle, scale, or pose.

[0017] These inherent and external properties can affect the camera's view relative to objects and other aspects captured in the image, as well as artifacts and other effects (such as static objects appearing in the camera's view due to their position on the device or system). For example, a camera mounted on a vehicle may include the car's hood as part of its view, which will appear in many images and for all cameras in this configuration mounted in the same way on the same model of car. As another example, these camera properties can also include reflections emitted from objects within the camera's view. Reflections can be a consistent type of property that is included in many images captured by the camera.

[0018] By training a data model while preserving, saving, storing, or using the camera properties of images, and still adding augmented images to the training data, the resulting model can be useful across many different devices with the same camera properties. Moreover, augmentation can provide generality and greater robustness to model predictions, especially when images are blurred, occluded, or otherwise fail to provide a clear view of detectable objects. These methods can be particularly useful for object detection and autonomous vehicles. This approach may also be beneficial in other situations where the same camera configuration can be deployed across many devices. Since these devices may have a consistent set of sensors in a consistent orientation, training data can be collected using a given configuration, the model can be trained using augmented data from the collected training data, and the trained model can be deployed to devices with the same configuration. Therefore, these techniques avoid unnecessary generalizations in such cases and allow generalization of other variables through some data augmentation.

[0019] To preserve camera attributes, image manipulation functions used to generate augmented images are functions designed to maintain these attributes. For example, these manipulations can avoid affecting the camera's angle, scale, or pose relative to the capture environment. In this embodiment, images augmented by image manipulation functions that affect camera attributes are not used during training. Image manipulation functions that can be used to preserve camera attributes include cropping, chroma / saturation / value jitter, salt-and-pepper noise, and domain shifting (e.g., day / night modification). Functions that can modify camera attributes and are therefore not used in some embodiments include cropping, padding, flipping (horizontal or vertical), or affine transformations (such as shearing, rotation, translation, and tilting).

[0020] As another example, an augmented image can be created using a "cut" function that removes a portion of the original image. The removed portion can then be replaced with other image content (such as specified colors, blur, noise, or another image). The number, size, area, and replacement content of the cut can vary and can be based on image labels (e.g., regions of interest or bounding boxes of objects in the image).

[0021] Therefore, a computer model can be trained using images and augmented images, and the computer model can be assigned to a device with the camera characteristics that capture the images for use in sensor analysis. In particular, this data augmentation and model training can be used for models trained to detect objects or object bounding boxes in images.

[0022] Figure 1 This is an environment for training and deploying computer models according to one embodiment. One or more image collection systems 140 capture images that can be used by a model training system when training a computer model, which can be deployed and used by a model application system. These systems are connected via a network 120 (such as the Internet), which represents various wireless or wired communication links through which these devices communicate.

[0023] Model training system 130 trains a computer model with a set of trainable parameters to predict output given a set of inputs. In this example, model training system 130 typically trains the model based on image input to generate outputs that predict information about the image. For example, in various embodiments, these outputs may identify objects in the image (identified by bounding boxes or segments), identify conditions of the image (e.g., time of day, weather), or other labels or descriptors of the image.

[0024] Although images are used as an example type of sensor data in this paper for convenience, the augmentation and model development described herein can be applied to a variety of sensor types to augment the training data captured from these sensors while preserving sensor configuration characteristics.

[0025] Image collection system 140 has a set of sensors that capture information from the environment. Although one image collection system 140 is shown, many image collection systems 140 can capture images for model training system 130. The sensors used in image collection system 140 have the same or substantially the same sensor characteristics as those in image collection system 140. In one embodiment, the image collection system is a vehicle or other system that moves through an environment and captures images of the environment with a camera. Image collection system 140 can be manually operated or can be partially or fully automated. Thus, as image collection system 140 traverses the environment, it can capture images of the environment and transmit them to model training system 130.

[0026] Model application system 110 is a system with a set of sensors that have the same or substantially the same sensor characteristics as the image acquisition system. In some examples, model application system 110 also functions as image acquisition system 130 and provides the captured sensor data (e.g., images) to model training system 130 for use as additional training data. Model application system 110 receives a trained model from model training system 130 and uses the model with the data sensed by its sensors. Because the images captured from image acquisition system 140 and model application system 110 have the same camera configuration, model application system 110 can capture its environment in the same manner and from the same (or substantially similar) angle as the image acquisition system. After applying the model, model application system 110 can use the model's output for various purposes. For example, when model application system 110 is a vehicle, the model can predict the presence of objects in the images, and model application system 110 can use it as part of a safety system or an autonomous (or semi-autonomous) control system.

[0027] Figure 2Example images captured using the same camera characteristics are shown. In this example, image 200A was captured by a camera on image collection system 130. Another image 200B could also be captured by image collection system 130, which may be the same or a different image collection system 130. When capturing different environments and different objects within those environments, these images retain camera properties relative to the captured environment. Camera properties refer to the camera's configuration and orientation properties that affect how the environment appears in the camera. For example, these camera properties may include the camera's angle, scale, and pose (e.g., viewing position) relative to the environment. Modifying the camera's angle, scale, or position relative to the same environment in which the image was captured will cause the image of the environment to change. For example, a camera placed at a higher position will view an object from a different height and will show parts of the object that differ from those at a lower position. Similarly, these images include consistent artifacts and effects in the image due to camera configurations that are not part of the environment being analyzed. For example, both images 200A and 200B include glare and other effects from a windshield, an object on the lower right side of the image obscuring the environment, and the windshield obscuring the bottom of the image. Therefore, images captured from the same camera characteristics typically exhibit the same artifacts and distortions, and capture the environment in the same way.

[0028] Figure 3 Components of a model training system 130 according to one embodiment are shown. The model training system includes various modules for training a computer model and a data storage unit. The model training system 130 trains a model for use by a model application system 110 by augmenting images from an image collection system 140 to improve the model's generalization. The augmented images are generated using image manipulation functions that do not affect (e.g., preserve) the camera configuration of the images. This allows for more efficient modeling while allowing generalization of the model parameters, thereby more selectively avoiding overfitting to aspects of images that may differ between images, while allowing the model parameters to learn more closely the weights associated with consistent camera characteristics.

[0029] The model training system includes a data input module 310 that receives images from the image collection system 140. The data input module 310 can store these images in an image data storage unit 350. The data input module 310 can receive images generated or provided by the data collection system 140, or it can request images from the image collection system 140.

[0030] The labeling module 320 can identify labels or apply labels to images in the image data 350. In some examples, the images may already have the identified characteristics. Labels can also represent data that will be predicted or output by a trained model. For example, a label can specify a particular object in the environment shown in the image, or it can include a descriptor or "label" associated with the image. Depending on the application of the model, the label can represent this information in various ways. For example, an object can be associated with a bounding box within the image, or the object can be segmented from other parts of the image. Thus, the labeled image can represent the ground truth on which the training model is based. Images can be labeled in any suitable way, and typically, images can be labeled through a supervised labeling process (e.g., by a user viewing the image and assigning a label to it). These labels can then be associated with images in the image data storage unit 350.

[0031] Image augmentation module 330 can generate additional images based on images captured by image collection system 140. These images can be generated as part of the training pipeline of model training module 340, or the augmented images can be generated before training begins in model training module 340. Augmented images can be generated based on images captured by image collection system 140.

[0032] Figure 4 An example of augmented image generation based on labeled training image 400 is shown according to one embodiment. The labeled training image may be an image captured by image collection system 140. Training image 410 may include unaugmented training image 410A having an associated training output 420A corresponding to the labeled data in labeled training image 400.

[0033] Image augmentation module 330 generates augmented images by applying image manipulation functions to labeled training images 400. The image manipulation functions generate modified versions of the labeled training images 400 to alter the characteristics of the images used to train the model. The image manipulation functions used to generate the training images preserve the camera properties of the labeled training images 400. Therefore, when capturing an environment that may be consistent across various devices, the manipulation functions can preserve the scale, perspective, orientation, and other characteristics of the view of the environment that may be affected by the camera's physical capture characteristics or camera position. Thus, image manipulation functions can affect the visibility or sharpness of objects or other features in the scene, but not the position or size of objects in the image. Examples of image manipulation functions that can be applied to preserve camera characteristics include cropping, jitter (e.g., for chroma, saturation, or color values), salt-and-pepper noise (introducing black and white points), blurring, and domain shifting. One or more of these image manipulation functions can be combined to generate augmented images. Cropping is an image manipulation function that removes a portion of an image and replaces the removed portion with other image content. Domain transfer refers to image manipulation functions that modify an image to correspond to another environmental condition within the image. For example, an image captured during the day can be modified to approximate how the image would look at night, or an image captured in sunlight can be modified to add the effect of rain or snow.

[0034] These augmented images can be associated with the same training output as the labeled training images 400. Figure 4 In the example shown, the augmented image 410B is generated by applying a cropping technique to the labeled training image 400, and the augmented image 410B can be associated with the training output 420B. Similarly, to generate the training image 410C, multiple cropping techniques are applied to modify different parts of the image. In this example, the cropping techniques applied to generate the training image 410C fill the cropped regions of the image with different patterns.

[0035] In various embodiments, various parameters and configurations can be applied to the cuts, which can vary based on the position of the training image and the training output within the image. Therefore, the number, size, location, and replaced image content of the cuts can vary in different embodiments based on the position of the training output. As an example, the cut function can apply multiple cuts of similar sizes, or it can apply multiple cuts of varying semi-randomized sizes within a range. By using multiple cuts and varying their sizes, the cuts can more closely simulate the effect of real-world obstacles (of various sizes) when viewing an object, and can prevent the trained model from learning to compensate for cuts of any particular size.

[0036] The size range of the cut can be based on a portion of the size of the object or other label within the image. For example, the cut might not exceed 40% of the object's bounding box size in the image, or be smaller than the size of the smallest object bounding box. This ensures that the cut does not completely occlude the target object, and therefore ensures that the image continues to include image data of objects from which the model can learn. The number of cuts can also be randomized and can be chosen from distributions such as uniform, Gaussian, or exponential distributions.

[0037] Additionally, the location of the cut can be selected based on the position of the object in the image. This can provide some, but not excessive, overlap with the bounding box. The intersection between the object and the cut region can be measured by the portion of the object that is replaced by the cut, or by combining the intersection (IoU), which is measured by dividing the intersection of the object and the cut region by the union of the object region and the cut region. For example, the cut region can be positioned to have an intersection on a union value in the range of 20% to 50%. By including some, but not overwhelming, amount of object in the cut, the cut can thus create more “challenging” instances that partially occlude the object without removing too much relevant image data. Similarly, certain parts of the image can be selected for cut based on the camera’s expected view in the image. For example, the cut could be primarily located in the lower half of the image or in the center of the image, as the bottom can often include artifacts that are always present, while the center of the image can be the region of most interest (e.g., for vehicles, typically the direction of travel).

[0038] The replacement image data used for the cut-off region can be a solid color (e.g., a constant) or another pattern, such as Gaussian noise. As another example, to represent occlusion or other obstacles, the cut-off can be replaced with a patch of image data from another image with the same image type or label. Finally, the cut-off can be blended with the region near the cut-off, for example, using Poisson blending. By using various blending methods (such as background patching or blending), it is possible to ensure that the replacement data in the cut-off is more difficult to distinguish from the environment, and thus provides an example that is more similar to a real-world obstacle.

[0039] Despite Figure 4 The area is shown as a rectangular region, but in other embodiments, the cropping applied when generating the augmented image can vary in shape. After generating the augmented images 410B and 410C and associating the augmented images with the associated training outputs 420B and 420C, the image augmentation module 330 can add the images to the image data storage unit 350.

[0040] The model training module 340 trains a computer model based on images captured by the image collection system 140 and augmented images generated by the image augmentation module 330. These images can be used as an image training set for model training. In one embodiment, the machine learning model is a neural network model, such as a feedforward network, convolutional neural network (CNN), deep neural network (DNN), recurrent neural network (RNN), self-organizing map (SOM), etc., trained by the model training module 340 based on training data. After training, the computer model can be stored in the trained computer model storage unit 370. The model receives sensor data (e.g., images) as input and outputs output predictions based on the model's training. During model training, the model learns (or "trains") a set of parameters that predict the output based on an input image evaluated by a loss function of the training data. That is, during training, the training data is evaluated based on the current set of parameters to generate predictions. This prediction for the training input can be compared with a specified output (e.g., a label) to evaluate the loss (e.g., with a loss function), and the parameters can be modified by an optimization algorithm to optimize the set of parameters, thereby reducing the loss function. Although called "optimization," these algorithms can reduce overhead relative to a set of parameters, but may not guarantee finding the "optimal" values ​​of the parameters given a set of inputs. For example, gradient descent optimization algorithms may find local minima rather than global minima.

[0041] By training a computer model on augmented training data, the model can perform with improved accuracy when applied to sensor data from physical sensors operating in an environment with sensor characteristics that capture the data. Because augmentation preserves these characteristics, these sensor characteristics (e.g., camera characteristics) are represented in the images used during training. In one embodiment, the training data does not include augmented images generated by image manipulation functions that modify camera properties of the images, such as cropping, padding, flipping (vertical or horizontal), or applying affine transformations (e.g., shearing, rotation, translation, tilting) to the images.

[0042] After training, the model distribution module 380 can distribute the trained model to the system for application. Specifically, the model distribution module 380 can send the trained model (or its parameters) to the model application system 110 for use in detecting image features based on the sensors of the model application system 110. Therefore, predictions from the model can be used for the operation of the model application system 110, such as for object detection and control within the model application system 110.

[0043] For illustrative purposes, the foregoing description of embodiments of the invention has been provided; it is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Those skilled in the art will understand that many modifications and variations are possible based on the above disclosure.

[0044] Certain portions of this description describe embodiments of the invention based on algorithms and symbolic representations of operations on information. These descriptions and representations of algorithms are typically used by those skilled in the art of data processing to effectively convey the essence of their work to others skilled in the art. Although these operations are described functionally, computationally, or logically, they should be understood as being implemented through computer programs or equivalent circuits, microcode, etc. Furthermore, without loss of generality, it is sometimes convenient to arrange these operations as modules. The described operations and their associated modules can be embodied in software, firmware, hardware, or any combination thereof.

[0045] Any step, operation, or process described herein may be performed or implemented using one or more hardware or software modules, either alone or in combination with other devices. In one embodiment, the software module is implemented with a computer program product comprising a computer-readable medium containing computer program code that can be executed by a computer processor to perform any or all of the described steps, operations, or processes.

[0046] Embodiments of the invention may also relate to means (e.g., systems) for performing the operations described herein. Such means may be specifically configured for a desired purpose and / or may include general-purpose computing devices that are selectively activated or reconfigured by a computer program stored in a computer. A computing device may be a system or device of one or more processors and / or computer systems. Such computer programs may be stored in a non-transitory tangible computer-readable storage medium or in any type of medium suitable for storing electronic instructions that can be coupled to a computer system bus. Furthermore, any computing system mentioned in the specification may include a single processor or may be an architecture employing multiple processors to enhance computing power.

[0047] Embodiments of the present invention may also relate to products produced by the computational processes described herein. Such products may include information generated by the computational processes, wherein the information is stored on a non-transitory tangible computer-readable storage medium, and may include any embodiment of a computer program product or other combinations of data described herein.

[0048] Finally, the language used in this specification has been chosen primarily for readability and instructional purposes and may not have been chosen to depict or limit the subject matter of the invention. Therefore, it is intended that the scope of the invention is not limited by this detailed description, but rather by any claims based thereon. Thus, the disclosure of embodiments of the invention is intended to illustrate, rather than limit, the scope of the invention, which is set forth in the appended claims.

Claims

1. A method for training a set of parameters for a predictive computer model, the method comprising: Acquire multiple images and associated training outputs, the images being captured by cameras of one or more vehicles; For an individual image among the plurality of images, an augmented image is generated by modifying the individual image using one or more image manipulation functions that preserve the camera attributes of the individual image, such that the angle, scale, and / or pose associated with the individual image are preserved, wherein the augmented image is associated with the associated training output of the individual image. And the one or more image manipulation functions mentioned above include a cropping function that adjusts portions of the image based on regions of interest corresponding to artifacts that are always present in the image; as well as The predictive computer model is trained based at least on the individual image and the augmented image, wherein the trained predictive computer model is configured to predict the presence of objects in the input image for specific vehicle autonomy or semi-autonomy.

2. The method according to claim 1, wherein the region of interest corresponds to the direction of travel.

3. The method of claim 1, wherein the image is acquired via a specific camera, and wherein the artifact that is always present in the image is acquired via the specific camera.

4. The method of claim 1, wherein the portion of the image partially overlaps with an object in the image, the replacement data corresponding to the cut is a solid color, and the replacement data corresponding to the cut is blended with the region near the cut.

5. The method of claim 1, wherein the adjusted portion includes replacing image data.

6. The method of claim 5, wherein the replacement image data is a solid color or a pattern.

7. The method of claim 1, wherein the one or more vehicles and the particular vehicle have at least one camera, the at least one camera having the same configuration.

8. The method of claim 7, wherein the identical configuration indicates the same location and / or orientation with respect to the one or more vehicles and the particular vehicle.

9. A system for training a set of parameters of a predictive computer model, the system comprising one or more processors and a non-transitory computer storage medium storing instructions, the instructions causing the processors, when executed by the one or more processors, to: Acquire multiple images and associated training outputs, the images being captured by cameras of one or more vehicles; For an individual image among the plurality of images, an augmented image is generated by modifying the individual image using one or more image manipulation functions that preserve the camera attributes of the individual image, such that the angle, scale, and / or pose associated with the individual image are preserved, wherein the augmented image is associated with the associated training output of the individual image. And the one or more image manipulation functions mentioned above include a cropping function that adjusts portions of the image based on regions of interest corresponding to artifacts that are always present in the image; as well as The predictive computer model is trained based at least on the individual image and the augmented image, wherein the trained predictive computer model is configured to predict the presence of objects in the input image for specific vehicle autonomy or semi-autonomy.

10. The system of claim 9, wherein the region of interest corresponds to the direction of travel.

11. The system of claim 9, wherein the image is acquired via a specific camera, and wherein the artifacts that are always present in the image are acquired via the specific camera.

12. The system of claim 9, wherein the portion of the image partially overlaps with an object in the image, the replacement data corresponding to the cut is a solid color, and the replacement data corresponding to the cut is blended with the region near the cut.

13. The system of claim 9, wherein the adjusted portion includes replacement image data, the replacement image data including solid colors or patterns.

14. The system of claim 9, wherein the one or more vehicles and the particular vehicle have at least one camera, the at least one camera having the same configuration.

15. The system of claim 14, wherein the identical configuration indicates the same location and / or orientation with respect to the one or more vehicles and a particular vehicle.

16. A non-transitory computer-readable medium having instructions for execution by a processor, the instructions causing the processor, when executed by the processor, to: Acquire multiple images and associated training outputs, the images being captured by cameras of one or more vehicles; For an individual image among the plurality of images, an augmented image is generated by modifying the individual image using one or more image manipulation functions that preserve the camera attributes of the individual image, such that the angle, scale, and / or pose associated with the individual image are preserved, wherein the augmented image is associated with the associated training output of the individual image. And the one or more image manipulation functions mentioned above include a cropping function that adjusts portions of the image based on regions of interest corresponding to artifacts that are always present in the image; as well as The predictive computer model is trained based at least on the individual image and the augmented image, wherein the trained predictive computer model is configured to predict the presence of objects in the input image for specific vehicle autonomy or semi-autonomy.

17. The computer-readable medium of claim 16, wherein the region of interest corresponds to the direction of travel.

18. The computer-readable medium of claim 16, wherein the image is acquired via a specific camera, and wherein the artifact that is always present in the image is acquired via the specific camera.

19. The computer-readable medium of claim 16, wherein the portion of the image partially overlaps with an object in the image, the replacement data corresponding to the cut is a solid color, and the replacement data corresponding to the cut is blended with the region near the cut.

20. The computer-readable medium of claim 16, wherein the adjusted portion includes replacement image data, the replacement image data including solid colors or patterns.

Images