Systems and methods for training machine models with augmented data

JP2023162341A5Inactive Publication Date: 2024-11-11TESLA INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2023139431
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2018-10-11
Filing Date
2023-08-30
Publication Date
2024-11-11
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing machine learning models struggle to generalize across different sensor configurations due to variations in sensor characteristics such as focal length, position, and environmental conditions, leading to overfitting and reduced robustness when deployed on devices with varying camera setups.

Method used

A method and system for training computer models by preserving camera characteristics through image manipulation functions that maintain intrinsic and extrinsic properties, generating augmented images that simulate real-world sensor conditions, thereby enhancing model robustness and generalization.

Benefits of technology

The approach allows models to be trained effectively across devices with consistent camera configurations, improving prediction accuracy and generalization, particularly in applications like object detection and autonomous vehicles.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

To provide systems and methods for training machine models with augmented data.SOLUTION: A method in a system comprising one or more processors and non-transitory computer storage media storing instructions, the one or more processors executing the instructions perform are configured for: identifying a set of images captured by a set of cameras while affixed to one or more image collection systems; for each image in the set of images, identifying training output for the image; for one or more images in the set of images, modifying the image with an image manipulation function that maintains camera properties of the image so as to generate an augmented image for a set of augmented images, and associating the augmented training image with the training output of the image, thereby generating the augmented image for the set of augmented images; training a set of parameters of a predictive computer model to predict the training output on the basis of an image training set including the images and the set of augmented images.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] [Cross - Reference to Related Applications] This application claims priority to U.S. Provisional Patent Application No. 62 / 744,534, filed October 11, 2018 、entitled "TRAINING MACHINE MODELS WITH DATA AUGMENTATION THAT RETAINS SENSOR CHARAC TERISTICS". U.S. Provisional Patent Application No. 62 / 744,53 4 is hereby incorporated by reference in its entirety.

[0002] Embodiments of the present invention generally relate to systems and methods for training data in a machine learning environment, and more particularly to augmenting training data by including additional data such as sensor characteristics in a training dataset.

Background Art

[0003] In typical machine learning applications, data can be augmented in various ways to avoid the model from over - fitting to the characteristics of the capture devices used to obtain training data. For example, in a typical set of images used to train a computer model, the images can represent objects captured in many different capture environments that have various sensor characteristics with respect to the object being captured. For example, such images may be captured with various sensor characteristics such as various scales (e.g., significantly different distances within the image), various focus distances, various lens types, various pre - processing or post - processing, various software environments, sensor array hardware, etc. In this way, the images can be captured under different conditions. These sensors also determine the position of the imaging sensor relative to the environment when the image is captured. Various extrinsic parameters such as direction may also differ. All of the sensor characteristics of the device capture images within a set of multiple different images. By presenting the body in different and varied ways, the computer model can be properly trained. This could make things more difficult.

[0004] Many applications of neural networks involve capturing data under various conditions. It learns from data and works with various different sensor configurations (for example, on multiple types of mobile phones). (In a moving app) it is deployed. Difference in the sensor used to capture the image. To take differences into account, developers consider factors such as focal length, axial skew, position, and rotation. Regarding camera properties, the developed model can be generalized by inverting, rotating, or cropping the image. Image training data can be expanded through modifications such as those mentioned above.

[0005] To take these fluctuations into account and deploy the trained network on various sources Furthermore, the robustness of the trained model can be increased by expanding or manipulating the training data. However, these techniques typically alter camera characteristics in the augmented image. By applying a transformation, the model can effectively learn about any particular camera configuration. It hinders learning. [Overview of the project] [Means for solving the problem]

[0006] One embodiment is a method for training a set of parameters for a predictive computer model. Yes. This embodiment involves a camera being fixed to one or more image acquisition systems. Identifying the set of images captured by the set, and each of the images within the set Regarding images, the training output of an image is identified, and one or more images in a set of images Regarding the image, by modifying the image using an image manipulation function that maintains the camera characteristics of the image, This generates an augmented image from a set of augmented images, and associates the augmented training image with the training output of the image. This generates an augmented image of a set of augmented images, and the set of images and augmented images A predictive computer model that predicts the training output based on an image training set that includes the set. This can include training a set of parameters.

[0007] An additional embodiment includes one or more processors and non-temporary containers that store instructions. A system having a computer storage medium, wherein instructions are transmitted to one or more processors. Therefore, when executed, the processor is fixed to one or more image acquisition systems. Identifying the set of images captured by the set of cameras during the process, and the images For each image in the set, identify the training output of the image, and one of the images in the set Alternatively, for multiple images, modify the images using an image manipulation function that maintains the camera characteristics of the images. By doing so, an augmented image is generated from the set of augmented images, and the augmented training image is used as the training output for the image. By associating with this, it is possible to generate an extended image of a set of extended images, and the image and Predictive Computing to Predict Training Outputs Based on an Image Training Set Including an Augmented Image Set The system performs actions that include training a set of parameters for a computer model. It can include...

[0008] Another embodiment is a non - transient computer - readable medium having instructions for execution by a processor, the instructions, when executed by the processor, cause the processor to identify a set of images captured by a set of cameras while fixed to one or more image collection systems, identify, for each image in the set of images, a training output of the image, generate an extended set of images by modifying an image using an image manipulation function that maintains camera characteristics of the image for one or more images in the set of images, generate an extended set of training images by associating the extended images with the training output of the image, and train a computer model to predict a training output based on an image training set that includes the set of images and the extended set of images. The non - transient computer - readable medium can include this.

Brief Description of the Drawings

[0009] [Figure 1] It is a block diagram of an environment for training and deploying a computer model according to an embodiment.

[0010] [Figure 2] It is a diagram showing exemplary images captured by the same camera characteristics.

[0011] [Figure 3] It is a block diagram of components of a model training system according to an embodiment.

[0012] [Figure 4] It is a data flow diagram showing an example of generating extended images based on labeled training images according to an embodiment.

Mode for Carrying Out the Invention

[0013] The drawings illustrate various embodiments of the present invention for illustrative purposes only. Those skilled in the art will be able to read the following discussion. Therefore, without departing from the principles of the present invention as described herein, the structure shown herein It will be readily apparent that alternative embodiments of the construction and method may be available.

[0014] One embodiment is extended to maintain the camera characteristics of the originally captured image. This is a system that trains computer models using images. These camera characteristics are: The camera may have intrinsic or extrinsic properties. Such intrinsic properties include This includes characteristics of the sensor itself, such as dynamic range, field of view, focal length, and lens distortion. This is possible. Extrinsic characteristics include the camera angle, scale, or posture, etc., which are captured. This can represent the camera configuration for different environments.

[0015] These intrinsic and extrinsic characteristics are related to the objects and other aspects captured in the image. , and appear within the camera's view due to its position on the device or system. This affects the camera view of artifacts and other effects, such as stationary objects. For example, a camera mounted on a vehicle captures many images as part of its view. And, for all cameras in that configuration that are mounted in the same way on the same model of car This may include the car hood that appears along with it. As another example, these camera characteristics are also, This may include reflections originating from objects within the camera's view. Reflections are captured by the camera. This could be one type of consistent characteristic that will be present in many of the images being created.

[0016] To train a data model, images are used to further add to the training data. By maintaining, saving, remembering, or using the camera characteristics of the image, the resulting The model can be useful across many different devices that have the same camera characteristics. Furthermore, the extension allows for the detection of objects, especially when the image is cloudy, obscured, or otherwise undetectable. If a clear view is not provided, generalization and greater robustness to model predictions will not be achieved. These methods can be provided. These techniques may be particularly useful for object detection and autonomous vehicles. This method is also useful in other situations where the same camera configuration may be deployed in many devices. It is possible. These devices may have a consistent set of sensors in a consistent orientation. Therefore, training data can be collected by a given configuration, and the model is collected The model can be trained using augmented data from the training data, and the trained model will have the same configuration. These techniques can be applied to devices that have [this feature]. Therefore, in this context, This leads to unnecessary generalizations and allows for the generalization of other variables with some kind of data expansion. Avoid expansion.

[0017] The image manipulation functions used to generate augmented images in order to maintain camera characteristics are: This function maintains camera characteristics. For example, these operations are tailored to the environment being captured. This can avoid affecting the camera angle, scale, or attitude. In the embodiment, Images enhanced by image manipulation functions that affect camera characteristics are not used for training. For example, image manipulation functions that may be used to maintain camera characteristics include cropping and hue. Includes / saturation / value jitter, salt-and-pepper effect, and domain migration (e.g., changing from day to night). Hmm. Camera characteristics may be changed, and therefore not used in some embodiments. The functions include trimming, padding, flipping (horizontal or vertical), or affine transformation (slash). This includes cross-sectional, rotational, translational, and skewed motions.

[0018] As a further example, an image can be expanded by using a "crop" function to remove a portion of the original image. This can be done. Then, the removed portion of the image will have the specified color, blur, noise, and It can be replaced with other image content, such as from another image. Number of crops, The size, area, and replacement content may be changed, and the image labels (for example, related to the image) may be changed. It may also be based on the cardiac region or the bounding box of an object.

[0019] Therefore, computer models can be trained using images and augmented images. The captured images are distributed to a device with camera characteristics, and the model is used for sensor analysis. It can be used for this purpose. In particular, this data augmentation and model training can be used for objects in images or This can be used with models trained to detect object bounding boxes. ru.

[0020] Figure 1 shows an environment for training and deploying a computer model according to one embodiment. One or more image acquisition systems 140 are deployed and used by the model application system. Can be used by a model training system when training a computer model that may be used. Capture images. These systems are designed to allow these devices to communicate through them. via a network 120 such as the Internet, which represents various wireless or wired communication links. Connected.

[0021] The model training system 130 is trainable to predict the output given a set of inputs. Train a computer model with a set of parameters. The model in this example... The training system 130 typically uses image input to generate output prediction information about an image. The model is trained based on force. For example, in various embodiments, these outputs are pictures. Objects within the image can be identified (bounding box or segmentation). Objects can be identified by one of these methods, and the state of the image (for example, time of day, weather can be identified). Alternatively, it can identify other tags or descriptors of the image.

[0022] For convenience, images are used as an exemplary type of sensor data in this specification. However, the extensions and model developments described herein maintain the sensor configuration characteristics while To expand the training data captured from the sensor, suitable for various types of sensors. It can be used.

[0023] The image acquisition system 140 captures information from the environment of the image acquisition system 140. It has a set of sensors. A single image acquisition system 140 is shown, but many images The collection system 140 can capture images for the model training system 130. Yes, it is possible. The sensor for the image acquisition system 140 spans the image acquisition system 140. The sensor has the same or substantially the same sensor characteristics. Image acquisition system in one embodiment The stem is a vehicle or other vehicle that moves within the environment and captures images of the environment with its camera. It is a system. The image acquisition system 140 may be operated manually or partially. Alternatively, it may be operated by a fully automated vehicle. Therefore, the image acquisition system As Tem 140 moves through the environment, the image acquisition system 140 captures images of the environment. This data can then be transmitted to the model training system 130.

[0024] The model application system 110 has the same or substantially the same sensor characteristics as the image acquisition system. It is a system having a set of sensors. In some examples, the model application system M110 also functions as an image acquisition system 130, and the sensor data that is captured ( For example, to use an image as further training data, the model training system 130 The model application system 110 provides the model trained from the model training system 130. Dell receives data from its sensors and uses the model along with the data it senses. Image acquisition Images captured from system 140 and model-applied system 110 are from the same camera. Because it has the same configuration, the model application system 110 is the same as the image acquisition system. The environment can be captured from a viewpoint (or substantially equivalent). Apply the model. Afterward, the model application system 110 uses the model output for various purposes. This is possible. For example, if the model application system 110 is a vehicle, the model will be an object in the image. The presence of can be predicted, and this can be as part of a safety system, or autonomous (or It can be used by the model application system 110 as part of a semi-autonomous control system.

[0025] Figure 2 shows an exemplary image captured by the same camera characteristics. In this example, image 200A is captured by a camera on the image acquisition system 130. Another image 200B may also be captured by the image acquisition system 130. This image acquisition system may be the same image acquisition system, or different image acquisition systems It may also be stem 130. It captures different environments and different objects within those environments. During this time, these images maintain the camera characteristics for capturing the environment. The characteristics of the camera are the camera's configuration and orientation, which affect how the environment appears within the camera. This refers to the characteristics of the camera. For example, these camera characteristics include the camera angle relative to the environment, scale, This can include posture (e.g., viewing position) and the same environment in which the image is captured. Changing the camera angle, scale, or position relative to the environment will change the image of the environment. For example, a camera positioned higher up will view an object from a different height, and the underside of that object... It shows parts that are not in the same location. Similarly, these images are not part of the environment that should be analyzed. This includes consistent artifacts and effects within the image resulting from the camera configuration. For example, Both images 200A and 200B include glare and other effects from the windshield. The object in the lower right of the image obscures the environment, and the windshield obscures the bottom of the image. Therefore Images captured from the same camera characteristics typically exhibit the same artifacts and distortions. Present the image and capture the environment using the same method.

[0026] Figure 3 shows the components of a model training system 130 according to one embodiment. The system includes various modules and data stores for training computer models. The model training system 130 includes an image acquisition system to improve the generalization of the model. By expanding the image from M140, it is used by the model application system 110. Train a model that can do this. Augmented images do not affect the camera configuration of the image (for example, maintenance It is generated using the image manipulation function (which it possesses). This is because the model parameters are consistent with the camera. While enabling closer learning of weights related to characteristics, images that may differ between images This enables the generalization of model parameters, allowing for more selective avoidance of overfitting regarding image characteristics. This enables more effective modeling.

[0027] The model training system is a data input module that receives images from the image acquisition system 140. Includes module 310. The data input module 310 stores these images in the image data store 3 It can be stored in 50. The data input module 310 is in the data acquisition system 14 Images may be received as generated or provided by 0, or image collection You may request an image from system 140.

[0028] The labeling module 320 identifies labels in the image data 350 and It can be applied. In some cases, the image already has identified characteristics. This is also fine. Labels can also represent the data predicted or output by the trained model. This is possible. For example, a label can specify a particular object in the environment shown in the image. It can include descriptors or "tags" associated with the image. Depending on the context, labels can represent this information in various ways. For example, an object can be an image. It may be associated with an internal bounding box, or the object may be associated with other parts of the image. It may be segmented. Therefore, the labeled image is trained by the model. The ground truth can be represented by any appropriate means. Labeling is also acceptable, typically through a supervised labeling process (e.g., user Labelling is done by examining the image and specifying the label for the image. These labels may be associated with images in the image data store 350. It is possible to do so.

[0029] The image extension module 330 receives images captured by the image acquisition system 140. Additional images can be generated based on these images. These images are used in the model training module. These augmented images may be generated as part of a 340 training pipeline, or they may be generated as part of a training pipeline. The extended image may be generated in the model training module 340 before training begins. It can be generated based on images captured by the image acquisition system 140. ru.

[0030] Figure 4 shows an example of generating an augmented image based on a labeled training image 400 according to one embodiment. Labeled training images are images captured by the image acquisition system 140. This is acceptable. Training image 410 corresponds to the labeled data in labeled training image 400. Includes an unenhanced training image 410A having an associated training output 420A. It is possible.

[0031] The image extension module 330 applies image manipulation functions to the labeled training images 400. It generates an augmented image. The image manipulation function uses the characteristics of the image to train the model. To make the changes, a modified version of the 400 labeled training images is generated. The image manipulation function used for generation maintains the camera characteristics of the 400 labeled training images. Therefore, the operating functions capture an environment that can be consistent across various devices. When shooting, the results may be affected by the camera's physical capture characteristics or the camera's position. It is possible to maintain the scale, viewpoint, orientation, and other characteristics of the environment's view. Therefore, the image manipulation function can determine how other features of an object or environment are visible, These can affect how clearly they appear in the scene, but within the image It can be done without affecting the position or size of the object. Camera characteristics are maintained. Examples of image manipulation functions that may be applied include cropping, jitter (e.g., hue, saturation, or Includes color values, salt and pepper (introducing black and white dots), blur, and domain transitions. By combining and applying several of these image manipulation functions, an enhanced image can be generated. This is possible. Cropping involves removing a portion of an image and replacing the removed portion with other image content. This refers to image manipulation functions. Domain migration involves changing the image to accommodate different environmental conditions within the image. This refers to image manipulation functions that change the image. For example, changing how a daytime image looks at night. You can modify it to approximate, or images taken in sunny places, even in the rain. Alternatively, you can modify it to add a snow effect.

[0032] These augmented images can be associated with the same training output as the labeled training images 400. In the example shown in Figure 4, the extended image 410B is cropped to the labeled training image 400. By applying the extended image 410B, the extended image 410B can be associated with the training output 420B. Similarly, in order to generate the training image 410C, multiple parts of the image are modified. Number truncation is applied. In this example, it is applied to generate training image 410C. Cropping fills the cropped area of ​​the image with a different pattern.

[0033] In various embodiments, cropping is based on the training image and the position of the training output within the image. It may be applied using various parameters and configurations that can change accordingly. The number, size, position, and replacement image content of the extracts are different in different embodiments of the training output. It may change based on the position. For example, the cropping function may change the cropping function to multiple crops of similar size. You can apply a set of semi-randomized sizes within a certain range. You can apply several crops. Use multiple crops and change the size. By doing so, the cropping affects how (various sizes) real-world obstacles view the object. It is possible to simulate the effects more closely, and the trained model can perform any one characteristic This can prevent the system from learning to compensate for cropping to a fixed size.

[0034] The cropping size range is based on a portion of the size of objects or other labels within the image. This is also acceptable. For example, the cropping may be 40% of the size of the bounding box of the object in the image. It may be less than or equal to a percent, or smaller than the bounding box of the smallest object. This ensures that the cropping does not completely obscure the subject object, and therefore the image is Ensure that the model continues to include image data of objects from which it can learn. The number of cuts can also be randomized, uniformly distributed, Gaussian, or indexed. You may choose from distributions such as number distributions.

[0035] Furthermore, the cropping position may be selected based on the position of objects within the image. This can result in some, but not excessive, overlap with the bounding box. The intersection between the object and the cut-out region is the part of the object that is being replaced by the cut-out. Therefore, it may be measured, or the intersection of the object and the cut-out region may be used to determine the area of ​​the object and the cut-out region. The cross overunion (Io) can be measured by dividing by the union with the domain. It may be measured by U). For example, the cut area is within the range of 20% to 50%. They may be arranged to have a difference over union value. Therefore, how much to cut By including a small but significant amount of objects, cropping allows for the inclusion of the relevant image data. A more "difficult" example than partially obscuring an object without removing too much data. It can be created. Similarly, based on the camera's predicted view within the image, a specific image can be created. You can also choose to crop a portion. For example, the bottom is typically always an arch. While it may contain facts, the focus of the image is on the area of ​​greatest interest (for example, in the case of a vehicle, many In this case, the cropping may be the lower half of the image or the direction of travel of the vehicle. It can be located at the center of the image.

[0036] The replacement image data for the cropped area may be a solid color (for example, a constant color), Alternatively, it could be another pattern such as Gaussian noise. Another example is occlusion or other To represent an obstacle, cropping is done from another image with the same image type or label. Image data patches may be used as replacements. Finally, the cropping can be performed, for example, by Poisson synthesis. This allows for compositing with nearby areas of the cutout, such as background patches or composites. By using various synthesis techniques, these allow the replacement data within the cutout to be distinguished from the environment. This ensures that it is more difficult to do, and therefore provides examples that are more similar to obstacles in the real world. We can provide this.

[0037] Although shown as a rectangular region in Figure 4, the cuts applied when generating the extended image The trimmer may vary in shape in other embodiments. (See extended images 410B and 410C) After generating the image and associating the augmented image with the relevant training outputs 420B and 420C, the image augmentation is performed. Joule 330 can add these images to the image data store 350.

[0038] The model training module 340 uses images captured by the image acquisition system 140. Based on the image and the augmented image generated by the image augmentation module 330, the computer Train the model. These images will be used as the image training set for model training. This is possible. In one embodiment, the machine learning model is trained based on the training data. Feedforward networks, convolutional nuclei, trained by Joule 340. Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), Recurrent Neural networks (RNNs), self-organizing maps (SOMs), etc. This is a twerk model. After training, the computer model is a trained computer model. It can be stored in the TOA370. The model takes sensor data (e.g., images) as input and It receives the data and outputs output predictions according to the model's training. When training the model, the model The output is evaluated based on the input image, with the loss function of the training data determining the output. Learn (or "train") the set of parameters to predict. That is, during training, The training data is evaluated according to the current parameter set to generate predictions. The prediction about the force is specified to evaluate the loss (for example, using a loss function). The force (e.g., label) can be compared, and the parameters are designed to reduce the loss function. The parameter set can be modified through an optimization algorithm to optimize it. These algorithms are called "optimization," but they are not loss-inducing in terms of the parameter set. Losses can be reduced, but given a set of inputs, it is not possible to find the "optimal" value for the parameters. There are cases where this is not guaranteed. For example, gradient descent optimization algorithms are not globally efficient. It is possible to find the local minimum, not just the smallest value.

[0039] By training a computer model on augmented training data, the computer model The system operates in an environment with sensor characteristics of the captured data. When applied to sensor data from the source, it can be performed with improved accuracy. The extension maintains these characteristics, so these sensor characteristics (e.g., camera characteristics) are The data is represented within images used for training. In one embodiment, the training data is represented within images. Rimming, padding, inversion (vertical or horizontal), or affine transformation (e.g., shearing) Image manipulation devices that change the camera characteristics of an image, such as applying operations (rotation, translation, skew). It does not include augmented images generated by the function.

[0040] After training, the model distribution module 380 applies the trained model. The model can be distributed to the system. In particular, the model distribution module 380 To be used when detecting image characteristics based on the sensor of the model application system 110 The trained model (or its parameters) is sent to the model application system 110. Therefore, predictions from the model can be applied to the operation of the model application system 110. For example, it can be used in object detection and control of the model application system 110. Cut.

[0041] The above description of embodiments of the present invention is provided for illustrative purposes only and is not exhaustive. Or, the invention is not intended to be limited to the exact form disclosed. Those skilled in the art will know. It can be understood that many changes and modifications are possible in light of the above disclosures. .

[0042] Several parts of this specification relate to algorithms and symbolic representations of operations on information. Embodiments of the present invention are described below. These algorithmic descriptions and expressions are data processing In order to effectively communicate the content of those operations to others skilled in the field, These are used in a specific way. These actions are described functionally, computationally, or logically, It is implemented by computer programs or equivalent electrical circuits, microcode, etc. This should be understood as follows. Furthermore, without losing generality, the configuration of these operations can be modularized. It has also been proven that it is sometimes convenient to refer to the described actions and those Modules related to this include software, firmware, hardware, or related components. It can be realized in any combination of these.

[0043] Any of the steps, actions, or processes described herein, either alone or in combination with others, may be used. Combined with the device, one or more hardware or software modules This can be implemented or carried out by a software module. In one embodiment, a software module to perform any or all of the steps, actions, or processes described. Computer programs that can be executed by a computer processor Implemented by computer program products that include computer-readable media containing D. .

[0044] Embodiments of the present invention also include apparatus (e.g., systems) for carrying out the operations described herein. This may relate to the following. This device may be specially configured for a required purpose, and / or , selectively launched by computer programs stored on the computer or It may include a reconfigurable general-purpose computing device. The chair is a system of one or more processors and / or computer systems. Or it may be a device. Such computer programs are non-temporary tangible devices. A computer-readable storage medium, or a computer system bus, that can be used to record electronic instructions. It may be stored in any type of medium suitable for storage. Furthermore, as referred to herein Any computing system may include a single processor, or An architecture that utilizes multiple processor designs to enhance computing power. That's fine.

[0045] Embodiments of the present invention also relate to products generated by the calculation process described herein. It is possible. Such products may contain information resulting from the calculation process, and the information is non Stored in a temporary tangible computer-readable storage medium, the computer program described herein Any embodiment may include a combination of ram products or other data.

[0046] Finally, the language used in this specification is primarily for readability and explanatory purposes. Selected and not selected to describe or limit the subject matter of the present invention Therefore, the scope of the present invention is not defined by this detailed description, but rather by the foregoing. It is intended to be limited by any patent claims issued in relation to the application. However Therefore, the disclosure of embodiments of the present invention is within the scope of the present invention as described in the appended claims. This is an example, but it is not intended to be an exhaustive list.

Claims

1. A method for detecting a set of images captured by a set of cameras associated with a first vehicle, comprising: generating, by the processor, for at least one image representing an object in the set of images, at least one augmented image by modifying at least one visual element of the at least one image; training, by the one or more processors, a predictive computer model using a portion of the set of images and the at least one augmented image, wherein training the predictive computer model configures the predictive computer model to predict a presence of an object in an input image for use in autonomous or semi-autonomous control of a second vehicle; A method comprising:

2. The step of generating at least one augmented image comprises: The method of claim 1 , comprising performing, by the one or more processors, an image manipulation function that preserves camera characteristics of the at least one image to generate the at least one augmented image.

3. Performing the image manipulation function to maintain camera characteristics of the at least one image comprises: The method of claim 2 , comprising performing, by the one or more processors, an image manipulation function that maintains one or more angles, scales, or attitudes associated with the at least one image.

4. The step of generating at least one augmented image comprises: The method of claim 1 , comprising performing, by the one or more processors, an image manipulation function that adjusts a portion of the at least one image based on a region of interest.

5. Performing the image manipulation function comprises:

5. The method of claim 4, comprising modifying camera characteristics by the one or more processors: cropping the at least one image, padding the at least one image, horizontally flipping the at least one image, vertically flipping the at least one image, or applying an affine transformation to the at least one image.

6. Performing the image manipulation function comprises: The method of claim 4 , comprising performing, by the one or more processors, at least one of cropping, hue, saturation, value jittering, salt and pepper, or domain transition on the at least one image.

7. The step of generating at least one augmented image comprises:

2. The method of claim 1, further comprising performing, by the one or more processors, a cropping on the at least one image, a location of the cropping selected based on a predicted view of at least one camera associated with the first vehicle.

8. The step of generating at least one augmented image comprises: The method of claim 1 , comprising performing, by the one or more processors, a crop on the at least one image, the location of the crop comprising a center of the at least one image representing a direction of travel.

9. The step of generating at least one augmented image comprises: performing, by the one or more processors, a cropping applied to a portion of the at least one image, the location of the cropping being selected based on a predicted view of at least one camera associated with the first vehicle; The method of claim 1 , wherein the locations include artifacts that are always present in images captured by the set of cameras.

10. A system comprising one or more processors and a non-transitory computer storage medium storing instructions that, when executed by the one or more processors, cause the processor to: Identifying a set of images captured by a set of cameras associated with the first vehicle; generating, for at least one image representing an object in the set of images, at least one augmented image by modifying at least one visual element of the at least one image; training a predictive computer model using a portion of the set of images and the at least one augmented image, where training the predictive computer model configures the predictive computer model to predict a presence of an object in an input image for use in autonomous or semi-autonomous control of a second vehicle.

11. The instructions for causing the one or more processors to generate the at least one augmented image include instructions for causing the one or more processors to: The system of claim 10 , further comprising: performing an image manipulation function that preserves camera characteristics of the at least one image to generate the at least one augmented image.

12. The instructions for causing the one or more processors to perform the image manipulation function that maintains camera characteristics of the at least one image include instructions for causing the one or more processors to: The system of claim 11 , further comprising: causing the image manipulation function to be performed such that one or more of an angle, scale, or pose associated with the at least one image is maintained.

13. The instructions for causing the one or more processors to generate the at least one augmented image include instructions for causing the one or more processors to: The system of claim 10 , further comprising: a processor configured to perform an image manipulation function that adjusts a portion of the at least one image based on a region of interest.

14. The instructions causing the one or more processors to generate the image manipulation function include instructions to the one or more processors:

14. The system of claim 13, further comprising: modifying camera characteristics by cropping the at least one image, padding the at least one image, horizontally flipping the at least one image, vertically flipping the at least one image, or applying an affine transformation to the at least one image.

15. The instructions for causing the one or more processors to perform the image manipulation function include instructions for causing the one or more processors to: The system of claim 13 , further comprising: performing at least one of cropping, hue, saturation, value jittering, salt and pepper, or domain transition on the at least one image.

16. The instructions for causing the one or more processors to generate the at least one augmented image include instructions for causing the one or more processors to: The system of claim 10 , further comprising: performing a crop on the at least one image, the location of the crop being selected based on a predicted view of at least one camera associated with the first vehicle.

17. The instructions for causing the one or more processors to generate the at least one augmented image include instructions for causing the one or more processors to: The system of claim 10 , further comprising: performing a crop on the at least one image, the location of the crop comprising a center of the at least one image representing a direction of travel.

18. The instructions for causing the one or more processors to generate the at least one augmented image include instructions for causing the one or more processors to: performing a cropping applied to a portion of the at least one image, the location of the cropping being selected based on a predicted view of at least one camera associated with the first vehicle; The system of claim 10 , wherein the locations include artifacts that are always present in images captured by the set of cameras.

19. A non-transitory computer storage medium having instructions for execution by the one or more processors, the instructions, when executed by the one or more processors, causing the one or more processors to: Identifying a set of images captured by a set of cameras associated with the first vehicle; generating, for at least one image representing an object in the set of images, at least one augmented image by modifying at least one visual element of the at least one image; and training a predictive computer model using a portion of the set of images and the at least one augmented image, wherein training the predictive computer model configures the predictive computer model to predict a presence of an object in an input image for use in autonomous or semi-autonomous control of a second vehicle.

20. The instructions for causing the one or more processors to generate the at least one augmented image include instructions for causing the one or more processors to:

20. The non-transitory computer storage medium of claim 19, further comprising: a processor configured to perform image manipulation functions that preserve camera characteristics of the at least one image to generate the at least one augmented image.