Electronic device and method for augmenting training data

Abstract

Disclosed are an electronic device and a method for augmenting training data. A method according to an embodiment may comprise an operation of obtaining first driving data indicating a route along which a vehicle has traveled over continuous time and information on the driving environment of the vehicle. The method may comprise an operation of randomly changing the location of the vehicle and the heading of the vehicle at a specific time point during the continuous time. The method may comprise an operation of generating second driving data for training a neural network to cause the vehicle to return to the center of a lane if the vehicle deviates from the center of the lane, by reflecting changed driving environment information of the vehicle according to the changed location and the changed heading.

Description

Methods for augmenting electronic devices and training data

[0001] The following disclosure relates to an electronic device and a method for augmenting learning data.

[0002] Autonomous driving refers to a technology in which a vehicle independently perceives its driving environment, determines its route, and moves without driver intervention. Autonomous vehicles utilize sensors (e.g., cameras, LiDAR, and / or radar) to detect their surroundings and can derive optimal driving strategies through artificial intelligence (AI)-based models (e.g., neural networks). Training neural networks to derive driving strategies for autonomous driving may require a large amount of training data.

[0003] Training data for autonomous driving can consist of driving data collected from actual roads, and this driving data may include not only the vehicle's location, speed, and driving path, but also surrounding environment information such as surrounding vehicles, pedestrians, and lane information. If sufficient training data is not secured, a problem of bias toward specific driving patterns may occur.

[0004] The background technology described above is possessed or acquired by the inventor in the process of deriving the content of the disclosure of the present application, and cannot necessarily be considered as prior art disclosed to the general public prior to the filing of this application.

[0005] A method for augmenting training data for training a neural network for autonomous driving according to one embodiment may include the operation of acquiring first driving data representing a path traveled by a vehicle over a continuous period of time and driving environment information of said vehicle. The method may include the operation of randomly changing the position of said vehicle and the heading of said vehicle at a specific point in time during said continuous period of time. The method may include the operation of generating second driving data to train the neural network to return to the center of the lane when the vehicle deviates from the center of the lane by reflecting the changed driving environment information of said vehicle according to the changed position and changed heading.

[0006] The above path may include a path where the vehicle traveled in the same lane.

[0007] The operation of randomly changing the position of the vehicle and the heading of the vehicle may include the operation of randomly changing the position of the vehicle at the specific point in time in a lateral orientation.

[0008] The operation of randomly changing the position of the vehicle and the heading of the vehicle may include the operation of changing the heading of the vehicle at a specific point in time such that the changed heading maintains continuity with the heading of the vehicle at a point in time prior to the specific point in time.

[0009] The operation of changing the heading of the vehicle at the specific point in time to maintain the above continuity may include the operation of randomly selecting the heading of the vehicle at the specific point in time within a preset range and the heading of the vehicle at the previous point in time to maintain the above continuity.

[0010] The operation of generating the second driving data may include an operation of restoring the path shown in the first driving data based on the changed position and the changed heading. The operation of generating the second driving data may include an operation of generating the second driving data by reflecting changes in the vehicle's driving environment according to the changed position and the changed heading into the restored path.

[0011] The operation of restoring the above path may include the operation of restoring the path to satisfy physical constraints of the vehicle. The physical constraints of the vehicle may be intended to prevent the vehicle from following a physically impossible path.

[0012] The operation of restoring the path shown in the first driving data based on the changed position and the changed heading may include the operation of estimating a second past path by restoring a first past path at a point in time earlier than the specific point in time among the paths, based on the changed position and the changed heading. The operation of restoring the path shown in the first driving data based on the changed position and the changed heading may include the operation of predicting a second future path by restoring a first future path at a point in time later than the specific point in time among the paths, based on the changed position and the changed heading.

[0013] In one embodiment, an electronic device for augmenting training data to train a neural network for autonomous driving may include a processor. The electronic device may include a memory for storing instructions. The instructions may be executed individually or collectively by the processor to enable the electronic device to acquire first driving data representing a path traveled by a vehicle over a continuous period of time. The instructions may be executed individually or collectively by the processor to enable the electronic device to randomly change the position of the vehicle and the heading of the vehicle at a specific point in time during the continuous period of time. The instructions may be executed individually or collectively by the processor to enable the electronic device to generate second driving data to train the neural network to return to the center of the lane if the vehicle deviates from the center of the lane, by reflecting the changed driving environment information of the vehicle according to the changed position and changed heading.

[0014] The above path may include a path where the vehicle traveled in the same lane.

[0015] The above instructions may be executed individually or collectively by the processor to cause the electronic device to randomly change the position of the vehicle at the specific point in time in a lateral orientation.

[0016] The above instructions may be executed individually or collectively by the processor to cause the electronic device to change the heading of the vehicle at a specific point in time such that the changed heading maintains continuity with the heading of the vehicle at a point in time prior to the specific point in time.

[0017] The above instructions may be executed individually or collectively by the processor to allow the electronic device to randomly select the heading of the vehicle at the specific point in time within a preset range and the heading of the vehicle at the previous point in time in order to maintain the continuity.

[0018] The above instructions may be executed individually or collectively by the processor to enable the electronic device to restore the path shown in the first driving data based on the changed location and the changed heading. The above instructions may be executed individually or collectively by the processor to enable the electronic device to generate the second driving data by reflecting changes in the vehicle's driving environment according to the changed location and the changed heading into the restored path.

[0019] The above instructions may be executed individually or collectively by the processor to cause the electronic device to restore the path to satisfy the physical constraints of the vehicle. The physical constraints of the vehicle may be intended to prevent the vehicle from following a physically impossible path.

[0020] The above instructions may be executed individually or collectively by the processor to enable the electronic device to estimate a second past path by restoring a first past path at a point in time earlier than the specific point in time among the paths, based on the changed position and the changed heading. The above instructions may be executed individually or collectively by the processor to enable the electronic device to predict a second future path by restoring a first future path at a point in time later than the specific point in time among the paths, based on the changed position and the changed heading.

[0021] FIG. 1 is a drawing for explaining the driving situation of an autonomous vehicle according to one embodiment.

[0022] Figure 2a is a diagram illustrating a deep learning computation method using an artificial neural network.

[0023] FIG. 2b is a diagram illustrating a method for learning and inferring an artificial neural network model according to one embodiment.

[0024] FIG. 3 is a diagram illustrating an autonomous driving framework according to one embodiment.

[0025] FIG. 4 is a diagram illustrating a method for augmenting driving data according to one embodiment.

[0026] FIG. 5 is a diagram illustrating the operation of restoring a path based on a changed position and a changed heading according to one embodiment.

[0027] FIG. 6 is a diagram illustrating driving data generated through augmentation according to one embodiment.

[0028] FIG. 7 is an example of a flowchart of a learning data augmentation method according to one embodiment.

[0029] FIG. 8 is an example of an electronic device according to one embodiment.

[0030] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, actual implementations are not limited to the specific embodiments disclosed, and the scope of this specification includes modifications, equivalents, or substitutions included in the technical concept described by the embodiments.

[0031] Terms such as "first" or "second" may be used to describe various components, but these terms should be interpreted solely for the purpose of distinguishing one component from another. For example, the first component may be named the second component, and similarly, the second component may be named the first component.

[0032] When it is stated that a component is "connected" to another component, it should be understood that it may be directly connected to or coupled with that other component, or that there may be other components in between.

[0033] The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, terms such as "comprising" or "having" are intended to specify the existence of the described features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0034] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0035] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are given the same reference numeral regardless of the drawing number, and redundant descriptions thereof will be omitted.

[0036]

[0037] FIG. 1 is a drawing for explaining the driving situation of an autonomous vehicle according to one embodiment.

[0038] Referring to FIG. 1, an autonomous driving device (100) refers to a device capable of driving autonomously without driver intervention. The autonomous driving device (100) may be implemented as a vehicle, but is not necessarily limited thereto, and may be implemented as various means of transportation such as a two-wheeled vehicle, a robot, an aircraft, etc. For convenience of explanation, the present specification assumes and describes the case where it is implemented as a vehicle.

[0039] The autonomous driving device (100) can drive in an autonomous mode according to the perceived driving environment. The driving environment may be perceived through one or more sensors attached to or installed on the autonomous driving device (100). For example, one or more sensors may include a camera, LIDAR, RADAR, and voice recognition sensors, but are not limited to the examples described. The driving environment may include a road, road condition, type of lane, presence or absence of surrounding vehicles, distance to nearby vehicles, weather, presence or absence of obstacles, etc., but is not limited to the examples described.

[0040] The autonomous driving device (100) can recognize the driving environment and generate a driving path suitable for the driving environment. The autonomous driving device (100) can control internal and external mechanical elements to follow the driving path. The autonomous driving device (100) can periodically generate a driving path.

[0041] Artificial intelligence (AI) algorithms may be used to generate a driving path. The autonomous driving device (100) may generate a driving path through a neural network. For example, the autonomous driving device (100) may generate a driving path by running a neural network that has completed training.

[0042] The autonomous driving device (100) may deviate from the center of the lane (110) while driving, and in such case, it may generate a driving path (130) that returns to the center of the lane (110).

[0043] In order for neural network-based driving path generation to be performed more precisely, a large amount of driving data may be required as training data. In particular, in order to generate a driving path (130) that deviates from the center of the lane (110) and returns to the center of the lane (110) more precisely, a large amount of driving data such as the driving path (130) may be required. In the present disclosure, by generating (or augmenting) large-scale training data including various lane deviation and return situations and training the neural network, it is possible to make driving path determination (or prediction) more precise.

[0044] Before describing the method for augmenting learning data according to one embodiment, an artificial intelligence algorithm will be described with reference to FIGS. 2a and 2b.

[0045]

[0046] Figure 2a is a diagram illustrating a deep learning computation method using an artificial neural network.

[0047] Artificial intelligence (AI) algorithms, including deep learning, are characterized by inputting input data (10) into an artificial neural network (ANN) and learning output data (30) through operations such as convolution. An artificial neural network may refer to a computational architecture that models a biological brain. Within an artificial neural network, nodes corresponding to neurons of the brain are connected to each other and operate collectively to process input data. Examples of various types of neural networks include, but are not limited to, Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Deep Belief Networks (DBN), and Restricted Boltzmann Machines (RBM). In a feed-forward neural network, the neurons of the neural network have links with other neurons. These connections can be extended in one direction, for example, in the forward direction, through a neural network.

[0048] Referring to FIG. 2a, an artificial neural network structure is illustrated in which input data (10) is input into the artificial neural network, and output data (30) is output through the artificial neural network (e.g., Convolutional Neural Network (CNN) (20)) which includes one or more layers. The artificial neural network may be a deep neural network having two or more layers.

[0049] A convolutional neural network (20) can be used to extract "features" such as borders, line colors, etc. from input data (10). The convolutional neural network (20) may include multiple layers. Each layer can receive data and process the data input to that layer to generate data output from that layer. The data output from the layer may be a feature map generated by convolutional operation with the weight values ​​of one or more filters on an image or feature map input to the convolutional neural network (20). The initial layers of the convolutional neural network (20) may be operated to extract low-level features such as edges or gradients from the input. The subsequent layers of the convolutional neural network (20) may extract progressively more complex features such as eyes, noses, etc. within the image.

[0050]

[0051] FIG. 2b is a diagram illustrating a method for learning and inferring an artificial neural network model according to one embodiment.

[0052] Referring to FIG. 2b, the autonomous driving system may include a learning device (200) and an inference device (250). A learning device (200) according to one embodiment corresponds to a computing device having various processing functions, such as generating a neural network, training (or learning) a neural network, or retraining a neural network. For example, the learning device (200) may be implemented as various types of devices, such as a PC (personal computer), a server device, or a mobile device.

[0053] The learning device (200) can generate one or more trained neural networks (210) by repeatedly training (learning) a given initial neural network. Generating one or more trained neural networks (210) may mean determining neural network parameters. Here, the parameters may include various types of data input to / output to the neural network, such as input / output activations of the neural network, weights, biases, etc. As the repeated training of the neural network progresses, the parameters of the neural network may be tuned to compute a more accurate output for a given input.

[0054] The learning device (200) can transmit one or more trained neural networks (210) to the inference device (250). The inference device (250) may be included in a mobile device, an embedded device, etc. According to one embodiment, the inference device (250) may be an electronic device comprising at least one of a processor, memory, an input / output (I / O) interface, a display, a communication interface, or a sensor as dedicated hardware for operating the neural network.

[0055] The inference device (250) may be a concept that includes all digital devices equipped with memory means and equipped with a microprocessor to have computational capabilities, such as tablet PCs, smartphones, personal computers (e.g., laptop computers, etc.), artificial intelligence speakers, smart TVs, mobile phones, navigation systems, web pads, PDAs, workstations, etc.

[0056] The inference device (250) may drive one or more trained neural networks (210) as is, or drive one or more trained neural networks (210) that have been processed (e.g., quantized) into a neural network (260). The inference device (250) that drives the processed neural network (260) may be implemented in a separate, independent device from the learning device (200). However, it is not limited thereto, and the inference device (250) may also be implemented within the same device as the learning device (200). For example, the learning device (200) and / or the inference device (250) may be implemented as an electronic device (e.g., the autonomous driving device (100) of FIG. 1). That is, the operations of the learning device (200) and / or the inference device (250) described below may be performed by the autonomous driving device (100).

[0057]

[0058] FIG. 3 is a diagram illustrating an autonomous driving framework according to one embodiment.

[0059] Referring to FIG. 3, the autonomous driving framework may include a data generation process (310), a learning process (320), and an inference process (330).

[0060] The data generation process (310) may refer to the process of generating training data required in the learning process (320). The data generation process (310) may be performed by a data generation device. According to one embodiment, the data generation device may be a learning device (200) or a separate device (not shown). For convenience of explanation, it is assumed below that the data generation process (310) is performed by the learning device (200), but it is not limited thereto. For example, the data generation process (310) may be performed by a separate device, and the learning device (200) may receive the training data generated through the data generation process (310) and perform learning.

[0061] The learning device (200) can generate (or collect) a learning data set for training a neural network. The learning data set may include driving data. The driving data may represent a path traveled by a vehicle over a continuous period of time (e.g., driving path) and / or driving environment information of the vehicle. The driving environment information of the vehicle may include various environmental information around the vehicle. For example, the driving environment information of the vehicle may include information such as objects around the vehicle (e.g., other vehicles and pedestrians, etc.), lanes, and / or traffic lights.

[0062] The driving path may include a path where the vehicle travels along the same lane. This can be utilized to generate a driving path so that the vehicle returns to the center of the lane (110) if it deviates while traveling along the same lane. For example, the driving path may include a path where the vehicle travels continuously along a single lane without changing lanes and / or deviating.

[0063] Driving data can be implemented in the form of a feature map, but is not limited thereto. For example, driving data may be in an analog form. However, if the driving data is analog, it may be necessary to preprocess the driving data to generate training data in the form of a feature map.

[0064] In the learning process (320), the learning device (200) can train the neural network (210) to generate an autonomous driving path. For example, the learning device (200) can train the neural network (210) to generate a path (e.g., driving path (130) of FIG. 1) to return to the center of the lane when the vehicle (e.g., autonomous driving device (100) of FIG. 1) deviates from the center of the lane (e.g., lane center (110) of FIG. 1).

[0065] In the inference process (330), the inference device (250) can generate a driving path using a learned neural network (e.g., the neural network (210) and / or the neural network (260) of FIG. 2b). For example, the inference device (250) can generate a driving path to return the vehicle to the center of the lane if the vehicle deviates from the center of the lane.

[0066] Below, the data generation process (310) will be described in detail with reference to FIGS. 4 to 7.

[0067]

[0068] FIG. 4 is a diagram illustrating a method for augmenting driving data according to one embodiment.

[0069] Referring to FIG. 4, a vehicle (e.g., the autonomous driving device (100) of FIG. 1) can identify a driving path (460, 470) from a starting point (440) to a target point (450). The driving paths (460, 470) may be driving paths that travel along the same lane. A learning device (e.g., the learning device (200) of FIG. 2b) can generate various learning data by augmenting input data (or original data). Although the data before augmentation is referred to as input data (or original data) to distinguish between the data before augmentation and the augmented data, the input data can also be used as learning data.

[0070] The learning device (200) can augment input driving data (e.g., first driving data) to generate various learning data (e.g., second driving data). For convenience of explanation, the driving data corresponding to the path (460) is assumed to be the first driving data, and the driving data corresponding to the path (470) is assumed to be the second driving data augmented from the first driving data.

[0071] In operation 410, the learning device (200) can select a specific time point of the first driving data. The first driving data may represent a path (460) that the vehicle has traveled over a continuous period of time. The path (460) shown in the first driving data may be a path that travels along the same lane without deviating from the center of the lane. The learning device (200) can select a specific point in time among the continuous periods included in the first driving data. For example, the first driving data may represent a path (460) that connects the vehicle's position at each time point when the vehicle travels from the starting point (440) to the target point (450). The learning device (200) can select a specific point in time (e.g., any point in time within the time taken to reach the target point (450) from the starting point (440)) when the vehicle travels from the starting point (440) to the target point (450).

[0072] The first driving data may represent information regarding the driving environment of the vehicle. For example, the first driving data may include information regarding the driving environment of the vehicle at each time point when the vehicle travels from a starting point (440) to a target point (450). The driving environment of the vehicle may include various environmental information such as objects, lanes, and / or traffic lights around the vehicle.

[0073] In operation 420, the learning device (200) can randomly change the position of the vehicle and the heading of the vehicle at a specific point in time. The learning device (200) can randomly change the position of the vehicle at a specific point in time in a lateral orientation.

[0074] The learning device (200) can randomly change the vehicle's heading at a specific point in time. However, the learning device (200) can randomly change the vehicle's heading within a range that does not violate the vehicle's physical limits, taking into account the vehicle's physical limits. For example, let us assume that a continuous time is from t=1 to t=N, and that t=m is selected in operation 410. Changing the vehicle's heading to the rear at t=m, even though the vehicle's heading was facing forward at a point in time prior to the selected point (e.g., t=m-1), may violate the vehicle's physical limits. If the vehicle's physical limits are violated in this way, unrealistic data may be generated. To take into account the vehicle's physical limits, the learning device (200) can change the vehicle's heading while maintaining continuity with the vehicle's heading at a point in time prior to the specific point in time.

[0075] The learning device (200) can change the vehicle's heading at a specific point in time so that the changed heading maintains continuity with the vehicle's heading at a point in time prior to the specific point in time. To maintain continuity, the learning device (200) can randomly select the vehicle's heading at the specific point in time within a preset range with the vehicle's heading at the previous point in time. For example, if the vehicle's heading is facing forward at a point in time prior to the selected point in time (e.g., t=m-1), the learning device (200) may, at a preset angle (e.g., t=m-1) relative to the front of the vehicle (e.g., The vehicle's heading can be changed randomly within the range of ). The preset angle can be set by the learning device (200) and / or the user.

[0076] In operation 430, the learning device (200) can generate new driving data (e.g., second driving data) using a kinematic model. The learning device (200) can generate second driving data by reflecting information on the vehicle's changed driving environment according to the changed position and changed heading. The learning device (200) can restore the path shown in the first driving data (e.g., a path from the starting point (440) to the target point (450)) based on the changed position and changed heading. The learning device (200) can restore the path to satisfy the physical constraints of the vehicle using a kinematic model. The physical constraints of the vehicle may be intended to prevent the vehicle from following a physically impossible path. For example, the learning device (200) can model the vehicle through a kinematic model (e.g., a kinematic bicycle model). The learning device (200) can model the vehicle to satisfy physical constraints of the vehicle (e.g., kinematic constraints of the vehicle such as simplifying the four wheels of the vehicle into one front wheel and one rear wheel, not considering tire slip, or having a structure where only the front wheels of the vehicle can be steered and the rear wheels simply rotate). The learning device (200) can reconstruct the path from the starting point (440) to the target point (450) based on the changed position and changed heading, taking into account the movement of the modeled vehicle.

[0077] The learning device (200) can generate second driving data by reflecting changes in the vehicle's driving environment according to the changed position and changed heading in the restored path. That is, in the second driving data, not only is the vehicle's position heading changed, but the driving environment that has changed according to the changes in the vehicle's position and heading can also be reflected. This will be explained in detail with reference to FIG. 6.

[0078]

[0079] FIG. 5 is a diagram illustrating the operation of restoring a path based on a changed position and a changed heading according to one embodiment.

[0080] Referring to FIG. 5, the changed position and heading of the vehicle at a specific point in time (510) can be identified. A learning device (e.g., the learning device (200) of FIG. 2b) can restore a path (e.g., the path shown in the first driving data) based on the changed position and the changed heading.

[0081] The learning device (200) can restore the paths of a point in time (520) (or past point in time) and a point in time (530) (or future point in time) before a specific point in time (510), respectively, through an optimization method. Below, a method for restoring the path of a previous point in time (520) will be described.

[0082] The learning device (200) can estimate a second past path by restoring a first past path from a point in time prior to a specific point in time (510) based on the changed position and changed heading. The first past path may refer to a path from a point in time prior to a specific point in time (510) among the paths appearing in the first driving data. The learning device (200) can generate (or design) a cost function for estimating the second past path through a kinematic model (e.g., a kinematic cycle model). The learning device (200) can set hard constraints (e.g., constraints such as steering angle limits and / or speed limits) to prevent the vehicle from suddenly changing direction or speed by considering the kinematic limits of the vehicle's actual movement. The learning device (200) can estimate the second past path by performing backward optimization to satisfy the hard constraints.

[0083] The learning device (200) can predict a second future path by restoring a first future path at a later point in time than a specific point in time (510) based on the changed position and the changed heading. The first future path may refer to a path at a later point in time than the specific point in time (510) among the paths shown in the first driving data. For example, the learning device (200) can generate (or design) a cost function to estimate the second future path through a kinematic model (e.g., a kinematic cycle model). The learning device (200) can set forced constraints (e.g., steering angle limit and / or speed limit) by considering the kinematic limits at which the vehicle can actually move. The learning device (200) can estimate the second future path by performing forward optimization to satisfy the forced constraints.

[0084]

[0085] FIG. 6 is a diagram illustrating driving data generated through augmentation according to one embodiment.

[0086] Referring to FIG. 6, the first driving data (610) and the second driving data (630) can be seen. The first driving data (610) and the second driving data (630) can be implemented in the form of a feature map. For example, the first driving data (610) and the second driving data (630) may be three-dimensional (e.g., height, width, channel) feature maps.

[0087] The second driving data (630) may represent an augmented driving path in which the position and heading of the vehicle at a specific point in time are changed from the first driving data (610). However, the second driving data (630) may be data that further reflects not only the driving path according to the changed position and changed heading, but also the driving environment of the vehicle that has changed according to the changed position and changed heading. As a result, as shown in FIG. 6, the second driving data (630) may not only have the position and / or heading of the vehicle (e.g., indicated by white dots included in the driving data (610) and / or driving data (630) changed from the first driving data (610), but also have a change in height and / or width dimensions compared to the first driving data (610). For example, due to the change in height and / or width dimensions, it can be seen that the second driving data (630) is slightly angled apart from the first driving data (610).

[0088]

[0089] FIG. 7 is an example of a flowchart of a learning data augmentation method according to one embodiment.

[0090] Referring to FIG. 7, operations 710 through 750 may be performed sequentially, but are not limited thereto. For example, two or more operations may be performed in parallel. Operations 710 through 750 may be substantially identical to the operations of the learning device (e.g., the learning device (200) of FIG. 2b) described with reference to FIG. 1 through 6. Accordingly, a detailed description is omitted.

[0091] In operation 710, the learning device (200) can acquire first driving data representing the path traveled by the vehicle and the driving environment information of the vehicle over a continuous period of time. The first driving data can be used for data augmentation as original data. The first driving data can represent a driving path traveled in the same lane without deviating from the center of the lane.

[0092] In operation 730, the learning device (200) can randomly change the position of the vehicle and the heading of the vehicle at a specific point in time during a continuous period. The learning device (200) can randomly change the heading of the vehicle within a range that does not violate the physical limits of the vehicle (e.g., a range that maintains continuity with the heading of the vehicle at a point in time prior to the specific point in time). The learning device (200) can change the position of the vehicle in the lateral direction.

[0093] In operation 750, the learning device (200) can generate second driving data to train a neural network to return to the center of the lane when the vehicle deviates from the center of the lane by reflecting information on the vehicle's changed driving environment according to the changed position and changed heading. The learning device (200) can restore the path shown in the first driving data based on the changed position and changed heading. The learning device (200) can generate second driving data by reflecting changes in the driving environment in the restored path. That is, as described in FIG. 6, the second driving data may reflect changes in angles, etc., due to changes in the vehicle's driving environment, rather than simply changing the vehicle's position and heading.

[0094]

[0095] FIG. 8 is an example of an electronic device according to one embodiment.

[0096] Referring to FIG. 8, the electronic device (800) may include a memory (810) and a processor (830). The description with reference to FIG. 1 through 7 may also apply to FIG. 8. For example, the learning device (200) and / or inference device (250) of FIG. 2b may be the electronic device (800).

[0097] Memory (810) can store instructions (e.g., programs) executable by the processor (830). For example, the instructions may include instructions for executing the operation of the processor (830) and / or the operation of each component of the processor (830).

[0098] The memory (810) can be implemented as a volatile memory device or a non-volatile memory device.

[0099] Volatile memory devices can be implemented as DRAM (dynamic random access memory), SRAM (static random access memory), T-RAM (thyristor RAM), Z-RAM (zero capacitor RAM), or TTRAM (Twin Transistor RAM).

[0100] Non-volatile memory devices can be implemented as EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, MRAM (Magnetic RAM), Spin-Transfer Torque (STT)-MRAM, Conductive Bridging RAM (CBRAM), FeRAM (Ferroelectric RAM), PRAM (Phase change RAM), Resistive RAM (RRAM), Nanotube RRAM, Polymer RAM (PoRAM), Nano Floating Gate Memory (NFGM), holographic memory, Molecular Electronic Memory Device, or Insulator Resistance Change Memory.

[0101] The processor (830) can process data stored in memory (810). The processor (830) can execute computer-readable code (e.g., software) stored in memory (810) and instructions triggered by the processor (830).

[0102] The processor (830) may be a data processing device implemented in hardware having a circuit having a physical structure for executing desired operations. For example, the desired operations may include code or instructions included in a program.

[0103] For example, a data processing device implemented in hardware may include a microprocessor, a central processing unit, a processor core, a multi-core processor, a multiprocessor, an Application-Specific Integrated Circuit (ASIC), and a Field Programmable Gate Array (FPGA).

[0104] The processor (830) can cause the electronic device (800) to perform one or more operations by executing code and / or instructions stored in memory (810). The operations performed by the electronic device (800) may be substantially the same as the operations performed by the learning device (200) and / or inference device (250) described with reference to FIGS. 2b through 7. Such redundant descriptions are omitted.

[0105]

[0106] The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0107] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave in order to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on computer-readable recording media.

[0108] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may store program instructions, data files, data structures, etc., either individually or in combination, and the program instructions recorded on the medium may be those specifically designed and configured for the embodiment or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0109] The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0110] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can apply various technical modifications and variations based thereon. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0111] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

Claims

1. A method for augmenting training data for training a neural network for autonomous driving, An operation to acquire first driving data representing the path traveled by the vehicle over a continuous period of time and driving environment information of the vehicle; An operation to randomly change the position of the vehicle and the heading of the vehicle at a specific point in time during the aforementioned continuous time; An operation to generate second driving data to train the neural network to return the vehicle to the center of the lane when the vehicle deviates from the center of the lane, by reflecting the changed driving environment information of the vehicle according to the changed position and changed heading. A method including 2. In Paragraph 1, The above path is, The path the above vehicle traveled in the same lane A method including 3. In Paragraph 1, The operation of randomly changing the position of the vehicle and the heading of the vehicle is, The operation of randomly changing the position of the vehicle at the aforementioned specific point in time in the lateral orientation. A method including 4. In Paragraph 1, The operation of randomly changing the position of the vehicle and the heading of the vehicle is, The operation of changing the heading of the vehicle at a specific point in time such that the changed heading maintains continuity with the heading of the vehicle at a point in time prior to the specific point in time. A method including 5. In Paragraph 4, The operation of changing the heading of the vehicle at the specific point in time to maintain the above continuity is, The operation of randomly selecting the heading of the vehicle at the specific point in time within a preset range and the heading of the vehicle at the previous point in time to maintain the above continuity. A method including 6. In Paragraph 1, The operation of generating the above second driving data is, An operation to restore the path shown in the first driving data based on the changed position and the changed heading; and Operation of generating the second driving data by reflecting changes in the driving environment of the vehicle in the restored path according to the changed position and the changed heading. A method including 7. In Paragraph 6, The operation of restoring the above path is, The operation of restoring the path to satisfy the physical constraints of the vehicle. Includes, The physical constraints of the above vehicle are, A method for preventing the above vehicle from following a physically impossible path.

8. In Paragraph 6, The operation of restoring the path shown in the first driving data based on the changed position and the changed heading is, Based on the above-mentioned changed position and the above-mentioned changed heading, the operation of estimating a second past path by restoring a first past path at a point in time earlier than the above-mentioned specific point in time among the paths; and Based on the above-mentioned changed location and the above-mentioned changed heading, the operation of predicting a second future path by restoring a first future path at a point in time later than the above-mentioned specific point in time among the above paths. A method including 9. An electronic device for augmenting training data for training a neural network for autonomous driving, processor; and Memory that stores instructions Includes, The above instructions are executed individually or collectively by the processor, causing the electronic device, Acquire first driving data representing the path traveled by the vehicle over a continuous period of time, and Randomly changing the position of the vehicle and the heading of the vehicle at a specific point in time during the aforementioned continuous time, and An electronic device that generates second driving data to train a neural network to return to the center of the lane when the vehicle deviates from the center of the lane, by reflecting information on the changed driving environment of the vehicle according to the changed position and changed heading.

10. In Paragraph 9, The above path is, The path the above vehicle traveled in the same lane An electronic device including 11. In Paragraph 9, The above instructions are executed individually or collectively by the processor, causing the electronic device, An electronic device that randomly changes the position of the vehicle at the aforementioned specific point in time in a lateral orientation.

12. In Paragraph 9, The above instructions are executed individually or collectively by the processor, causing the electronic device, An electronic device that changes the heading of the vehicle at a specific point in time so that the changed heading maintains continuity with the heading of the vehicle at a point in time prior to the specific point in time.

13. In Paragraph 12, The above instructions are executed individually or collectively by the processor, causing the electronic device, An electronic device that selects the heading of the vehicle at a specific point in time randomly within a preset range and the heading of the vehicle at the previous point in time to maintain the above continuity.

14. In Paragraph 9, The above instructions are executed individually or collectively by the processor, causing the electronic device, Restore the path shown in the first driving data based on the above changed position and the above changed heading, and An electronic device that generates the second driving data by reflecting changes in the driving environment of the vehicle according to the changed position and the changed heading in the restored path.

15. In Paragraph 14, The above instructions are executed individually or collectively by the processor, causing the electronic device, Restoring the path to satisfy the physical constraints of the vehicle, and The physical constraints of the above vehicle are, An electronic device intended to prevent the above vehicle from following a physically impossible path.

16. In Paragraph 14, The above instructions are executed individually or collectively by the processor, causing the electronic device, Based on the above changed location and the above changed heading, a second past path is estimated by restoring the first past path at a point in time earlier than the above specific point in time among the above paths, and An electronic device that predicts a second future path by restoring a first future path at a later point in time than a specific point in time among the paths, based on the above-mentioned changed position and the above-mentioned changed heading.

Images