Method for estimating heading of object and device for performing same

Abstract

A method for estimating a heading of an object is disclosed. The method for estimating a heading of an object located in proximity to a vehicle, according to one embodiment, may comprise the operations of: calculating a loss on the basis of the location of the object, a predicted value for the heading of the object, and ground truth; and estimating the heading by using a model trained on the basis of the loss. The loss can be calculated on the basis of a first weighted value related to the distance between the vehicle and the object and a second weighted value related to the length of the object.

Description

Method for estimating object headings and device for performing the same

[0001] The following disclosure relates to a method for estimating the heading of an object and an apparatus for performing the same.

[0002] In technologies utilizing artificial intelligence (AI) models, inference can be performed using the trained model. During the model training process, the model can easily and accurately learn data types that appear frequently, whereas data types that appear infrequently may be learned relatively inaccurately.

[0003] In the driving environment of autonomous driving vehicles (ADVs), accurately estimating the location and / or heading of other vehicles merging while changing lanes can be important; however, since data on lane changes appears relatively infrequently, training the model may be difficult.

[0004] The background technology described above is possessed or acquired by the inventor in the process of deriving the content of the disclosure of the present application, and cannot necessarily be considered as prior art disclosed to the general public prior to the filing of this application.

[0005] One embodiment can accurately estimate the heading of other vehicles located around a vehicle.

[0006] One embodiment can accurately estimate the heading of an object to enable the vehicle to safely perform autonomous driving.

[0007] However, technical challenges are not limited to the technical challenges described above, and other technical challenges may exist.

[0008] A method for estimating the heading of an object located around a vehicle according to one embodiment may include an operation of calculating a loss based on the location of the object, a predicted value for the heading of the object, and ground truth, and an operation of estimating the heading using a model learned based on the loss. The loss may be calculated based on a first weight regarding the distance between the vehicle and the object and a second weight regarding the length of the object.

[0009] According to one embodiment, the first weight may converge to 1 as the distance decreases and converge to 0 as the distance increases.

[0010] According to one embodiment, the second weight may converge to 0 as the length becomes smaller in response to the length being less than or equal to a predetermined threshold value.

[0011] According to one embodiment, the second weight may converge to 1 as the length increases in response to the length being greater than a predetermined threshold value.

[0012] According to one embodiment, a computer-readable recording medium storing one or more computer programs may include instructions for performing the method in a processor.

[0013] An apparatus according to one embodiment may include at least one processor and a memory comprising instructions. Based on the instructions being executed individually or collectively by the at least one processor, the apparatus may calculate a loss based on the location of the object, a predicted value for the heading of the object, and ground truth, and estimate the heading using a model learned based on the loss. The loss may be calculated based on a first weight for the distance between the vehicle and the object and a second weight for the length of the object.

[0014] According to one embodiment, the first weight may converge to 1 as the distance decreases and converge to 0 as the distance increases.

[0015] According to one embodiment, the second weight may converge to 0 as the length becomes smaller in response to the length being less than or equal to a predetermined threshold value.

[0016] According to one embodiment, the second weight may converge to 1 as the length increases in response to the length being greater than a predetermined threshold value.

[0017] In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0018] FIG. 1 is a drawing for explaining an autonomous driving system according to one embodiment.

[0019] FIGS. 2a and FIGS. 2b are drawings for explaining weights according to one embodiment.

[0020] FIG. 3 is a drawing for illustrating an example of an estimated heading according to one embodiment.

[0021] FIG. 4 is a flowchart illustrating a method according to one embodiment.

[0022] FIG. 5 is a schematic block diagram of an electronic device according to one embodiment.

[0023] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, actual implementations are not limited to the specific embodiments disclosed, and the scope of this specification includes modifications, equivalents, or substitutions included in the technical concept described by the embodiments.

[0024] Terms such as "first" or "second" may be used to describe various components, but these terms should be interpreted solely for the purpose of distinguishing one component from another. For example, the first component may be named the second component, and similarly, the second component may be named the first component.

[0025] When it is stated that a component is "connected" to another component, it should be understood that it may be directly connected to or coupled with that other component, or that there may be other components in between.

[0026] Singular expressions include plural expressions unless the context clearly indicates otherwise. In this document, phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B or C,” “at least one of A, B and C,” and “at least one of A, B, or C” may each include any one of the items listed together with the corresponding phrase, or all possible combinations thereof. In this specification, terms such as “comprising” or “having” are intended to designate the existence of the described feature, number, step, action, component, part, or combination thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0027] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0028] As used herein, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0029] As used in this document, the term "part" refers to software or hardware components, such as FPGAs or ASICs, and the "part" performs certain roles. However, the meaning of "part" is not limited to software or hardware. The "part" may be configured to reside in an addressable storage medium or configured to operate one or more processors. For example, the "part" may include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and "parts" may be combined into a smaller number of components and "parts" or further separated into additional components and "parts." Furthermore, the components and "parts" may be implemented to operate one or more CPUs within a device or secure multimedia card. Additionally, '~part' may include one or more processors.

[0030] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are given the same reference numeral regardless of the drawing number, and redundant descriptions thereof will be omitted.

[0031]

[0032] FIG. 1 is a drawing for explaining an autonomous driving system according to one embodiment.

[0033] Referring to FIG. 1, according to one embodiment, an autonomous driving system (e.g., autonomous driving system (10)) may be a system that generates a driving path and controls the driving of a vehicle (e.g., vehicle (110)) based on sensing data about the surroundings of a vehicle (e.g., vehicle (110)). The autonomous driving system (10) may include a heading estimation device (100), a vehicle (110), and a server (130). The autonomous driving system (10) may generate a driving path of the vehicle (110) based on an end-to-end model. The end-to-end model may be a model that performs motion planning by directly processing input data without complex intermediate steps. The vehicle (110) means a vehicle that carries people and / or goods and may include a vehicle such as an automobile. The vehicle (110) may be an autonomous driving vehicle.

[0034] A heading estimating device (100) can estimate the heading of an object (e.g., another vehicle) located in the surroundings of a vehicle (110). The heading may be an angle indicating the direction in which a specific object is moving. For example, the heading may be an angle indicating the direction in which a specific object directs relative to the front-side. The heading estimating device (100) can obtain data sensing the surroundings of the vehicle (110) from multiple sensors mounted on the vehicle (110). For example, the heading estimating device (100) can obtain data sensing the surroundings of the vehicle (110) (e.g., images, point clouds) from various sensors such as image sensors, LiDAR sensors, radar sensors, event sensors, light sensors, GPS devices, and accelerometer sensors.

[0035] The heading estimation device (100) can estimate the heading of an object (e.g., another vehicle) located around the vehicle using a model (e.g., a heading estimation model). The model can be trained using a loss derived based on a predicted value and ground truth data. For example, the model can be trained so that the predicted value approaches the ground truth data using a loss, which is the difference between the predicted value and the ground truth data for predicting the object's heading. The training of the model may be performed internally within the heading estimation device (100), or the heading estimation device (100) may use a model trained in a separate device. The heading estimation device (100) can calculate the loss based on the location of the object located around the vehicle, the predicted value for the object's heading, and the ground truth data. The heading estimation device (100) can estimate the object's heading using a model trained based on the loss (e.g., a heading estimation model). The loss may be calculated based on a first weight regarding the distance between a vehicle (e.g., vehicle (110)) and an object (e.g., another vehicle) and a second weight regarding the length of the object (e.g., another vehicle). The first weight and the second weight will be described in detail later with reference to FIGS. 2a and FIGS. 2b.

[0036] The heading of an object (e.g., another vehicle) estimated by the heading estimation device (100) is transmitted to a device (not shown) that controls the autonomous driving of the vehicle (110), so that the vehicle can drive safely without colliding with the object (e.g., another vehicle). The heading estimation device (100) is mounted inside the vehicle (110) and can estimate the heading of an object located around the vehicle (110) in real time. The heading estimation device (100) may be mounted on a server (130) to estimate the heading of an object located around the vehicle (110) and transmit it to the vehicle (110).

[0037] The heading estimation device (100), vehicle (110), and server (130) can communicate using a network (not shown). For example, the network may include a Local Area Network (LAN), a Wide Area Network (WAN), a Value Added Network (VAN), a mobile radio communication network, a satellite communication network, and combinations thereof. The network is a comprehensive data communication network that enables the area of ​​interest setting device (100) and the server (130) to communicate seamlessly with each other, and may include wired internet, wireless internet, and mobile wireless communication networks. Additionally, the wireless communication network may include, for example, Wi-Fi, Bluetooth, Bluetooth Low Energy, Zigbee, Wi-Fi Direct (WFD), Ultra-Wideband (UWB), Infrared Data Association (IrDA), Near Field Communication (NFC), but is not limited thereto.

[0038]

[0039] FIGS. 2a and FIGS. 2b are drawings for explaining weights according to one embodiment.

[0040] Referring to FIGS. 2a and 2b, according to one embodiment, a heading estimation device (e.g., the heading estimation device (100) of FIG. 1) can estimate the heading of an object (e.g., another vehicle) located around a vehicle (e.g., the vehicle (110) of FIG. 1). The heading estimation device (100) can estimate the heading of an object (e.g., another vehicle) using a model (e.g., a heading estimation model) learned using a loss calculated based on a first weight regarding the distance between the vehicle (110) and the object (e.g., another vehicle) and a second weight regarding the length of the object. The first weight may converge to 1 as the distance from the vehicle (110) decreases and converge to 0 as the distance increases. FIG. 2a may illustrate the value of the first weight according to the distance between the vehicle (e.g., the vehicle (110)) and the object. As the distance between the vehicle (110) and the object (e.g., another vehicle) decreases (e.g., as it gets closer to the vehicle (110)), the first weight may converge to 1. For example, if the other vehicle is within n meters (e.g., 20 meters) of the vehicle (110), the first weight may converge to 1 as it gets closer to the vehicle (110). As the distance between the vehicle (110) and the object (e.g., another vehicle) increases (e.g., as it gets further away from the vehicle (110)), the first weight may converge to 0. For example, if the other vehicle is located further away from the vehicle (110) than n meters (e.g., 20 meters), the first weight may converge to 0 as it gets further away from the vehicle (110). The first weight can be calculated as shown in Equation 1 below.

[0041]

[0042] Here, x,y[m] can represent the location (e.g., coordinates of the center position) of an object (e.g., another vehicle) based on the ground truth data. can represent the distance (e.g., Euclidean distance) from the vehicle (110) to the object when the position (e.g., coordinates) of the vehicle (110) is assumed to be 0, 0[m].

[0043]

[0044] The second weight may relate to the length of an object (e.g., another vehicle). Since longer objects (e.g., other vehicles) can cause larger lateral errors even when the heading changes at a small angle, the second weight may be set to have a value that varies depending on the length of the object (e.g., another vehicle). The second weight may converge to 0 as the length decreases in response to the length of the object (e.g., another vehicle) being below a predetermined threshold. The second weight may converge to 1 as the length increases in response to the length of the object (e.g., another vehicle) being greater than a predetermined threshold. FIG. 2b may illustrate the value of the second weight according to the length of the object. When the length of the object (e.g., another vehicle) is below a predetermined threshold, the second weight may converge to 0 as the length of the object decreases. For example, when the length of the other vehicle is m meters (e.g., 7 meters) or less, the second weight may converge to 0 as the length of the other vehicle decreases. The second weight may converge to 1 as the length increases in response to the length of an object (e.g., another vehicle) being greater than a predetermined threshold. For example, if the length of another vehicle is greater than k meters (e.g., 10 meters), the second weight may converge to 1 as the length of the other vehicle increases. The second weight can be calculated as shown in Equation 2 below.

[0045]

[0046] Here, l[m] can represent the length of an object (e.g., another vehicle) according to the correct answer data for the length of the object (e.g., another vehicle).

[0047]

[0048] The heading estimation device (100) can calculate a loss based on the position (e.g., x, y[m]) of an object (e.g., another vehicle), a predicted value for the object's heading, and ground truth data. The loss can be calculated by the following mathematical formula 3.

[0049]

[0050] Here, is the predicted value for the heading of an object (e.g., another vehicle), represents the heading of the object based on the ground truth, and L2_loss is the mean squared error (MSE), which is the loss derived by squaring the difference between the predicted value and the actual heading based on the ground truth and averaging it.

[0051]

[0052] The heading estimation device (100) can estimate the heading more accurately by training a model (e.g., a heading estimation model) using a loss that reflects the distance of the object (e.g., another vehicle) from the vehicle (110) and the length of the object (e.g., another vehicle).

[0053]

[0054] FIG. 3 is a drawing for illustrating an example of an estimated heading according to one embodiment.

[0055] Referring to FIG. 3, according to one embodiment, FIG. 3 may illustrate a heading estimated by a heading estimation device (e.g., the heading estimation device of FIG. 1) using a model learned based on a loss with a first weight and a second weight reflected.

[0056] The heading estimation device (100) can estimate the heading of a bus located in front of a vehicle (e.g., vehicle (110) of FIG. 1). The LiDAR top view shown on the left can show the heading according to a model learned using L2 loss without reflecting the first weight and the second weight. The LiDAR top view shown on the right can show the heading according to a model learned using loss with reflected the first weight and the second weight. In the top view on the left, the heading of the bus (310) was estimated to be 3 degrees, but in the top view on the right, the heading of the same bus (330) was estimated to be 6 degrees, so it can be seen that the accuracy of the prediction has increased when using loss with reflected the first weight and the second weight.

[0057] The heading estimation device (100) can better predict heading in specific situations (e.g., lane change situations) by using a model trained using first weights and second weights for heading estimation. The heading estimation device (100) can use a model trained to be specialized for specific situations by setting an object to adjust the weights or by setting a situation.

[0058]

[0059] FIG. 4 is a flowchart illustrating a method according to one embodiment.

[0060] Referring to FIG. 4, according to one embodiment, operations 410 to 430 may be operations performed by the heading estimation device (100) of FIG. 1 described with reference to FIG. 1 to FIG. 3.

[0061] According to one embodiment, operations 410 to 430 may be understood to be performed in a processor (e.g., processor (530) of FIG. 5) of the heading estimation device (100) described with reference to FIG. 1 (e.g., electronic device (500) of FIG. 5).

[0062] In operation 410, the heading estimation device (100) can calculate the loss based on the position of the object, the predicted value for the object's heading, and the correct answer data.

[0063] In operation 430, the heading estimation device (100) can estimate the heading using a model learned based on the loss. The loss may be calculated based on a first weight regarding the distance between the vehicle and the object and a second weight regarding the length of the object.

[0064] Operations 410 through 430 may be performed sequentially, but are not limited thereto. For example, operations 410 and 430 may be performed in parallel.

[0065]

[0066] FIG. 5 is a schematic block diagram of an electronic device according to one embodiment.

[0067] Referring to FIG. 5, according to one embodiment, an electronic device (500) (e.g., the heading estimation device (100) of FIG. 1) may include a memory (510) and a processor (530).

[0068] The memory (510) can store instructions (or programs) executable by the processor (530). For example, the instructions may include instructions for executing the operation of the processor (530) and / or the operation of each component of the processor (530).

[0069] The memory (510) may include one or more computer-readable storage media. The memory (510) may include non-volatile storage devices (e.g., magnetic hard disc, optical disc, floppy disc, flash memory, EPROM (electrically programmable memories), EEPROM (electrically erasable and programmable)).

[0070] The memory (510) may be a non-transitory medium. The term "non-transitory" may indicate that the storage medium is not implemented by a carrier wave or a propagated signal. However, the term "non-transitory" should not be interpreted as meaning that the memory (510) is immobile.

[0071] The processor (530) can process data stored in memory (510). The processor (530) can execute computer-readable code (e.g., software) stored in memory (510) and instructions triggered by the processor (530).

[0072] The processor (530) may be a data processing device implemented in hardware having a circuit having a physical structure for executing desired operations. For example, the desired operations may include code or instructions included in a program.

[0073] For example, a data processing device implemented in hardware may include a microprocessor, a central processing unit, a processor core, a multi-core processor, a multiprocessor, an Application-Specific Integrated Circuit (ASIC), and a Field Programmable Gate Array (FPGA).

[0074] The processor (530) can cause the electronic device (500) to perform one or more operations by executing code and / or instructions stored in memory (510). The operations performed by the electronic device (500) may be substantially the same as the operations performed by the anomaly detection device (100) described with reference to FIGS. 1 through 5. Such redundant descriptions are omitted.

[0075]

[0076] The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0077] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or instruct the processing unit independently or collectively. Software and / or data may be stored on any type of machine, component, physical device, virtual equipment, computer storage medium, or device so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on computer-readable recording media.

[0078] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may store program instructions, data files, data structures, etc., either individually or in combination, and the program instructions recorded on the medium may be those specifically designed and configured for the embodiment or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0079] The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0080] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can apply various technical modifications and variations based thereon. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0081] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

Claims

1. A method for estimating the heading of an object located around a vehicle, An operation to calculate a loss based on the position of the object, a predicted value for the heading of the object, and ground truth data; and The operation of estimating the heading using a model trained based on the above loss Includes, The above loss is, A method calculated based on a first weight for the distance between the vehicle and the object and a second weight for the length of the object.

2. In Paragraph 1, The above first weight is, As the above distance decreases, it converges to 1, and A method that converges to 0 as the distance increases.

3. In Paragraph 1, The above second weight is, A method in which, in response to the length being less than or equal to a predetermined threshold value, the length converges to 0 as it becomes smaller.

4. In Paragraph 1, The above second weight is, A method in which, in response to the length being greater than a predetermined threshold value, the length converges to 1 as it increases.

5. A computer program stored on a computer-readable recording medium in combination with hardware to execute the method of any one of claims 1 to 4.

6. A device for estimating the heading of an object located around a vehicle, At least one processor; and memory that stores instructions Includes, Based on the above instructions being executed individually or collectively by the at least one processor, the device, Based on the location of the object, the predicted value for the heading of the object, and the ground truth, the loss is calculated, and Using a model trained based on the above loss, the above heading is estimated, and The above loss is, A device calculated based on a first weighting factor regarding the distance between the vehicle and the object and a second weighting factor regarding the length of the object.

7. In Paragraph 6, The above first weight is, As the above distance decreases, it converges to 1, and A device that converges to 0 as the above distance increases.

8. In Paragraph 6, The above second weight is, A device that converges to 0 as the length becomes smaller in response to the length being less than or equal to a predetermined threshold value.

9. In Paragraph 6, The above second weight is, A device that converges to 1 as the length increases in response to the length being greater than a predetermined threshold value.

Images