Method and system for generating synthetic radar data

Abstract

The present disclosure provides a method and a system for generating synthetic radar data. The method for generating synthetic radar data comprises the steps of: receiving point cloud data generated by a light detection and ranging (LiDAR) sensor; receiving three-dimensional bounding box information associated with the point cloud data; and generating synthetic radar data corresponding to the point cloud data on the basis of the point cloud data and the three-dimensional bounding box information, wherein the synthetic radar data can simulate data generated by a radar sensor.

Description

Method and system for generating synthetic radar data

[0001] The present disclosure relates to a method and system for generating synthetic radar data, and more particularly, to a method and system for generating synthetic radar data using point cloud data generated by lidar sensing.

[0002]

[0003] Radar (RADAR: RAdio Detection And Ranging) sensors can easily collect data such as distance and speed regardless of weather conditions. Furthermore, radar sensors are suitable for long-distance data collection. The radar data generated by these radar sensors can be used for autonomous driving or vehicle driving simulations.

[0004] However, radar data is relatively less publicly available than data generated by other sensors, such as cameras and lidar sensors. Furthermore, even using virtual reality simulators, the complexity of radar sensors makes it difficult to ensure real-time data availability. This makes it difficult to generate and secure large amounts of radar data.

[0005]

[0006] The present disclosure provides a method for generating a synthetic radar, a computer program stored in a recording medium, and a system (device) for solving the above-described problems.

[0007]

[0008] The present disclosure can be implemented in various ways, including a method, a device (system), and / or a computer program stored in a computer-readable storage medium, and a computer-readable storage medium having a computer program stored therein.

[0009] According to one embodiment of the present disclosure, a method for generating synthetic radar (RADAR) data, performed by at least one processor, comprises the steps of: receiving point cloud data generated by a LiDAR (Light Detection And Ranging) sensor; receiving three-dimensional bounding box information associated with the point cloud data; and generating synthetic radar data corresponding to the point cloud data based on the point cloud data and the three-dimensional bounding box information, wherein the synthetic radar data can simulate data generated by the radar sensor.

[0010] According to one embodiment of the present disclosure, the three-dimensional bounding box information may include at least one of position information, orientation information, class information, width information, height information, depth information, or relative velocity information.

[0011] According to one embodiment of the present disclosure, the point cloud data includes a first set of points, and the step of generating synthetic radar data may include the steps of receiving a first extrinsic parameter associated with the lidar sensor, receiving a second extrinsic parameter associated with the radar sensor, and identifying a second set of points within a field of view (FOV) of the radar sensor among the first set of points based on the first extrinsic parameter and the second extrinsic parameter.

[0012] According to one embodiment of the present disclosure, the step of identifying the second set of points includes the step of converting the first set of points into a spherical coordinate system based on the first external parameter and the second external parameter, wherein positional information of the second set of points can be expressed in the spherical coordinate system.

[0013] According to one embodiment of the present disclosure, the step of generating synthetic radar data may further include the step of determining a third set of points included in a three-dimensional bounding box among the second set of points as being associated with a dynamic object, and the step of determining a fourth set of points not included in the three-dimensional bounding box among the second set of points as being associated with a static object.

[0014] According to one embodiment of the present disclosure, each of the fourth set of points includes distance, elevation angle, and azimuth angle information, and the step of generating synthetic radar data may further include the step of generating Radar Cross Section (RCS) information associated with the third set of points based on three-dimensional bounding box information, and the step of generating RCS information for each of the fourth set of points based on the elevation angle information.

[0015] According to one embodiment of the present disclosure, the step of generating synthetic radar data may further include the step of calculating relative velocity information of a third set of points based on three-dimensional bounding box information, the first external parameter, and the second external parameter, and the step of calculating relative velocity information of each of the fourth set of points based on position information of each of the fourth set of points, the first external parameter, and the second external parameter.

[0016] According to one embodiment of the present disclosure, the step of identifying the second set of points may include the steps of receiving FOV information of the radar sensor, dividing an area corresponding to the FOV of the radar sensor into a plurality of bins, mapping the first set of points to the plurality of bins, and, for bins containing two or more points among the plurality of bins, removing the remaining points except for one point within the bin.

[0017] According to one embodiment of the present disclosure, the step of identifying the second set of points may include the steps of receiving FOV information of the radar sensor, dividing an area corresponding to the FOV of the radar sensor into a plurality of bins, mapping the points of the first set to the plurality of bins, and removing points for some of the bins that include a point of one of the plurality of bins.

[0018] According to one embodiment of the present disclosure, the synthetic radar data includes a plurality of points, each point including at least one of position information, distance information, relative velocity information, or RCS information.

[0019] A computer program stored in a computer-readable recording medium for executing a method according to one embodiment of the present disclosure on a computer can be provided.

[0020] According to one embodiment of the present disclosure, an information processing system comprises a communication module, a memory, and at least one processor connected to the memory and configured to execute at least one computer-readable program included in the memory, wherein the at least one program comprises instructions for receiving point cloud data generated by a lidar sensor, receiving three-dimensional bounding box information associated with the point cloud data, and generating synthetic radar data corresponding to the point cloud data based on the point cloud data and the three-dimensional bounding box information, wherein the synthetic radar data can mimic data generated by the radar sensor.

[0021]

[0022] According to some embodiments of the present disclosure, a large amount of radar data can be secured by generating synthetic radar data based on data generated by a relatively easy-to-obtain lidar sensor. This large amount of radar data can be used to train an artificial neural network model associated with the radar sensor.

[0023] According to some embodiments of the present disclosure, a method for generating synthetic radar data can easily generate radar data, thereby reducing data acquisition costs.

[0024] According to some embodiments of the present disclosure, the method for generating synthetic radar data utilizes point cloud data, enabling the generation of synthetic radar data regardless of the type of lidar sensor used. Furthermore, since the method utilizes point cloud data and 3D bounding box information, it can be equally applied to not only real-world environments but also virtual reality, simulation environments, and the like.

[0025] According to some embodiments of the present disclosure, the information about points included in the synthetic radar data may include elements identical to the information about points generated by an actual radar sensor. Thus, the synthetic radar data can simulate data generated by a radar sensor by including data elements that can be generated by the radar sensor.

[0026] According to some embodiments of the present disclosure, points can be removed using probability during the filtering and / or dropping step for multiple bins. In this case, by finely adjusting the probability, the method for generating synthetic radar data can be made more similar to data generated by a radar sensor.

[0027] According to some embodiments of the present disclosure, the generated synthetic radar data can more closely mimic data generated by the radar sensor by reducing the density of points measured by the lidar sensor.

[0028] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure belongs (referred to as “one skilled in the art”) from the description of the claims.

[0029]

[0030] Embodiments of the present disclosure will be described below with reference to the accompanying drawings, wherein like reference numerals represent similar elements, but are not limited thereto.

[0031] FIG. 1 is a schematic diagram illustrating an example of generating synthetic radar data according to one embodiment of the present disclosure.

[0032] FIG. 2 is a block diagram showing the internal configuration of a computing device according to one embodiment of the present disclosure.

[0033] FIG. 3 is a flowchart illustrating an example of a process for generating synthetic radar data according to one embodiment of the present disclosure.

[0034] FIG. 4 is a diagram illustrating an example of point cloud data according to one embodiment of the present disclosure.

[0035] FIG. 5 is a diagram illustrating an example of a mapping, filtering, and dropping process for multiple bins according to one embodiment of the present disclosure.

[0036] FIG. 6 is a diagram illustrating an example of a mapping, filtering, and dropping process for multiple bins according to one embodiment of the present disclosure.

[0037] FIG. 7 is a diagram illustrating a process for calculating a Doppler velocity associated with a dynamic object according to one embodiment of the present disclosure.

[0038] FIG. 8 is a diagram illustrating a process for calculating a Doppler velocity associated with a static object according to one embodiment of the present disclosure.

[0039] FIG. 9 is a flowchart illustrating an example of a method for generating synthetic radar data according to one embodiment of the present disclosure.

[0040]

[0041] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions of widely known functions or configurations will be omitted if they may unnecessarily obscure the gist of the present disclosure.

[0042] In the attached drawings, identical or corresponding components are assigned the same reference numerals. Furthermore, in the description of the embodiments below, duplicate descriptions of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0043] The advantages and features of the disclosed embodiments, and methods for achieving them, will become clearer with reference to the embodiments described below, along with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure the completeness of the disclosure and to fully inform those skilled in the art of the scope of the invention.

[0044] The terms used in this specification will be briefly explained, followed by a detailed description of the disclosed embodiments. The terms used in this specification have been selected from widely used, current terms, taking into account the functions of the present disclosure. However, these terms may vary depending on the intentions of engineers working in the relevant field, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on their meanings and the overall content of the present disclosure.

[0045] In this specification, singular expressions include plural expressions unless the context clearly indicates otherwise. Furthermore, plural expressions include singular expressions unless the context clearly indicates otherwise. When a part of the specification is said to include a component, this does not exclude other components, but rather implies that other components may be included, unless otherwise specifically stated.

[0046] Also, the term 'module' or 'part' used in the specification means a software or hardware component, and the 'module' or 'part' performs certain roles. However, the 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to reside on an addressable storage medium and may be configured to execute one or more processors. Thus, as an example, the 'module' or 'part' may include at least one of components such as software components, object-oriented software components, class components, and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. The functionality provided within the components and 'modules' or 'parts' may be combined into a smaller number of components and 'modules' or 'parts', or further separated into additional components and 'modules' or 'parts'.

[0047] According to one embodiment of the present disclosure, a 'module' or 'unit' may be implemented as a processor and a memory. 'Processor' should be broadly construed to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, and the like. In some circumstances, a 'processor' may also refer to an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field-programmable gate array (FPGA), and the like. A 'processor' may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors in conjunction with a DSP core, or any other such combination of configurations. In addition, 'memory' should be broadly construed to include any electronic component capable of storing electronic information. 'Memory' may refer to various types of processor-readable media, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, etc. Memory is said to be in electronic communication with the processor if the processor can read information from, and / or write information to, the memory. Memory integrated in a processor is in electronic communication with the processor.

[0048] In addition, terms such as first, second, A, B, (a), (b), etc. used in the following embodiments are only used to distinguish certain components from other components, and the nature, order, or sequence of the components are not limited by the terms.

[0049] Additionally, in the embodiments below, when it is described that a component is 'connected', 'coupled' or 'connected' to another component, it should be understood that the component may be directly connected or connected to the other component, but another component may also be 'connected', 'coupled' or 'connected' between each component.

[0050] In the present disclosure, 'each of the plurality of As' may refer to each of all components included in the plurality of As, or may refer to each of some components included in the plurality of As.

[0051] Additionally, the terms 'comprises' and / or 'comprising' used in the following embodiments do not exclude the presence or addition of one or more other components, steps, operations and / or elements.

[0052] In the present disclosure, the "system" may include, but is not limited to, at least one of a server device and a cloud device. For example, the system may be comprised of one or more server devices. As another example, the system may be comprised of one or more cloud devices. As yet another example, the system may be configured and operated by a combination of a server device and a cloud device.

[0053] FIG. 1 is a schematic diagram illustrating an example of generating synthetic radar data (150) according to one embodiment of the present disclosure. A synthetic radar data generating system (140) can receive point cloud data (110), information associated with a lidar sensor (120), and information associated with a radar sensor (130). The synthetic radar data generating system (140) can generate synthetic radar data (150) based on the point cloud data (110), information associated with a lidar sensor (120), and information associated with a radar sensor (130).

[0054] In one embodiment, point cloud data (110) may be data generated by a LiDAR (Light Detection And Ranging) sensor. Here, point cloud data (110) may refer to a set of data for representing three-dimensional space and objects. For example, point cloud data (110) may be generated by a LiDAR sensor mounted on a vehicle while the vehicle is driving. Three-dimensional virtual reality may be implemented using point cloud data (110). Each point in the point cloud data (110) may include location information (e.g., three-dimensional coordinate values) and signal intensity information. A visualized appearance of point cloud data (110) will be described in detail with reference to FIG. 4.

[0055] In one embodiment, the synthetic radar data generation system (140) may receive three-dimensional bounding box information associated with point cloud data (110). Here, the three-dimensional bounding box may refer to a minimum hexahedral region surrounding an object detected in an image. The three-dimensional bounding box information may be generated by a three-dimensional object recognition model. In another embodiment, the synthetic radar data generation system (140) may calculate and utilize three-dimensional bounding box information based on the point cloud data (110). In yet another embodiment, the synthetic radar data generation system (140) may receive three-dimensional bounding box information from a user, an external device, or the like.

[0056] In one embodiment, the 3D bounding box information may include at least one of position information, orientation information, class information, width information, height information, depth information, or relative velocity information. Here, the class information may be associated with the type of object. For example, the class information may include a class associated with a vehicle among dynamic objects. Additionally, the synthetic radar data generation system (140) may modify the class information (e.g., when the class information and the actual information of the recognized object are different) by receiving a specific input. In addition, the position information or relative velocity information may be information generated based on a lidar coordinate system (e.g., a coordinate system whose origin is the point where the lidar sensor is located).

[0057] In one embodiment, the information (120) associated with the lidar sensor may be information associated with the lidar sensor used to generate point cloud data (110). The information (120) associated with the lidar sensor may include a first extrinsic parameter associated with the lidar sensor. Here, the first extrinsic parameter may include position information of the lidar sensor within a specific coordinate system (e.g., an absolute coordinate system, a global coordinate system, etc.), direction information (e.g., viewing direction information, 6 DoF (Degree of Freedom) information, etc.), etc.

[0058] In one embodiment, the information (130) associated with the radar sensor may be information about a virtual radar sensor to be simulated. For example, the virtual radar sensor may be a subject that generates synthetic radar data (150). In one example, the information (130) associated with the radar sensor may include a second external parameter associated with the radar sensor, information on a field of view (FOV) of the radar sensor, an azimuth resolution (e.g., about 0.5 degrees), a detectable range (e.g., about 100 m), a detectable range resolution (e.g., about 0.5 m), etc. Here, the FOV of the radar sensor may mean a field of view that can be detected by the radar sensor. For example, the FOV information of the radar sensor may be 90 degrees. In addition, the second external parameter may include position information of the radar sensor within a specific coordinate system (e.g., an absolute coordinate system, a global coordinate system, etc.), direction information (e.g., view direction information), etc. Information (130) associated with such radar sensors can be determined / selected based on the synthetic radar data (150) to be generated.

[0059] The synthetic radar data generation system (140) can generate synthetic radar data (150) corresponding to the point cloud data (110) based on point cloud data (110) and 3D bounding box information. At this time, information (120) associated with the lidar sensor and information (130) associated with the radar sensor can be used together. Here, the synthetic radar data (150) can mimic data generated by the radar sensor. That is, the synthetic radar data (150) can be similar to data generated by direct measurement with the radar sensor. A specific process for generating the synthetic radar data (150) will be described with reference to FIG. 3.

[0060] With this configuration, synthetic radar data (150) can be generated based on data generated by a lidar sensor. Data generated by a radar sensor can be relatively more difficult to obtain than data generated by a lidar sensor. In this case, by generating synthetic radar data (150) based on data generated by a lidar sensor, which is easier to obtain, a large amount of radar data can be secured. This large amount of radar data can be used to train an artificial neural network model associated with the radar sensor.

[0061] To generate actual radar data, operation of a vehicle, drone, or other device equipped with a radar sensor may be required. The method for generating synthetic radar data according to the present disclosure can generate synthetic radar data without such operation, thereby significantly reducing data acquisition costs. Additionally, since the method for generating synthetic radar data according to the present disclosure utilizes point cloud data, it can generate synthetic radar data regardless of the type of lidar sensor. Furthermore, since the method for generating synthetic radar data according to the present disclosure utilizes point cloud data and 3D bounding box information, it can be equally applied not only to the real world but also to virtual reality, simulation environments, and the like.

[0062] FIG. 2 is a block diagram illustrating an internal configuration of a computing device (200) according to one embodiment of the present disclosure. The computing device (200) may include a memory (210), a processor (220), a communication module (230), and an input / output interface (240). As illustrated in FIG. 3, the computing device (200) may be configured to communicate information and / or data via a network using the communication module (230). Specifically, a user may utilize a synthetic radar data generation system (140) to generate synthetic radar data (150) of FIG. 1, and the synthetic radar data generation system (140) may include one or more computing devices (200).

[0063] The memory (210) may include any non-transitory computer-readable recording medium. According to one embodiment, the memory (210) may include a non-permanent mass storage device such as a random access memory (RAM), a read only memory (ROM), a disk drive, a solid state drive (SSD), a flash memory, etc. As another example, a non-permanent mass storage device such as a ROM, an SSD, a flash memory, a disk drive, etc. may be included in the computing device (200) as a separate permanent storage device distinct from the memory. In addition, the memory (210) may store an operating system and at least one program code (e.g., code for generating synthetic radar data installed and operated in the computing device (210).

[0064] These software components may be loaded from a computer-readable recording medium separate from the memory (210). This separate computer-readable recording medium may include a recording medium directly connectable to the computing device (200), for example, a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. As another example, the software components may be loaded into the memory (210) through a communication module (230) other than a computer-readable recording medium. For example, at least one program may be loaded into the memory (210) based on a computer program (e.g., a program for generating synthetic radar data, etc.) that is installed by files provided by developers or a file distribution system that distributes installation files of applications through the communication module (230).

[0065] The processor (220) may be configured to process commands of a computer program by performing basic arithmetic, logic, and input / output operations. The commands may be provided to a user terminal (not shown) or another external system via a memory (210) or a communication module (230). For example, the processor (220) may generate synthetic radar data corresponding to the point cloud data based on point cloud data and 3D bounding box information. The generated synthetic radar data may be provided to the user terminal or another external system.

[0066] The communication module (230) may provide a configuration or function for a user terminal (not shown) and a computing device (200) to communicate with each other via a network, and may provide a configuration or function for the computing device (200) to communicate with an external system (e.g., a separate cloud system, etc.). For example, control signals, commands, data, etc. provided under the control of the processor (220) of the computing device (200) may be transmitted to the user terminal and / or the external system via the communication module (230) and the network via the communication module of the user terminal and / or the external system.

[0067] In addition, the input / output interface (240) of the computing device (200) may be a means for interfacing with a device (not shown) for input or output that is connected to the computing device (200) or that the computing device (200) may include. In FIG. 2, the input / output interface (240) is illustrated as an element configured separately from the processor (220), but is not limited thereto, and the input / output interface (240) may be configured to be included in the processor (220). The computing device (200) may include more components than those illustrated in FIG. 2. However, there is no need to explicitly illustrate most of the conventional components.

[0068] The processor (220) of the computing device (200) may be configured to manage, process, and / or store information and / or data received from a plurality of user terminals and / or a plurality of external systems. According to one embodiment, the processor (220) may receive point cloud data generated by a lidar sensor. The processor (220) may receive three-dimensional bounding box information associated with the point cloud data. The processor (220) may generate synthetic radar data corresponding to the point cloud data based on the point cloud data and the three-dimensional bounding box information. At this time, the processor (220) may use information associated with the lidar sensor and information associated with the radar sensor stored in the memory (210) of the computing device (200).

[0069] FIG. 3 is a flowchart illustrating an example of a process (S300) for generating synthetic radar data according to one embodiment of the present disclosure. First, a processor (e.g., at least one processor included in the synthetic radar data generation system (140) of FIG. 1 ) may receive point cloud data generated by a lidar sensor. Here, the point cloud data may include a first set of points. Additionally, the processor may receive / generate 3D bounding box information associated with the point cloud data.

[0070] Thereafter, the processor can transform the first set of points into a spherical coordinate system (S310). The spherical coordinate system can be expressed by distance, azimuth angle, and elevation angle. Specifically, the spherical coordinate system can be expressed by r, Ф(0<Ф<2π), θ(-π / 2<θ<π / 2). For example, each of the first set of points can be expressed by the coordinates x, y, and z. Each of the transformed first set of points can be expressed by r, Ф, and θ.

[0071] The processor can identify a second set of points within the FOV of the radar sensor among the first set of points (S320). At this time, the first set of points may be converted into a spherical coordinate system. Here, the FOV of the radar sensor may refer to a field of view that can be detected by the radar sensor at a specific point in time. Specifically, the processor can identify points included within the FOV of the radar sensor expressed in a spherical coordinate system among the first set of points. The processor can identify the identified points as points of the second set. Thereafter, the identified points of the second set can be stored separately. Additionally or alternatively, points of the first set that are not identified as points of the second set can be removed.

[0072] The processor can divide an area corresponding to the FOV of the radar sensor into a plurality of bins (S330). Here, the area corresponding to the FOV may refer to an area that can be detected by the radar sensor at a specific point in time. The area corresponding to the FOV may be calculated based on the location information of the radar sensor, the FOV information, and the detectable distance information of the radar sensor. In addition, one bin may refer to a specific area that is divided and classified into segments of the target area. For example, one bin may be an area where the azimuth is 'the FOV of the radar sensor ÷ the azimuth resolution of the radar sensor' and the distance is 'the detectable distance of the radar sensor ÷ the detectable distance resolution of the radar sensor'. Each of the plurality of bins may have its own location information. A specific example of dividing the area corresponding to the FOV of the radar sensor into a plurality of bins will be described with reference to FIGS. 5 and 6.

[0073] The processor may map the points of the second set to a plurality of bins (S340). Specifically, each point of the second set may be mapped to a plurality of bins based on the location information of each point of the second set and the location information of each of the plurality of bins. Each mapped point of the second set may correspond to a single bin. For example, some of the plurality of bins may each include one or more points. The remaining bins may not include any points.

[0074] Although FIG. 3 illustrates mapping the points of the second set to multiple bins (S340), the present invention is not limited thereto. For example, the points of the first set may be mapped to multiple bins. At this time, the points of the first set may be converted to a spherical coordinate system. That is, the points of the first set may be mapped to multiple bins without being identified as points of the second set. At this time, by identifying only points included within the FOV of the radar sensor while performing the mapping, the points of the second set may be identified together with the mapping.

[0075] The processor can perform filtering on multiple bins (S350). Here, the multiple bins may be multiple bins in which mapping of multiple points has been performed. Specifically, the processor may remove only one point from bins containing two or more points among the multiple bins, leaving the remaining points. For example, if a bin contains three points, only one point may be removed from the three points, leaving two points. Here, the point to be removed may be determined in various ways. For example, the point to be removed may be determined arbitrarily. As another example, the point to be removed may be determined as the point closest to the center position of the bin. As another example, the point to be removed may be determined as the centroid point, i.e., the point corresponding to the mean coordinates of the points included in the bin.

[0076] The processor can perform dropping on a plurality of bins (S360). Here, the plurality of bins may be a plurality of bins for which mapping of a plurality of points has been performed or a plurality of bins for which filtering has been performed. Specifically, the processor may remove points for some of the bins containing one or more points among the plurality of bins. For example, some of the bins containing one or more points may be selected at random or at a predetermined probability. In this case, selecting the bins at random or at a predetermined probability may be performed independently for each bin. Point(s) included in some of the selected bins containing one or more points may be removed.

[0077] In one embodiment, the processor may perform dropping on a plurality of filtered bins. Some of the filtered bins may each contain a point, while others of the filtered bins may not contain any points. In this case, dropping may be performed by removing points from some of the bins containing the point. For example, some of the bins containing the point may be selected at random or predetermined probability. Point(s) contained in the selected portion of the bins containing the point may be removed.

[0078] The processor can determine whether a point is included in a three-dimensional bounding box (S370). For example, the point may be one of the points in the first set described above. As another example, the point may be one of the points in the second set described above. As another example, the point may be one of the points included in a plurality of bins on which filtering has been performed. As another example, the point may be one of the points included in a plurality of bins on which dropping has been performed. Specifically, the processor can determine whether a point is included in a three-dimensional bounding box based on position information of the point and position information of the three-dimensional bounding box. For example, the processor can identify an interior region of the three-dimensional bounding box based on the position information of the three-dimensional bounding box. The processor can determine whether a point is included in an interior region of the identified three-dimensional bounding box based on the position information of the point.

[0079] The processor may determine a point not included in the three-dimensional bounding box as one of the points of the third set (S372). Furthermore, the processor may determine a point included in the three-dimensional bounding box as one of the points of the fourth set (S374). That is, the points of the third set may include points not included in the three-dimensional bounding box, and the points of the fourth set may include points included in the three-dimensional bounding box. Here, the points of the fourth set may be a set of points associated with a dynamic object, and the points of the third set may be a set of points associated with a static object.

[0080] The processor may generate Radar Cross Section (RCS) information associated with a third set of points (S382). Here, the RCS information may indicate electromagnetic wave reflection characteristics of a radar sensor for an object. For example, a target object may be inferred based on the RCS information. As such, the RCS information may be information generated by the radar sensor. Specifically, the processor may determine that the third set of points are associated with a dynamic object. Thereafter, RCS information for the third set of points may be generated based on information from a three-dimensional bounding box. The unit of the RCS information may be dBm2 or dBsm (decibel square meter).

[0081] In one example, a first 3D bounding box may include some of the points in a third set. In this case, the class information of the first 3D bounding box may be associated with a pedestrian. In this case, each point in the third set of points included in the first 3D bounding box may be assigned a value of -5 to 0 as RCS information. For example, each point in the third set of points included in the first 3D bounding box may be assigned 0 as RCS information. In another example, a second 3D bounding box may include some of the points in the third set. In this case, the class information of the second 3D bounding box may be associated with a vehicle. In this case, each point in the third set of points included in the second 3D bounding box may be assigned a value of -5 to 25 as RCS information. For example, each point included in the second three-dimensional bounding box among the third set of points may be assigned 10 or 20 as RCS information.

[0082] The RCS information may be determined based on the orientation or position information of the 3D bounding box or the object associated with the 3D bounding box. For example, if the radar sensor is oriented toward the left, right, or rear of the vehicle (or the second 3D bounding box in the example described above), each point in the third set of points included in the second 3D bounding box may be assigned a value of 23 as RCS information. As another example, if the radar sensor is oriented toward the front of the vehicle (or the second 3D bounding box in the example described above), each point in the third set of points included in the second 3D bounding box may be assigned a value of 15 as RCS information. The processor may read a memory (e.g., at least one memory included in the synthetic radar data generation system (140) of FIG. 1) to identify RCS information corresponding to the orientation or position information of an object such as a vehicle, and assign RCS information to points based on the identified information.

[0083] The processor may generate Radar Cross Section (RCS) information associated with a fourth set of points (S384). Specifically, the processor may determine that the fourth set of points are associated with static objects. Thereafter, the processor may generate RCS information for each of the fourth set of points based on the elevation information of each of the fourth set of points. For example, if the elevation information for a point in the fourth set is less than 0 degrees, the point may be assumed to be associated with a road, a pedestrian, etc. In this case, -40 may be assigned as RCS information for the point. As another example, if the elevation information for a point in the fourth set is greater than 0 degrees, the point may be assumed to be associated with a building, a stationary obstacle (e.g., a streetlight, a tree, etc.), etc. In this case, -10 may be assigned as RCS information for the point.

[0084] Thereafter, the processor can calculate relative velocity information for each of the third set of points or the fourth set of points for which RCS information was generated. The method for calculating relative velocity information for each of the third set of points determined to be associated with a dynamic object is described with reference to FIG. 7. The method for calculating relative velocity information for each of the fourth set of points determined to be associated with a static object is described with reference to FIG. 8.

[0085] Referring to FIG. 3, the step (S370) of determining whether a point is included in a 3D bounding box is illustrated as being performed after dropping is performed on a plurality of bins, but is not limited thereto. For example, the step (S370) of determining whether a point is included in a 3D bounding box may be performed before the step (S350) of performing filtering on a plurality of bins. That is, after generating RCS information by determining whether a point mapped to a plurality of bins is included in a 3D bounding box, filtering and / or dropping may be performed on the plurality of bins. As another example, the step (S370) of determining whether a point is included in a 3D bounding box may be performed before the step (S330) of dividing an area corresponding to the FOV of the radar sensor into a plurality of bins.

[0086] The generated synthetic radar data includes a plurality of points, and each point may include one of position information, distance information, relative velocity information, and RCS information. By the above-described configuration, position information, distance information, relative velocity information (described with reference to FIGS. 7 and 8), and RCS information of each point can be calculated / generated. The information of the points included in the synthetic radar data may include the same elements as the information of the points generated by an actual radar sensor. In this way, the synthetic radar data can simulate data generated by the radar sensor by including data elements that the radar sensor can generate.

[0087] FIG. 4 is a diagram illustrating an example of point cloud data according to an embodiment of the present disclosure. The first example (410) may be a camera image captured by a camera positioned within a virtual reality. Here, the virtual reality may be rendered using a 3D model. In addition, the camera may be a virtual camera positioned within the virtual reality. The second example (420) may be an image visualizing point cloud data generated by a lidar sensor at the same location as the camera positioned within the virtual reality. The white line shown in the second example (420) may indicate an area at the same distance from the lidar sensor.

[0088] In one embodiment, a three-dimensional bounding box (422) may be generated based on a 3D model of an object in virtual reality. For example, the three-dimensional bounding box (422) may be generated based on width information, height information, depth information, class information, etc. of the 3D model of the object. In another embodiment, the three-dimensional bounding box (422) may be generated by performing object recognition on point cloud data. The three-dimensional bounding box information may include at least one of position information, orientation information, class information, width information, height information, depth information, or relative velocity information for the three-dimensional bounding box.

[0089] In another embodiment, a 3D bounding box may be generated based on images associated with point cloud data. For example, a camera may be used to generate multiple images at multiple points in time when the point cloud data was generated. At this time, object recognition may be performed on the multiple images to generate a 3D bounding box. The generated 3D bounding box may be projected onto the point cloud data based on information associated with the lidar sensor and information associated with the image sensor, corresponding to each point in time. This 3D bounding box may be represented as a 3D bounding box (422) in the second example (420).

[0090] In one embodiment, 3D bounding boxes can be generated only for dynamic objects. Alternatively, 3D bounding boxes can also be generated for static objects. In this case, the class information of the static object can be the same regardless of its target. For example, the class information for a static object being a road and a static object being a building can be the same because they correspond to information associated with "not interested."

[0091] Referring to FIG. 4, in the first example (410), a 3D bounding box can be inserted into virtual reality, and the inserted 3D bounding box (412) can be displayed. In the second example (420), a 3D bounding box (422) generated based on point cloud data can be displayed. In FIG. 4, a 3D bounding box is displayed for a dynamic object (e.g., a cyclist, a vehicle), but the present invention is not limited thereto. For example, a 3D bounding box can be displayed for a static object (e.g., a building, a tree, a streetlight, etc.).

[0092] The first example (410) and the second example (420) are merely images visualizing a 3D bounding box and point cloud data, and synthetic radar data may not be generated based on these 3D bounding boxes and point cloud data. For example, the processor may generate synthetic radar data based on unvisualized 3D bounding boxes and point cloud data.

[0093] FIG. 5 is a diagram illustrating an example of a mapping, filtering, and dropping process for a plurality of bins according to one embodiment of the present disclosure. A first example (510) may represent a first set of points mapped to a plurality of bins. A second example (520) may represent a second set of filtered points. A third example (530) may represent a third set of dropped points. In the first example (510), the second example (520), and the third example (530), the X-axis may represent azimuth, and the Y-axis may represent distance.

[0094] In the first example (510), the first set of points may be points included in the point cloud data. For example, the first set of points may be points of the first set transformed into the spherical coordinate system described with reference to FIG. 3. As another example, the first set of points may be points of the second set identified as being within the FOV described with reference to FIG. 3. As another example, the first set of points may be points of the first set or points of the second set determined to be included in the three-dimensional bounding box described with reference to FIG. 3.

[0095] Referring to FIG. 5, a radial area may be displayed in the first example (510). The area corresponds to the FOV of the radar sensor and may be determined by the location information, FOV information, and detectable distance of the radar sensor. In addition, it can be confirmed that the area corresponding to the FOV in the first example (510) is divided into multiple grid patterns. One divided area in the grid patterns may represent one bin. That is, it can be confirmed that the area corresponding to the FOV in the first example (510) is divided into multiple bins. Similar to the first example (510), the second example (520) and the third example (530) may also display areas corresponding to the FOV.

[0096] The second example (520) may represent a filtered second set of points. Here, the points of the second set may be some of the points of the mapped first set illustrated in the first example (510). Filtering of the points of the first set may be performed by performing the step (S350) of performing filtering on multiple bins described with reference to FIG. 3. As can be seen in the second example (520), some of the points of the first set may be removed by performing the filtering.

[0097] The third example (530) may represent a third set of dropped points. Here, the third set of points may be some of the filtered second set of points illustrated in the second example (520). By performing the step (S360) of performing dropping on multiple bins described with reference to FIG. 3, dropping on the second set of points may be performed. As can be seen in the third example (530), performing dropping on the second set of points may result in the removal of some of the points in the second set.

[0098] As explained with reference to Figure 3, some points can be removed using probability during the filtering and / or dropping process for multiple bins. In this case, by finely adjusting the probability, the synthetic radar data generation method can be made more similar to data generated by radar sensors. For example, the probability can be finely adjusted by receiving user input, utilizing statistical values, or utilizing an artificial neural network model.

[0099] The density of points in point cloud data can be relatively higher than in typical radar data. The synthetic radar data generated by the above-described configuration can more closely mimic data generated by radar sensors by reducing the density of points measured by the lidar sensor.

[0100] FIG. 6 is a diagram illustrating an example of a mapping, filtering, and dropping process for a plurality of bins according to one embodiment of the present disclosure. A first example (610) may represent a first set of points mapped to a plurality of bins. A second example (620) may represent a second set of filtered points. A third example (630) may represent a third set of dropped points. In the first example (610), the second example (620), and the third example (630), the X-axis may represent azimuth, and the Y-axis may represent distance. Here, the Y-axis may be displayed in a logarithmic scale. FIG. 6 may be understood based on the description of FIG. 5. FIG. 6 will be described focusing on differences from FIG. 5.

[0101] In one embodiment, the first set of points illustrated in the first example (610) may include the third set of points and the fourth set of points described with reference to FIG. 3 . That is, each point in the first set may be determined to be associated with a dynamic object or a static object. The fifth set of points (612) may be a set of points determined to be included in a specific three-dimensional bounding box, as part of the third set of points. In this case, each point in the fifth set (612) may have the same RCS value.

[0102] By performing filtering on multiple bins, some of the points (612) of the fifth set may be removed. Referring to the second example (620), the appearance of the filtered points (622) of the fifth set can be confirmed. It can be visually confirmed that the density of the points (622) of the filtered fifth set is lower than that of the points (612) of the fifth set.

[0103] By performing dropping on multiple bins, some of the filtered fifth set of points (622) may be removed. Referring to the third example (630) of FIG. 6 , the appearance of the dropped fifth set of points (632) can be confirmed. It can be visually confirmed that the dropped fifth set of points (632) has a lower density than the filtered fifth set of points (622).

[0104] FIG. 7 is a diagram illustrating a process of calculating the Doppler velocity of a point associated with a dynamic object (710) according to an embodiment of the present disclosure. Specifically, FIG. 7 may show a radar sensor (730) measuring a dynamic object (710) at a specific point in time. FIG. 7 may assume a situation in which a dynamic object is detected by a virtual radar sensor (730), rather than actually detected by the radar sensor. In this case, the subject (720) equipped with the radar sensor is v ego moving at a speed of v, and the dynamic object (710) -target- It can move at a speed of . Here, the subject (720) equipped with a radar sensor may be equipped with a lidar sensor (not shown) together with a radar sensor (730). The position and speed described in Fig. 7 may mean a vector.

[0105] The radar sensor (730) can sense multiple points (712_1 to 712_n) for a dynamic object (710). The distance and direction to the first point (712_1) measured by the radar sensor (730) can be represented as r1. Similarly, the distance and direction to the n-th point (712_n) measured by the radar sensor (730) can be represented as r n can be expressed as

[0106] Velocity (e.g., Doppler velocity) may mean a relative velocity with respect to a radial direction from the radar sensor (730) toward the point. For example, for the first point (712_1), the Doppler velocity is associated with r1, which is a radial direction from the radar sensor (730) toward the first point (712_1). It can mean the relative velocity with respect to the direction. Also, for the n-th point (712_n), the Doppler velocity is r, which is the radial direction from the radar sensor (730) toward the n-th point (712_n). n Associated with It can mean relative speed with respect to direction.

[0107] Two assumptions can be made to derive the Doppler velocity. First, it can be assumed that the dynamic object (710) moves in a planar manner. For example, only the motion of the dynamic object (710) when the elevation angle is 0 degrees can be considered in the data of the radar sensor (730). Second, when the rotational velocity of the dynamic object (710), such as a vehicle, is slow (for example, when the rotational velocity of the dynamic object (710) is less than a predetermined rotational velocity), it can be assumed that the velocities of the respective reflected / scattered points of the dynamic object (710) are the same. For example, the velocity of the first point (712_1) and the velocity of the n-th point (712_n) can be the same as the velocity of the dynamic object reference point observed in the radar coordinate system. As another example, the velocity of the subject (720) equipped with the radar sensor can be the same as the velocity of the radar sensor (730) and the velocity of the lidar sensor. Here, equal velocity can mean equal absolute velocity in the same coordinate system (e.g. radar coordinate system, lidar coordinate system).

[0108] The Doppler velocity of the nth point (712_n) can be defined as in the following mathematical expression 1. In FIG. 7, the dynamic object (710) is depicted as moving away from the radar sensor (730), but the following mathematical expression 1 can be defined based on the case where the dynamic object (710) approaches the radar sensor (730).

[0109]

[0110] [Correction pursuant to Rule 91, March 27, 2025] [Formula 1]

[0111]

[0112] The relative velocity of point n in the radar's coordinate system is v n radar , and the position of the nth point in the radar coordinate system is r n radar It can be. Also, v target radar is the velocity of the dynamic object in the radar's coordinate system, v radar radar is the velocity of the radar sensor (730) in the radar's coordinate system, v ego radar can refer to the speed of a subject (720) equipped with a radar sensor in the coordinate system of the radar. By the second assumption described above, v radar radar = v ego radar Since it satisfies , the relative velocity of the nth point in the radar's coordinate system is v n radar = v target radar - v radar radar = v target radar - v ego radar It can be calculated as in (Formula 1).

[0113] R lidar radar is a rotation matrix that rotates the lidar coordinate system into the radar coordinate system, and t radarlidar If is a translation vector that translates the lidar coordinate system to the radar coordinate system, then the equation for converting the velocity from the lidar coordinate system to the radar coordinate system is v target radar - v ego radar = R lidar radar (v target lidar - v ego lidar )(Equation 2) can be satisfied. In addition, the equation for converting the position from the lidar coordinate system to the radar coordinate system is r n radar = R lidar radar (r n lidar - t radar lidar )(Equation 3) can be satisfied. At this time, the relative velocity of the 3D bounding box described in the lidar coordinate system is v box lidar If you say, v box lidar = v target lidar - v lidar lidar = v target lidar - v ego lidar (Equation 4) can be calculated as follows. Using Equations 1, 2, and 4, v n radar = R lidar radar (v target lidar - v ego lidar ) can satisfy (Equation 5)

[0114] By substituting Equations 3 and 5 into Equation 1, Equation 2 below can be derived.

[0115]

[0116] [Correction pursuant to Rule 91, March 27, 2025] [Formula 2]

[0117]

[0118] In this way, the processor can calculate the Doppler velocity of each point associated with a dynamic object using Equation 2. The process of calculating the Doppler velocity of each point associated with a static object is described with reference to FIG. 8.

[0119] FIG. 8 is a diagram illustrating a process of calculating a Doppler velocity associated with a static object according to an embodiment of the present disclosure. Specifically, FIG. 8 may show a situation in which a radar sensor (830) measures a static object at a specific point in time. FIG. 8 may assume a situation in which a static object is detected by a virtual radar sensor (830), rather than actually detected by a radar sensor. In this case, a subject (820) equipped with a radar sensor is v ego It can move at a speed of . Here, the subject (820) equipped with a radar sensor may be equipped with a lidar sensor (not shown) together with the radar sensor (730). The position and speed described in Fig. 8 may mean a vector.

[0120] The radar sensor (830) can sense multiple points (810_1 to 810_n) for a static object. The distance and direction of the first point (810_1) measured by the radar sensor (830) are denoted as r1. can be expressed. Similarly, the distance and direction of the nth point (810_n) measured by the radar sensor (830) are r n can be expressed as

[0121] Referring to FIG. 8, the plurality of points (810_1 to 810_n) measured by the radar sensor (830) are illustrated as being associated with separate static objects, but are not limited thereto. For example, at least some of the plurality of points (810_1 to 810_n) may be points measured by the radar sensor (830) for separate points associated with a single static object.

[0122] The velocity (e.g., Doppler velocity) may mean the relative velocity with respect to the radial direction from the radar sensor (830) toward the point. For example, for the first point (810_1), the Doppler velocity may mean the relative velocity with respect to the direction associated with r1, which is the radial direction from the radar sensor (830) toward the first point (810_1). In addition, the Doppler velocity for the n-th point (810_n) may be calculated in the same manner as in Equation 2. In order to calculate the Doppler velocity, the two assumptions described above in FIG. 7 may be assumed. That is, the velocity of a static object in the coordinate system with respect to the ground is 0, and by the second assumption described in FIG. 7, v lidar lidar = v ego lidar can satisfy. The relative velocity of the three-dimensional bounding box described above in the lidar coordinate system is v box lidar If so, then v as in Equation 4 described above in Fig. 7 box lidar can be produced. At this time, it can be assumed that a 3D bounding box can be generated even for static objects.

[0123] R ground lidar is a rotation matrix that rotates the ground coordinate system to the lidar coordinate system, and v box ground can be 0 as the velocity of the 3D bounding box in the ground coordinate system. Also, v lidar ground -is the velocity of the lidar sensor in the ground coordinate system, and R ego lidar may be a rotation matrix that rotates the coordinate system of the subject (820) equipped with the radar sensor to the lidar coordinate system. In addition, R ground ego can be a rotation matrix that rotates the ground coordinate system into the lidar coordinate system. At this time, if Equation 4 described above in Fig. 7 is rearranged, v box lidar = v target lidar - v lidar lidar = R grond lidar (v box ground - v lidar ground - ) = - R grond lidar v ego ground = - R ego lidar R ground ego v ego ground (Equation 6) can be organized as shown below. If Equation 6 is substituted into Equation 2 above, it can be organized as shown in Equation 3 below.

[0124]

[0125] [Correction pursuant to Rule 91, March 27, 2025] [Formula 3]

[0126]

[0127] In this way, the processor can calculate the Doppler velocity of each point associated with the static object using Equation 3.

[0128] In summary, the processor can calculate the Doppler velocity of each point associated with a dynamic object using Equation 2, and can calculate the Doppler velocity of each point associated with a static object using Equation 3. Accordingly, the processor can calculate the Doppler velocity of each point based on at least one of point cloud data (e.g., position information of each point, relative velocity information, etc.), information associated with a lidar sensor (e.g., a first external parameter associated with a lidar sensor, etc.), information associated with a radar sensor (e.g., a second external parameter associated with a radar sensor, etc.), and / or 3D bounding box information (e.g., relative velocity information of a 3D bounding box). The processor can generate relative velocity information of points included in synthetic radar data based on the Doppler velocity of each point.

[0129] FIG. 9 is a flowchart illustrating an example of a synthetic radar data generation method (900) according to one embodiment of the present disclosure. According to one embodiment, the synthetic radar data generation method (900) may be performed by at least one processor of the synthetic radar data generation system (140) of FIG. 1. The synthetic radar data generation method (900) may be initiated by the processor receiving point cloud data generated by a lidar sensor (S910).

[0130] In one embodiment, the processor may receive 3D bounding box information associated with point cloud data (S920). For example, the 3D bounding box information may include at least one of position information, orientation information, class information, width information, height information, depth information, or relative velocity information.

[0131] In one embodiment, the processor may generate synthetic radar data corresponding to the point cloud data based on the point cloud data and 3D bounding box information (S930). Here, the synthetic radar data may mimic data generated by a radar sensor. Additionally, the point cloud data may include a first set of points. For example, the synthetic radar data may include a plurality of points, and each point may include at least one of position information, distance information, relative velocity information, or RCS information.

[0132] Specifically, the processor may receive a first external parameter associated with the lidar sensor. Furthermore, the processor may receive a second external parameter associated with the radar sensor. Then, based on the first external parameter and the second external parameter, a second set of points within a field of view (FOV) of the radar sensor may be identified among the first set of points. Here, the processor may convert the first set of points into a spherical coordinate system based on the first external parameter and the second external parameter. At this time, positional information of the second set of points may be expressed in the spherical coordinate system.

[0133] Specifically, the processor may receive field of view (FOV) information from the radar sensor. Thereafter, the processor may divide the area corresponding to the field of view (FOV) of the radar sensor into a plurality of bins. Furthermore, the processor may map the first set of points to the plurality of bins. For bins containing two or more points among the plurality of bins, the processor may remove all but one point from the bin.

[0134] Additionally or alternatively, the processor may map the points of the first set to a plurality of bins. Furthermore, the processor may remove points from some of the bins that contain a point from one of the plurality of bins.

[0135] In one embodiment, the processor may determine that a third set of points, among the second set of points, are included in the three-dimensional bounding box and are associated with a dynamic object. Furthermore, the processor may determine a fourth set of points, among the second set of points and not included in the three-dimensional bounding box, are associated with a static object. Each point in the fourth set may include distance, elevation, and azimuth information.

[0136] In one embodiment, the processor may generate RCS information associated with a third set of points based on the three-dimensional bounding box information. Additionally, the processor may generate RCS information for each of the fourth set of points based on the elevation information.

[0137] In one embodiment, the processor may calculate relative velocity information for a third set of points based on three-dimensional bounding box information, a first external parameter, and a second external parameter. Furthermore, the processor may calculate relative velocity information for each of the fourth set of points based on positional information for each of the fourth set of points, the first external parameter, and the second external parameter.

[0138] The above-described method may be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program instructions, including ROM, RAM, and flash memory. In addition, examples of other media may include recording or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.

[0139] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will appreciate that the various exemplary logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, various exemplary components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software will depend on the particular application and the design requirements imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each particular application, but such implementations should not be construed as departing from the scope of the present disclosure.

[0140] In a hardware implementation, the processing units used to perform the techniques may be implemented within one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, a computer, or a combination thereof.

[0141] Accordingly, the various exemplary logical blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed by any combination of a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0142] In a firmware and / or software implementation, the techniques may be implemented as instructions stored on a computer-readable medium, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, a compact disc (CD), a magnetic or optical data storage device, etc. The instructions may be executable by one or more processors and may cause the processor(s) to perform certain aspects of the functionality described herein.

[0143] When implemented in software, the techniques described above may be stored on or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media, including any medium that facilitates transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium.

[0144] For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, digital subscriber line, or wireless technologies such as infrared, radio, and microwave are included within the definition of media. Disk and disc, as used herein, includes compact discs, laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks usually reproduce data magnetically, whereas discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0145] A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium may be coupled to the processor such that the processor can read information from, and write information to, the storage medium. Alternatively, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. Alternatively, the processor and the storage medium may reside as discrete components in the user terminal.

[0146] While the embodiments described above have been described as utilizing aspects of the presently disclosed subject matter in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or distributed computing environment. Furthermore, aspects of the present disclosure may be implemented in multiple processing chips or devices, and storage may be similarly affected across multiple devices. Such devices may include personal computers, network servers, and portable devices.

[0147] While the present disclosure has been described in connection with certain embodiments herein, various modifications and variations may be made without departing from the scope of the present disclosure, which would be apparent to those skilled in the art. Furthermore, such modifications and variations are intended to fall within the scope of the claims appended to this specification.

Claims

1. A method for generating synthetic radar (RADAR: RAdio Detection And Ranging) data, performed by at least one processor, A step of receiving point cloud data generated by a LiDAR (Light Detection And Ranging) sensor; A step of receiving 3D bounding box information associated with the above point cloud data; and A step of generating synthetic radar data corresponding to the point cloud data based on the point cloud data and the 3D bounding box information. Including, The above synthetic radar data is a method for generating synthetic radar data that simulates data generated by a radar sensor.

2. In paragraph 1, A method for generating synthetic radar data, wherein the three-dimensional bounding box information includes at least one of position information, orientation information, class information, width information, height information, depth information, or relative velocity information.

3. In paragraph 1, The above point cloud data includes a first set of points, The step of generating the above synthetic radar data is: A step of receiving a first extrinsic parameter associated with the lidar sensor; A step of receiving a second external parameter associated with the radar sensor; and A step of identifying a second set of points within the FOV (Field Of View) of the radar sensor among the first set of points based on the first external parameter and the second external parameter. A method for generating synthetic radar data, comprising:

4. In paragraph 3, The step of identifying the points of the second set is: A step of converting the points of the first set into a spherical coordinate system based on the first external parameter and the second external parameter. Including, A method for generating synthetic radar data, wherein the position information of the second set of points is expressed in a spherical coordinate system.

5. In paragraph 3, The step of generating the above synthetic radar data is: A step of determining a third set of points included in the three-dimensional bounding box among the second set of points as being associated with a dynamic object; and A step of determining that a fourth set of points among the second set of points that are not included in the three-dimensional bounding box are associated with a static object. A method for generating synthetic radar data, further comprising:

6. In paragraph 5, Each of the points in the fourth set above includes distance information, elevation information, and azimuth information, The step of generating the above synthetic radar data is: A step of generating RCS (Radar Cross Section) information associated with the third set of points based on the three-dimensional bounding box information; and A step of generating RCS information for each point of the fourth set based on the elevation information. A method for generating synthetic radar data, further comprising:

7. In paragraph 5, The step of generating the above synthetic radar data is: A step of calculating relative velocity information of the third set of points based on the three-dimensional bounding box information, the first external parameter, and the second external parameter; and A step of calculating relative velocity information of each point of the fourth set based on the position information of each point of the fourth set, the first external parameter, and the second external parameter. A method for generating synthetic radar data, further comprising:

8. In paragraph 3, The step of identifying the points of the second set is: A step of receiving FOV information of the above radar sensor; A step of dividing an area corresponding to the FOV of the radar sensor into a plurality of bins; a step of mapping the points of the first set to the plurality of bins; and A step of removing points from bins containing two or more points among the above plurality of bins, leaving only one point within the bin. A method for generating synthetic radar data, comprising:

9. In paragraph 3, The step of identifying the points of the second set is: A step of receiving FOV information of the above radar sensor; A step of dividing an area corresponding to the FOV of the radar sensor into a plurality of bins; a step of mapping the points of the first set to the plurality of bins; and A step of removing points for some of the bins that contain one point among the above plurality of bins. A method for generating synthetic radar data, comprising:

10. In paragraph 1, The above synthetic radar data includes multiple points, A method for generating synthetic radar data, wherein each point includes at least one of position information, distance information, relative velocity information, or RCS information.

11. A computer-readable non-transitory recording medium recording commands for executing the method according to paragraph 1 on a computer.

12. As an information processing system, Communication module; memory; and At least one processor connected to said memory and configured to execute at least one computer-readable program contained in said memory Including, At least one program above, Receive point cloud data generated by the lidar sensor, Receive 3D bounding box information associated with the above point cloud data, Based on the point cloud data and the 3D bounding box information, it includes commands for generating synthetic radar data corresponding to the point cloud data, The above synthetic radar data is an information processing system that simulates data generated by a radar sensor.

Images