Data generation method, data learning method, and computing device

Abstract

A data generation method of a computing device according to an embodiment of the present invention comprises the steps of: generating first noise data for an intensity image and second noise data for a depth image by using the physical characteristics of a scene including a retro-reflective material; generating a synthesized intensity image by adding the first noise data to an original intensity image; and generating a synthesized depth image by adding the second noise data to an original depth image corresponding to the original intensity image.

Description

Data generation method, data learning method, and computing device

[0001] The present invention relates to a data generation method, a data learning method using the same, and a computing device.

[0002] LiDAR (Light Detection and Ranging) sensors are essential for acquiring 3D spatial information, and they acquire 3D spatial information by measuring the time of flight (TOF) of a light-emitting signal. LiDAR sensors are widely used in robots and autonomous vehicles. Typically, single photon avalanche diode (SPAD) image sensors are used for direct TOF measurements and can have high sensitivity. However, SPAD image sensors can suffer from various noises such as pile-up, dark current, stray light, and crosstalk. Crosstalk includes interference between multiple LiDAR sensors and interference between adjacent pixels within a SPAD image sensor. Crosstalk between adjacent pixels is called the blooming effect. The blooming effect occurs when the light-emitting signal from a LiDAR sensor is reflected from a strongly reflective surface or bright object, such as a retro-reflective (RR) material such as a traffic sign. This causes internal stray light to spread to adjacent pixels, resulting in distortion of point cloud information.

[0003] Because the blooming effect distorts the original shape and size of an object, thereby increasing the likelihood of incorrect target detection or classification, it is necessary to obtain point cloud information without the blooming effect.

[0004] Training-based techniques have been attempted to remove blooming effects from acquired images, but they require a large number of pairs of blooming-corrupted and blooming-free images. However, obtaining these dataset pairs is expensive and time-consuming.

[0005] The technical problem to be solved by the present invention is to provide a data generation method, a data learning method using the same, and a computing device.

[0006] The technical problem to be achieved by the present invention is to provide a method for efficiently acquiring a large number of pairs of blooming corrupted images and blooming free images and training them.

[0007] A data generation method of a computing device according to one embodiment of the present invention includes the steps of generating first noise data for an intensity image and second noise data for a depth image by using physical characteristics of a scene including a retroreflective material, generating a synthetic intensity image by adding the first noise data to an original intensity image, and generating a synthetic depth image by adding the second noise data to an original depth image corresponding to the original intensity image.

[0008] A scene including the above retroreflective material may include a target area, a first peripheral area disposed on one side of the target area, and a second peripheral area disposed on the other side of the target area.

[0009] The first noise data may be generated such that an intensity value of the target area is greater than an intensity value of the first peripheral area and an intensity value of the second peripheral area, and the second noise data may be generated such that a depth value of the target area is equal to a depth value of the first peripheral area and a depth value of the second peripheral area.

[0010] The intensity value of the first peripheral area and the intensity value of the second peripheral area may decrease exponentially as they move away from the target area.

[0011] The intensity value of the first peripheral region may exponentially decrease from the intensity value of the target region to the intensity value outside the first peripheral region, and the intensity value of the second peripheral region may exponentially decrease from the intensity value of the target region to the intensity value outside the second peripheral region.

[0012] The length of the first peripheral region in the direction from the target region toward the first peripheral region may be different from the length of the second peripheral region in the direction from the target region toward the second peripheral region.

[0013] The above original intensity image and the above original depth image may be images acquired by a lidar device.

[0014] The above synthetic intensity image and the above synthetic depth image may include blooming artifacts.

[0015] A data learning method of a computing device according to one embodiment of the present invention includes the steps of receiving a synthetic intensity image generated by adding first noise data for an intensity image to an original intensity image and a synthetic depth image generated by adding second noise data for a depth image to an original depth image corresponding to the original intensity image, a step of training the synthetic intensity image and the synthetic depth image, and a step of outputting a noise mask indicating a possibility of distortion due to noise for each pixel, wherein the first noise data and the second noise data are generated using physical characteristics of a scene including a retroreflective material.

[0016] A scene including the above retroreflective material may include a target area, a first peripheral area disposed on one side of the target area, and a second peripheral area disposed on the other side of the target area.

[0017] The first noise data may be generated such that an intensity value of the target area is greater than an intensity value of the first peripheral area and an intensity value of the second peripheral area, and the second noise data may be generated such that a depth value of the target area is equal to a depth value of the first peripheral area and a depth value of the second peripheral area.

[0018] The intensity value of the first peripheral area and the intensity value of the second peripheral area may decrease exponentially as they move away from the target area.

[0019] The above synthetic intensity image and the above synthetic depth image can be trained using a deep learning-based network.

[0020] A computing device according to one embodiment of the present invention comprises: a communication unit for obtaining an original intensity image and an original depth image corresponding to the original intensity image; and a processor connected to the communication unit, wherein the processor is configured to generate first noise data for the intensity image and second noise data for the depth image using physical characteristics of a scene including a retroreflective material, generate a synthetic intensity image by adding the first noise data to the original intensity image, and generate a synthetic depth image by adding the second noise data to the original depth image.

[0021] The processor may be configured to train the synthetic intensity image and the synthetic depth image to output a noise mask indicating a possibility of distortion due to pixel-by-pixel noise.

[0022] According to an embodiment of the present invention, a pair of blooming-corrupted and blooming-free images can be efficiently acquired. Accordingly, blooming effects can be effectively removed from images acquired by a lidar sensor, thereby obtaining accurate 3D information.

[0023] Figure 1 is an example of the blooming effect according to a retroreflective traffic sign.

[0024] Figure 2 illustrates the principle of how the blooming effect occurs within a single photon avalanche diode (SPAD) array.

[0025] FIG. 3 is a flowchart illustrating a method for a computing device to synthesize a blooming damage image according to one embodiment of the present invention.

[0026] Figure 4 is an example of a scene containing a retroreflective material acquired by a lidar sensor.

[0027] Figures 5a and 5b are drawings for explaining the physical characteristics of the blooming effect for an intensity image.

[0028] Figures 6a and 6b are drawings for explaining the physical characteristics of the blooming effect for a depth image.

[0029] FIG. 7 is a flowchart of a method for generating noise data by a computing device according to an embodiment of the present invention.

[0030] Fig. 8a is an example of noise data generated by the method of Fig. 7, and Fig. 8b is a physical characteristic of noise data generated by the method of Fig. 7.

[0031] FIG. 9a is an example of an original intensity image and an original depth image, and FIG. 9b is an example of a synthetic intensity image and a synthetic depth image generated according to an embodiment of the present invention.

[0032] Figure 10 is a block diagram of a deep learning-based network according to one embodiment of the present invention.

[0033] Figure 11 is a flowchart illustrating a data learning method of a computing device according to one embodiment of the present invention.

[0034] Figure 12 is a conceptual diagram of a deep learning-based network according to one embodiment of the present invention.

[0035] FIG. 13 and FIG. 14(a) to FIG. 14(d) are diagrams for explaining the results of verifying data generation and learning effects according to an embodiment of the present invention.

[0036] FIG. 15 is a block diagram of a computing device according to one embodiment of the present invention.

[0037] Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the attached drawings.

[0038] However, the technical idea of ​​the present invention is not limited to some of the embodiments described, but can be implemented in various different forms, and within the scope of the technical idea of ​​the present invention, one or more of the components between the embodiments can be selectively combined or substituted for use.

[0039] In addition, terms (including technical and scientific terms) used in the embodiments of the present invention may be interpreted as having a meaning that can be generally understood by a person of ordinary skill in the technical field to which the present invention belongs, unless explicitly and specifically defined and described, and terms that are commonly used, such as terms defined in a dictionary, may be interpreted in consideration of the contextual meaning of the relevant technology.

[0040] Additionally, the terms used in the embodiments of the present invention are intended to describe the embodiments and are not intended to limit the present invention.

[0041] In this specification, the singular may also include the plural unless specifically stated otherwise in the phrase, and when it is described as “A and / or at least one (or more) of B, C”, it may include one or more of all combinations that can be combined with A, B, C.

[0042] Additionally, in describing components of embodiments of the present invention, terms such as first, second, A, B, (a), (b), etc. may be used.

[0043] These terms are intended only to distinguish one component from another, and are not intended to limit the nature, order, or sequence of the component.

[0044] And, when a component is described as being 'connected', 'coupled' or 'connected' to another component, it may include not only cases where the component is directly connected, coupled or connected to the other component, but also cases where the component is 'connected', 'coupled' or 'connected' by another component between the component and the other component.

[0045] Additionally, when described as being formed or arranged "above or below" each component, "above" or "below" includes not only cases where the two components are in direct contact with each other, but also cases where one or more other components are formed or arranged between the two components. Furthermore, when expressed as "above" or "below", it can include the meaning of a downward direction as well as an upward direction based on one component.

[0046] Hereinafter, embodiments will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or corresponding components are given the same reference numbers, and redundant descriptions thereof will be omitted.

[0047] Figure 1 illustrates an example of a blooming effect due to a retroreflective traffic sign. Referring to Figure 1, when a vehicle-mounted camera, such as a lidar sensor, captures a forward image of a traffic sign coated with a retroreflective material, the acquired depth and intensity images contain distorted point cloud information due to the blooming effect.

[0048] Figure 2 illustrates the principle of how the blooming effect occurs within a single photon avalanche diode (SPAD) array. Referring to Figure 2, stray light within a target area overflows and spreads to adjacent pixels, thereby causing the blooming effect.

[0049] Training-based techniques are being attempted to remove blooming effects from acquired images, but this requires a large number of blooming-corrupted image and blooming-free image pairs, and it may be realistically difficult to acquire a large number of blooming-corrupted image and blooming-free image pairs by capturing with a lidar sensor.

[0050] In an embodiment of the present invention, noise data is generated using the physical characteristics of the blooming effect, and the generated noise data is synthesized with an original image to generate a blooming-damaged image. In this specification, the blooming effect may be used interchangeably with crosstalk noise or noise.

[0051] FIG. 3 is a flowchart illustrating a method for a computing device to synthesize a blooming damage image according to an embodiment of the present invention, FIG. 4 is an example of a scene including a retroreflective material acquired by a lidar sensor, FIGS. 5A and 5B are diagrams for explaining the physical characteristics of a blooming effect for an intensity image, FIGS. 6A and 6B are diagrams for explaining the physical characteristics of a blooming effect for a depth image, FIG. 7 is a flowchart illustrating a method for a computing device to generate noise data according to an embodiment of the present invention, FIG. 8A is an example of noise data generated by the method of FIG. 7, FIG. 8B is a physical characteristic of noise data generated by the method of FIG. 7, FIG. 9A is an example of an original intensity image and an original depth image, and FIG. 9B is an example of a synthetic intensity image and a synthetic depth image generated according to an embodiment of the present invention.

[0052] Referring to FIG. 3, the computing device (200) obtains an original image (S300), generates noise data (S310), and generates a synthetic image (S320).

[0053] In step S300, the original image includes an original intensity image and an original depth image. Here, the original intensity image and the original depth image are images captured from the same scene. Accordingly, the original intensity image and the original depth image may be referred to as having a corresponding relationship. In this specification, the intensity image may refer to an image representing light intensity, and the depth image may refer to an image representing distance information. The intensity image may represent light intensity per pixel, and the depth image may represent distance information per pixel. Referring to FIG. 9A, an example of an original intensity image and a corresponding original depth image can be seen.

[0054] According to an embodiment of the present invention, the original intensity image and the original depth image can be acquired by a lidar sensor mounted on a vehicle. For example, the original intensity image and the original depth image can be acquired by a fixed lidar sensor mounted on a vehicle. For example, the fixed lidar sensor can measure depth using a direct TOF method, include a SPAD array, and include a 52*192 point cloud of a sequential flash type, but is not limited thereto. The embodiment of the present invention can be applied to an original image acquired by a mechanical lidar sensor, and can also be applied to an original image acquired by a lidar sensor that measures depth using an in-direct TOF method and includes an image sensor other than a SPAD array. In this case, the number of original intensity images and original depth images acquired by the lidar sensor can be 1K or more, preferably 10K or more. In the embodiment of the present invention, 17K original intensity images and original depth images were acquired for 9 scenes.

[0055] Next, in step S310, the computing device (200) generates first noise data for an intensity image and second noise data for a depth image using physical characteristics of a scene including a retroreflective material. Referring to FIG. 4, the scene including a retroreflective material may include a target area (400). Here, the target area (400) may refer to a surface having a strong reflectivity or an object that is significantly brighter than its surroundings. For example, the target area (400) may be a traffic sign coated with a retroreflective material. Accordingly, the target area (400) may be referred to as a retroreflective area or an RR (retro-reflective) area. In addition, the scene including a retroreflective material may further include a first peripheral area (410) disposed on one side of the target area (400) and a second peripheral area (420) disposed on the other side of the target area (400). In this specification, the first peripheral region (410) and the second peripheral region (420) refer to regions where a blooming effect according to the target region (400) affects. Accordingly, the first peripheral region (410) and the second peripheral region (420) may also be referred to as blooming artifact regions. At this time, the first peripheral region (410) and the second peripheral region (420) may be asymmetrical with respect to the target region (400). For example, the length (l) of the first peripheral region (410) in the direction from the target region (400) toward the first peripheral region (410) may be different from the length (r) of the second peripheral region (420) in the direction from the target region (400) toward the second peripheral region (420).

[0056] In this specification, the first peripheral region (410) and the second peripheral region (420) are formed in the horizontal direction of the target region (400), but this is not limited thereto. The direction in which the first peripheral region (410) and the second peripheral region (420) are formed may vary depending on the scan direction of the lidar sensor. For example, the first peripheral region (410) and the second peripheral region (420) may be formed in the vertical direction of the target region (400).

[0057] Examples of the first peripheral region (410) and the second peripheral region (420) formed around the target region (400) can be seen with reference to FIGS. 5A and 6A. According to FIGS. 5A and 6A, it can be seen that a band-shaped blooming artifact is observed around the target region (400) due to internal crosstalk. At this time, referring to FIGS. 5A and 5B, it can be seen that the intensity value of the pixel corresponding to the target region (400) is saturated and is greater than the intensity values ​​of the pixels corresponding to the first peripheral region (410) and the second peripheral region (420). More specifically, the intensity values ​​of the pixels corresponding to the first peripheral region (410) and the second peripheral region (420) can decrease exponentially as they get farther away from the target region (400). In contrast, referring to FIGS. 6A and 6B , it can be seen that the depth value of the pixel corresponding to the target area (400) is the same as the depth values ​​of the pixels corresponding to the first peripheral area (410) and the second peripheral area (420). In this way, the physical characteristics of the blooming effect in the intensity image are different from the physical characteristics of the blooming effect in the depth image. In the embodiment of the present invention, noise data is generated by utilizing the difference between the physical characteristics of the blooming effect in the intensity image and the physical characteristics of the blooming effect in the depth image.

[0058] More specifically, referring to FIG. 7, the computing device (200) according to the embodiment of the present invention sets the number N of target areas in one scene (S700) to generate noise data of step S310, and sets the center coordinates (x) of the two-dimensional space for each target area. i , y i ) and sets the width and height (w) for each target area (S710). i , h i ) is set (S720). Here, i may mean the ith target area among the total N target areas. Next, the depth value (d) of each target area i ) is set (S730), and the length (l) of the first peripheral area placed on one side of each target area i ) and the length (r) of the second peripheral region placed on the other side of each target region i ) is set (S740), and the minimum intensity value (I) of the first peripheral area and the second peripheral area of ​​each target area i,left , I i,right ) is set (S750), and the exponential decay parameter (α) of each target area i,left , α i,right ) is set (S760). Steps S700 to S760 can be modeled based on the physical characteristics of the blooming effect included in the intensity image and the physical characteristics of the blooming effect included in the depth image. As described above, since the first peripheral region and the second peripheral region are asymmetrical with respect to the target region, the length (li) of the first peripheral region and the length (ri) of the second peripheral region are set differently, and accordingly, the minimum intensity value (I) of the first peripheral region and the second peripheral region of each target region i,left , I i,right ) and the exponential decay parameter (α) of each target region i,left , α i,right ) can be set independently.

[0059] The noise data of Fig. 8a can be generated by the modeling of Fig. 7. Referring to Fig. 8a, the noise data can be first noise data for the intensity image and second noise data for the depth image. Referring to Figs. 8a and 8b, the first noise data for the intensity image can be the length (w) of the target area. i ), the length of the first peripheral region (l i ) and the length of the second peripheral region (r i ), the intensity value of the target area is greater than the intensity value of the first peripheral area and the intensity value of the second peripheral area, and the intensity value of the first peripheral area and the intensity value of the second peripheral area have an exponential attenuation parameter (α) as they get farther away from the target area. i,left , α i,right ) can be seen to decrease according to the intensity value of the first peripheral region, which decreases exponentially. For example, the intensity value of the first peripheral region decreases from the intensity value of the target region to the minimum intensity value (I) of the first peripheral region. i,left ) and the intensity value of the second peripheral area, which decreases exponentially, decreases from the intensity value of the target area to the minimum intensity value of the second peripheral area (I i,right ) can be reduced to. Here, the minimum intensity value (I) of the first peripheral area i,left ) is the value of the outer region of the first peripheral region, and the minimum intensity value of the second peripheral region (I i,right ) may be the value of the outer region of the second peripheral region. In addition, the second noise data for the depth image may also be the length (w) of the target region. i ), the length of the first peripheral region (l i ) and the length of the second peripheral region (r i ) and the depth value (d) of the target area i ) may be equal to the depth value of the first peripheral area and the depth value of the second peripheral area.

[0060] In this way, according to an embodiment of the present invention, first noise data for an intensity image and second noise data for a depth image are generated using physical properties of a scene including a retroreflective material.

[0061] Referring back to FIG. 3, in step S320, the computing device according to the embodiment of the present invention generates a synthetic intensity image by adding the first noise data generated in step S310 to the original intensity image acquired in step S300, and generates a synthetic depth image by adding the second noise data generated in step S310 to the original depth image acquired in S300. FIG. 9a is an example of the original intensity image and the original depth image acquired in step S300, and FIG. 9b is an example of the synthetic intensity image generated in step S320 using the first noise data generated in step S310, and the synthetic depth image generated in step S320 using the second noise data generated in step S310.

[0062] In this way, according to an embodiment of the present invention, a blooming-corrupted image corresponding to a blooming-free image can be acquired without actual shooting by a lidar sensor, thereby efficiently obtaining a data set pair of blooming-corrupted images and blooming-free images. In particular, when synthesizing blooming-corrupted images according to an embodiment of the present invention, a large number of blooming-corrupted images, 10K or more, can be efficiently generated, and can be applied to training-based techniques.

[0063] Meanwhile, according to an embodiment of the present invention, a synthetic image generated by the above-described method is trained to output a noise mask.

[0064] FIG. 10 is a block diagram of a deep learning-based network according to one embodiment of the present invention, FIG. 11 is a flowchart showing a data learning method of a computing device according to one embodiment of the present invention, and FIG. 12 is a conceptual diagram of a deep learning-based network according to one embodiment of the present invention.

[0065] Referring to FIGS. 10 to 12, a data learning method of a computing device according to one embodiment of the present invention includes a synthetic image input step (S1100), a synthetic image training step (S1110), and a noise mask output step (S1120).

[0066] The synthetic image input in step S1100 is a synthetic intensity image and a synthetic depth image generated by the method of FIG. 7 according to an embodiment of the present invention. As illustrated in FIG. 10, the method of FIG. 7 according to an embodiment of the present invention can be implemented by a data synthesizer (1000). The synthetic intensity image and the synthetic depth image generated by the data synthesizer (1000) are input to a deep learning-based network (1010) together with the original intensity image and the original depth image used to generate the synthetic intensity image and the synthetic depth image.

[0067] In step S1110, a deep learning-based network (1010) is trained to generate a noise mask. The deep learning-based network (1010) is trained using a synthetic intensity image and a synthetic depth image and corresponding original intensity images and original depth images. The synthetic intensity image may be concatenated with the corresponding original intensity image, and the synthetic depth image may be concatenated with the corresponding original depth image. Here, the deep learning-based network (1010) may be referred to as a deep learning-based segmentation network, and may be, for example, SqueezeSegV2. The inventor of the present application acquired 17K depth and intensity images for 9 scenes using an S lidar sensor mounted on a vehicle, and trained the deep learning-based network (1010) with 100 epochs and a batch size of 128.

[0068] In step S1120, the deep learning-based network (1010) outputs a noise mask. Here, the pixel values ​​of the noise mask may represent the probability of each pixel belonging to a blooming artifact. The noise mask may have the form illustrated in FIG. 12.

[0069] A noise mask generated according to an embodiment of the present invention can be used to acquire three-dimensional information from a lidar sensor. For example, if a lidar sensor captures a scene containing a traffic sign coated with a retroreflective material, the blooming effect can be removed by a noise mask generated according to an embodiment of the present invention.

[0070] To verify the data generation and learning effect according to an embodiment of the present invention, an actual shooting scene including a target area was acquired as shown in Fig. 13. The actual shooting scene was acquired in an indoor environment with a single retroreflective film placed at a distance of 25 m. Accordingly, the depth image of Fig. 14(a) and the intensity image of Fig. 14(b) were acquired. Thereafter, the predicted blooming artifact probability map of Fig. 14(d) was extracted. In the map of Fig. 14(d), the pixel value of each pixel represents the probability of belonging to a blooming artifact from 0 to 1. Comparing the ground true blooming artifact of Fig. 14(c) with the map of Fig. 14(d), the true positive rate (TPR) was found to be 71.4%. A positive sample of the TPR indicates that the pixel is a blooming artifact, and a negative sample indicates that the pixel is a real object.

[0071] FIG. 15 is a block diagram of a computing device according to one embodiment of the present invention.

[0072] Referring to FIG. 15, a computing device (200) includes a communication unit (210) and a processor (220), and can directly or indirectly communicate with an external computing device (not shown) through the communication unit (210).

[0073] Specifically, the computing device (200) may achieve desired system performance by using a combination of typical computer hardware (e.g., devices that may include a computer processor, memory, storage, input devices and output devices, and other components of conventional computing devices; electronic communication devices such as routers, switches, etc.; electronic information storage systems such as network-attached storage (NAS) and storage area networks (SAN)) and computer software (i.e., instructions that cause the computing device to function in a particular manner).

[0074] The communication unit (210) of the computing device (200) can transmit and receive requests and responses with other computing devices to which it is connected. As an example, such requests and responses may be made through the same transmission control protocol (TCP) session, but are not limited thereto. For example, they may be transmitted and received as user datagram protocol (UDP) datagrams. In addition, in a broad sense, the communication unit (210) may include a keyboard, a pointing device such as a mouse, other external input devices, a printer, a display, or other external output devices for transmitting commands or instructions.

[0075] Additionally, the processor (220) of the computing device (200) may include hardware configurations such as a micro processing unit (MPU), a central processing unit (CPU), a graphics processing unit (GPU) or a tensor processing unit (TPU), a cache memory, and a data bus. In addition, it may further include a software configuration of an operating system and an application that performs a specific purpose.

[0076] Embodiments of the present invention may be configured based on hardware / software, and a processor (220) of a computing device may be configured to perform / control operations according to embodiments of the present invention.

[0077] Based on the description of the above embodiments, it will be apparent to those skilled in the art that the methods and / or processes of the present invention, and their steps, may be implemented by hardware, software, or any combination of hardware and software suitable for a specific application. The hardware may include a general-purpose computer and / or a dedicated computing device, or a specific computing device or a specific aspect or component of a specific computing device. The processes may be implemented by one or more microprocessors, microcontrollers, embedded microcontrollers, programmable digital signal processors, or other programmable devices having internal and / or external memory. Additionally, or alternatively, the processes may be implemented by an application specific integrated circuit (ASIC), a programmable gate array, a programmable array logic (PAL), or any other device or combination of devices that can be configured to process electronic signals. Furthermore, the objects of the technical solution of the present invention or the parts contributing to prior art can be implemented in the form of program commands that can be executed by various computer components and recorded on a machine-readable recording medium. The machine-readable recording medium can include program commands, data files, data structures, etc., either singly or in combination. The program commands recorded on the machine-readable recording medium may be those specifically designed and constructed for the present invention or may be known and usable by those skilled in the art of computer software.Examples of machine-observable storage media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical recording media such as CD-ROMs, DVDs, and Blu-ray; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memories. Examples of program instructions include, but are not limited to, any of the aforementioned devices, as well as a processor, a heterogeneous combination of processor architectures or different combinations of hardware and software, or any other program instructions that can be stored and compiled or interpreted using a structured programming language such as C, an object-oriented programming language such as C++, or a high-level or low-level programming language (assembler, hardware description languages, and database programming languages ​​and technologies), and include not only machine code and byte code, but also high-level language code that can be executed by a computer using an interpreter, etc.

[0078] Accordingly, in one aspect according to the present disclosure, when the methods and combinations thereof described above are performed by one or more computing devices, the methods and combinations thereof may be implemented as executable code performing the respective steps. In another aspect, the methods may be implemented as systems performing the steps, and the methods may be distributed in various ways across the devices, or all functions may be integrated into a single dedicated, standalone device or other hardware. In yet another aspect, the means for performing the steps associated with the processes described above may comprise any of the hardware and / or software described above. All such sequential combinations and arrangements are intended to fall within the scope of the present disclosure.

[0079] For example, the hardware device may be configured to operate as one or more software modules to perform processing according to the present specification, and vice versa. The hardware device may include a processor, such as an MPU, a CPU, a GPU, or a TPU, coupled with a memory, such as a ROM / RAM, for storing program instructions and configured to execute the instructions stored in the memory, and may include a communication unit capable of sending and receiving signals with an external device. In addition, the hardware device may include a keyboard, a mouse, or other external input devices for receiving instructions written by developers.

[0080] Although the present invention has been described above with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various modifications and changes may be made to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below.

Claims

1. In a method for generating data of a computing device, A step of generating first noise data for an intensity image and second noise data for a depth image by using physical properties of a scene including a retroreflective material, A step of generating a synthetic intensity image by adding the first noise data to the original intensity image, and A step of generating a synthetic depth image by adding the second noise data to the original depth image corresponding to the original intensity image. A method of generating data including .

2. In paragraph 1, A data generation method wherein a scene including the above retroreflective material includes a target area, a first peripheral area disposed on one side of the target area, and a second peripheral area disposed on the other side of the target area.

3. In paragraph 2, The first noise data is generated such that the intensity value of the target area is greater than the intensity value of the first peripheral area and the intensity value of the second peripheral area, The second noise data is generated so that the depth value of the target area is equal to the depth value of the first peripheral area and the depth value of the second peripheral area. How to generate data.

4. In paragraph 3, A data generation method in which the intensity value of the first peripheral area and the intensity value of the second peripheral area decrease exponentially as they move away from the target area.

5. In paragraph 4, The intensity value of the first peripheral area decreases exponentially from the intensity value of the target area to the intensity value outside the first peripheral area, A data generation method in which the intensity value of the second peripheral area exponentially decreases from the intensity value of the target area to the intensity value outside the second peripheral area.

6. In paragraph 4, A data generation method wherein the length of the first peripheral area in the direction from the target area toward the first peripheral area is different from the length of the second peripheral area in the direction from the target area toward the second peripheral area.

7. In paragraph 1, A method for generating data in which the original intensity image and the original depth image are images acquired by a lidar device.

8. In paragraph 7, A method for generating data in which the above synthetic intensity image and the above synthetic depth image include blooming artifacts.

9. In a data learning method of a computing device, A step of receiving a synthetic intensity image generated by adding first noise data for an intensity image to an original intensity image and a synthetic depth image generated by adding second noise data for a depth image to an original depth image corresponding to the original intensity image, a step of training the above synthetic intensity image and the above synthetic depth image, and A step of outputting a noise mask indicating the possibility of distortion due to pixel-by-pixel noise, A data learning method in which the first noise data and the second noise data are generated using physical characteristics of a scene including a retroreflective material.

10. A communication unit for obtaining an original intensity image and an original depth image corresponding to the original intensity image; and A processor connected to the above communication unit, wherein the processor comprises: A method for generating a first noise data for an intensity image and a second noise data for a depth image by using physical properties of a scene including a retroreflective material, generating a synthetic intensity image by adding the first noise data to the original intensity image, and generating a synthetic depth image by adding the second noise data to the original depth image. Computing device.

Images