Object information generation method and apparatus using radar

Abstract

A method and apparatus for generating information about an object included in a package using radar are disclosed. The disclosed method for generating object information using radar includes the steps of: acquiring a radar signal reflected from a package containing at least one object from a plurality of receiving channels; generating a distance-angle map for each channel group including two or more of the plurality of receiving channels from the radar signal; and inputting the distance-angle map into a pre-trained deep learning model to generate information about the object.

Description

Technology Field

[0001] The present invention relates to a method and apparatus for generating object information using radar, and more specifically, to a method and apparatus for generating information about an object included in a package using radar. Background Technology

[0003] Products manufactured in the factory are packaged and shipped to customers. During this process, errors in the packaging process may result in products not being included in the package, or fewer products being included than required. Additionally, products may be damaged if placed incorrectly within the package, or products may be included with foreign substances.

[0004] Consequently, there has recently been an increasing need for technology capable of inspecting products not only during the packaging stage but also afterward. Since products become invisible after packaging, making visual verification difficult, research and development are being conducted in the direction of using radar to verify information about objects contained within the packaging.

[0005] Related prior art includes patent documents, Korean Published Patent No. 2020-0091141 and No. 2024-0086345, and Korean Registered Patent No. 10-2280190 and No. 10-2280190, as well as non-patent document, "3D Object Detection Based on YOLO Using Radar Data, Son Ha-young, Lee Sang-won, Choi Gye-won, Proceedings of the 2021 Summer Conference of the Korean Institute of Communication Sciences." The problem to be solved

[0007] The present invention is intended to provide a method and apparatus for generating object information that can generate information about an object using radar while the object is in a non-visible state, such as being included in a package. means of solving the problem

[0009] According to one embodiment of the present invention for achieving the above-mentioned purpose, a method for generating object information using radar is provided, comprising: a step of acquiring a radar signal reflected from a packaging containing at least one object from a plurality of receiving channels; a step of generating a distance-angle map for each channel group containing two or more of the plurality of receiving channels from the radar signal; and a step of inputting the distance-angle map into a pre-trained deep learning model to generate information about the object.

[0010] In addition, according to another embodiment of the present invention for achieving the above-mentioned purpose, a method for generating object information using radar is provided, comprising the steps of: selecting receiving channels for generating information about the object among a plurality of receiving channels that receive a radar signal reflected from a packaging including at least one object; acquiring the radar signal from the selected receiving channels; generating a distance-angle map for each channel group including two or more of the selected receiving channels from the radar signal; and inputting the distance-angle map into a pre-trained deep learning model to generate information about the object.

[0011] In addition, according to another embodiment of the present invention for achieving the above-mentioned purpose, an object information generation device using a radar is provided, comprising: a radar sensor including at least one transmitter that radiates a radar signal to a package including at least one object, and a plurality of receivers that receive a radar signal reflected from the package; a memory; and a processor that generates a distance-angle map for each of a channel group including two or more of a plurality of receiving channels from the reflected radar signal, and inputs the distance-angle map into a pre-trained deep learning model to generate information about the object. Effects of the invention

[0013] According to one embodiment of the present invention, information about an object located in a non-visible state can be accurately obtained by using radar and deep learning. Brief explanation of the drawing

[0015] FIGS. 1 and FIGS. 2 are drawings for explaining an object information generation device using a radar according to an embodiment of the present invention. FIG. 3 is a flowchart of a method for generating object information using radar according to an embodiment of the present invention. Figure 4 is a diagram showing the distance spectrum for a receiving channel. Figure 5 is a diagram showing a distance-angle map for a channel group. Specific details for implementing the invention

[0016] The present invention is susceptible to various modifications and may have various embodiments, and specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the invention to specific embodiments, and it should be understood that the invention includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention. Similar reference numerals have been used for similar components in the description of each drawing.

[0017] Hereinafter, embodiments according to the present invention will be described in detail with reference to the attached drawings.

[0019] FIGS. 1 and FIGS. 2 are drawings for explaining an object information generation device using a radar according to an embodiment of the present invention.

[0020] Referring to FIGS. 1 and 2, an object information generating device (110) according to an embodiment of the present invention includes a radar sensor (111), a memory (112), and a processor (113).

[0021] A radar sensor (111) emits a radar signal to at least one object, namely a package (120) containing a product, and receives a radar signal reflected from the package (120). In one embodiment, the radar sensor (111) may emit a radar signal from the bottom of the package (120) toward the package, and depending on the embodiment, the direction of radar radiation may be varied.

[0022] The radar sensor (111) includes at least one transmitter (210) that emits a radar signal through packaging and a plurality of receivers (221 to 225) that receive the reflected radar signal. The receivers (221 to 225) may be arranged according to a preset arrangement pattern. An embodiment in which five receivers (221 to 225) are used is shown in FIG. 2, and the number of receivers may vary depending on the embodiment.

[0023] The processor (113) performs a series of processes to generate information about the object included in the package from the reflected radar signal.

[0024] The processor (113) generates a range-angle map for each channel group comprising two or more of the plurality of receiving channels from the reflected radar signal. Here, the receiving channel represents a channel in which the receiver (221 to 225) receives the reflected radar signal, and the number of receiving channels corresponds to the number of receivers (221 to 225). The channel group is a group comprising two or more of the receiving channels, for example, each channel group may include a different pair of receiving channel combinations generated from the plurality of receiving channels. That is, each channel group may include a receiving channel combination composed of a different pair of receiving channels.

[0025] As shown in Fig. 2, when there are 5 receiving channels, 10 channel groups can be used. The first channel group is a receiving channel combination corresponding to the first and second receivers (221, 222), the second channel group is a receiving channel combination corresponding to the first and third receivers (221, 223), the third channel group is a receiving channel combination corresponding to the first and fourth receivers (221, 224), the fourth channel group is a receiving channel combination corresponding to the first and fifth receivers (221, 225), the fifth channel group is a receiving channel combination corresponding to the second and third receivers (222, 223), the sixth channel group is a receiving channel combination corresponding to the second and fourth receivers (222, 224), the seventh channel group is a receiving channel combination corresponding to the second and fifth receivers (222, 225), the eighth channel group is a receiving channel combination corresponding to the third and fourth receivers (223, 224), and the ninth channel group is the third and A receiving channel combination corresponding to the 5th receiving unit (223, 225), and a 10th channel group may include a receiving channel combination corresponding to the 4th and 5th receiving units (224, 225).

[0026] Then, the processor (113) inputs the distance-angle map into a pre-trained deep learning model to generate information about the object. Here, the information about the object may include information about the number of objects, information about the arrangement pattern of objects, or information about the material of objects. The deep learning model may include multiple models that receive the distance-angle map and output information about the number of objects, information about the arrangement pattern of objects, and information about the material of objects, respectively, and the processor (113) may input the distance-angle map into a model that generates information according to a user's request to generate information about the object.

[0027] A more specific method for generating object information is explained in more detail in FIGS. 3 to 5.

[0029] FIGS. 3 to 5 are drawings for explaining a method for generating object information using radar according to an embodiment of the present invention, where FIG. 3 is a flowchart of a method for generating object information using radar according to an embodiment of the present invention, FIG. 4 is a distance spectrum for a receiving channel, and FIG. 5 is a distance-angle map for a channel group.

[0030] An object information generation method according to one embodiment of the present invention may be performed in a computing device including memory and a processor, and the aforementioned object information generation device may be an example of such a computing device.

[0031] Referring to FIG. 3, a computing device according to an embodiment of the present invention acquires (S310) a radar signal reflected from a package containing at least one object from a plurality of receiving channels. As described above, the computing device can acquire each of the reflected radar signals from a plurality of receiving units that receive the radar signal reflected from the package.

[0032] Then, the computing device generates a distance-angle map for each channel group comprising two or more of the plurality of receiving channels from the reflected radar signal (S320). To generate the distance-angle map, the computing device may use a distance spectrum and a digital beamforming algorithm.

[0033] The computing device first generates a distance spectrum for each receiving channel from the reflected radar signal. The computing device can obtain a distance spectrum as shown in FIG. 4 by processing the reflected radar signal using FFT. FIG. 4 shows two distance spectra generated from radar signals, obtained from each receiving channel included in the channel group. In the distance spectrum, the distance represents the distance between the radar sensor and the object included in the packaging, and the magnitude represents the magnitude of the distance component calculated through FFT.

[0034] Then, the computing device uses digital beamforming to generate a distance-angle map from the distance spectrum for a combination of receiving channels included in a channel group. If a channel group includes a pair of receiving channels, a distance-angle map can be generated from the distance spectrum for the pair of receiving channels. Digital beamforming is a technique that digitizes signals received through an antenna array and processes them on a computer, and a distance-angle map is a map that visually displays the magnitudes of the distance and angle components in radar signals acquired from different receiving channels. Because radar signals are received through different channels, a phase difference in the radar signals occurs between the receiving channels, and this phase difference corresponds to the angle component. Figure 5 shows a distance-angle map generated from the distance spectrum of Figure 4.

[0035] Returning to Fig. 3, the computing device inputs the distance-angle map generated in step S320 into a pre-trained deep learning model to generate information about the object (S330).

[0036] The computing device can generate a tensor by concatenating distance-angle maps generated for each channel group, and input the generated tensor into a deep learning model to generate information about an object.

[0037] A deep learning model is a model trained to receive a distance-angle map as input and output information about an object, and may include, for example, a CNN and a fully connected neural network. Training data for training the deep learning model may include a training distance-angle map and ground truth values ​​labeled on the training distance-angle map. The ground truth values ​​may be information on the number of objects, information on the arrangement pattern of objects, or information on the material of objects.

[0038] Meanwhile, according to an embodiment, the receiving channel combination included in each channel group may include different pairs of receiving channel combinations generated from receiving channels selected among a plurality of receiving channels. In other words, the receiving channel combination included in each channel group may include different pairs of receiving channel combinations generated from receiving channels corresponding to some of the receiving parts among the plurality of receiving parts included in the radar sensor. Furthermore, the receiving channels selected among the plurality of receiving channels may be receiving channels separated by a preset threshold interval or greater.

[0039] As the size of the tensor increases as the number of receiving channels used increases, the amount of computation for generating object information increases; therefore, an embodiment of the present invention may selectively use some of the receiving channels among a plurality of receiving channels in order to efficiently perform computations while generating accurate object information. At this time, since the generation of object information may be inaccurate if the selected receiving channels correspond to a receiving section concentrated in a specific area, the computing device may selectively use receiving channels separated by more than a threshold interval.

[0040] For example, in an embodiment such as FIG. 2, when three receiving channels are selected, a receiving channel corresponding to the first, third, and fifth receiving units (221, 223, 225) that are spaced apart from each other may be selected, rather than the first to third receiving units (221 to 223) or the third to fifth receiving units (223 to 225) that are arranged adjacently.

[0042] FIG. 6 is a diagram illustrating a method for generating object information using radar according to another embodiment of the present invention.

[0043] Referring to FIG. 6, a computing device according to an embodiment of the present invention selects (S610) receiving channels for generating information about an object among a plurality of receiving channels that receive a radar signal reflected from a package containing at least one object. Here, the computing device may select a receiving channel that is spaced apart by a preset interval among the plurality of receiving channels.

[0044] The computing device acquires the reflected radar signal from the selected receiving channels (S620) and, from the reflected radar signal, generates a distance-angle map for each channel group including two or more of the selected receiving channels (S630).

[0045] Then, the computing device inputs the distance-angle map into a pre-trained deep learning model to generate information about the object (S640).

[0047] The technical details described above may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiments, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. Hardware devices may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

[0049] As described above, the present invention has been explained by specific details such as specific components, limited embodiments, and drawings; however, this is provided merely to aid in a more comprehensive understanding of the invention, and the invention is not limited to the above embodiments. A person skilled in the art to which the invention pertains can make various modifications and variations from this description. Therefore, the scope of the invention should not be limited to the described embodiments, and all things equivalent to or having equivalent variations to the claims set forth below, as well as the claims themselves, shall be considered to fall within the scope of the concept of the invention.

Claims

Claim 1 A method for generating object information using radar, comprising: a step of selecting some receiving channels for generating information about an object among a plurality of receiving channels that receive a radar signal reflected from a package containing at least one object; a step of acquiring the radar signal from the selected receiving channels; a step of generating a distance-angle map for each channel group including two or more receiving channels among the selected receiving channels from the radar signal; and a step of inputting the distance-angle map into a pre-trained deep learning model to generate information about the object, wherein the step of generating information about the object includes a step of generating a tensor by concatenating the distance-angle maps generated for each channel group; and a step of inputting the tensor into the deep learning model to generate information about the object, wherein the plurality of receiving channels are channels in which a plurality of receivers arranged according to a pre-set arrangement pattern receive the radar signal, and the selected receiving channels are receiving channels corresponding to receivers that are not arranged adjacently among the plurality of receivers but are separated by a pre-set threshold interval or more. Claim 2 A method for generating object information using radar according to claim 1, wherein the step of generating the distance-angle map comprises: generating a distance spectrum for each of the receiving channels from the radar signal; and generating the distance-angle map from the distance spectrum for the receiving channels included in the channel group using digital beamforming. Claim 3 A method for generating object information using a radar, wherein each of the channel groups comprises a different pair of receiving channel combinations generated from the plurality of receiving channels. Claim 4 delete Claim 5 delete Claim 6 A method for generating object information using radar according to claim 1, wherein information about the object includes information on the number of the object, information on the arrangement pattern of the object, or information on the material of the object. Claim 7 delete Claim 8 A radar sensor comprising at least one transmitter that emits a radar signal through a packaging that includes at least one object, and a plurality of receivers that receive a radar signal reflected from the packaging; a memory; A radar-based object information generation device comprising a processor, wherein the processor selects some receiving channels for generating information about the object from among a plurality of receiving channels that receive a radar signal reflected from the packaging, acquires the radar signal from the selected receiving channels, generates a distance-angle map for each channel group including two or more receiving channels among the selected receiving channels from the radar signal, inputs the distance-angle map into a pre-trained deep learning model to generate information about the object, wherein the plurality of receiving channels are channels in which the plurality of receivers arranged according to a pre-set arrangement pattern receive the radar signal, and the selected receiving channels are receiving channels corresponding to receivers that are not arranged adjacently among the plurality of receivers and are separated by a pre-set threshold interval or more, and wherein the processor generates a tensor by concatenating the distance-angle maps generated for each channel group, and inputs the tensor into the deep learning model to generate information about the object. Claim 9 In claim 8, the above processor generates a distance spectrum for each of the receiving channels from the radar signal, and generates the distance-angle map from the distance spectrum for the receiving channels included in the channel group using digital beamforming, an object information generation device using a radar. Claim 10 An object information generating device using radar, wherein each of the channel groups comprises a different pair of receiving channel combinations generated from receiving channels selected from among the plurality of receiving channels.

Images