Information processing apparatus, system, method, and program

Abstract

To provide an information processing apparatus, a system, a method, and a program that can easily prepare a radar image used for learning a learning model.SOLUTION: An information processing apparatus according to an embodiment is provided with acquisition means, first creation means, and second creation means. The acquisition means acquires, from a radar device including a plurality of antennas configured to transmit radar signals and receive reflected wave signals based on reflected waves of the radar signals, a first observation signal based on the radar signals and reflected wave signals. The first creation means creates a first radar image by executing signal processing based on a first condition on the first observation signal. The second creation means creates a second radar image by executing signal processing based on a second condition on the first observation signal. The first and second radar images are used as teacher data used for learning a learning model for detecting an object to which the radar signals are transmitted.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] An embodiment of the present invention relates to an information processing device, a system, a method, and a program. [Background technology]

[0002] In recent years, radar systems have been developed that image an object by utilizing reflected waves of a radar signal (transmitted wave) (that is, generate a radar image including the object).

[0003] Here, it is being considered to use a learning model (radar sensing AI) prepared in advance to detect (identify) objects included in a radar image generated by the above-mentioned radar system.

[0004] A learning model is generated by learning from training data (a set of radar images and label information representing objects contained in the radar images), but a large amount of training data is required to generate a learning model capable of detecting objects with high accuracy (i.e., a good learning model). In other words, when the amount of training data (especially radar images) is small, it is difficult to generate a good learning model, and a mechanism for preparing (padding) radar images necessary for learning using the learning model is required. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2022-078753 Summary of the Invention [Problem to be solved by the invention]

[0006] Therefore, the problem to be solved by the present invention is to provide an information processing device, system, method, and program that can easily prepare radar images used for learning a learning model. [Means for solving the problem]

[0007] The information processing device according to the embodiment includes an acquisition means, a first generation means, and a second generation means. The acquisition means acquires a first observation signal based on a radar signal and the reflected wave signal from a radar device having a plurality of antennas configured to transmit a radar signal and receive a reflected wave signal based on a reflected wave of the radar signal. The first generation means generates a first radar image by executing signal processing on the acquired first observation signal based on a first condition that is predetermined. The second generation means generates a second radar image by executing signal processing on the acquired first observation signal based on a second condition that is different from the first condition. The first and second radar images are used as teacher data for learning a learning model for detecting an object from which the radar signal is transmitted. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram for explaining an overview of an FMCW system adopted in a radar device. [Diagram 2] FIG. 1 is a diagram for explaining an overview of a MIMO radar. [Diagram 3] FIG. 1 is a diagram for explaining an overview of synthetic aperture processing. [Figure 4] FIG. 1 is a diagram for explaining an overview of synthetic aperture processing. [Diagram 5] FIG. 1 is a diagram for explaining an overview of synthetic aperture processing. [Figure 6] FIG. 1 is a diagram showing an example of the configuration of a radar system according to an embodiment of the present invention. [Figure 7] FIG. 2 is a diagram showing an example of a hardware configuration of an information processing apparatus. [Figure 8] 11 is a flowchart showing an example of a processing procedure of an information processing device. [Figure 9] 1 is a diagram for explaining an observation point realized by a combination of multiple transmitting antennas and multiple receiving antennas. [Figure 10]5A and 5B are diagrams for explaining synthetic aperture processing executed by a signal processing unit. [Figure 11] FIG. 1 is a diagram for explaining an overview of a mixing process. [Figure 12] FIG. 4 is a diagram for explaining noise addition to a radar image. [Figure 13] FIG. 4 is a diagram for explaining the dynamic range of a radar image. [Figure 14] 1A to 1C are diagrams for explaining shifting, inversion, rotation, enlargement, and reduction of a radar image. [Figure 15] FIG. 1 is a diagram for explaining the shift of a radar image reproduced by selecting an antenna. [Figure 16] 3A and 3B are diagrams showing examples of first and second radar images generated in the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment will be described with reference to the drawings. The radar system according to this embodiment is configured to measure the direction of the target and the distance to the target by utilizing the reflected wave from the target of the radar signal transmitted (irradiated) to the target, and to image the target (generate a radar image including the target). Note that the radar signal transmitted to the target in such a radar system is, for example, a radio wave such as a millimeter wave (EHF: Extra High Frequency).

[0010] The following is a brief overview of a radar system. First, the Frequency Modulated Continuous Wave (FMCW) method will be described as an example of a radar modulation method that performs frequency modulation.

[0011] According to the FMCW method, a radar signal (transmitted wave) is transmitted that has been modulated so that its frequency changes over time, as shown in FIG. 1, and the distance to the target is measured based on the frequency difference (hereinafter referred to as the beat frequency) between the radar signal and a reflected wave signal (received wave) that is based on the reflected wave of the radar signal.

[0012] Specifically, in the FMCW system, an intermediate frequency (IF) signal (hereinafter, referred to as an IF signal) is obtained by mixing a radar signal and a reflected wave signal. This IF signal corresponds to the time waveform (sine wave) of the above-mentioned beat frequency. The beat frequency is defined as f b If the slope of the radar signal (the rate of change of frequency) called the chirp rate is γ (Hz / s) and the delay time of the reflected wave signal relative to the radar signal is τ, then b , γ and τ are f b There is a relationship such as =γτ.

[0013] When the IF signal is subjected to, for example, a Fast Fourier Transform (FFT), the IF signal is transformed into a frequency domain representation. The result of the FFT contains a beat frequency f b A peak appears at this position, and the distance according to the position of the peak (that is, the distance to the object that reflected the radar signal) can be obtained.

[0014] Incidentally, the FMCW method in which frequency modulation is performed linearly as shown in FIG. 1 is specifically called Linear-FMCW.

[0015] Here, the radar system includes a radar device having a transmitting antenna and a receiving antenna, and a MIMO (Multiple-Input Multiple-Output) radar can be used as the radar device. The MIMO radar has multiple transmitting antennas (transmitting antenna array) and multiple receiving antennas (receiving antenna array), and each transmitting antenna transmits a radar signal in a time-division manner, and the reflected wave signals based on the reflected waves of the radar signals are received by multiple receiving antennas, thereby realizing reception of many reflected wave signals (i.e., radar observation) with a small number of radar signal transmissions (i.e., radar irradiations).

[0016] Specifically, as shown in FIG. 2, it is assumed that the MIMO radar has, for example, two transmitting antennas 1a and 1b and four receiving antennas 2a to 2d linearly arranged in a predetermined spatial direction.

[0017] In this case, for example, if a radar signal is transmitted from the transmitting antenna 1a, a reflected wave signal based on the reflected wave of the radar signal is received by the receiving antenna 2a. Although the receiving antenna 2a has been described here, the reflected wave signal based on the reflected wave of the radar signal transmitted from the transmitting antenna 1a is similarly received by the receiving antennas 2b to 2d.

[0018] Similarly, for example, if a radar signal is transmitted from transmitting antenna 1b, a reflected wave signal based on the reflected wave of the radar signal is received by receiving antenna 2a. Although the receiving antenna 2a has been described here, the reflected wave signal based on the reflected wave of the radar signal transmitted from transmitting antenna 1b is similarly received by receiving antennas 2b to 2d.

[0019] That is, in the above-mentioned MIMO radar, when a radar signal is transmitted from the transmitting antenna 1a, radar observation is performed at each of the receiving antennas 2a to 2d, and when a radar signal is transmitted from the transmitting antenna 1b, radar observation is similarly performed at the receiving antennas 2a to 2d.

[0020] According to this, a MIMO radar having two transmitting antennas 1a and 1b and four receiving antennas 2a to 2d as shown in FIG. 2 can realize eight observation points 3a to 3h arranged in the spatial direction by only irradiating a radar signal once each from the transmitting antennas 1a and 1b. For example, the observation point 3a is an observation point realized by the receiving antenna 2a receiving a reflected wave signal based on the reflected wave of the radar signal transmitted from the transmitting antenna 1a. Similarly, the observation points 3b to 3d are observation points realized by the receiving antennas 2b to 2d receiving a reflected wave signal based on the reflected wave of the radar signal transmitted from the transmitting antenna 1a. Furthermore, the observation point 3e is an observation point realized by the receiving antenna 2a receiving a reflected wave signal based on the reflected wave of the radar signal transmitted from the transmitting antenna 1b. Similarly, the observation points 3f to 3h are observation points realized by the receiving antennas 2b to 2d receiving a reflected wave signal based on the reflected wave of the radar signal transmitted from the transmitting antenna 1b. That is, in a MIMO radar, one observation point is realized by a combination of one transmitting antenna and one receiving antenna.

[0021] Such a MIMO radar makes it possible to measure the distance to an object and perform imaging of the object using observation signals observed at each of the observation points 3a to 3h (radar signals transmitted in a time-division manner from the transmitting antennas 1a and 1b and IF signals acquired based on reflected wave signals received by each of the receiving antennas 2a to 2d).

[0022] In this embodiment, a MIMO radar is used as the radar device. However, the radar device may be a single-input single-output (SISO) radar having a single antenna for transmitting radar signals and receiving reflected wave signals.

[0023] In the radar system according to this embodiment, it is possible to image an object (generate a radar image) using the observation signals acquired from the above-mentioned radar device, but it is desirable to improve the spatial resolution when imaging the object.

[0024] As described above, when a MIMO radar is used, observation signals are acquired at each observation point realized for each combination of transmitting antennas and receiving antennas. In this embodiment, however, in order to image the target object, synthetic aperture processing is performed to combine the observation signals acquired at each observation point, thereby improving the spatial resolution described above.

[0025] Synthetic aperture processing corresponds to signal processing performed in a synthetic aperture radar (SAR). Generally, the larger the antenna aperture length, the higher the spatial resolution. Synthetic aperture processing is a signal processing method that performs multiple observations while moving a low-resolution small aperture antenna, and synthesizes the observation results to realize high-resolution imaging as if it were observed by a virtual large aperture antenna.

[0026] The synthetic aperture processing is generally described below. The synthetic aperture processing includes two steps: extraction of echo components and calculation of convolution integrals.

[0027] First, extraction of echo components means measuring the distance to an object as shown in Figure 3. Specifically, in the above-mentioned FMCW method, when FFT is applied to the observation signal (IF signal), a power peak appears in the FFT result at the distance where the object exists (range peak). This is the echo component, and is called a range compressed echo signal. Note that this echo component is extracted for each observation point.

[0028] Next, the convolution integral means calculating the convolution integral of the echo components extracted for each observation point and the matched filter as shown in Figure 4. A radar image is generated by calculating such a convolution integral.

[0029] Specifically, the echo component is a(x I ,y I ,z I ), and the matched filter is b(x I ,y I ,z I ), then the high-resolution radar image c(x I ,y I ,z I ) is generated by calculating the following equation (1):

number

[0030] Note that N is a set of observation points, and m indicates an observation point in the set N. I ,y I ,z I b represents the width, height, and depth pixels (spatial coordinates) of the radar image generated by synthetic aperture processing. * is the complex conjugate of the matched filter b.

[0031] Incidentally, spatial resolution varies not only with the antenna aperture length but also with the distance to an object located in the direction of irradiation of the radar signal (i.e., an object that reflects the radar signal). When generating a high-resolution radar image at a distance z0 in the above-mentioned synthetic aperture processing, a convolution integral is calculated between the echo components extracted for each observation point and a matched filter (z0). Specifically, each echo component is multiplied by a matched filter that extracts the component at distance z0, and the sum is calculated. In other words, it can be said that the matched filter used in the synthetic aperture processing is set to a distance for generating a high-resolution radar image.

[0032] As described above, the echo components (echo signals) extracted for each observation point have slight delay times depending on the positions of the transmitting and receiving antennas and the target object. Therefore, by calculating the convolution integral of the echo components and the matched filter, it is possible to achieve high resolution in radar images.

[0033] As shown in Figure 5, the above-mentioned synthetic aperture processing (synthetic aperture radar) collects many observation signals and creates an image, so it can be likened to the imaging of an optical image by a lens. In this case, the size of the lens corresponds to the aperture length of the virtual large aperture antenna obtained by the synthetic aperture processing, and the screen that projects the image corresponds to the matched filter.

[0034] In this embodiment, a radar image is generated by performing synthetic aperture processing on the observation signal obtained from the radar device, and it is possible to detect objects contained in the radar image (perform object identification) by using, for example, a learning model equivalent to radar sensing AI.

[0035] Specifically, for example, when the radar system is applied to a security inspection system installed in facilities such as airports, stations, shopping malls, concert halls, and exhibition centers, it is possible to use the learning model to detect whether or not a person under inspection moving near the radar device is carrying a specified object (e.g., a dangerous object).

[0036] The learning model described above is generated by learning, for example, teacher data (learning data) including a set of a radar image and label information representing the radar image (or objects contained in the radar image). The learning model that has learned such teacher data is constructed so as to output label information representing objects contained in the radar image when the radar image is input. The label information (objects represented by the label information) output from the learning model corresponds to the detection result.

[0037] The learning model is generated based on technologies such as artificial intelligence (AI), machine learning, or deep learning. Specifically, the learning model can be generated by applying various machine learning algorithms such as a neural network or a random forest.

[0038] Here, in order to improve the detection accuracy in the learning model, it is necessary to have the learning model learn a large amount of teacher data, but it is difficult to prepare such a large amount of teacher data (especially radar images). Generally, in this embodiment, it is considered to apply an optical image padding technique to prepare a large amount of teacher data, but since the nature of a radar image (radio waves) generated by signal processing using an observation signal that is complex information representing amplitude and phase is significantly different from that of a general image (optical), if the optical image padding technique is simply applied, padding that appropriately reproduces the class distribution of the radar image (i.e., features that represent the radar image-likeness) cannot be realized.

[0039] Therefore, in this embodiment, unlike the optical image augmentation technology, we propose an augmentation technology (i.e., radar image augmentation technology) that generates a large amount of training data that reproduces the radar image-like appearance during the signal processing process that generates the radar image.

[0040] The radar system according to this embodiment will be described in detail below. An example of the configuration of a radar system is shown in Fig. 6. As shown in Fig. 6, the radar system 10 includes a radar device 20 and an information processing device 30 that is communicably connected to the radar device 20.

[0041] The radar device 20 is a MIMO radar that employs the above-mentioned FMCW system, and includes, for example, a plurality of transmitting antennas 1 and a plurality of receiving antennas 2. Furthermore, the radar device 20 includes a synthesizer 21, a mixer 22, and an A / D converter .

[0042] The synthesizer 21 generates a radar signal based on the FMCW method (a transmission wave modulated so that the frequency changes over time). The radar signal generated by the synthesizer 21 is output to the multiple transmitting antennas 1 and the mixer 22.

[0043] Each of the multiple transmitting antennas 1 transmits (radiates) the radar signal output from the synthesizer 21 in a time-division manner. Also, each of the multiple receiving antennas 2 receives a reflected wave signal based on a reflected wave of the radar signal transmitted from each of the multiple transmitting antennas 1. The reflected wave signals received by the multiple receiving antennas 2 are output to a mixer 22.

[0044] The mixer 22 mixes the radar signal output from the synthesizer 21 with the reflected wave signals output from the multiple receiving antennas 2. By such mixing by the mixer 22, the above-mentioned IF signal is generated (obtained).

[0045] In addition, when the radar device 20 is configured to include the transmitting antennas 1a and 1b and the receiving antennas 2a to 2d shown in Fig. 2, the IF signal is generated for each of the observation points 3a to 3h (i.e., combinations of the transmitting antennas 1a and 1b and the receiving antennas 2a to 2d). The IF signal generated by the mixer 22 is output to the A / D converter 23.

[0046] The A / D converter 23 generates (acquires) an observation signal by converting the IF signal output from the mixer 22 from analog to digital (i.e., performing A / D conversion on the IF signal). The observation signal generated by the A / D converter 23 corresponds to the observation signal observed at each of the above-mentioned observation points 3a to 3h, and is output (transmitted) to the information processing device 30.

[0047] As shown in FIG. 6, the information processing device 30 includes a learning model holding unit 31, a signal processing unit 32, an augmentation processing unit 33, and a learning processing unit .

[0048] The learning model holding unit 31 holds a learning model (i.e., a learning model for detecting an object from which a radar signal is transmitted) that is constructed to output label information representing an object contained in a radar image by inputting the radar image as described above.

[0049] The signal processing unit 32 includes a synthetic aperture processing unit 321 and a detection unit 322. The synthetic aperture processing unit 321 acquires an observation signal output from the radar device 20 (the A / D converter 23) and generates a radar image (hereinafter referred to as a first radar image) by executing synthetic aperture processing on the observation signal. The first radar image generated by the synthetic aperture processing unit 321 is output to the detection unit 322.

[0050] The detection unit 322 detects an object included in the first radar image output from the synthetic aperture processing unit 321 by using the learning model held in the learning model holding unit 31. Specifically, the detection unit 322 inputs the first radar image output from the synthetic aperture processing unit 321 to the learning model and acquires label information output from the learning model. The label information acquired in this manner corresponds to information (i.e., detection result) representing the object detected (estimated) by the learning model. The first radar image output from the synthetic aperture processing unit 321 and the label information acquired by the detection unit 322 are output to the learning processing unit 34.

[0051] Here, it is assumed that the synthetic aperture processing unit 321 executes synthetic aperture processing (radar signal processing) based on a first condition that is determined in advance. In this case, the padding processing unit 33 acquires an observation signal output from the radar device 20 (A / D converter 23) and acquires a second condition different from the first condition by changing the first condition. The padding processing unit 33 executes synthetic aperture processing (radar signal processing) based on the second condition on the observation signal to pad the radar image. In the following description, the radar image generated by the padding processing unit 33 (i.e., the padded radar image) is referred to as a second radar image for convenience.

[0052] The padding processor 33 includes a signal mixer 331 , an antenna selector 332 , a matched filter setting unit 333 , and a radar image padding unit 334 .

[0053] The signal mixer 331 generates a composite signal by adding, to an observation signal acquired from the radar device 20, another observation signal that is acquired at a different timing from the observation signal.

[0054] Further, the observation signals acquired from the above-mentioned radar device 20 are signals observed at observation points realized for each combination of, for example, the transmitting antenna 1 and the receiving antenna 2, and the antenna selection unit 332 selects some of the combinations realized by the multiple transmitting antennas 1 and the multiple receiving antennas 2 (i.e., some of the multiple antennas), and acquires the observation signals observed at the observation points realized by the selected combinations. In other words, the antenna selection unit 332 acquires some of the observation signals output from the radar device 20.

[0055] The matched filter setting unit 333 sets (generates) a matched filter used in the convolution integral calculation with the echo components of the selected antenna combination performed in the synthetic aperture processing described above.

[0056] The radar image augmentation unit 334 generates a second radar image by performing synthetic aperture processing using the composite signal generated by the signal mixer 331.

[0057] In other words, if the above-mentioned first condition includes that the target of the synthetic aperture processing is the observation signal acquired from the radar device 20, the signal mixing unit 331 can be said to be a functional unit that changes the first condition to a second condition that the target of the synthetic aperture processing is the synthetic signal generated by the signal mixing unit 331.

[0058] In addition, the radar image padding unit 334 generates a second radar image by performing synthetic aperture processing using the observation signals acquired by the antenna selection unit 332.

[0059] Here, if it is assumed that the first condition includes that the target of the synthetic aperture processing is the observation signal acquired from the radar device 20 as described above, then the antenna selector 332 is a functional unit that changes the first condition to a second condition that the target of the synthetic aperture processing is the observation signal acquired by the antenna selector 332 (i.e., a part of the observation signal acquired from the radar device 20). In other words, the antenna selector 332 has a function of estimating a pattern of a combination of antennas different from the observation signal used by the signal processor 32 (synthetic aperture processor 321).

[0060] Furthermore, when performing the synthetic aperture processing, the radar image augmentation unit 334 uses the matched filter set by the matched filter setting unit 333. According to this, the radar image augmentation unit 334 generates a second radar image by calculating a convolution integral between the observation signal acquired from the radar device 20 and the matched filter set by the matched filter setting unit 333.

[0061] Here, if the first condition includes the use of a matched filter (hereinafter referred to as a first matched filter) prepared in advance in the synthetic aperture processing as described above, the matched filter setting unit 333 can be said to be a functional unit that changes the first condition to a second condition that uses a matched filter (hereinafter referred to as a second matched filter) different from the first matched filter. In other words, the matched filter setting unit 333 has a function of estimating a pattern of a second matched filter having different conditions from the first matched filter used by the signal processing unit 32 (synthetic aperture processing unit 321).

[0062] In other words, the above-mentioned signal mixing unit 331, antenna selection unit 332 and matched filter setting unit 333 function as a condition changing unit that changes the conditions related to the synthetic aperture processing, and in the radar image augmentation unit 334, a second radar image is generated under conditions different from those of the signal processing unit 32.

[0063] The second radar image (a radar image augmented from the first radar image) generated by the radar image augmentation unit 334 is output to the learning processing unit .

[0064] The learning processing unit 34 includes a teacher data generating unit 341 , a teacher data storage unit 342 , and a learning unit 343 .

[0065] The teacher data generating unit 341 generates teacher data by processing the second radar image and label information output from the signal processing unit 32 (detection unit 322). The teacher data generating unit 341 also generates teacher data by processing the second radar image output from the padding processing unit 33 (radar image padding unit 334) and the label information output from the signal processing unit 32. In this case, the teacher data generating unit 341 generates, as teacher data, the first radar image to which label information has been added (i.e., a set of the first radar image and label information) and the second radar image to which label information has been added (i.e., a set of the second radar image and label information). The teacher data generated by the teacher data generating unit 341 is stored in the teacher data storage unit 342.

[0066] The learning unit 343 uses the teacher data stored in the teacher data storage unit 342 to learn the learning model stored in the learning model storage unit 31 .

[0067] Fig. 7 shows an example of a hardware configuration of the information processing device 30 shown in Fig. 3. As shown in Fig. 7, the information processing device 30 includes a CPU 30a, a non-volatile memory 30b, a main memory 30c, a communication device 30d, and the like.

[0068] The CPU 30a is a processor for controlling the operation of various components in the information processing device 30. The CPU 30a may be a single processor or may be composed of multiple processors. The CPU 30a executes various programs loaded from the non-volatile memory 30b to the main memory 30c. The communication device 30d is a device configured to perform wireless communication or wired communication.

[0069] 7, only the non-volatile memory 30b and the main memory 30c are shown, but the information processing device 30 may include other storage devices such as a hard disk drive (HDD) and a solid state drive (SSD). Furthermore, the information processing device 30 may further include an input device (such as a mouse and a keyboard) and a display device (such as a display).

[0070] In this embodiment, some or all of the signal processing unit 32, the padding processing unit 33, and the learning processing unit 34 (teacher data generating unit 341 and learning unit 343) shown in Fig. 6 can be realized by causing the CPU 30a (i.e., the computer of the information processing device 30) to execute a predetermined program, that is, by software. This program may be distributed by being stored in a computer-readable storage medium, or may be downloaded to the information processing device 30 via a network.

[0071] Here, the signal processing unit 32, the padding processing unit 33 and the learning processing unit 34 shown in FIG. 6 have been described as being partly or entirely realized by software, but the signal processing unit 32, the padding processing unit 33 and the learning processing unit 34 may be partly or entirely realized by hardware such as an IC (Integrated Circuit), or may be realized by a combination of software and hardware.

[0072] Moreover, the learning model holding unit 31 and the teacher data accumulation unit 342 included in the learning processing unit 34 shown in FIG. 6 can be realized by, for example, the non-volatile memory 30b or other storage devices.

[0073] Hereinafter, an example of a processing procedure of the information processing device 30 according to this embodiment will be described with reference to the flowchart of FIG.

[0074] As described above, the radar device 20 operates to transmit a radar signal from each of the multiple transmitting antennas 1, receive a reflected wave signal based on the reflected wave of the radar signal at each of the multiple receiving antennas 2, and output an observation signal.

[0075] In this case, the information processing device 30 (the signal processing unit 32 and the padding processing unit 33) acquires the observation signal output from the radar device 20 (step S1).

[0076] After the process of step S1 is executed, the signal processing unit 32 executes processes of steps S2 and S3.

[0077] First, the synthetic aperture processor 321 included in the signal processor 32 executes synthetic aperture processing using the observation signal acquired in step S1 (step S2). In this step S2, the synthetic aperture processor 321 executes synthetic aperture processing based on, for example, a first condition.

[0078] Here, for example, if a radar device 20 is equipped with (an antenna module consisting of) multiple transmitting antennas 1 and multiple receiving antennas 2 arranged as shown on the left side of Figure 9, the radar device 20 can output observation signals observed at each of the observation points 3 arranged in a matrix as shown on the right side of Figure 9.

[0079] In this case, assuming that observation point 3 is realized for each combination of transmitting antenna 1 and receiving antenna 2 as described above, in step S2, a process is executed to capture a reflected image at distance z0 by adopting a combination of transmitting antennas and receiving antennas and a matched filter setting that are predetermined as shown in Fig. 10. In other words, in step S2, synthetic aperture processing is executed with fixed antenna and matched filter settings as the first condition.

[0080] In Fig. 10, it is assumed that the observation points are arranged in a 6 × 7 matrix, for example. Also, in Fig. 10, it is assumed that some of the observation signals obtained at all the observation points realized by the multiple transmitting antennas 1 and the multiple receiving antennas 2 are used, but all of the observation signals may be used.

[0081] The synthetic aperture processing unit 321 executes the process of step S2 (that is, the synthetic aperture process based on the first condition) to generate a first radar image including the target object.

[0082] Next, the detection unit 322 inputs the first radar image generated by the synthetic aperture processing unit 321 into the learning model stored in the learning model storage unit 31, thereby acquiring label information (label information representing the object included in the first radar image) output from the learning model (step S3).

[0083] Here, the processes of steps S2 and S3 executed by the signal processing unit 32 have been described. After the process of step S1 described above is executed, the padding processing unit 33 executes the processes of steps S4 to S7.

[0084] First, the signal mixer 331 included in the padding processor 33 executes a process (hereinafter referred to as a mixing process) of adding an observed signal different from the observed signal acquired in step S1 to the observed signal (step S4).

[0085] Specifically, as shown in FIG. 11, assuming that an observation signal acquired in step S1 is s1 and another observation signal acquired at a timing different from that of the observation signal s1 is s2, in step S4, a synthetic signal s is used as an observation signal to be used for generating a second radar image (i.e., a target for synthetic aperture processing). mix =(s1+s2) / 2 is generated. The observed signal s2 to be added to the observed signal s1 may be stored in advance in the signal mixer 331, or may be obtained from outside the information processing device 30.

[0086] Next, the antenna selection unit 332 selects some antennas from among the multiple transmitting antennas 1 and multiple receiving antennas provided in the radar device 20 (step S5). The antenna selection unit 332 acquires, from among the observation signals acquired in step S1, observation signals observed at observation points realized by the selected antennas (combinations of transmitting antennas 1 and receiving antennas 2) as observation signals to be used for generating a second radar image (i.e., targets of synthetic aperture processing). Note that the antennas selected in step S5 are at least partially different from the antennas (combinations) that realize the observation signals used in the synthetic aperture processing in step S2 described above.

[0087] Furthermore, the matched filter setting unit 333 sets a second matched filter used to generate a second radar image (step S6). Note that the setting of the second matched filter in step S6 is at least partially different from the setting of the first matched filter used in the synthetic aperture processing in step S2 described above.

[0088] After the processes of steps S4 to S6 are performed, the radar image padding unit 334 performs synthetic aperture processing based on the results of the processes of steps S4 to S6 (step S7).

[0089] Specifically, for example, in step S2, synthetic aperture processing is performed on the observation signal acquired from the radar device 20, while in step S7, synthetic aperture processing is performed on the synthetic signal generated in step S4.

[0090] Also, for example, in step S2, synthetic aperture processing is performed on the observation signals acquired from the radar device 20, while in step S7, synthetic aperture processing is performed on the observation signals observed at the observation point realized by the antenna selected in step S5 (i.e., a portion of the observation signals acquired from the radar device 20).

[0091] Furthermore, in step S2, the synthetic aperture processing is performed using a fixed first matched filter, but in step S7, the synthetic aperture processing is performed using the second matched filter set in step S6.

[0092] That is, in step S7, unlike step S2, synthetic aperture processing is executed based on a second condition in which the target of synthetic aperture processing and the setting of the matched filter are changed.

[0093] The radar image padding unit 334 executes the process of step S7 to generate a second radar image including the target object.

[0094] In the synthetic aperture processing in step S2, only one radar image is generated from the observation signal acquired from the radar device 20 (i.e., one radar measurement by the radar device 20), whereas in this embodiment, synthetic aperture processing is performed based on the processing results of steps S4 to S6, making it possible to generate second radar images in the number of (whether or not mixing processing is performed) × (the number of selected antenna patterns) × (the number of matched filter setting patterns).

[0095] Here, it has been described that synthetic aperture processing is performed (i.e., a second radar image is generated) based on the processing results of steps S4 to S6. However, in this embodiment, as long as a sufficient number of second radar images are generated, some of the processing of steps S4 to S6 may be omitted.

[0096] In this embodiment, the processes in steps S4 to S7 described above are performed to generate a second radar image in which general optical image padding techniques such as noise addition, brightness and contrast adjustment, shifting, inversion, enlargement, reduction, and mixing are reproduced by radar signal processing. The second radar image generated in this embodiment will be described below.

[0097] In the technique for augmenting an optical image, noise addition is performed by adding Gaussian noise to the image. In contrast, when noise addition is reproduced in a radar image, it is sufficient to degrade the signal-to-noise ratio (SNR) of the radar image. Specifically, for example, when a first radar image is generated using observation signals observed at the observation points shown on the right side of FIG. 10, as shown in FIG. 12, it is possible to generate a second radar image in which the SNR of the first radar image is degraded by selecting only a part of the observation points (the multiple antennas that realize the observation points) and reducing the number of the observation points (i.e., thinning out the number of observation signals used in synthetic aperture processing).

[0098] In synthetic aperture processing, many observation signals are integrated by convolution of echo components with a matched filter. In this case, the more observation signals are integrated, the more the SNR is improved. Therefore, by not using some observation signals (antennas) in the calculation, it is possible to degrade the SNR of the radar image.

[0099] As described above, in the second radar image with a degraded SNR (i.e., the second radar image with noise added to the first radar image), the power difference between the image of the object and the background noise becomes smaller, and the background noise becomes visible compared to the first radar image.

[0100] That is, in this embodiment, a second radar image different from the first radar image can be generated by changing the observation signal (i.e., the observation point or the antenna that realizes the observation point) that is the subject of the synthetic aperture processing as a condition when the synthetic aperture processing is executed.

[0101] The second radar image may be generated, for example, by adding complex noise to the observation signal or the first radar image.

[0102] Although the case where noise is added (deteriorating the SNR) has been described here, a second radar image may be generated in which the SNR of the first radar image has been improved. Specifically, for example, a second radar image in which the SNR of the first radar image has been improved can be generated by performing synthetic aperture processing using a larger number of observation signals. Furthermore, when generating the second radar image using the observation signals, signal processing for improving the SNR (signal processing technology that realizes noise suppression and side lobe suppression) may be applied.

[0103] The second radar image may be generated by adjusting (changing) the brightness or contrast of the first radar image.

[0104] From the viewpoint of changing the brightness or contrast of the first radar image, the second radar image may be generated with a dynamic range different from that of the first radar image. As shown in FIG. 13, the dynamic range corresponds to the width between the maximum power (upper limit) and the minimum power (lower limit) of the observation signal for visualizing the object in the radar image. By adjusting the dynamic range to the power of the object and background noise in the observation signal, the brightness and contrast of the radar image can be changed. For example, when the maximum power of the dynamic range is higher than the power of the object, a radar image including the object visualized in a relatively dark state is generated. Also, when the minimum power of the dynamic range is higher than the power of the background noise, a radar image in which the noise is not visualized is generated. Furthermore, when the minimum power of the dynamic range is lower than the power of the background noise, a radar image with a lot of background noise is generated.

[0105] In this embodiment, as described above, a second radar image different from the first radar image may be generated by changing the dynamic range as a condition when the synthetic aperture processing is executed.

[0106] As described above, the matched filter corresponds to a screen that projects an image. When shifting, inverting, rotating, enlarging, and reducing are to be reproduced in a radar image, the matched filter is shifted, inverted, rotated, enlarged, and reduced before radar signal processing (synthetic aperture processing) is performed, as shown in FIG. 14.

[0107] Specifically, a radar image is generated by calculating the convolution integral of the echo components extracted from the observation signal in synthetic aperture processing and a matched filter, and the shift of the radar image is reproduced by shifting the coordinates of the matched filter (screen).

[0108] The inversion of the radar image is reproduced by inverting the polarity of the top / bottom and left / right with respect to the coordinates of the matched filter. However, in the inversion of the radar image, only one of the polarities of the top / bottom and left / right may be inverted.

[0109] The rotation of the radar image is reproduced by rotating the coordinates of the matched filter (i.e., generating the matched filter in rotated coordinates).

[0110] The enlargement of the radar image is reproduced by reducing the size of the matched filter. Specifically, when the size of the matched filter is reduced, a radar image is generated in which the size of the target object is relatively enlarged. Here, the size of the matched filter corresponds to the imaging range in which the radar image is generated by synthetic aperture processing.

[0111] The reduction in the radar image is reproduced by increasing the size of the matched filter: if the size of the matched filter is increased, a radar image is generated in which the size of the object is reduced relative to the original size.

[0112] In this embodiment, a second radar image different from the first radar image can be generated by changing the settings (coordinates, size, etc.) of the matched filter as a condition for executing the above-mentioned synthetic aperture processing.

[0113] The shift of the radar image can also be reproduced by selecting an antenna (combination of antennas) that shifts the observation point for observing the observation signal used to generate the radar image, as shown in FIG. 15.

[0114] Specifically, for example, a first radar image is generated using an observation signal observed at an observation point realized by an antenna installed in the range of -10 cm to +10 cm in the x direction. In this case, if an object exists at a position of x=±0 cm, the object is visualized near the center of the first radar image. In contrast, if a second radar image is generated using an observation signal observed at an observation point realized by an antenna installed in the range of ±0 cm to +10 cm in the x direction, the object is visualized in a position to the left of the first radar image in the second radar image. Similarly, if a second radar image is generated using an observation signal observed at an observation point realized by an antenna installed in the range of -10 cm to ±0 cm in the x direction, the object is visualized in a position to the right of the first radar image in the second radar image.

[0115] That is, in this embodiment, a second radar image in which the first radar image (or the object contained in the first radar image) is shifted may be generated by changing the observation signal (i.e., the observation point or the antenna that realizes the observation point) that is the subject of the synthetic aperture processing as a condition when the synthetic aperture processing is executed.

[0116] Here, changing the settings of the matched filter has been described, and as described above, the matched filter is set to a distance for generating a high-resolution image. If the distance set in the matched filter is misaligned, the SNR of the radar image generated by executing synthetic aperture processing (i.e., generated using the matched filter) will be degraded. Specifically, if the distance to the target is 2 m and synthetic aperture processing is executed using a matched filter set to a distance of 2.5 m, the calculated correlation will be smaller (i.e., a radar image with a degraded SNR will be generated) compared to when a matched filter set to a distance of 2 m is used.

[0117] Therefore, in this embodiment, the setting (distance) of the matched filter may be changed in order to degrade the SNR (for example, by adding noise).

[0118] In addition, the radar image is mixed by generating a composite signal s from an observation signal s1 acquired from the radar device 20 and an observation signal s2 different from the observation signal s1. mix and generate the composite signal s mix The image can be recreated by performing synthetic aperture processing using

[0119] That is, in this embodiment, the composite signal s mix A second radar image different from the first radar image can be generated by executing synthetic aperture processing based on a condition targeting the first radar image. In this case, for example, one object is visualized in the first radar image, whereas a second radar image in which the object and two objects different from the object are visualized (i.e., a second radar image including an object visualized by the observation signal s1 and an object visualized by the observation signal s2) can be generated.

[0120] Here, the augmentation techniques including noise addition, brightness / contrast adjustment, shifting, inversion, enlargement, reduction, and mixing are reproduced in the radar image by changing the conditions when the synthetic aperture processing is performed, but the second radar image may be generated by reproducing other augmentation techniques in the radar image. Below, smoothing and high resolution are described as examples of other augmentation techniques.

[0121] First, the smoothing of the radar image can be reproduced by changing the resolution of the matched filter. Specifically, since the resolution of the radar image depends on the resolution of the matched filter (the number of pixels per inch), by changing the resolution of the matched filter when synthetic aperture processing (digital signal processing) is performed, it is possible to generate, for example, a second radar image with a higher resolution (i.e., smoother) than the first radar image.

[0122] For example, if a matched filter in which pixels are spaced at 1 cm intervals is used to map a range of -10 cm to +10 cm in the x and y directions, the number of pixels in the matched filter will be 21 x 21. On the other hand, if a matched filter in which pixels are spaced at 2 mm intervals is used to map a range of -10 cm to +10 cm in the x and y directions, the number of pixels in the matched filter will be 101 x 101.

[0123] According to this, by changing the matched filter in which pixels are spaced 1 cm apart to a matched filter in which pixels are spaced 2 mm apart, a second radar image that is smoother than the first radar image can be generated.

[0124] Next, the high resolution of the radar image can be reproduced by changing the combination of antennas that realize the observation points that observe the observation signals used in the synthetic aperture processing (i.e., the antennas used in the synthetic aperture processing). Here, the spatial resolution that can be realized in the radar image generated by executing the synthetic aperture processing is defined by the aperture length of the virtual antenna as described above. The aperture length of the virtual antenna corresponds to the distance between the antennas located at both ends of the antennas used in the synthetic aperture processing.

[0125] In this case, by changing the combination of antennas used in the synthetic aperture processing so that the aperture length of the virtual antenna changes, it is possible to generate a second radar image having a spatial resolution different from that of the first radar image.

[0126] For example, if the first radar image is generated using an observation signal observed at an observation point realized by an antenna installed in the range of -10 cm to +10 cm in the x direction, the aperture length of the virtual antenna is 20 cm. On the other hand, if the second radar image is generated using an observation signal observed at an observation point realized by an antenna installed in the range of -5 cm to +5 cm in the x direction, the aperture length of the virtual antenna is 10 cm.

[0127] According to this, by changing the combination of antennas installed in the range of -10 cm to +10 cm in the x direction to a combination of antennas installed in the range of -5 cm to +5 cm in the x direction, it is possible to generate a second radar image having a lower spatial resolution than the first radar image.

[0128] For example, a second radar image having a higher spatial resolution than the first radar image may be generated by changing the combination of antennas installed in the range of -5 cm to +5 cm in the x direction to a combination of antennas installed in the range of -10 cm to +10 cm in the x direction.

[0129] Fig. 16 shows examples of the first and second radar images generated in this embodiment. Fig. 16 shows examples of a first radar image, a second radar image obtained by shifting the first radar image, a second radar image obtained by flipping the first radar image vertically and horizontally, a second radar image obtained by rotating the first radar image, a second radar image obtained by enlarging an object included in the first radar image, a second radar image obtained by adding noise to the first radar image, a second radar image obtained by changing the brightness and contrast of the first radar image, and a second radar image obtained by changing the resolution of the first radar image.

[0130] 8, the teacher data generating unit 341 included in the learning processing unit 34 generates teacher data based on the first radar image generated by executing the synthetic aperture processing in the above-mentioned step S2, the label information acquired in step S3, and the second radar image generated by executing the synthetic aperture processing in step S7 (step S8). In step S8, a plurality of teacher data including a set of radar images (each of the first and second radar images) and label information is generated. The plurality of teacher data generated in step S8 is stored in the teacher data storage unit 342.

[0131] The learning unit 343 executes a process of making the learning model held in the learning model holding unit 31 learn the multiple pieces of teacher data stored in the teacher data storage unit 342 (step S9).

[0132] In learning the learning model, for example, a radar image included in the teacher data is input to the learning model to obtain label information output from the learning model, and the error between the obtained label information and the label information included in the teacher data is fed back to the learning model (i.e., a parameter such as a weighting factor of the learning model is updated so that the error is reduced) is processed for each teacher data. The learning model that has learned the teacher data as described above is stored again in the learning model storage unit 31.

[0133] 8, it is assumed that the processes of steps S1 to S9 are executed when an observation signal is output from the radar device 20, but the processes of steps S1 to S9 are executed repeatedly every time an observation signal is output from the radar device 20. This allows the learning model to learn a large amount of teacher data, making it possible to obtain a learning model with high detection accuracy.

[0134] Also, in FIG. 8, it has been described that after the processing of step S1 is executed, the processing of steps S2 and S3 is executed by the signal processing unit 32, and the processing of steps S4 to S7 is executed by the padding processing unit 33, but the processing of steps S2 and S3 and the processing of steps S4 to S7 may be executed in parallel or in a predetermined order.

[0135] 8, the teacher data is generated using the label information acquired in step S3, but if the detection accuracy of the learning model is insufficient, the label information (label information representing the object included in the first radar image) output from the learning model by inputting the first radar image to the learning model may not be appropriate, and it is highly likely that appropriate learning cannot be performed with the teacher data generated using such label information. In such a case, in step S3, for example, label information designated (input) by the user who uses the information processing device 30 may be acquired.

[0136] In addition, in FIG. 8, the process of step S9 is described as being executed after the process of step S8 is executed, but the process of step S9 may be executed at any timing.

[0137] Specifically, the processing of step S9 may be executed when a predetermined number (amount) of teacher data has been accumulated in the teacher data storage unit 342, for example, by repeatedly executing the processing of steps S1 to S8, or may be executed in response to instructions from a user using the information processing device 30.

[0138] Furthermore, for example, when a learning model is actually used to detect an object included in a radar image, the detection accuracy of the learning model may decrease due to a change in the object or the environment in which the object is placed. The above-mentioned process of step S9 may be executed at the timing when it is determined that the detection accuracy of the learning model has decreased. Specifically, when the learning model is constructed to output a score for the label information (a value indicating the reliability of the label information) in addition to the label information, it may be determined whether the detection accuracy of the learning model has decreased based on the score. In addition, when a radar image including a known object can be prepared, it may be determined whether the detection accuracy of the learning model has decreased based on the accuracy rate calculated by inputting the radar image into the learning model and comparing the label information output from the learning model with the known object (the label information representing the known object).

[0139] As described above, the information processing device 30 according to the present embodiment transmits a radar signal and acquires an observation signal (first observation signal) based on the radar signal and the reflected wave signal from the radar device 20 having a plurality of antennas (transmitting antenna 1 and receiving antenna 2) configured to receive a reflected wave signal based on the reflected wave of the radar signal. The information processing device 30 according to the present embodiment generates a first radar image by executing synthetic aperture processing (radar signal processing) based on a first condition that is predetermined for the observation signal acquired from the radar device 20. In addition, the information processing device 30 according to the present embodiment generates a second radar image by executing synthetic aperture processing (radar signal processing) based on a second condition different from the first condition for the observation signal acquired from the radar device 20. The first and second radar images generated in the present embodiment are used as teacher data for learning a learning model for detecting an object from which the radar signal is transmitted.

[0140] In this embodiment, the above-mentioned configuration makes it possible to further generate a second radar image equivalent to an augmented radar image from an observation signal from which generally only one radar image (first radar image) is generated, thereby making it possible to easily prepare teacher data (radar images) to be used for learning the learning model.

[0141] In this embodiment, the first and second conditions described above include, for example, the target (observation signal) on which synthetic aperture processing is performed and the (settings of) matched filters used when generating the first and second radar images.

[0142] Specifically, the first radar image is generated using an observation signal acquired from the radar device 20, and the second radar image is generated using a composite signal (third observation signal) that is generated by adding an observation signal (second observation signal) different from the observation signal to the observation signal. With this configuration, it is possible to prepare, as teacher data, a second radar image that includes, for example, an object included in the first radar image and an object different from the first object, in addition to the first radar image.

[0143] Moreover, the first radar image is generated using an observation signal acquired from the radar device 20, and the second radar image is generated using an observation signal (second observation signal) that is a part of the observation signal. With this configuration, it is possible to prepare, as teacher data, a second radar image in which, for example, the SNR of the first radar image is degraded (that is, noise is added to the first radar image) in addition to the first radar image. Note that the observation signal that is a part of the observation signal acquired from the radar device 20 corresponds to an observation signal observed at an observation point realized by, for example, a part of antennas selected from a plurality of antennas. Also, although the second radar image has been described here as a radar image in which the SNR is degraded more than that of the first radar image, the second radar image may be a radar image in which the SNR of the first radar image is improved.

[0144] Moreover, the first radar image is generated by executing a synthetic aperture process that calculates a convolution integral between an observation signal acquired from the radar device 20 and a first matched filter prepared in advance, and the second radar image is generated by executing a synthetic aperture process that performs a convolution operation between the observation signal and a second matched filter different from the first matched filter. With this configuration, it is possible to prepare a second radar image, for example, by shifting, inverting, rotating, enlarging or reducing the first radar image, as teacher data, in addition to the first radar image.

[0145] When generating the second radar image by shifting, inverting, rotating, enlarging or reducing the first radar image as described above, a second matched filter is used that is generated by shifting, inverting, rotating, enlarging or reducing the first matched filter that was used when the first radar image was generated, but the second radar image may be generated using a second matched filter in which a distance different from the distance set for the first matched filter is set. When a second matched filter in which a distance different from the distance set for the first matched filter is set is used in this manner, for example, a second radar image in which the SNR of the first radar image is degraded can be prepared as teacher data.

[0146] Furthermore, the second radar image may be generated using a second matched filter having a different resolution from the first matched filter, whereby the second radar image having a different resolution from the first radar image can be prepared as teacher data.

[0147] Here, the above-mentioned first and second conditions have been described as including a target for synthetic aperture processing and (the settings of) a matched filter, but the first and second conditions may also include a dynamic range. Specifically, when the first radar image is generated based on a first dynamic range, the second radar image may be generated based on a second dynamic range different from the first dynamic range. With this configuration, it is possible to prepare, as teacher data, a second radar image in which the brightness and contrast of the first radar image are changed.

[0148] In addition, in this embodiment, the first and second radar images generated as described above are used as teacher data (i.e., the learning model learns the teacher data), thereby making it possible to improve the detection accuracy of the learning model, and the teacher data includes the first and second radar images and label information representing the object contained in the first radar image.

[0149] The label information included in the teacher data in this manner may be, for example, label information output from the learning model by inputting the first radar image to the learning model. With this configuration, for example, it is possible to reduce the effort required for the user to prepare label information. On the other hand, the label information included in the teacher data may be specified by the user who uses the information processing device 30. With this configuration, it is possible to prevent the learning model from learning teacher data including inappropriate label information, thereby preventing the improvement of the detection accuracy of the learning model from being hindered.

[0150] In the present embodiment, the radar signal is described as being transmitted based on the FMCW method, but may be transmitted based on other methods. Also, in the present embodiment, the radar device 20 is described as being a MIMO radar having a plurality of transmitting antennas 1 and a plurality of receiving antennas 2, but the radar device 20 (radar system) may be realized using various other radars.

[0151] Furthermore, in the present embodiment, the radar signal processing executed to generate a radar image has been described as synthetic aperture processing (signal processing performed in a synthetic aperture radar), but the radar signal processing is not limited to synthetic aperture processing, and various radar imaging processes can be applied to the present embodiment. Specifically, the radar signal processing described in the present embodiment may be signal processing based on beamforming. For example, in signal processing based on beamforming, convolution integration is performed using a mode vector instead of a matched filter, and in this case, the radar image can be amplified by changing the conditions of the mode vector.

[0152] In the present embodiment, the radar system 10 has been described as including the radar device 20 and the information processing device 30 shown in FIG. 6, but the radar system 10 may have other configurations. Specifically, in the radar system 10, some of the units 31 to 34 included in the information processing device 30 may be incorporated in the radar device 20, or the radar device 20 and the information processing device 30 may be configured as a single unit. Furthermore, some of the units 31 to 34 included in the information processing device 30 (e.g., the learning processing unit 34, etc.) may be arranged in a server device or the like outside the radar system 10. In this case, the processing result (the learning model that has learned the teacher data or the parameters of the learning model) by the learning processing unit 34 arranged in the server device may be fed back from the server device to the information processing device 30. In addition, the information processing device 30 according to the present embodiment may be configured to operate as various server devices.

[0153] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents described in the claims, as well as in the scope and spirit of the invention.

[0154] The following supplementary notes are further disclosed regarding the above-described embodiment. [1] an acquisition means for acquiring a first observation signal based on a radar signal and a reflected wave signal from a radar device including a plurality of antennas configured to transmit the radar signal and receive a reflected wave signal based on a reflected wave of the radar signal; a first generation means for generating a first radar image by performing signal processing on the acquired first observation signal based on a first condition that is determined in advance; a second generating means for generating a second radar image by performing signal processing on the acquired first observation signal based on a second condition different from the first condition; Equipped with The first and second radar images are used as training data for a learning model for detecting an object from which the radar signal is transmitted. Information processing device. [2] the first radar image is generated using the first observation signal; The second radar image is generated using a third observation signal that is generated by adding a second observation signal different from the first observation signal to the first observation signal. [1] An information processing device according to the present invention. [3] the first radar image is generated using the first observation signal; The second radar image is generated using a second observation signal that is a part of the first observation signal. [1] or [2]. [4] the first radar image is generated by performing a synthetic aperture process that performs a convolution integral or a convolution operation between the first observation signal and a first matched filter that is prepared in advance, The second radar image is generated by performing a synthetic aperture process that performs a convolution integral of the first observation signal with a second matched filter different from the first matched filter. The information processing device according to any one of [1] to [3]. [5] The information processing device according to [4], wherein the second matched filter is generated by shifting, inverting, rotating, enlarging or reducing the first matched filter. [6] The information processing device according to [4] or [5], wherein a distance set for the second matched filter is different from a distance set for the first matched filter. [7] The information processing device according to any one of [4] to [6], wherein a resolution of the second matched filter is different from a resolution of the first matched filter. [8] The information processing device according to any one of [1] to [7], wherein the second radar image is a radar image obtained by degrading a signal-to-noise ratio of the first radar image. [9] The information processing device according to any one of [1] to [8], wherein the second radar image is a radar image obtained by improving a signal-to-noise ratio of the first radar image.

[10] The first radar image is generated based on a first dynamic range; The second radar image is generated based on a second dynamic range different from the first dynamic range. The information processing device according to any one of [1] to [9].

[11] The information processing device according to any one of [1] to

[10] , wherein the teacher data includes the first and second radar images, and label information representing an object included in the first radar image output from the learning model by inputting the first radar image into the learning model, or label information representing the object specified by a user.

[12] The information processing device according to any one of [1] to

[11] , wherein the radar signal is transmitted based on a Frequency Modulated Continuous Wave (FMCW) method.

[13] An information processing device according to any one of [1] to

[12] , The radar device; A system comprising:

[14] A method executed by an information processing device, comprising: acquiring a first observation signal based on a radar signal and a reflected wave signal from a radar device including a plurality of antennas configured to transmit the radar signal and receive a reflected wave signal based on a reflected wave of the radar signal; generating a first radar image by performing signal processing on the acquired first observation signal based on a first condition that is determined in advance; generating a second radar image by performing signal processing on the acquired first observation signal based on a second condition different from the first condition; Equipped with The first and second radar images are used as training data for a learning model for detecting an object from which the radar signal is transmitted. method.

[15] A program executed by a computer of an information processing device, The computer includes: acquiring a first observation signal based on a radar signal and a reflected wave signal from a radar device including a plurality of antennas configured to transmit the radar signal and receive a reflected wave signal based on a reflected wave of the radar signal; generating a first radar image by performing signal processing on the acquired first observation signal based on a first condition that is determined in advance; generating a second radar image by performing signal processing on the acquired first observation signal based on a second condition different from the first condition; Run the command, The first and second radar images are used as training data for a learning model for detecting an object from which the radar signal is transmitted. program. [Explanation of symbols]

[0155] 1...transmitting antenna, 2...receiving antenna, 10...radar system, 20...radar device, 21...synthesizer, 22...mixer, 23...A / D converter, 30...information processing device, 30a...CPU, 30b...non-volatile memory, 30c...main memory, 30d...communication device, 31...learning model holding unit, 32...signal processing unit (first generation unit), 33...padding processing unit (second generation unit), 34...learning processing unit, 321...synthetic aperture processing unit, 322...detection unit, 331...signal mixing unit, 332...antenna selection unit, 333...matched filter setting unit, 334...radar image padding unit, 341...teacher data generation unit, 342...teacher data storage unit, 343...learning unit.

Claims

1. an acquisition means for acquiring a first observation signal based on a radar signal and a reflected wave signal from a radar device including a plurality of antennas configured to transmit the radar signal and receive a reflected wave signal based on a reflected wave of the radar signal; a first generating means for generating a first radar image by performing signal processing on the acquired first observation signal based on a predetermined first condition; a second generating means for generating a second radar image by performing signal processing on the acquired first observation signal based on a second condition different from the first condition; Equipped with The first and second radar images are used as teacher data for learning a learning model for detecting an object from which the radar signal is transmitted. Information processing device.

2. the first radar image is generated using the first observation signal; The second radar image is generated using a third observation signal that is generated by adding a second observation signal different from the first observation signal to the first observation signal.

2. The information processing device according to claim 1.

3. the first radar image is generated using the first observation signal; The second radar image is generated using a second observation signal that is a part of the first observation signal.

2. The information processing device according to claim 1.

4. the first radar image is generated by executing a synthetic aperture process that calculates a convolution integral of the first observation signal and a first matched filter that is prepared in advance; The second radar image is generated by performing a synthetic aperture process that calculates a convolution integral of the first observation signal with a second matched filter different from the first matched filter.

2. The information processing device according to claim 1.

5. 5. The information processing apparatus according to claim 4, wherein the second matched filter is generated by shifting, inverting, rotating, enlarging or reducing the first matched filter.

6. 5. The information processing apparatus according to claim 4, wherein the distance set for the second matched filter is different from the distance set for the first matched filter.

7. 5. An information processing apparatus according to claim 4, wherein the resolution of the second matched filter is different from the resolution of the first matched filter.

8. The information processing apparatus according to claim 1 , wherein the second radar image is a radar image obtained by degrading a signal-to-noise ratio of the first radar image.

9. The information processing apparatus according to claim 1 , wherein the second radar image is a radar image obtained by improving a signal-to-noise ratio of the first radar image.

10. The first radar image is generated based on a first dynamic range; The second radar image is generated based on a second dynamic range different from the first dynamic range.

2. The information processing device according to claim 1.

11. 2. The information processing device according to claim 1, wherein the teacher data includes the first and second radar images, and label information representing an object included in the first radar image output from the learning model by inputting the first radar image into the learning model, or label information representing the object specified by a user.

12. 2. The information processing device according to claim 1, wherein the radar signal is transmitted based on a Frequency Modulated Continuous Wave (FMCW) method.

13. An information processing device according to any one of claims 1 to 12, The radar device; A system comprising:

14. A method executed by an information processing device, comprising: acquiring a first observation signal based on a radar signal and a reflected wave signal from a radar device including a plurality of antennas configured to transmit the radar signal and receive a reflected wave signal based on a reflected wave of the radar signal; generating a first radar image by performing signal processing on the acquired first observation signal based on a first condition that is determined in advance; generating a second radar image by performing signal processing on the acquired first observation signal based on a second condition different from the first condition; Equipped with The first and second radar images are used as teacher data for learning a learning model for detecting an object from which the radar signal is transmitted. method.

15. A program executed by a computer of an information processing device, The computer includes: acquiring a first observation signal based on a radar signal and a reflected wave signal from a radar device including a plurality of antennas configured to transmit the radar signal and receive a reflected wave signal based on a reflected wave of the radar signal; generating a first radar image by performing signal processing on the acquired first observation signal based on a first condition that is determined in advance; generating a second radar image by performing signal processing on the acquired first observation signal based on a second condition different from the first condition; Run the command, The first and second radar images are used as teacher data for learning a learning model for detecting an object from which the radar signal is transmitted. program.

Images