Information processing device, system, method, and program

Abstract

The objective is to provide an information processing device, system, method, and program that can easily prepare radar images used for training a learning model. [Solution] The information processing device according to the embodiment comprises an acquisition means, a first generation means, and a second generation means. The acquisition means acquires a radar signal and a first observation signal based on the reflected wave signal from a radar device equipped with a plurality of antennas configured to transmit a radar signal and receive a reflected wave signal based on the reflected wave of the radar signal. The first generation means generates a first radar image by performing a convolution integral of the first observation signal and a first matched filter. The second generation means generates a second radar image by performing a convolution integral of the first observation signal and a second matched filter. The first and second radar images are used as training data for a learning model to learn how to detect an object on which a radar signal is transmitted.

Description

[Technical Field]

[0001] Embodiments of the present invention relate to information processing devices, systems, methods, and programs. [Background technology]

[0002] In recent years, radar systems have been developed that use reflected waves from radar signals (transmitted waves) to image objects (i.e., generate radar images that include the object in question).

[0003] Here, it is conceivable to use a pre-prepared learning model (radar sensing AI) to detect (identify) objects included in the radar images generated by the radar system described above.

[0004] A learning model is generated by training it with training data (a pair of radar images and label information representing the objects contained in those radar images). However, generating a learning model that can detect objects with high accuracy (i.e., a high-quality learning model) requires a large amount of training data. In other words, if the amount of training data (especially radar images) is small, it is difficult to generate a high-quality learning model, and a mechanism is needed to provide (amplify) the radar images necessary for training the learning model. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2022-078753 [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] Therefore, the problem that the present invention aims to solve is to provide an information processing device, system, method, and program that can easily prepare radar images used for training a learning model. [Means for solving the problem]

[0007] The information processing device according to the embodiment comprises an acquisition means, a first generation means, and a second generation means. The acquisition means acquires a radar signal and a first observation signal based on the reflected wave signal from a radar device equipped with a plurality of antennas configured to transmit a radar signal and receive a reflected wave signal based on the reflected wave of the radar signal. The first generation means generates a first radar image by performing a synthetic aperture process that calculates the convolution integral of the acquired first observation signal and a pre-prepared first matched filter. The second generation means generates a second radar image by performing a synthetic aperture process that calculates the convolution integral of the acquired first observation signal and a second matched filter different from the first matched filter. The first and second radar images are used as training data for a learning model to learn how to detect an object on which the radar signal is transmitted. [Brief explanation of the drawing]

[0008] [Figure 1] A diagram illustrating the FMCW system used in radar equipment. [Figure 2] A diagram illustrating the overview of MIMO radar. [Figure 3] A diagram illustrating the overview of synthetic aperture processing. [Figure 4] A diagram illustrating the overview of synthetic aperture processing. [Figure 5] A diagram illustrating the overview of synthetic aperture processing. [Figure 6] A diagram showing an example of the configuration of the radar system according to this embodiment. [Figure 7] A diagram showing an example of the hardware configuration of an information processing device. [Figure 8] A flowchart illustrating an example of the processing procedure for an information processing device. [Figure 9] A diagram illustrating observation points realized by a combination of multiple transmitting antennas and multiple receiving antennas. [Figure 10] A diagram for explaining synthetic aperture processing executed by a signal processing unit. [Figure 11] A diagram for explaining an outline of mixing processing. [Figure 12] A diagram for explaining noise addition to a radar image. [Figure 13] A diagram for explaining the dynamic range of a radar image. [Figure 14] A diagram for explaining shift, inversion, rotation, enlargement, and reduction of a radar image. [Figure 15] A diagram for explaining the shift of a radar image reproduced by selecting an antenna. [Figure 16] A diagram showing an example of the first and second radar images generated in this embodiment.

Best Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments will be described with reference to the drawings. The radar system according to this embodiment measures the azimuth of an object where the object is located and the distance to the object by using the reflected wave of a radar signal transmitted (irradiated) to the object from the object, and is configured to image the object (generate a radar image including the object). Note that the radar signal transmitted to the object in such a radar system is, for example, a radio wave such as millimeter wave (EHF: Extra High Frequency).

[0010] Hereinafter, an outline of the radar system will be briefly described. First, as an example of a radar modulation method that performs frequency modulation, the FMCW (Frequency Modulated Continuous Wave) method will be described.

[0011] According to the FMCW method, as shown in Figure 1, a radar signal (transmitted wave) is transmitted that is modulated so that its frequency changes over time, and the distance to the object is measured based on the frequency difference (hereinafter referred to as the beat frequency) between the radar signal and the reflected wave signal (received wave) based on the reflected wave of the radar signal.

[0012] Specifically, in the FMCW system, an intermediate frequency (IF) signal (hereinafter referred to as the IF signal) is obtained by mixing the radar signal and the reflected wave signal. This IF signal corresponds to the time waveform (sine wave) of the beat frequency described above. Note that the beat frequency is f b If we let γ (Hz / s) be the slope (rate of change of frequency) of the radar signal, called the chart rate, and τ be the delay time of the reflected wave signal relative to the radar signal, then the f b γ and τ have f b There is a relationship like =γτ.

[0013] When the above-mentioned IF signal is subjected to a Fast Fourier Transform (FFT), for example, the IF signal is converted into a frequency domain representation. In the result of this FFT, the beat frequency f b A peak is observed at this position, and the distance corresponding to the position of this peak (i.e., the distance to the object that reflected the radar signal) can be obtained.

[0014] Furthermore, the FMCW method, which performs linear frequency modulation as shown in Figure 1, is specifically called Linear-FMCW.

[0015] Here, the radar system includes a radar device having transmitting antennas and receiving antennas, and a MIMO (Multiple-Input Multiple-Output) radar can be used as the radar device. A MIMO radar has multiple transmitting antennas (transmitting antenna arrays) and multiple receiving antennas (receiving antenna arrays), and each transmitting antenna transmits radar signals in time division, and the reflected wave signals based on the reflected waves of the radar signals are received by multiple receiving antennas, thereby enabling the reception of many reflected wave signals (i.e., radar observation) with a small number of radar signal transmissions (i.e., radar irradiations).

[0016] Specifically, as shown in Figure 2, we assume that the MIMO radar has, for example, two transmitting antennas 1a and 1b and four receiving antennas 2a to 2d arranged linearly in a predetermined spatial direction.

[0017] In this case, for example, if a radar signal is transmitted from transmitting antenna 1a, a reflected signal based on the reflected wave of that radar signal is received by receiving antenna 2a. Here, we have described receiving antenna 2a, but the reflected signal based on the reflected wave of the radar signal transmitted from transmitting antenna 1a is similarly received by receiving antennas 2b to 2d.

[0018] Similarly, if, for example, a radar signal is transmitted from transmitting antenna 1b, a reflected signal based on the reflected wave of that radar signal is received by receiving antenna 2a. Although receiving antenna 2a has been described here, reflected signals based on the reflected wave of the radar signal transmitted from transmitting antenna 1b are similarly received by receiving antennas 2b to 2d.

[0019] In other words, in the MIMO radar described above, when a radar signal is transmitted from transmitting antenna 1a, radar observation is performed at each of the receiving antennas 2a to 2d, and similarly when a radar signal is transmitted from transmitting antenna 1b, radar observation is performed at the receiving antennas 2a to 2d.

[0020] According to this, a MIMO radar having two transmitting antennas 1a and 1b and four receiving antennas 2a to 2d, as shown in Figure 2, can realize eight observation points 3a to 3h arranged in the spatial direction by emitting radar signals only once each from transmitting antennas 1a and 1b. For example, observation point 3a is realized when receiving antenna 2a receives a reflected wave signal based on the reflected wave of the radar signal transmitted from transmitting antenna 1a. Similarly, observation points 3b to 3d are realized when receiving antennas 2b to 2d receive reflected wave signals based on the reflected wave of the radar signal transmitted from transmitting antenna 1a. Furthermore, observation point 3e is realized when receiving antenna 2a receives a reflected wave signal based on the reflected wave of the radar signal transmitted from transmitting antenna 1b. Similarly, observation points 3f to 3h are realized when receiving antennas 2b to 2d receive reflected wave signals based on the reflected wave of the radar signal transmitted from transmitting antenna 1b. In other words, in MIMO radar, one observation point is realized by combining one transmitting antenna and one receiving antenna.

[0021] With such a MIMO radar, it becomes possible to measure the distance to an object and image the object using the observation signals observed at each of the observation points 3a to 3h (radar signals transmitted in time division from transmitting antennas 1a and 1b and IF signals acquired based on reflected wave signals received by each of the receiving antennas 2a to 2d).

[0022] In this embodiment, a MIMO radar is described as being used as the radar device, but the radar device may also be a SISO (Single-Input Single-Output) radar having a single antenna for transmitting radar signals and receiving reflected wave signals.

[0023] Incidentally, in the radar system according to this embodiment, it is possible to perform imaging of an object (generation of a radar image) using the observation signal acquired from the radar device described above, but it is desirable to improve the spatial resolution when imaging the object.

[0024] As described above, when using MIMO radar, observation signals are acquired at each observation point realized for each combination of transmitting and receiving antennas. In this embodiment, in order to image the target object, the spatial resolution can be improved by performing synthetic aperture processing, which combines the observation signals acquired at each observation point.

[0025] Synthetic aperture processing is equivalent to the signal processing performed in synthetic aperture radar (SAR). Generally, the larger the antenna aperture length, the higher the spatial resolution. Synthetic aperture processing is a signal processing technique that achieves high-resolution imaging, similar to that obtained with a large-aperture antenna, by performing multiple observations while moving a low-resolution, small-aperture antenna and combining the results of these observations.

[0026] The following is an overview of synthetic aperture processing. Synthetic aperture processing involves two steps: the extraction of echo components and the calculation of convolution integrals.

[0027] First, the extraction of echo components refers to measuring the distance to an object as shown in Figure 3. Specifically, in the FMCW method described above, when an FFT is applied to the observation signal (IF signal), a power peak appears at the distance where the object is located in the result of the FFT (range peak). This is the echo component, also known as the range-compressed echo signal. This echo component is extracted for each observation point.

[0028] Next, in the convolution integral, as shown in FIG. 4, it means calculating the convolution integral of the echo component extracted for each of the above-described observation points and the matched filter. The radar image is generated by calculating such a convolution integral.

[0029] Specifically, if the echo component is a(x I ,y I ,z I ) and the matched filter is b(x I ,y I ,z I ), the high-resolution radar image c(x I ,y I ,z I ) is generated by calculating the following formula (1).

Equation

[0030] Here, N is a set of observation points, and m indicates an observation point in the set N. x I ,y I ,z I represent the width, height, and depth-direction pixels (spatial coordinates) of the radar image generated by synthetic aperture processing. b * is the complex conjugate of the matched filter b.

[0031] By the way, the spatial resolution varies not only with the antenna aperture length but also with the distance to the object located in the irradiation direction of the radar signal (that is, the object that reflects the radar signal). When generating a high-resolution radar image at a distance z0 in the above-described synthetic aperture processing, the convolution integral of the echo component extracted for each observation point and the matched filter (z0) is calculated. Specifically, each echo component is multiplied by a matched filter that extracts the component at the distance z0, and the sum thereof is calculated. That is, it can be said that the distance for generating a high-resolution radar image is set in the matched filter used in the synthetic aperture processing.

[0032] As described above, the echo components (echo signals) extracted at each observation point have a slight delay time depending on the position of the transmitting and receiving antennas and the object. Therefore, by calculating the convolution integral of these echo components and the matched filter, it is possible to achieve higher resolution in radar images.

[0033] As shown in Figure 5, the synthetic aperture processing (synthetic aperture radar) described above can be likened to imaging by a lens in an optical image, since it collects many observation signals and forms an image. In this case, the size of the lens corresponds to the aperture length of the virtual large-aperture antenna obtained by the synthetic aperture processing, and the screen on which the image is projected corresponds to the matched filter.

[0034] In this embodiment, a radar image is generated by performing synthetic aperture processing on the observation signal acquired from the radar device. For example, it is possible to detect objects included in the radar image (perform object identification) by using a learning model equivalent to a radar sensing AI.

[0035] Specifically, when applying a radar system to security inspection systems installed in facilities such as airports, train stations, shopping malls, concert halls, and exhibition venues, it is possible to use a learning model to detect whether a person being inspected is carrying a predetermined object (e.g., a dangerous object) as they move near the radar device.

[0036] The learning model described above is generated by training on training data, for example, which includes a pair of radar images and label information representing the objects contained in those radar images. A learning model trained on such training data is constructed to output label information representing the objects contained in a radar image when a radar image is input. The object represented by the label information output from the learning model corresponds to the detection result.

[0037] Furthermore, learning models are generated based on technologies such as artificial intelligence (AI), machine learning, or deep learning. Specifically, learning models can be generated by applying various machine learning algorithms, such as neural networks or random forests.

[0038] In order to improve the detection accuracy of the learning model, it is necessary to train the model with a large amount of training data, but it is difficult to prepare such a large amount of training data (especially radar images). Generally, in this embodiment, it is conceivable to apply optical image augmentation techniques to prepare a large amount of training data, but since radar images (radio waves) generated by signal processing using observation signals, which are complex number information representing amplitude and phase, have very different properties from general images (optical), simply applying optical image augmentation techniques will not be able to achieve augmentation that appropriately reproduces the class distribution of radar images (i.e., features that represent the characteristics of a radar image).

[0039] Therefore, in this embodiment, unlike optical image augmentation techniques, we propose an augmentation technique that generates a large amount of training data that reproduces the characteristics of the radar image during the signal processing process that generates the radar image (i.e., radar image augmentation technique).

[0040] The radar system according to this embodiment will now be described in detail. Figure 6 shows an example of the configuration of the radar system. As shown in Figure 6, the radar system 10 comprises a radar device 20 and an information processing device 30 that is communicatively connected to the radar device 20.

[0041] The radar system 20 is a MIMO radar employing the FMCW method described above, and includes, for example, a plurality of transmitting antennas 1 and a plurality of receiving antennas 2. Furthermore, the radar system 20 includes a synthesizer 21, a mixer 22, and an A / D converter 23.

[0042] The synthesizer 21 generates a radar signal based on the FMCW method (a transmitted wave that has been modulated so that its frequency changes over time). The radar signal generated by the synthesizer 21 is output to multiple transmitting antennas 1 and mixer 22.

[0043] Each of the multiple transmitting antennas 1 transmits (irradiates) the radar signal output from the synthesizer 21 in a time-division manner. Each of the multiple receiving antennas 2 receives a reflected wave signal based on the reflected waves of the radar signals transmitted from each of the multiple transmitting antennas 1. The reflected wave signals received by the multiple receiving antennas 2 are output to the mixer 22.

[0044] Mixer 22 mixes the radar signal output from synthesizer 21 with the reflected wave signals output from multiple receiving antennas 2. Through this mixing by mixer 22, the above-mentioned IF signal is generated (acquired).

[0045] In the case where the radar device 20 is configured to include transmitting antennas 1a and 1b and receiving antennas 2a to 2d as shown in Figure 2, an IF signal is generated for each of the observation points 3a to 3h (i.e., each combination of transmitting antennas 1a and 1b and receiving antennas 2a to 2d). The IF signal generated by the mixer 22 is output to the A / D converter 23.

[0046] The A / D converter 23 generates (acquires) an observation signal by converting the IF signal output from the mixer 22 from analog to digital (i.e., performing A / D conversion on the IF signal). The observation signal generated by the A / D converter 23 corresponds to the observation signal observed at each of the observation points 3a to 3h described above, and is output (transmitted) to the information processing device 30.

[0047] As shown in Figure 6, the information processing device 30 includes a learning model holding unit 31, a signal processing unit 32, an augmentation processing unit 33, and a learning processing unit 34.

[0048] As described above, the learning model holding unit 31 holds a learning model (i.e., a learning model for detecting objects to which radar signals are transmitted) that is constructed to output label information representing objects contained in a radar image when a radar image is input.

[0049] The signal processing unit 32 includes a synthetic aperture processing unit 321 and a detection unit 322. The synthetic aperture processing unit 321 acquires the observation signal output from the radar device 20 (A / D converter 23) and generates a radar image (hereinafter referred to as the first radar image) by performing synthetic aperture processing on the observation signal. The first radar image generated by the synthetic aperture processing unit 321 is output to the detection unit 322.

[0050] The detection unit 322 uses the learning model held in the learning model holding unit 31 to detect objects included in the first radar image output from the synthetic aperture processing unit 321. Specifically, the detection unit 322 inputs the first radar image output from the synthetic aperture processing unit 321 into the learning model and obtains label information output from the learning model. The label information obtained in this way corresponds to information representing the object detected (estimated) by the learning model (i.e., the detection result). The first radar image output from the synthetic aperture processing unit 321 and the label information obtained by the detection unit 322 are output to the learning processing unit 34.

[0051] Here, the synthetic aperture processing unit 321 described above is assumed to be performing synthetic aperture processing (radar signal processing) based on a predetermined first condition. In this case, the augmentation processing unit 33 acquires the observation signal output from the radar device 20 (A / D converter 23) and acquires a second condition different from the first condition by changing the first condition described above. The augmentation processing unit 33 performs synthetic aperture processing (radar signal processing) based on the second condition on the observation signal to augment the radar image. For the purposes of the following explanation, the radar image generated by the augmentation processing unit 33 (i.e., the augmented radar image) will be referred to as the second radar image.

[0052] The image augmentation processing unit 33 includes a signal mixing unit 331, an antenna selection unit 332, a matching filter setting unit 333, and a radar image augmentation unit 334.

[0053] The signal mixing unit 331 generates a composite signal by adding another observation signal acquired at a different timing to the observation signal acquired from the radar device 20.

[0054] Furthermore, the observation signals acquired from the radar device 20 are, for example, signals observed at observation points realized for each combination of transmitting antenna 1 and receiving antenna 2. The antenna selection unit 332 selects a portion of the combinations realized by multiple transmitting antennas 1 and multiple receiving antennas 2 (i.e., some of the antennas among the multiple antennas) and acquires the observation signals observed at the observation points realized by the selected combination. In other words, the antenna selection unit 332 acquires a portion of the observation signals output from the radar device 20.

[0055] The matching filter setting unit 333 sets (generates) a matching filter used in the convolution integral operation with the echo component of the selected antenna combination performed in the synthetic aperture processing described above.

[0056] The radar image augmentation unit 334 generates a second radar image by performing synthetic aperture processing using the composite signal generated by the signal mixing unit 331.

[0057] In other words, if the first condition described above includes making the target of synthetic aperture processing the observation signal acquired from the radar device 20, then the signal mixing unit 331 can be said to be a functional unit that changes the first condition to a second condition in which the target of synthetic aperture processing is the composite signal generated by the signal mixing unit 331.

[0058] Furthermore, the radar image augmentation unit 334 generates a second radar image by performing synthetic aperture processing using the observation signals acquired by the antenna selection unit 332.

[0059] Here, assuming that the first condition includes making the target of synthetic aperture processing the observation signal acquired from the radar device 20, as described above, the antenna selection unit 332 can be said to be a functional unit that changes the first condition to a second condition, which makes the target of synthetic aperture processing the observation signal acquired by the antenna selection unit 332 (i.e., a part of the observation signal acquired from the radar device 20). In other words, the antenna selection unit 332 has the function of estimating a combination pattern of antennas different from the observation signal used by the signal processing unit 32 (synthetic aperture processing unit 321).

[0060] Furthermore, when the radar image augmentation unit 334 performs synthetic aperture processing, it uses the matched filter set by the matched filter setting unit 333. In this case, the radar image augmentation unit 334 generates a second radar image by calculating the convolution integral of the observation signal acquired from the radar device 20 and the matched filter set by the matched filter setting unit 333.

[0061] Here, assuming that the first condition includes using a pre-prepared matched filter in the synthetic aperture processing (hereinafter referred to as the first matched filter), as described above, the matched filter setting unit 333 can be said to be a functional unit that changes the first condition to a second condition that uses a matched filter different from the first matched filter (hereinafter referred to as the second matched filter). In other words, the matched filter setting unit 333 has the function of estimating a pattern for the second matched filter, which has different conditions from the first matched filter used by the signal processing unit 32 (synthetic aperture processing unit 321).

[0062] In other words, the signal mixing unit 331, antenna selection unit 332, and matching filter setting unit 333 described above function as condition changing units that change the conditions related to synthetic aperture processing, and in the radar image augmentation unit 334, a second radar image is generated under different conditions than those of the signal processing unit 32.

[0063] The second radar image (a radar image augmented from the first radar image) generated by the radar image augmentation unit 334 is output to the learning processing unit 34.

[0064] The learning processing unit 34 includes a teacher data generation unit 341, a teacher data storage unit 342, and a learning unit 343.

[0065] The training data generation unit 341 generates training data by processing the second radar image and label information output from the signal processing unit 32 (detection unit 322). The training data generation unit 341 also generates training data by processing the second radar image output from the augmentation processing unit 33 (radar image augmentation unit 334) and the label information output from the signal processing unit 32. In this case, the training data generation unit 341 generates a first radar image with label information attached (i.e., a pair of the first radar image and label information) and a second radar image with label information attached (i.e., a pair of the second radar image and label information) as training data. The training data generated by the training data generation unit 341 is stored in the training data storage unit 342.

[0066] The learning unit 343 uses the training data stored in the training data storage unit 342 to train the learning model stored in the learning model holding unit 31.

[0067] Figure 7 shows an example of the hardware configuration of the information processing device 30 shown in Figure 3. As shown in Figure 7, the information processing device 30 includes a CPU 30a, non-volatile memory 30b, main memory 30c, and communication device 30d, etc.

[0068] The CPU 30a is a processor for controlling the operation of various components within the information processing device 30. The CPU 30a may be a single processor or may consist of multiple processors. The CPU 30a executes various programs that are loaded from the non-volatile memory 30b into the main memory 30c. The communication device 30d is a device configured to perform wireless or wired communication.

[0069] Although only the non-volatile memory 30b and main memory 30c are shown in Figure 7, the information processing device 30 may also include other storage devices such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive). Furthermore, the information processing device 30 may also include input devices (such as a mouse and keyboard) and display devices (such as a display).

[0070] In this embodiment, some or all of the signal processing unit 32, augmentation processing unit 33, and learning processing unit 34 (training data generation unit 341 and learning unit 343) shown in Figure 6 can be implemented by having the CPU 30a (i.e., the computer of the information processing device 30) execute a predetermined program, i.e., by software. This program may be stored and distributed on a computer-readable storage medium, or it may be downloaded to the information processing device 30 via a network.

[0071] In this explanation, the signal processing unit 32, augmentation processing unit 33, and learning processing unit 34 shown in Figure 6 are described as being implemented in part or all by software. However, the signal processing unit 32, augmentation processing unit 33, and learning processing unit 34 may also be implemented in hardware such as an IC (Integrated Circuit), or in combination with software and hardware.

[0072] Furthermore, the training data storage unit 342 included in the learning model holding unit 31 and the learning processing unit 34 shown in Figure 6 can be implemented by, for example, a non-volatile memory 30b or other storage device.

[0073] Below, an example of the processing procedure of the information processing device 30 according to this embodiment will be described with reference to the flowchart in Figure 8.

[0074] As described above, the radar device 20 operates by transmitting a radar signal from each of the multiple transmitting antennas 1 and receiving a reflected wave signal based on the reflected wave of the radar signal at each of the multiple receiving antennas 2, thereby outputting an observation signal.

[0075] In this case, the information processing device 30 (signal processing unit 32 and augmentation processing unit 33) acquires the observation signal output from the radar device 20 (step S1).

[0076] Once the process in step S1 is executed, the signal processing unit 32 executes the processes in steps S2 and S3.

[0077] First, the synthetic aperture processing unit 321 included in the signal processing unit 32 performs synthetic aperture processing using the observed signal acquired in step S1 (step S2). In step S2, the synthetic aperture processing unit 321 performs synthetic aperture processing based on, for example, a first condition.

[0078] Here, for example, if the radar device 20 comprises an antenna module consisting of multiple transmitting antennas 1 and multiple receiving antennas 2 arranged as shown on the left side of Figure 9, the radar device 20 can output observation signals observed at each of the observation points 3 arranged in a matrix as shown on the right side of Figure 9.

[0079] In this case, assuming that an observation point 3 is realized for each combination of transmitting antenna 1 and receiving antenna 2 as described above, in step S2, a predetermined combination of transmitting and receiving antennas and matching filter settings are adopted as shown in Figure 10, and a process is executed to capture a reflected image at distance z0. In other words, in step S2, a composite aperture process is executed with a fixed antenna and matching filter settings as the first condition.

[0080] In Figure 10, for example, the observation points are arranged on a 6x7 matrix. Also, in Figure 10, it is assumed that only a portion of the observation signals acquired at all observation points realized by multiple transmitting antennas 1 and multiple receiving antennas 2 are used, but all of the observation signals may also be used.

[0081] The synthetic aperture processing unit 321 generates a first radar image including the object by performing the processing in step S2 (i.e., synthetic aperture processing based on the first condition).

[0082] Next, the detection unit 322 inputs the first radar image generated by the synthetic aperture processing unit 321 into the learning model held in the learning model holding unit 31, thereby acquiring label information (label information representing objects included in the first radar image) output from the learning model (step S3).

[0083] Here, we have described the processes of steps S2 and S3 performed by the signal processing unit 32. However, once the process of step S1 described above is executed, the augmentation processing unit 33 executes the processes of steps S4 to S7.

[0084] First, the signal mixing unit 331 included in the augmentation processing unit 33 performs a process (hereinafter referred to as the mixing process) in which it adds an observation signal different from the observation signal acquired in step S1 to the observation signal (step S4).

[0085] Specifically, as shown in Figure 11, if the observation signal acquired in step S1 is s1, and another observation signal acquired at a different timing than the said observation signal s1 is s2, then in step S4, the composite signal s is used as the observation signal (i.e., the target of the composite aperture processing) to generate the second radar image. mix The equation =(s1+s2) / 2 is generated. Note that the observation signal s2, which is added to the observation signal s1, may be stored in advance within the signal mixing unit 331, or it may be acquired from outside the information processing device 30.

[0086] Next, the antenna selection unit 332 selects some antennas from among the multiple transmitting antennas 1 and multiple receiving antennas provided in the radar device 20 (step S5). The antenna selection unit 332 acquires the observation signals observed at the observation points realized by the selected antennas (combination of transmitting antenna 1 and receiving antenna 2) from among the observation signals acquired in step S1 as observation signals used to generate the second radar image (i.e., targets for synthetic aperture processing). The antennas selected in step S5 shall be at least partially different from the antennas (combinations) that realize the observation signals used in the synthetic aperture processing in step S2 described above.

[0087] Furthermore, the matching filter setting unit 333 sets a second matching filter used to generate the second radar image (step S6). The setting of the second matching filter in step S6 is at least partially different from the setting of the first matching filter used in the synthetic aperture processing of step S2 described above.

[0088] Once the processes described in steps S4 to S6 are executed, the radar image augmentation unit 334 performs synthetic aperture processing based on the processing results of steps S4 to S6 (step S7).

[0089] Specifically, for example, in step S2, synthetic aperture processing is performed on the observation signal acquired from the radar device 20, but in step S7, synthetic aperture processing is performed on the composite signal generated in step S4.

[0090] Furthermore, in step S2, for example, synthetic aperture processing is performed on the observation signals acquired from the radar device 20, but in step S7, synthetic aperture processing is performed on the observation signals observed at the observation point realized by the antenna selected in step S5 (i.e., a portion of the observation signals acquired from the radar device 20).

[0091] Furthermore, in step S2, synthetic aperture processing is performed using a fixed first matching filter, while in step S7, synthetic aperture processing is performed using a second matching filter set in step S6.

[0092] In other words, in step S7, unlike in step S2, synthetic aperture processing is performed based on a second condition in which the target of synthetic aperture processing and the settings of the matching filter are changed.

[0093] The radar image augmentation unit 334 generates a second radar image including the target object by performing the process in step S7.

[0094] In contrast to the synthetic aperture processing in step S2, where only one radar image is generated from the observation signal acquired from the radar device 20 (i.e., one radar measurement by the radar device 20), in this embodiment, the synthetic aperture processing is performed based on the processing results of steps S4 to S6, thereby generating a number of second radar images equal to (presence or absence of mixing processing) × (number of antenna patterns selected) × (number of matching filter setting patterns).

[0095] In this description, it is assumed that synthetic aperture processing is performed based on the processing results of steps S4 to S6 (i.e., a second radar image is generated). However, in this embodiment, if a sufficient number of second radar images are generated, some of the processing in steps S4 to S6 may be omitted.

[0096] In this embodiment, the processing described in steps S4 to S7 above is performed to generate a second radar image in which common optical image augmentation techniques such as noise addition, brightness and contrast adjustment, shift, inversion, enlargement, reduction, and mixing are reproduced by radar signal processing. The second radar image generated in this embodiment will be described below.

[0097] In optical image augmentation techniques, noise is added by adding Gaussian noise to the image. In contrast, to reproduce noise in a radar image, the signal-to-noise ratio (SNR) of the radar image can be degraded. Specifically, for example, if the first radar image is generated using observation signals observed at the observation point shown on the right side of Figure 10, then, as shown in Figure 12, a second radar image with a degraded SNR of the first radar image can be generated by selecting only a portion of the observation point (or the multiple antennas that realize it) and reducing the number of observation points (i.e., decimating the number of observation signals used in synthetic aperture processing).

[0098] In synthetic aperture processing, numerous observed signals are integrated through a convolution operation between the echo component and a matched filter. In this case, the more observed signals integrated, the better the signal-to-noise ratio (SNR). Therefore, by excluding some observed signals (antennas) from the calculation, it becomes possible to degrade the SNR of the radar image.

[0099] As described above, in the second radar image with a degraded SNR (i.e., the second radar image with noise added to the first radar image), the power difference between the image of the object and the background noise becomes smaller, and the background noise becomes more visible compared to the first radar image.

[0100] In other words, in this embodiment, a second radar image different from the first radar image can be generated by changing the observation signal that is the target of the synthetic aperture processing (i.e., the observation point or the antenna that realizes said observation point) as a condition when the synthetic aperture processing is performed.

[0101] The second radar image may be generated, for example, by adding complex noise to the observation signal or the first radar image.

[0102] Furthermore, while this explanation describes the case of adding noise (degrading the SNR), a second radar image with an improved SNR of the first radar image may also be generated. Specifically, for example, a second radar image with an improved SNR of the first radar image can be generated by performing synthetic aperture processing using more observation signals. In addition, when generating a second radar image using observation signals, signal processing that improves the SNR (signal processing techniques that achieve noise suppression or sidelobe suppression) may be applied.

[0103] The second radar image may be generated by adjusting (changing) the brightness or contrast of the first radar image.

[0104] From the perspective of changing the brightness or contrast of the first radar image, the second radar image may be generated with a different dynamic range than the first radar image. As shown in Figure 13, the dynamic range corresponds to the range between the maximum power (upper limit) and minimum power (lower limit) of the observation signal required to visualize an object in the radar image. By adjusting the dynamic range in relation to the power of the object and background noise in the observation signal, the brightness and contrast of the radar image can be changed. For example, if the maximum power of the dynamic range is higher than the power of the object, a radar image is generated that includes the object visualized in a relatively dark state. Also, if the minimum power of the dynamic range is higher than the power of the background noise, a radar image is generated in which the noise is not visualized. Furthermore, if the minimum power of the dynamic range is lower than the power of the background noise, a radar image with a lot of background noise is generated.

[0105] In this embodiment, as described above, a second radar image different from the first radar image may be generated by changing the dynamic range as a condition when the synthetic aperture processing is performed.

[0106] Furthermore, as mentioned above, the matching filter corresponds to the screen on which the image is projected. When reproducing shift, inversion, rotation, enlargement, and reduction in the radar image, the matching filter is shifted, inverted, rotated, enlarged, and reduced, and then radar signal processing (synthetic aperture processing) is performed.

[0107] Specifically, radar images are generated by calculating the convolution integral of the echo component extracted from the observation signal and the matched filter in synthetic aperture processing, but the shift in the radar image is reproduced by shifting the coordinates of the matched filter (screen).

[0108] The inversion of the radar image is reproduced by reversing the vertical and horizontal polarity of the coordinates of the matched filter. Note that in the inversion of the radar image, only one of the vertical or horizontal polarities may be reversed.

[0109] The rotation of the radar image is reproduced by rotating the coordinates of the matched filter (i.e., generating a matched filter in rotated coordinates).

[0110] The enlargement of the radar image is achieved by reducing the size of the matched filter. Specifically, when the size of the matched filter is reduced, a radar image is generated in which the size of the object is relatively enlarged. Here, the size of the matched filter corresponds to the imaging range in which the radar image is generated by synthetic aperture processing.

[0111] The reduction in radar image size is achieved by increasing the size of the matching filter. Specifically, increasing the size of the matching filter generates a radar image in which the size of the object is relatively reduced.

[0112] In this embodiment, a second radar image different from the first radar image can be generated by changing the settings of the matched filter (coordinates, size, etc.) as a condition for executing the above-described synthetic aperture processing.

[0113] Furthermore, as shown in Figure 15, the shift in the radar image can also be reproduced by selecting an antenna (or combination thereof) that shifts the observation point that observes the observation signal used to generate the radar image.

[0114] Specifically, suppose a first radar image is generated using observation signals from observation points realized by antennas installed in the range of -10cm to +10cm in the x-direction. In this case, if an object is located at x=±0cm, the object will be visible near the center of the first radar image. In contrast, if a second radar image is generated using observation signals from observation points realized by antennas located in the range of ±0cm to +10cm in the x-direction, the object will be visible to the left of the first radar image in the second radar image. Similarly, if a second radar image is generated using observation signals from observation points realized by antennas located in the range of -10cm to ±0cm in the x-direction, the object will be visible to the right of the first radar image in the second radar image.

[0115] In other words, in this embodiment, a second radar image in which the first radar image (and the objects contained therein) is shifted may be generated by changing the observation signal that is the target of the synthetic aperture processing (i.e., the observation point or the antenna that realizes said observation point) as a condition when the synthetic aperture processing is performed.

[0116] This section explains how to change the settings of a matched filter. As mentioned above, a distance is set for the matched filter to generate a high-resolution image. If the distance set for the matched filter is incorrect, the signal-to-noise ratio (SNR) of the radar image generated by the synthetic aperture processing (i.e., the image generated using that matched filter) will deteriorate. Specifically, if the distance to the object is 2m, but synthetic aperture processing is performed using a matched filter set to a distance of 2.5m, the calculated correlation will be smaller compared to using a matched filter set to a distance of 2m (i.e., a radar image with a degraded SNR will be generated).

[0117] Therefore, in this embodiment, the settings (distance) of the matching filter may be changed in order to degrade the SNR (for example, to add noise).

[0118] Furthermore, as described above, radar image mixing is performed by combining the observation signal s1 acquired from the radar device 20 and an observation signal s2 that is different from the said observation signal s1 to form a composite signal s mix The synthesized signal s is generated and mix This can be reproduced by performing synthetic aperture processing using [a specific method / tool].

[0119] In other words, in this embodiment, the combined signal s mix By performing synthetic aperture processing based on conditions targeting the first radar image, a second radar image different from the first radar image can be generated. In this case, while the first radar image visualizes, for example, one object, a second radar image can be generated in which two objects are visualized: that object and another object different from the first object (i.e., a second radar image including the object visualized by observation signal s1 and the object visualized by observation signal s2).

[0120] Here, we have described how to reproduce image augmentation techniques, including noise addition, brightness / contrast adjustment, shifting, inversion, scaling, and mixing, in a radar image by changing the conditions under which synthetic aperture processing is performed. However, a second radar image may be generated by reproducing other image augmentation techniques in a radar image. Below, we will describe smoothing and resolution enhancement as examples of other image augmentation techniques.

[0121] First, the smoothing of radar images can be reproduced by changing the resolution of the matched filter. Specifically, since the resolution of the radar image depends on the resolution of the matched filter (number of pixels per inch), by changing the resolution of the matched filter when synthetic aperture processing (digital signal processing) is performed, it is possible to generate a second radar image with higher resolution (i.e., smoother) than the first radar image.

[0122] For example, if a matched filter with pixels spaced 1 cm apart is used to map a range of -10 cm to +10 cm in the x and y directions, the number of pixels in that matched filter will be 21 × 21. On the other hand, if a matched filter with pixels spaced 2 mm apart is used to map a range of -10 cm to +10 cm in the x and y directions, the number of pixels in that matched filter will be 101 × 101.

[0123] According to this, by changing the matched filter, in which each pixel is spaced 1 cm apart, to a matched filter in which each pixel is spaced 2 mm apart, a second radar image that is smoother than the first radar image can be generated.

[0124] Next, the high resolution of radar images can be achieved by changing the combination of antennas that realize the observation points that observe the observation signals used in synthetic aperture processing (i.e., the antennas used in synthetic aperture processing). Here, the spatial resolution that can be achieved in the radar image generated by performing synthetic aperture processing is defined by the aperture length of the virtual antenna, as described above. The aperture length of the virtual antenna corresponds to the distance between the antennas located at both ends of the antenna used in synthetic aperture processing.

[0125] In this case, by changing the combination of antennas used in the synthetic aperture processing so that the aperture length of the virtual antenna changes, a second radar image with a different spatial resolution from the first radar image can be generated.

[0126] For example, if the first radar image is generated using observation signals from an observation point realized by an antenna installed in the range of -10cm to +10cm in the x-direction, the aperture length of the virtual antenna is 20cm. On the other hand, if the second radar image is generated using observation signals from an observation point realized by an antenna installed in the range of -5cm to +5cm in the x-direction, the aperture length of the virtual antenna is 10cm.

[0127] According to this, by changing the combination of antennas installed in the range of -10cm to +10cm in the x-direction to a combination of antennas installed in the range of -5cm to +5cm in the x-direction, it is possible to generate a second radar image with lower spatial resolution than the first radar image.

[0128] Furthermore, a second radar image with higher spatial resolution than the first radar image may be generated by changing, for example, the combination of antennas installed in the range of -5cm to +5cm in the x-direction to a combination of antennas installed in the range of -10cm to +10cm in the x-direction.

[0129] Figure 16 shows examples of first and second radar images generated in this embodiment. Figure 16 shows examples of a first radar image, a second radar image obtained by shifting the first radar image, a second radar image obtained by flipping the first radar image vertically and horizontally, a second radar image obtained by rotating the first radar image, a second radar image obtained by enlarging an object contained in the first radar image, a second radar image obtained by adding noise to the first radar image, a second radar image obtained by changing the brightness and contrast of the first radar image, and a second radar image obtained by changing the resolution of the first radar image.

[0130] Returning to Figure 8, the training data generation unit 341 included in the learning processing unit 34 generates training data (step S8) based on the first radar image generated by the synthetic aperture processing in step S2, the label information acquired in step S3, and the second radar image generated by the synthetic aperture processing in step S7. In step S8, multiple training data sets are generated, each containing a pair of radar images (the first and second radar images) and label information. The multiple training data sets generated in step S8 are stored in the training data storage unit 342.

[0131] The learning unit 343 executes a process to train the learning model held in the learning model holding unit 31 with multiple training data stored in the training data storage unit 342 (step S9).

[0132] During the training of the learning model, for example, by inputting radar images included in the training data into the learning model, label information output from the learning model is obtained, and the error between the obtained label information and the label information included in the training data is fed back to the learning model (i.e., parameters such as the weight coefficients of the learning model are updated so that the error decreases), and this process is performed for each set of training data. As described above, the learning model that has trained on the training data is again stored in the learning model holding unit 31.

[0133] In Figure 8, it is assumed that steps S1 to S9 are executed when, for example, an observation signal is output from the radar device 20. However, steps S1 to S9 are to be executed repeatedly each time an observation signal is output from the radar device 20. This allows the learning model to be trained with a large amount of training data, making it possible to obtain a learning model with high detection accuracy.

[0134] Furthermore, although Figure 8 describes the process as being executed in step S1, followed by the execution of steps S2 and S3 by the signal processing unit 32, and then the execution of steps S4 to S7 by the augmentation processing unit 33, the processes in steps S2 and S3 and steps S4 to S7 may be executed in parallel or in a predetermined order.

[0135] Furthermore, although Figure 8 explains that training data is generated using the label information acquired in step S3, if, for example, the detection accuracy of the learning model is insufficient, the label information output from the learning model (label information representing the objects contained in the first radar image) may not be appropriate when the first radar image is input to the learning model. In such cases, it is highly likely that proper learning cannot be performed with training data generated using such label information. In such cases, in step S3, label information specified (input) by the user using the information processing device 30 may be acquired.

[0136] Furthermore, although Figure 8 describes the process of step S9 as being executed after the process of step S8, the process of step S9 may be executed at any time.

[0137] Specifically, the process in step S9 may be executed, for example, when a predetermined number (amount) of training data has been accumulated in the training data storage unit 342 through the repeated execution of the processes in steps S1 to S8, or it may be executed in accordance with the instructions of a user using the information processing device 30.

[0138] Furthermore, if a learning model is actually being used to detect objects included in radar images, for example, the detection accuracy of the learning model may decrease due to changes in the object or the environment in which the object is located. However, the processing in step S9 described above may be executed when it is determined that the detection accuracy of the learning model has decreased. Specifically, if the learning model is constructed to output a score for the label information (a value indicating the confidence level of the label information) in addition to the label information, it may be determined whether or not the detection accuracy of the learning model has decreased based on the score. Also, if it is possible to prepare a radar image containing a known object, it may be determined whether or not the detection accuracy of the learning model has decreased based on the accuracy rate calculated by inputting the radar image into the learning model and comparing the label information output from the learning model with the known object (represented by the label information).

[0139] As described above, the information processing device 30 according to this embodiment acquires a radar signal and an observation signal (first observation signal) based on the reflected wave signal from a radar device 20 which is equipped with a plurality of antennas (transmitting antenna 1 and receiving antenna 2) configured to transmit a radar signal and receive a reflected wave signal based on the reflected wave of the radar signal. The information processing device 30 according to this embodiment generates a first radar image by performing synthetic aperture processing (radar signal processing) on ​​the observation signal acquired from the radar device 20 based on a predetermined first condition. Furthermore, the information processing device 30 according to this embodiment generates a second radar image by performing synthetic aperture processing (radar signal processing) on ​​the observation signal acquired from the radar device 20 based on a second condition different from the first condition. The first and second radar images generated in this embodiment are used as training data for a learning model to learn how to detect an object on which the radar signal is transmitted.

[0140] In this embodiment, the above configuration makes it possible to generate a second radar image, which is equivalent to an augmented radar image, from an observation signal that generally generates only one radar image (first radar image). Therefore, it becomes possible to easily prepare training data (radar images) used for training the learning model.

[0141] In this embodiment, the first and second conditions described above include, for example, the target (observation signal) on which synthetic aperture processing is performed, and the matching filters (settings) used when the first and second radar images are generated.

[0142] Specifically, the first radar image is generated using observation signals acquired from the radar device 20, and the second radar image is generated using a composite signal (third observation signal) produced by adding a different observation signal (second observation signal) to the first observation signal. With this configuration, it becomes possible to prepare a second radar image as training data, which includes, for example, objects included in the first radar image and objects different from those included in the first radar image, separately from the first radar image.

[0143] Furthermore, the first radar image is generated using observation signals acquired from the radar device 20, and the second radar image is generated using observation signals (second observation signals) which are a part of the said observation signals. With this configuration, it is possible to prepare a second radar image as training data, which is obtained separately from the first radar image, for example, by degrading the SNR of the first radar image (i.e., by adding noise to the first radar image). The observation signals which are a part of the observation signals acquired from the radar device 20 as described above correspond to observation signals observed at observation points realized by some antennas selected from among multiple antennas. In addition, although it has been explained here that the second radar image is a radar image with a degraded SNR compared to the first radar image, the second radar image may be a radar image with an improved SNR compared to the first radar image.

[0144] Furthermore, the first radar image is generated by performing a synthetic aperture process that calculates the convolution integral of the observation signal acquired from the radar device 20 and a pre-prepared first matched filter, and the second radar image is generated by performing a synthetic aperture process that performs a convolution operation between the observation signal and a second matched filter different from the first matched filter. With this configuration, it is possible to prepare a second radar image as training data, for example, by shifting, inverting, rotating, enlarging, or reducing the first radar image, in addition to the first radar image.

[0145] Furthermore, as described above, when generating a second radar image by shifting, inverting, rotating, scaling, or reducing the first radar image, a second matching filter is used, which is generated by shifting, inverting, rotating, scaling, or reducing the first matching filter used when the first radar image was generated. However, the second radar image may be generated using a second matching filter with a different distance set for it than the distance set for the first matching filter. In this case, a second matching filter with a different distance set for it than the distance set for the first matching filter is used, for example, a second radar image with a degraded SNR of the first radar image can be prepared as training data.

[0146] Furthermore, the second radar image may be generated using a second matched filter with a different resolution than the first matched filter. This allows for the preparation of a second radar image with a different resolution than the first radar image as training data.

[0147] Here, the first and second conditions described above are explained assuming they include the target of synthetic aperture processing and the matching filter (settings), but these first and second conditions may also include dynamic range. Specifically, if the first radar image is generated based on a first dynamic range, the second radar image may be generated based on a second dynamic range different from the first dynamic range. With such a configuration, a second radar image with altered brightness and contrast from the first radar image can be prepared as training data.

[0148] Furthermore, in this embodiment, it is possible to improve the detection accuracy of the learning model by using the first and second radar images generated as described above as training data (i.e., the learning model learns from the training data), and the training data includes the first and second radar images and label information representing the objects contained in the first radar image.

[0149] Thus, the label information included in the training data may be, for example, the label information output from the learning model by inputting the first radar image into the learning model. With this configuration, for example, the effort required of the user to prepare the label information can be reduced. On the other hand, the label information included in the training data may be specified by the user using the information processing device 30. With this configuration, it is possible to avoid a situation where the learning model learns from training data containing inappropriate label information, which hinders the improvement of the learning model's detection accuracy.

[0150] In this embodiment, the radar signal is described as being transmitted based on the FMCW method, but it may be transmitted based on other methods. Also, in this embodiment, the radar device 20 is described as a MIMO radar equipped with multiple transmitting antennas 1 and multiple receiving antennas 2, but the radar device 20 (radar system) may be implemented using various other radars.

[0151] Furthermore, although this embodiment describes the radar signal processing performed to generate the radar image as synthetic aperture processing (signal processing performed in synthetic aperture radar), the radar signal processing is not limited to synthetic aperture processing, and various radar imaging processes can be applied to this embodiment. Specifically, the radar signal processing described in this embodiment may be beamforming-based signal processing. For example, in beamforming-based signal processing, convolution integration is performed using mode vectors instead of matched filters, and in this case, radar image augmentation can be achieved by changing the conditions of the mode vectors.

[0152] Furthermore, although this embodiment describes the radar system 10 as comprising the radar device 20 and information processing device 30 shown in Figure 6, the radar system 10 may have other configurations. Specifically, in the radar system 10, some of the parts 31 to 34 included in the information processing device 30 may be incorporated into the radar device 20, or the radar device 20 and the information processing device 30 may be configured as a single unit. Moreover, some of the parts 31 to 34 included in the information processing device 30 (for example, the learning processing unit 34) may be located in a server device or the like outside the radar system 10. In this case, the processing results from the learning processing unit 34 located in the server device (a learning model that has learned training data or the parameters of the learning model) should be fed back from the server device to the information processing device 30. In addition, the information processing device 30 according to this embodiment may be configured to operate as various types of server devices.

[0153] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents.

[0154] With regard to the embodiments described above, the following additional information is disclosed. [1] An acquisition means for acquiring a first observation signal based on the radar signal and the reflected wave signal from a radar device equipped with multiple antennas configured to transmit a radar signal and receive a reflected wave signal based on the reflected wave of the radar signal, A first generation means that generates a first radar image by performing signal processing on the acquired first observation signal based on a predetermined first condition, A second generation means generates a second radar image by performing signal processing on the acquired first observation signal based on a second condition different from the first condition. It is equipped with, The first and second radar images are used as training data for a learning model to detect objects to which the radar signal is transmitted. Information processing device. [2] The first radar image is generated using the first observation signal, The second radar image is generated using a third observation signal, which is produced by adding a second observation signal different from the first observation signal to the first observation signal. [1] The information processing device described above. [3] The first radar image is generated using the first observation signal, The second radar image is generated using the second observation signal, which is a part of the first observation signal. The information processing device described in [1] or [2]. [4] The first radar image is generated by performing a synthetic aperture process that involves convolution integration or convolution operation between the first observation signal and a pre-prepared first matched filter. The second radar image is generated by performing a synthetic aperture process that convolves and integrates the first observation signal with a second matched filter that is different from the first matched filter. An information processing device as described in any one of the items [1] to [3]. [5] The information processing apparatus described in [4] is generated by shifting, inverting, rotating, scaling the first matching filter. [6] The information processing device described in [4] or [5], wherein the distance set for the second matching filter is different from the distance set for the first matching filter. [7] The resolution of the second matching filter is different from the resolution of the first matching filter. [4] to [6] The information processing device according to any one of these items. [8] The information processing device described in any one of the following [1] to [7], wherein the second radar image is a radar image obtained by degrading the signal-to-noise ratio of the first radar image. [9] The second radar image is a radar image obtained by improving the signal-to-noise ratio of the first radar image. [1] to [8] The information processing device described in any one of these paragraphs.

[10] The first radar image is generated based on the first dynamic range, The second radar image is generated based on a second dynamic range that is different from the first dynamic range. An information processing device as described in any one of items [1] to [9].

[11] The information processing device according to any one of [1] to

[10] , wherein the training data includes the first and second radar images and label information representing an object included in the first radar image or label information representing an object specified by a user, which is output from the learning model by inputting the first radar image into the learning model.

[12] The radar signal is transmitted based on the FMCW (Frequency Modulated Continuous Wave) method. [1] to

[11] Information processing device according to any one of the above.

[13] An information processing device described in any one of items [1] to

[12] , The radar device and A system that is equipped with [the following].

[14] A method executed by an information processing device, The steps include obtaining a first observation signal based on the radar signal and the reflected wave signal from a radar device equipped with a plurality of antennas configured to transmit a radar signal and receive a reflected wave signal based on the reflected wave of the radar signal, The steps include generating a first radar image by performing signal processing on the acquired first observation signal based on predetermined first conditions, and The steps include generating a second radar image by performing signal processing on the acquired first observation signal based on a second condition different from the first condition, and It is equipped with, The first and second radar images are used as training data for a learning model to detect objects to which the radar signal is transmitted. method.

[15] A program executed by a computer of an information processing device, To the aforementioned computer, The steps include obtaining a first observation signal based on the radar signal and the reflected wave signal from a radar device equipped with a plurality of antennas configured to transmit a radar signal and receive a reflected wave signal based on the reflected wave of the radar signal, The steps include generating a first radar image by performing signal processing on the acquired first observation signal based on predetermined first conditions, and The steps include generating a second radar image by performing signal processing on the acquired first observation signal based on a second condition different from the first condition, and Make it run, The first and second radar images are used as training data for a learning model to detect objects to which the radar signal is transmitted. program. [Explanation of symbols]

[0155] 1...Transmitting antenna, 2...Receiving antenna, 10...Radar system, 20...Radar device, 21...Synthesizer, 22...Mixer, 23...A / D converter, 30...Information processing device, 30a...CPU, 30b...Non-volatile memory, 30c...Main memory, 30d...Communication device, 31...Learning model holding unit, 32...Signal processing unit (first generation unit), 33...Augmentation processing unit (second generation unit), 34...Learning processing unit, 321...Synthetic aperture processing unit, 322...Detection unit, 331...Signal mixing unit, 332...Antenna selection unit, 333...Matching filter setting unit, 334...Radar image augmentation unit, 341...Training data generation unit, 342...Training data storage unit, 343...Learning unit.

Claims

1. An acquisition means for acquiring a first observation signal based on the radar signal and the reflected wave signal from a radar device equipped with multiple antennas configured to transmit a radar signal and receive a reflected wave signal based on the reflected wave of the radar signal, A first generation means generates a first radar image by performing a synthetic aperture process that calculates the convolution integral of the acquired first observation signal and a pre-prepared first matched filter, A second generation means generates a second radar image by performing a synthetic aperture process that calculates the convolution integral of the acquired first observation signal and a second matched filter different from the first matched filter. It is equipped with, The first and second radar images are used as training data for a learning model to detect objects to which the radar signal is transmitted. Information processing device.

2. The first matching filter is represented by a complex number in the information processing apparatus according to claim 1.

3. The information processing apparatus according to claim 1 or 2, wherein the second matching filter is generated by shifting, inverting, rotating, scaling up or down the first matching filter.

4. The information processing apparatus according to claim 1 or 2, wherein the distance set for the second matching filter is different from the distance set for the first matching filter.

5. The information processing apparatus according to claim 1 or 2, wherein the resolution of the second matching filter is different from the resolution of the first matching filter.

6. The first radar image is generated using the first observation signal, The second radar image is generated using a third observation signal, which is produced by adding a second observation signal different from the first observation signal to the first observation signal. The information processing apparatus according to claim 1.

7. The first radar image is generated using the first observation signal, The second radar image is generated using the second observation signal, which is a part of the first observation signal. The information processing apparatus according to claim 1.

8. The information processing apparatus according to claim 1, wherein the second radar image is a radar image obtained by degrading the signal-to-noise ratio of the first radar image.

9. The information processing apparatus according to claim 1, wherein the second radar image is a radar image in which the signal-to-noise ratio of the first radar image has been improved.

10. The first radar image is generated based on the first dynamic range, The second radar image is generated based on a second dynamic range that is different from the first dynamic range. The information processing apparatus according to claim 1.

11. The information processing apparatus according to claim 1, wherein the training data includes the first and second radar images and label information representing an object included in the first radar image or label information representing an object specified by a user, which is output from the learning model by inputting the first radar image into the learning model.

12. The information processing apparatus according to claim 1, wherein the radar signal is transmitted based on the FMCW (Frequency Modulated Continuous Wave) method.

13. An information processing apparatus according to any one of claims 1, 2 and 6 to 12, The radar device and A system that is equipped with [the following].

14. A method executed by an information processing device, A radar device comprising a plurality of antennas configured to transmit a radar signal and receive a reflected wave signal based on the reflected wave of the radar signal, a step of acquiring a first observation signal based on the radar signal and the reflected wave signal, The steps include generating a first radar image by performing a synthetic aperture process that calculates the convolution integral of the acquired first observation signal and a pre-prepared first matched filter, The steps include generating a second radar image by performing a synthetic aperture process that calculates the convolution integral of the acquired first observation signal and a second matched filter different from the first matched filter, and It is equipped with, The first and second radar images are used as training data for a learning model to detect objects to which the radar signal is transmitted. method.

15. A program executed by a computer of an information processing device, To the aforementioned computer, A radar device comprising a plurality of antennas configured to transmit a radar signal and receive a reflected wave signal based on the reflected wave of the radar signal, a step of acquiring a first observation signal based on the radar signal and the reflected wave signal, The steps include generating a first radar image by performing a synthetic aperture process that calculates the convolution integral of the acquired first observation signal and a pre-prepared first matched filter, The steps include generating a second radar image by performing a synthetic aperture process that calculates the convolution integral of the acquired first observation signal and a second matched filter different from the first matched filter, and Make it run, The first and second radar images are used as training data for a learning model to detect objects to which the radar signal is transmitted. program.

Images