Learning device, inference device, and learning method

Abstract

The learning device (100) includes an object information acquisition unit (120) that acquires object information related to an object including object presence / absence information indicating the presence or absence of an object; a delay profile generation unit (113) that generates a delay profile based on a received wave obtained by receiving an antenna element after a wave reflected by an object from a transmission wave transmitted from a transmitting antenna element is received; a learning data generation unit (130) that generates learning data for generating a trained model based on the object information acquired by the object information acquisition unit (120) and the delay profile generated by the delay profile generation unit (113); and a machine learning unit (140) that performs machine learning on the learning data generated by the learning data generation unit (130) and generates a trained model that outputs a new object detection result based on an input of the delay profile generated by the delay profile generation unit (113).

Description

[Technical Field]

[0001] The present disclosure relates to a learning device, an inference device, and a learning method. [Background technology]

[0002] A radar system has been disclosed in the past that detects objects around a vehicle by receiving and processing a beam emitted from a beam steering antenna provided on the vehicle, which is reflected by objects in the path of the vehicle and in the surrounding environment (see Patent Document 1). The radar system described in Patent Document 1 is configured to detect objects using radar data generated by a radar module and a machine learning module. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 153016 Summary of the Invention [Problem to be solved by the invention]

[0004] Generally, when attempting to detect an object using radar, the transmitted wave that is sent into space and reaches the object directly is reflected by the object, and a delayed wave is generated that propagates along a different path from the direct wave, which is the received wave that is the wave reflected by the object and received directly. This creates a problem in that it is difficult to improve the accuracy of object detection due to the influence of the delayed wave.

[0005] The present disclosure was made in recognition of the above-mentioned problems, and aims to provide a learning device, an inference device, and a learning method that can improve the accuracy of object detection compared to conventional methods. [Means for solving the problem]

[0006] The learning device according to the present disclosure includes object presence / absence information indicating the presence or absence of an object.The information includes one or more pieces of information including at least object presence / absence information among object position information indicating the position of an object, object speed information indicating the speed of an object, and object class identification information indicating the class into which the object is classified. an object information acquisition unit that acquires object information about the object; a delay profile generation unit that generates a delay profile based on a received wave that is a wave that is a transmission wave transmitted from a transmitting antenna element and is reflected by the object and received by a receiving antenna element; and Multiple Based on the delay profile and By assigning labels to the plurality of delay profiles that are set based on the object information, a learning data generation unit that generates learning data for generating a trained model; The learning data generated by the learning data generation unit is subjected to machine learning. By doing and outputs a new object detection result based on the input of the delay profile generated by the delay profile generating unit. for a machine learning unit that generates the trained model; and a unit that simulates noise contained in the received wave received by the receiving antenna element. White Gaussian noise, thermal noise, and multiple noises with different noise levels are shown. a noise information acquisition unit that acquires noise information, and the delay profile generation unit The plurality of delay profiles include: a plurality of second delay profiles in which a plurality of different noise information pieces acquired by the noise information acquisition unit are added to a first delay profile which is a delay profile of a received wave received by the receiving antenna element; 3 Generate a delay profile It is something that The learning data generation unit generates the learning data acquired by the object information acquisition unit. The aforementioned Object information and the delay profile generated by the delay profile generating unit The aforementioned Multiple 3 and generating training data for generating the trained model based on the delay profile. It is something that , The object information is information stored in a storage unit or acquired based on input from an input device. It is characterized by the following. [Effects of the Invention]

[0007] The learning device according to the present disclosure can improve the object detection accuracy compared to conventional devices. [Brief explanation of the drawings]

[0008] [Figure 1]1 is a block diagram showing a schematic configuration of an object detection system according to a first embodiment. [Figure 2] 1 is a block diagram showing an example of a hardware configuration of a learning device according to a first embodiment. [Figure 3] 1 is a block diagram showing an example of a hardware configuration of a learning device according to a first embodiment. [Figure 4] 1 is a flowchart showing an example of a trained model generation determination process performed by the learning device according to embodiment 1. [Figure 5] 1 is a flowchart showing an example of a trained model generation process performed by the learning device according to embodiment 1. [Figure 6] 4 is a diagram showing an example of a graph of a first delay profile generated by the learning device according to the first embodiment. FIG. [Figure 7] FIG. 2 is a diagram illustrating an example of beamforming performed by a learning device according to the first embodiment. [Figure 8] FIG. 4 is a diagram showing an example of a graph of a second delay profile generated by the learning device according to the first embodiment. [Figure 9] FIG. 10 is a diagram showing a process of generating a third delay profile by the learning device according to the first embodiment. [Figure 10] 4 is a flowchart showing an example of an inference process performed by the learning device according to the first embodiment. [Figure 11] FIG. 10 is a block diagram showing a schematic configuration of an object detection system according to a second embodiment. [Figure 12] FIG. 10 is a diagram illustrating an example of application of a time gate by the learning device according to the second embodiment. [Figure 13] FIG. 10 is a block diagram showing a schematic configuration of an object detection system according to a third embodiment. [Figure 14] FIG. 11 is a diagram illustrating an example of application of a power threshold value by the learning device according to the third embodiment. [Figure 15] FIG. 10 is a block diagram showing a schematic configuration of an object detection system according to a fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Embodiment 1 First, an object detection system 1 according to the first embodiment will be described with reference to FIG. 1. FIG. 1 is a block diagram showing a schematic configuration of the object detection system 1 according to the first embodiment. The object detection system 1 shown in FIG. 1 is a system for detecting an object based on a wave reflected by the object from a transmitted transmission wave. As shown in FIG. 1, the object detection system 1 according to the first embodiment includes a signal transmitting / receiving unit 10, a display device 20, and a learning device 100, which are connected wirelessly or by wire so as to be able to communicate with each other. Note that the signal transmitting / receiving unit 10, the display device 20, and the learning device 100 may be configured to be able to communicate information with each other via a device or communication line not shown, or may be configured integrally as a single device.

[0010] The signal transmitting / receiving unit 10 includes multiple transmitting antenna elements and multiple receiving antenna elements (not shown), and transmits transmission waves from the multiple transmitting antenna elements and receives waves reflected by objects from the transmitted waves using multiple receiving antennas. For example, the signal transmitting / receiving unit 10 generates radio frequency (RF) signals and transmits the high-frequency signals as transmission waves into space from the multiple transmitting antenna elements, and receives waves reflected by objects from space using the multiple receiving antenna elements. The signal transmitting / receiving unit 10 outputs the received waves received by the multiple receiving antenna elements to the learning device 100 as analog signals.

[0011] Display device 20 acquires information from learning device 100 and displays the acquired information as visual information to a user of object detection system 1. For example, display device 20 may be configured with a liquid crystal display panel, an organic or inorganic EL (Electro Luminescence) panel, or any other device that displays information from learning device 100 as visual information.

[0012] The learning device 100 is a device that performs processing to detect an object based on input from a signal transmitting / receiving unit 10. As shown in Fig. 1, the learning device 100 includes a signal processing unit 110, an object information acquiring unit 120, a learning data generating unit 130, a machine learning unit 140, an inference unit 160, and a memory unit 150 that stores information.

[0013] The signal processing unit 110 outputs a signal to the signal transmitting / receiving unit 10 and performs various processes based on the signal from the signal transmitting / receiving unit 10. The signal processing unit 110 includes a signal acquiring unit 111, a delay profile generating unit 113, a beamforming unit 114, and a noise information acquiring unit 115.

[0014] The signal acquiring unit 111 acquires a signal from the signal transmitting / receiving unit 10. For example, the signal acquiring unit 111 acquires an analog signal from the signal transmitting / receiving unit 10 and converts the acquired analog signal into a digital signal.

[0015] The noise information acquiring unit 115 acquires noise information that simulates noise included in the received wave received by the receiving antenna element of the signal transmitting / receiving unit 10. For example, the noise information acquiring unit 115 acquires a plurality of different noise information including first noise information, second noise information, third noise information, and fourth noise information as the noise information that simulates noise included in the received wave received by the receiving antenna element of the signal transmitting / receiving unit 10. For example, the noise information acquiring unit 115 acquires a plurality of pieces of noise information that respectively indicate white Gaussian noise, thermal noise, and noises having different noise levels.

[0016] The delay profile generation unit 113 generates a delay profile of the received waves received by the signal transmission / reception unit 10 based on the signal acquired by the signal acquisition unit 111. In other words, the delay profile generation unit 113 generates a delay profile based on received waves that are waves that are reflected by an object and received by the multiple receiving antenna elements of the signal transmission / reception unit 10 of the transmitted waves transmitted from the multiple transmitting antenna elements of the signal transmission / reception unit 10. For example, the delay profile generation unit 113 generates a first delay profile as a delay profile for each transmitting antenna element and receiving antenna element based on the transmitted waves transmitted from the multiple transmitting antenna elements of the signal transmission / reception unit 10 and the received waves received by each of the multiple receiving antenna elements of the signal transmission / reception unit 10.

[0017] Specifically, when the signal transmitting / receiving unit 10 has m transmitting antenna elements and n receiving antenna elements, the delay profile generating unit 113 generates m×n first delay profiles for each combination of transmitting antenna elements and receiving antenna elements. Note that in the first embodiment, m and n each represent an integer of 2 or more. For example, when the signal transmitting / receiving unit 10 has four transmitting antenna elements and four receiving antenna elements, the delay profile generating unit 113 generates 4×4=16 delay profiles for each combination of transmitting antenna elements and receiving antenna elements. Note that the number of transmitting antenna elements is not necessarily limited to multiple, and the number of transmitting antenna elements may be one.

[0018] Furthermore, the delay profile generating unit 113 generates a third delay profile, which is a delay profile to which noise has been added, based on the first delay profile and the noise information acquired by the noise information acquiring unit 115. Details of the third delay profile will be described later.

[0019] The beamforming unit 114 controls the phase of the transmission wave transmitted from each of the multiple transmitting antenna elements of the signal transmitting / receiving unit 10, and also controls the phase of the reception wave received by each of the multiple receiving antenna elements of the signal transmitting / receiving unit 10, thereby performing beamforming of the transmission waves transmitted from the multiple transmitting antenna elements of the signal transmitting / receiving unit 10 and the reception waves received by the multiple receiving antenna elements of the signal transmitting / receiving unit 10. For example, the beamforming unit 114 performs beamforming of the transmission waves transmitted from the multiple transmitting antenna elements and the reception waves received by the multiple receiving antenna elements based on the delay profile generated by the delay profile generating unit 113.

[0020] Specifically, the beamforming unit 114 performs beamforming on the transmission waves transmitted from the multiple transmitting antenna elements of the signal transmitting and receiving unit 10 and the reception waves received by the multiple receiving antenna elements of the signal transmitting and receiving unit 10, so that the delay profiles of the receiving antenna elements generated by the delay profile generating unit 113 are aligned, based on the delay profile generated by the delay profile generating unit 113. Note that in the first embodiment, the delay profile generated by the delay profile generating unit 113 as a result of the beamforming of the transmission waves and reception waves by the beamforming unit 114 so that the delay profiles of the receiving antenna elements are aligned is also referred to as a second delay profile.

[0021] Furthermore, for example, the beamforming unit 114 performs beamforming on the transmission waves so that the transmission waves transmitted from each of the multiple transmitting antenna elements are orthogonal to one another. In other words, the beamforming unit 114 performs beamforming on the transmission waves so that the phase differences between the transmission waves transmitted from each of the multiple transmitting antenna elements are π / 2. Furthermore, for example, the beamforming unit 114 performs beamforming on the reception waves so that the reception waves received by each of the multiple receiving antenna elements are orthogonal to one another. In other words, the beamforming unit 114 performs beamforming on the reception waves so that the phase differences between the reception waves received by each of the multiple receiving antenna elements are π / 2.

[0022] The object information acquisition unit 120 acquires object information related to an object, including object presence / absence information indicating the presence or absence of the object. For example, the object information acquisition unit 120 acquires object information including one or more of the following information: object presence / absence information indicating the presence or absence of the object, object position information indicating the position of the object, object position information indicating the position of the object, object velocity information indicating the velocity of the object, and object class identification information indicating the class into which the object is classified. Note that the object presence / absence information, object position information, object position information, object velocity information, and object class identification information included in the object information may be information indicating the presence / absence, position, velocity, and class into which a single object is classified, or information indicating the presence / absence, position, velocity, and class into which each of a plurality of objects is classified. For example, the object information acquisition unit 120 references information stored in the storage unit 150 and acquires object information from the storage unit 150. Note that the object information acquisition unit 120 may be configured to acquire object information based on input information from an input device (not shown) communicatively connected to the learning device 100.

[0023] The training data generation unit 130 generates training data for generating a trained model based on the object information acquired by the object information acquisition unit 120 and the delay profile generated by the delay profile generation unit 113. For example, the training data generation unit 130 assigns labels set based on the object information acquired by the object information acquisition unit 120 to the multiple delay profiles generated by the delay profile generation unit 113, thereby generating a data set consisting of the multiple labeled delay profiles as training data for generating a trained model.

[0024] The machine learning unit 140 performs machine learning on the training data generated by the training data generation unit 130, and generates a trained model that outputs a detection result of a new object based on an input of a delay profile generated by the delay profile generation unit 113. For example, the machine learning unit 140 generates a trained model that outputs a detection result of a new object based on an input of a third delay profile generated by the delay profile generation unit 113. For example, the machine learning unit 140 generates a trained model that outputs, in addition to a detection result of a new object, information on at least one of the position, velocity, and class identification result of the new object based on an input of the delay profile generated by the delay profile generation unit 113. Specifically, the machine learning unit 140 generates a trained model that outputs, in addition to a detection result of a new object, information indicating the probability that the new object will be classified into each of a plurality of preset classes as a class identification result of the new object based on an input of the delay profile generated by the delay profile generation unit 113. For example, the machine learning unit 140 generates a trained model using a convolutional neural network (CNN) algorithm. Note that the machine learning unit may be configured to generate a trained model using other algorithms.

[0025] The storage unit 150 stores information used in various processes performed by the learning device 100 and information indicating the results of the various processes performed by the learning device 100. For example, the storage unit 150 stores a trained model generated by the machine learning unit 140. In the first embodiment, the storage unit 150 constitutes a trained model acquisition unit that acquires a trained model. Each component of the learning device 100 refers to the information stored in the storage unit 150 when performing various processes.

[0026] The inference unit 160 performs inference using the trained model based on input of the delay profile acquired by the delay profile generation unit 113 to the trained model generated by the machine learning unit 140. For example, the inference unit 160 refers to information stored in the storage unit 150, and performs inference using the trained model by inputting the delay profile acquired by the delay profile generation unit 113 to the trained model generated by the machine learning unit 140 and stored in the storage unit 150, and acquires information indicating the object detection result, which is the output of the trained model.

[0027] Next, the hardware configuration of the learning device 100 will be described with reference to Figures 2 and 3. Figure 2 is a diagram showing an example of the hardware configuration of the learning device 100, and Figure 3 is a diagram showing an example of the hardware configuration of the learning device 100 that is different from that shown in Figure 2. For example, as shown in Figure 2, the learning device 100 is a computer having a processor 100a, a memory 100b, and an I / O port 100c, and is configured so that the processor 100a reads and executes a program stored in the memory 100b.

[0028] 3, the learning device 100 is a computer that has a processing circuit 100d, which is dedicated hardware, and an I / O port 100c, and executes a program. The processing circuit 100d is configured, for example, by a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. Each function of the learning device 100 is realized by the processor 100a or the processing circuit 100d, which is dedicated hardware, executing a program. The learning device 100 may also have hardware other than those described above, such as a hardware timer.

[0029] Next, details of the trained model generation determination process performed by the learning device 100 will be described with reference to Fig. 1 and Fig. 4. Fig. 4 is a flowchart showing an example of the trained model generation determination process performed by the learning device 100 according to embodiment 1. The trained model generation determination process performed by the learning device 100 shown in Fig. 4 is a process for determining whether to detect an object using an already generated trained model or to generate a new trained model.

[0030] 4, when the learning device 100 starts the trained model generation determination process, it first determines whether a valid trained model has been generated (step ST01). For example, in this process, the learning device 100 refers to information stored in the storage unit 150 and determines whether a trained model is stored in the storage unit 150. Note that in this process, the learning device 100 may be configured to acquire information indicating conditions for detecting a new object using a trained model, and determine whether a trained model corresponding to the acquired information is stored in the storage unit 150, thereby determining whether a valid trained model has been generated.

[0031] For example, in a state where the storage unit 150 stores information indicating conditions under which a trained model detects an object in association with the trained model, the learning device 100 may be configured to determine whether a valid trained model has been generated by comparing information indicating conditions under which a new object is detected using the trained model with information indicating conditions under which the trained model detects an object stored in the storage unit 150. For example, conditions under which the trained model detects an object include the type of object to be detected, the types and number of other objects present around the object to be detected, and weather conditions under which the object is detected.

[0032] If a valid trained model has not been generated in the processing of step ST01 (NO in step ST01), the learning device 100 performs a trained model generation process (step ST02). For example, in this process, the learning device 100 determines to perform a trained model generation process to generate a new trained model when no trained model is stored in the storage unit 150, or when a trained model is stored in the storage unit 150 but is not a valid trained model.

[0033] If a valid trained model has been generated in the processing of step ST01 (YES in step ST01), the learning device 100 performs inference processing (step ST03). In this processing, the learning device 100 determines to perform inference processing for detecting an object using the trained model based on the fact that a valid trained model has been generated.

[0034] After performing the trained model generation process in step ST02 or the inference process in step ST03, the learning device 100 ends the trained model generation determination process.

[0035] Next, details of the trained model generation process performed by the learning device 100 will be described with reference to FIG. 1 and FIG. 5 to FIG. 9. FIG. 5 is a flowchart showing an example of the trained model generation process performed by the learning device 100 according to embodiment 1. As described above, the trained model generation process performed by the learning device 100 shown in FIG. 5 is a process for generating a new trained model. As shown in FIG. 5, when the learning device 100 starts the trained model generation process, it first acquires a signal from the signal transmitter / receiver unit 10 (step ST11). In this process, the learning device 100 transmits a transmission wave from the transmitting antenna element of the signal transmitter / receiver unit 10, and acquires from the signal transmitter / receiver unit 10 a signal indicating a received wave that is a wave of the transmitted wave reflected by an object and received by the receiving antenna element of the signal transmitter / receiver unit 10.

[0036] After performing the process of step ST11, learning device 100 acquires object information (step ST12). In this process, learning device 100 acquires, by object information acquisition unit 120, object information about the object that reflected the transmission wave transmitted in the process of step ST11.

[0037] After performing the process of step ST12, the learning device 100 generates a first delay profile (step ST13). For example, in this process, the learning device 100 generates a delay profile of a received wave for each combination of a transmitting antenna element and a receiving antenna element based on the signal acquired in the process of step ST11, and thereby generates a plurality of first delay profiles according to the number of channels, which is the product of the number of transmitting antenna elements and the number of receiving antenna elements, by the delay profile generation unit 113.

[0038] 6 is a diagram showing an example of a graph of a first delay profile generated by learning device 100 according to Embodiment 1. As shown in FIG. 6, for example, the first delay profile is information including discrete data with delay time on the horizontal axis and power on the vertical axis. Note that the first delay profile generated by delay profile generating unit 113 may have propagation distance on the horizontal axis, or may have the power difference, amplitude difference, or complex power difference between two delay profiles on the vertical axis, or may be data itself indicating the relationship between delay time and power of a received wave, or may be partial information extracted from the data, or may be information indicating a specific feature extracted from the data.

[0039] After performing the process of step ST13, the learning device 100 performs beamforming of transmitted and received waves (step ST14). For example, in this process, the learning device 100 performs beamforming of the transmitted waves transmitted from the multiple transmitting antenna elements of the signal transmitting and receiving unit 10 and the received waves received by the multiple receiving antenna elements of the signal transmitting and receiving unit 10 based on the multiple first delay profiles generated in the process of step ST13, so that these multiple first delay profiles are aligned.

[0040] 7 is a diagram illustrating an example of beamforming performed by the learning device 100 according to the first embodiment. As illustrated in FIG. 7, for example, if there are four transmit beam patterns, B1, B2, B3, and B4, and four receive beam patterns, a total of 4×4=16 second delay profiles, which is the number of combinations of transmit beam patterns and receive beam patterns, are generated. Note that the learning device 100 may be configured to generate beam patterns by adjusting phase shifters (not shown) provided corresponding to each antenna element using an analog beamforming method, or may be configured to generate beam patterns by adjusting digital bit phase shifters (not shown) provided corresponding to each channel using a digital beamforming method.

[0041] After performing the processing of step ST14, the learning device 100 acquires a signal from the signal transmitting / receiving unit 10 (step ST15). In this processing, the learning device 100 acquires a signal indicating the received wave from the signal transmitting / receiving unit 10 by the signal acquiring unit 111 based on the transmitted wave and the received wave beamformed in the processing of step ST14.

[0042] After performing the process of step ST15, learning device 100 generates a second delay profile (step ST16). In this process, learning device 100 generates the second delay profile, which is the delay profile of the received wave, by delay profile generating unit 113 based on the signal acquired in the process of step ST15.

[0043] Fig. 8 is a diagram showing an example of a graph of a second delay profile generated by learning device 100 according to embodiment 1. For example, as shown in Fig. 8, beamforming is performed in the process of step ST14, and a first delay profile having N channels is generated as one second delay profile.

[0044] After performing the process of step ST16, the learning device 100 generates a third delay profile by adding noise to the second delay profile (step ST17). For example, in this process, based on the first noise information, the second noise information, the third noise information, and the fourth noise information acquired by the noise information acquisition unit 115 and the second delay profile, the learning device 100 generates, by the delay profile generation unit 113, a third delay profile in which the first noise indicated by the first noise information is added to the second delay profile, a third delay profile in which the second noise indicated by the second noise information is added to the second delay profile, a third delay profile in which the third noise indicated by the third noise information is added to the second delay profile, and a third delay profile in which the fourth noise indicated by the fourth noise information is added to the second delay profile.

[0045] Fig. 9 is a diagram showing a process of generating a third delay profile by learning device 100 according to embodiment 1. As shown in Fig. 9, for example, learning device 100 divides the second delay profile into multiple signals using a divider (not shown) included in learning device 100, and adds different noises to each of the second delay profiles, thereby generating multiple third delay profiles.

[0046] After performing the process of step ST17, the learning device 100 determines whether collection of a sufficient amount of data has been completed (step ST18). In this process, the learning device 100 determines whether generation of a sufficient number of third delay profiles for generating a trained model has been completed. For example, the learning device 100 determines whether generation of a sufficient number of third delay profiles has been completed based on whether the number of generated third delay profiles exceeds a preset threshold.

[0047] If the collection of a sufficient amount of data has not been completed in the processing of step ST18 (NO in step ST18), learning device 100 returns the processing to step ST11. In this processing, learning device 100 generates a further third delay profile by repeating the processing of steps ST11 to ST17 based on the fact that the collection of a sufficient amount of data has not been completed.

[0048] If a sufficient amount of data has been collected in the process of step ST18 (YES in step ST18), the learning device 100 generates training data based on the object information and the third delay profile (step ST19). In this process, the learning device 100 assigns a label based on the object information to each of the generated third delay profiles, and the training data generation unit 130 generates training data for generating a trained model.

[0049] After performing the process of step ST19, the learning device 100 generates a trained model based on the training data (step ST20). In this process, the learning device 100 performs machine learning on the training data generated in the process of step ST19, thereby generating a trained model that outputs a detection result of a new object based on an input of the delay profile generated by the delay profile generation unit 113.

[0050] After performing the processing of step ST20, the learning device 100 stores the trained model generated in the processing of step ST20 in the storage unit 150, and ends the trained model generation processing.

[0051] Next, details of the inference processing performed by the learning device 100 will be described with reference to Fig. 1 and Fig. 10. Fig. 10 is a flowchart showing an example of the inference processing performed by the learning device 100 according to embodiment 1. As described above, the inference processing performed by the learning device 100 shown in Fig. 10 is processing for detecting an object using a trained model generated by the trained model generation processing. As shown in Fig. 10, when the learning device 100 starts the inference processing, it first acquires a trained model (step ST31). In this processing, the learning device 100 acquires a trained model that has been determined to be a valid trained model in the processing of step ST01 of the trained model generation determination processing.

[0052] After performing the processing of step ST31, the learning device 100 acquires a signal from the signal transmitting / receiving unit 10 (step ST32). In this processing, the learning device 100 transmits a transmission wave and receives a reception wave from the signal transmitting / receiving unit 10 in order to detect a new object that is a detection target using the trained model, and acquires a signal corresponding to the reception wave based on the reflected wave from the new object from the signal transmitting / receiving unit 10.

[0053] After performing the processing of step ST32, the learning device 100 generates a fourth delay profile (step ST33). In this processing, the learning device 100 generates the fourth delay profile, which is the delay profile of the received wave acquired in the processing of step ST32, using the delay profile generation unit 113. Note that the details of the fourth delay profile generated in the processing of step ST33 are similar to the first delay profile generated in the processing of step ST13 of the trained model generation processing, and therefore will not be described again.

[0054] After performing the processing of step ST33, the learning device 100 performs beamforming of transmitted and received waves (step ST34). For example, in this processing, based on the multiple fourth delay profiles generated in the processing of step ST33, the learning device 100 performs beamforming of the transmitted waves transmitted from the multiple transmitting antenna elements of the signal transmitting and receiving unit 10 and the received waves received by the multiple receiving antenna elements of the signal transmitting and receiving unit 10 so that these multiple fourth delay profiles are aligned. Note that the details of the beamforming performed in the processing of step ST34 are similar to the beamforming performed in the processing of step ST14 of the trained model generation processing, and therefore will not be described again.

[0055] After performing the processing of step ST34, the learning device 100 acquires a signal from the signal transmitting / receiving unit 10 (step ST35). In this processing, the learning device 100 acquires a signal indicating the received wave from the signal transmitting / receiving unit 10 by the signal acquiring unit 111 based on the transmitted wave and the received wave beamformed in the processing of step ST34.

[0056] After performing the processing of step ST35, the learning device 100 generates a fifth delay profile (step ST36). In this processing, the learning device 100 generates the fifth delay profile, which is a delay profile of the received wave, based on the signal acquired in the processing of step ST35, using the delay profile generation unit 113. Note that the details of the fifth delay profile generated in the processing of step ST36 are similar to the second delay profile generated in the processing of step ST15 of the trained model generation processing, and therefore will not be described again.

[0057] After performing the process of step ST36, the learning device 100 inputs the fifth delay profile to the trained model (step ST37). In this process, the learning device 100 inputs the fifth delay profile generated in the process of step ST36 to the trained model acquired in the process of step ST31, thereby performing inference using the trained model.

[0058] After performing the process of step ST37, the learning device 100 acquires an inference result (step ST38). In this process, the learning device 100 inputs the fifth delay profile acquired in the process of step ST36 into the trained model, and acquires an object detection result as an inference result output by the trained model.

[0059] After performing the processing of step ST38, learning device 100 outputs the inference result (step ST39). In this processing, learning device 100 outputs, for example, the inference result acquired in the processing of step ST38 to display device 20, thereby causing display device 20 to display the inference result, which is the detection result of a new object. Note that in this processing, learning device 100 may be configured to output the inference result to storage unit 150, thereby storing the inference result in storage unit 150, or may be configured to output the inference result to another device (not shown) that is communicatively connected to learning device 100.

[0060] After performing the process of step ST39, learning device 100 ends the inference process.

[0061] As described above, the learning device 100 according to the first embodiment includes an object information acquisition unit 120 that acquires object information related to an object including object presence / absence information indicating the presence or absence of an object; a delay profile generation unit 113 that generates a delay profile based on a received wave obtained by receiving an antenna element after the reflected wave from the object of a transmitted wave transmitted from a transmitting antenna element is received by the receiving antenna element; a training data generation unit 130 that generates training data for generating a trained model based on the object information acquired by the object information acquisition unit 120 and the delay profile generated by the delay profile generation unit 113; and a machine learning unit 140 that performs machine learning on the training data generated by the training data generation unit 130 and generates a trained model that outputs a new object detection result based on the input of the delay profile generated by the delay profile generation unit 113.

[0062] With this configuration, learning device 100 can detect an object by taking into account the influence of delayed waves that have propagated along paths different from those of direct waves, using a trained model generated based on the delay profile, thereby improving object detection accuracy compared to conventional methods. Furthermore, by improving object detection accuracy compared to conventional methods, the detectable distance when detecting an object can be extended.

[0063] Furthermore, the learning device 100 according to the first embodiment includes a noise information acquisition unit 115 that acquires noise information that simulates noise contained in the received wave received by the receiving antenna element, and is configured to generate a second delay profile in which the noise information acquired by the noise information acquisition unit 115 is added to a first delay profile, which is the delay profile of the received wave received by the receiving antenna element, and to generate learning data for generating a trained model based on the object information acquired by the object information acquisition unit 120 and the second delay profile generated by the delay profile generation unit 113.

[0064] With this configuration, learning device 100 can detect objects taking into account the effects of noise in transmitted and received waves, thereby improving object detection accuracy compared to conventional methods. In particular, learning device 100 generates training data based on multiple delay profiles to which multiple preset, different noises are added, thereby improving noise resistance in object detection and improving object detection accuracy in environments with changing noise.

[0065] Furthermore, learning device 100 according to embodiment 1 includes beamforming unit 114 that performs beamforming on the transmitted waves transmitted from the plurality of transmitting antenna elements and the received waves received by the plurality of receiving antenna elements, based on the delay profile generated by delay profile generating unit 113. With this configuration, learning device 100 can prevent the reflected waves from the object to be detected from being buried in scattered waves from objects other than the object to be detected, such as surrounding structures, and other noise, thereby improving the object detection accuracy compared to conventional methods.

[0066] In the first embodiment, the learning device 100 is configured to perform beamforming of the transmitted waves and received waves by the signal transmitting / receiving unit 10 based on the delay profile generated by the delay profile generating unit 113, but is not limited to this. For example, the learning device may be configured to perform beamforming of the transmitted waves transmitted from the plurality of transmitting antenna elements and the received waves received by the plurality of receiving antenna elements based on the object information acquired by the object information acquiring unit 120. Specifically, when the object information acquired by the object information acquiring unit 120 includes object position information indicating the position of the object, the learning device may be configured to perform beamforming of the transmitted waves transmitted from the plurality of transmitting antenna elements so that the power of the transmitted waves transmitted from the signal transmitting / receiving unit 10 toward the position of the object is maximized, or may be configured to perform beamforming of the transmitted waves transmitted from the plurality of transmitting antenna elements so that the power of the transmitted waves transmitted from the signal transmitting / receiving unit 10 toward the position of the object exceeds a predetermined threshold.

[0067] Furthermore, for example, the learning device may be configured to perform beamforming of transmitted waves transmitted from a plurality of transmitting antenna elements so that the plurality of beams contained in the transmitted waves transmitted from the plurality of transmitting antenna elements are orthogonal to each other, and to perform beamforming of received waves received by a plurality of receiving antenna elements so that the received waves received by the plurality of receiving antenna elements are aligned in the direction of the beams contained in the transmitted waves transmitted from the plurality of transmitting antenna elements.

[0068] Embodiment 2 Next, an object detection system 2 according to embodiment 2 will be described with reference to Fig. 11 and Fig. 12. The object detection system 2 according to embodiment 2 differs from the object detection system 1 according to embodiment 1 in that the learning device includes a signal extraction unit, but other configurations are the same, and the same configurations as those in embodiment 1 are given the same names and symbols as those in embodiment 1, and descriptions thereof will be omitted.

[0069] As shown in FIG. 11, the object detection system 2 according to the second embodiment includes a signal transmitting / receiving unit 10, a display device 20, and a learning device 200, which are connected wirelessly or by wire so that they can communicate with each other.

[0070] The learning device 200 includes a signal processing unit 210, an object information acquisition unit 120, a learning data generation unit 130, a machine learning unit 140, an inference unit 160, and a storage unit 150. The signal processing unit 210 includes a signal acquisition unit 111, a signal extraction unit 212, a delay profile generation unit 213, a beamforming unit 114, and a noise information acquisition unit 115.

[0071] The signal extraction unit 212 extracts signals for a specific period from signals indicating received waves received by the receiving antenna elements using a time gate. For example, the signal extraction unit 212 calculates a period during which the received waves, which are direct waves from the object, are likely to be received by the receiving antenna elements based on the object information acquired by the object information acquisition unit 120, and extracts signals for that period using a time gate.

[0072] The delay profile generating unit 213 generates a delay profile based on the received waves for a specific period extracted by the signal extracting unit 212 from the received waves received by the receiving antenna elements.

[0073] Fig. 12 is a diagram showing an example of application of a time gate by learning device 200 according to embodiment 2. As shown in Fig. 12, learning device 200 generates, by delay profile generating unit 213, a delay profile corresponding to the signal of the received wave extracted by the time gate.

[0074] As described above, the learning device 200 according to the second embodiment is configured to generate a delay profile based on received waves for a specific period extracted from received waves received by the receiving antenna elements. With this configuration, the learning device 200 can exclude signals for a period in which received waves, which are direct waves from an object, are not received from the learning data used when generating a trained model, thereby improving the object detection accuracy compared to conventional methods.

[0075] Embodiment 3 Next, an object detection system 3 according to embodiment 3 will be described with reference to Fig. 13 and Fig. 14. The object detection system 3 according to embodiment 3 differs from the object detection system 1 according to embodiment 1 in the function of some of the delay profile generation units, but the other configurations are the same, and the same configurations as those in embodiment 1 are given the same names and symbols as those in embodiment 1, and description thereof will be omitted.

[0076] As shown in Figure 13, the object detection system 3 according to the third embodiment includes a signal transmission / reception unit 10, a display device 20, and a learning device 300, which are connected wirelessly or by wire so that they can communicate with each other.

[0077] The learning device 300 includes a signal processing unit 310, an object information acquisition unit 120, a learning data generation unit 130, a machine learning unit 140, an inference unit 160, and a storage unit 150. The signal processing unit 310 includes a signal acquisition unit 111, a delay profile generation unit 313, a beamforming unit 114, and a noise information acquisition unit 115.

[0078] The delay profile generating unit 313 has a threshold setting unit 313a that sets a power threshold that is a power threshold when generating a delay profile. For example, when generating a delay profile, the threshold setting unit 313a sets in advance an upper limit power of a signal that is clearly not a received wave that is a direct wave from an object as a power threshold, and the delay profile generating unit 313 generates a delay profile by extracting only profiles that exceed the power threshold.

[0079] Fig. 14 is a diagram showing an example of application of a power threshold by learning device 300 according to embodiment 3. As shown in Fig. 14, learning device 300 generates a delay profile by using delay profile generating section 213, in which a profile exceeding the power threshold is extracted.

[0080] As described above, the learning device 300 according to the third embodiment is configured to generate a delay profile based on received waves having a power equal to or greater than a preset threshold, extracted from received waves received by receiving antenna elements. With this configuration, the learning device 300 can exclude, from the learning data used in generating a trained model, delay profiles based on signals that are clearly not received waves that are direct waves from an object, such as signals due to noise and signals that are clearly delayed waves, thereby improving the accuracy of object detection compared to conventional methods.

[0081] Embodiment 4 Next, an object detection system 4 according to embodiment 4 will be described with reference to Fig. 15. The object detection system 4 according to embodiment 4 differs from the object detection system 1 according to embodiment 1 in that the learning device includes a simulated signal generation unit, but the other configurations are the same, and the same configurations as those in embodiment 1 are given the same names and symbols as those in embodiment 1, and description thereof will be omitted.

[0082] As shown in Figure 15, the object detection system 4 according to the fourth embodiment includes a signal transmission / reception unit 10, a display device 20, and a learning device 400, which are connected wirelessly or by wire so that they can communicate with each other.

[0083] The learning device 400 includes a signal processing unit 410, an object information acquisition unit 120, a learning data generation unit 130, a machine learning unit 140, an inference unit 160, a simulation signal generation unit 470, and a storage unit 150. The signal processing unit 410 includes a signal acquisition unit 411, a delay profile generation unit 313, a beamforming unit 114, and a noise information acquisition unit 115.

[0084] The simulation signal generator 470 generates a simulation signal simulating a received wave that is a wave of a transmission wave transmitted from a transmitting antenna element and reflected by an object and received by a receiving antenna element. For example, the simulation signal generator 470 generates a virtual propagation environment of the transmission wave based on the object information acquired by the object information acquirer 120, and calculates a signal indicating a received wave when a wave of a transmission wave transmitted from the transmitting antenna element and reflected by an object is received by the receiving antenna element in the generated propagation environment.

[0085] The signal acquisition unit 411 acquires a signal that indicates the received wave calculated by the simulation signal generation unit 470 .

[0086] As described above, the learning device 400 according to the fourth embodiment includes a simulation signal generator 470 that generates a simulation signal simulating a received wave that is a wave of a transmitted wave transmitted from a transmitting antenna element and reflected by an object and received by a receiving antenna element, and the delay profile generator 113 is configured to generate a delay profile based on the simulation signal generated by the simulation signal generator. This configuration enables the learning device 400 to generate a trained model without actually transmitting and receiving transmitted waves and received waves using the signal transmitter / receiver 10.

[0087] The learning device may have some or all of the components of the signal transmission / reception unit 10, or some or all of the components of the display device 20, or some of the components of the learning device may be provided in another device that is connected to the learning device so that it can communicate with the learning device.

[0088] In addition, the present disclosure allows for free combination of the respective embodiments, modification of any of the components of the respective embodiments, or omission of any of the components of the respective embodiments. [Industrial Applicability]

[0089] The learning device according to the present disclosure can be used, for example, in a radar device that detects an object based on a received wave that is a wave transmitted from a transmitting antenna element and reflected by the object and received by a receiving antenna element. [Explanation of symbols]

[0090] 1 object detection system, 2 object detection system, 3 object detection system, 4 object detection system, 10 signal transmission / reception unit, 20 display device, 100 learning device (inference device), 100a processor, 100b memory, 100c I / O port, 100d processing circuit, 110 signal processing unit, 111 signal acquisition unit, 113 delay profile generation unit, 114 beamforming unit, 115 noise information acquisition unit, 120 object information acquisition unit, 130 learning data generation unit, 140 machine learning unit, 150 memory unit (learned model acquisition unit), 160 inference unit, 200 learning device (inference device), 210 signal processing unit, 212 signal extraction unit, 213 delay profile generation unit, 300 learning device (inference device), 310 signal processing unit, 313 delay profile generation unit, 313a threshold setting unit, 400 Learning device (inference device), 410 signal processing unit, 411 signal acquisition unit, 470 simulated signal generation unit.

Claims

1. An object information acquisition unit acquires object information relating to the object, including object presence / absence information indicating the presence or absence of the object, A delay profile generation unit generates a delay profile based on the received wave received by a receiving antenna element, which is the reflected wave of the transmitted wave transmitted from the transmitting antenna element by the object. A training data generation unit generates training data for generating a trained model based on the object information acquired by the object information acquisition unit and the delay profile generated by the delay profile generation unit. A machine learning unit that performs machine learning on the training data generated by the training data generation unit and generates the trained model that outputs a new object detection result based on the input of the delay profile generated by the delay profile generation unit, The system includes a noise information acquisition unit that acquires noise information simulating noise contained in the received wave received by the receiving antenna element, The delay profile generation unit generates a plurality of second delay profiles by adding a plurality of different noise information acquired by the noise information acquisition unit to a first delay profile, which is the delay profile of the received wave received by the receiving antenna element. The training data generation unit generates training data for generating the trained model based on the object information acquired by the object information acquisition unit and a plurality of second delay profiles generated by the delay profile generation unit. A learning device characterized by the following features.

2. The system includes a beamforming unit that performs beamforming of the transmitted waves transmitted from the plurality of transmitting antenna elements and the received waves received by the plurality of receiving antenna elements based on the delay profile generated by the delay profile generation unit. The learning device according to claim 1, characterized by the features described above.

3. The system includes a beamforming unit that performs beamforming of the transmitted waves transmitted from the plurality of transmitting antenna elements and the received waves received by the plurality of receiving antenna elements based on the object information acquired by the object information acquisition unit. The learning device according to claim 1, characterized by the features described above.

4. The object information acquired by the object information acquisition unit includes information indicating the position of the object, The beamforming unit performs beamforming of the transmitted waves transmitted from the multiple transmitting antenna elements so that the power of the transmitted waves directed toward the object's position is maximized. The learning device according to claim 3, characterized in that it is a learning device.

5. The object information acquired by the object information acquisition unit includes information indicating the position of the object, The beamforming unit performs beamforming of the transmitted waves transmitted from the plurality of transmitting antenna elements so that the power of the transmitted waves directed toward the object's position exceeds a preset threshold. The learning device according to claim 3, characterized in that it is a learning device.

6. The beamforming unit performs beamforming of the transmitted waves transmitted from the plurality of transmitting antenna elements and the received waves received by the plurality of receiving antenna elements so that the transmitted waves transmitted from the plurality of transmitting antenna elements and the received waves received by the plurality of receiving antenna elements are orthogonal to each other. The learning device according to feature 2.

7. The beamforming unit beamforms the transmitted waves transmitted from the plurality of transmitting antenna elements so that the beams included in the transmitted waves transmitted from the plurality of transmitting antenna elements are orthogonal to each other, and beamforms the received waves received by the plurality of receiving antenna elements so that the received waves are aligned with the direction of the beams included in the transmitted waves transmitted from the plurality of transmitting antenna elements. The learning device according to feature 2.

8. The delay profile generation unit generates a delay profile based on the received wave over a specific period, which is extracted from the received wave received by the receiving antenna element. The learning device according to claim 1, characterized by the features described above.

9. The delay profile generation unit generates a delay profile based on the received wave, which is extracted from the received wave received by the receiving antenna element and is above a preset power threshold. The learning device according to claim 1, characterized by the features described above.

10. The system includes a simulated signal generation unit that generates a simulated signal that simulates a received wave received by a receiving antenna element, where the reflected wave of the transmitted wave transmitted from the transmitting antenna element by the object simulates the received wave. The delay profile generation unit generates a delay profile based on the simulated signal generated by the simulated signal generation unit. The learning device according to claim 1, characterized by the features described above.

11. The machine learning unit generates the trained model based on the input of the delay profile generated by the delay profile generation unit, which outputs information on at least one of the following: the detection result of the new object, as well as the position, velocity, and classification result of the new object. A learning device according to any one of claims 1 to 10.

12. A trained model acquisition unit acquires a trained model that acquires a trained model that uses machine learning on training data generated based on object information including object presence / absence information indicating the presence or absence of an object, and a plurality of second delay profiles which are obtained by adding noise information that simulates noise contained in the received wave received by the receiving antenna element, and which are different noise pieces of noise information, to a first delay profile generated based on a received wave that is received by a receiving antenna element from a transmitted antenna element, where the reflected wave from the object of the transmitted wave transmitted by the transmitting antenna element is added to the received wave, and which are different noise pieces of noise information that simulate noise contained in the received wave received by the receiving antenna element, and outputs a new object detection result based on the input of the delay profile. A delay profile generation unit generates a delay profile based on the received wave, which is the reflected wave of the transmitted wave transmitted from the transmitting antenna element by the object and received by the receiving antenna element. The system comprises: an inference unit that performs inference using the trained model based on the input of the delay profile obtained by the delay profile generation unit to the trained model; An inference device characterized by the following features.

13. A learning method performed by an apparatus comprising an object information acquisition unit, a delay profile generation unit, a learning data generation unit, a machine learning unit, and a noise information acquisition unit, The object information acquisition unit acquires object information relating to the object, including object presence / absence information indicating the presence or absence of the object. The delay profile generation unit generates a delay profile based on the received wave, which is the reflected wave of the transmitted wave transmitted from the transmitting antenna element by the object and received by the receiving antenna element. The training data generation unit generates training data for generating a trained model based on the object information acquired by the object information acquisition unit and the delay profile generated by the delay profile generation unit. The machine learning unit includes the steps of: machine learning the training data generated by the training data generation unit and generating a trained model that outputs a new object detection result based on the input of the delay profile generated by the delay profile generation unit; and the noise information acquisition unit includes the step of acquiring noise information that simulates the noise contained in the received wave received by the receiving antenna element. The delay profile generation unit generates a plurality of second delay profiles by adding a plurality of different noise information acquired by the noise information acquisition unit to a first delay profile, which is the delay profile of the received wave received by the receiving antenna element. The training data generation unit generates training data for generating the trained model based on the object information acquired by the object information acquisition unit and a plurality of second delay profiles generated by the delay profile generation unit. A learning method characterized by the following:

Images