DETECTION APPARATUS, MODEL GENERATION APPARATUS, DETECTION METHOD, AND PROGRAM

Abstract

The detection device (100) of the first embodiment includes a position identification unit (12), an extraction unit (14), a model selection unit (16), and a detection unit (18). The position identification unit (12) identifies a position of a subject in a 3D radar image. The extraction unit (14) extracts a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of the extraction sizes specified for each type of the subject. The model selection unit (16) selects at least one trained model based on at least one of the size of the 3D sub-image and the type of the subject included in the 3D radar image. The detection unit (18) detects an object in the 3D sub-image using the selected trained model.

Description

[Technical field]

[0001] The present invention relates to a radar image object detection system, where imaging is performed to measure radio waves scattered by a moving target object and generate a 3D radar image of the target being scanned, said 3D image is used by a deep learning module to detect, i.e. check for the presence of hidden threats. [Background technology]

[0002] An example of a conventional radar image object detection system is described in Non-Patent Document 1. This conventional object detection system includes a radar signal measurement means, an image generation means, and an object detection means. Specifically, the measurement means includes a radar antenna that transmits radar waves and receives reflected scattered waves. In addition, the generated 3D radar image is projected into 2D and used by a deep learning module to detect the presence or absence of a target object in the radar image. In Non-Patent Document 1, the object detection system using the radar image is used to detect concealed weapons.

[0003] Patent Document 1 describes setting an image processing area for extracting an image portion of a surveillance object according to the type of the surveillance object.

[0004] Patent Document 2 discloses that a part of an image based on the result of a preliminary inspection is used to determine whether or not a target person is carrying a prohibited item. Patent Document 2 also discloses that machine learning can be used as an example of a determination method based on the shape of an object reflected in a transparent image. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2008 / 139529 [Patent Document 2] International Publication No. 2021 / 166150 [Non-patent literature]

[0006] [Non-Patent Document 1] L. Carrer, "Concealed Weapon Detection: A microwave imaging approach", Master of Science Thesis, Delft University of Technology, 2012 Summary of the Invention [Problem to be solved by the invention]

[0007] However, the technology of Non-Patent Document 1 takes a long time to detect an object. The technology of Patent Document 1 cannot be applied to the use of a trained model by machine learning. Furthermore, Patent Document 2 does not disclose a technology for shortening the processing time without degrading the performance of the determination unit by using a trained model.

[0008] One example of an objective of the present invention is to reduce the processing time for object detection without compromising detection accuracy when a learned model is used for detection. [Means for solving the problem]

[0009] The present invention relates to a location identification unit for identifying a location of a subject in a 3D radar image; an extraction unit that extracts a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of the extraction sizes designated for each type of the subject; A model selection unit that selects at least one trained model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image; a detector that detects an object in the 3D sub-image using the selected trained model; A detection device comprising: to provide.

[0010] The present invention relates to a location identification unit for identifying a location of a subject in a 3D radar image; an extraction unit that extracts a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of the extraction sizes designated for each type of the subject; a model selector for selecting at least one model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image; a learning unit that performs machine learning on the selected model using a combination of the 3D sub-image and information indicating the position of the object within the 3D sub-image as training data; A model generating device comprising: to provide.

[0011] The present invention relates to Identifying the location of a subject in a 3D radar image; extracting a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of the extraction sizes designated for each type of subject; selecting at least one trained model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image; Detecting an object in the 3D sub-image using the selected trained model. A computer-implemented detection method comprising: to provide.

[0012] The present invention relates to Identifying the location of a subject in a 3D radar image; extracting a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of the extraction sizes designated for each type of subject; Selecting at least one trained model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image. Detecting an object in the 3D sub-image using the selected trained model. A program for causing a computer to execute the detection method including to provide. Effect of the Invention

[0013] An object of the present invention is to reduce the processing time for object detection without compromising detection accuracy when a trained model is used for detection. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram illustrating the setup of a radar image measurement system and the relative position of a target with respect to a radar. [Diagram 2] FIG. 2 is a block diagram illustrating a function-based configuration of the detection device according to the first embodiment. [Diagram 3] FIG. 3 is a block diagram illustrating a hardware configuration of a computer that realizes the detection device according to the first embodiment. [Figure 4] FIG. 4 is a flowchart illustrating a process flow by the detection device of the first embodiment. [Diagram 5] FIG. 5 is a block diagram showing an example of the functional base configuration of the detection device according to the first embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of a table of a subject DB of the detection device according to the first embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of the configuration of a network architecture DB of the detection device according to the first embodiment. [Figure 8]FIG. 8 is a block diagram illustrating a function-based configuration of a detection device including a first example subject finder. [Figure 9] FIG. 9 is a diagrammatic representation showing an example of the operation of the detection device according to the first embodiment. [Figure 10] FIG. 10 is a block diagram illustrating a function-based configuration of a detection device including a subject finder according to the second embodiment. [Figure 11] FIG. 11 is a block diagram illustrating a function-based configuration of a detection device including a subject finder according to the third embodiment. [Figure 12] FIG. 12 is a diagram showing another example of a table of the subject DB of the detection device according to the first embodiment. [Figure 13] FIG. 13 is a flowchart illustrating the operation of the detection device according to the first embodiment. [Figure 14] FIG. 14 is a block diagram illustrating a function-based configuration of the model generating device according to the second embodiment. [Figure 15] FIG. 15 is a flowchart illustrating the flow of processing performed by the model generating device according to the second embodiment. [Figure 16] FIG. 16 is a block diagram showing an example of the configuration of a function base of a model generating device according to the second embodiment. [Figure 17] FIG. 17 is a flowchart illustrating the operation of the model generating device according to the second embodiment. [Figure 18] FIG. 18 is a block diagram showing an example of the functional base configuration of a detection device according to the third embodiment. [Figure 19] FIG. 19 is a flowchart illustrating the operation of the detection device according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. For clarity of explanation, the same elements in the drawings will be denoted by the same reference numerals, and duplicate explanations will be omitted.

[0016] First embodiment

[0017] <Summary> First, the configuration of an object detection system 900 for radar images will be described with reference to FIG.

[0018] The object detection system 900 for radar images works as follows: First, radar signals are measured. In the measurement step, the radar antenna transmits radar signals one by one in a specific order, and the reflected waves are received by the antenna receiver. The measured radar signals are used to create a 3D radar image by the image generation means using the radar antenna information.

[0019] The specific purpose of this system is to check whether a person (target) 90 is carrying a concealed dangerous object. The system 900 measures the target 90 by a fixed antenna (radar 92) installed on a side panel 94 when the target 90 walks in a screening area (area) 96. The transmitters in the antenna transmit signals one by one, and the received scattered signals are acquired. The system 900 also acquires a camera image using a camera (camera 98) simultaneously with the radar signal. However, the system 900 according to this embodiment may not include a camera. The radar signal is processed using the antenna information to generate a radar image that is inherently 3D. If a concealed dangerous object is carried by the person (target 90), it will be displayed in the radar image. In this way, the radar image is used to detect the concealed dangerous object. To detect the presence of the dangerous object from the radar image, a trained model by machine learning is utilized. The trained model may be included in a deep learning module. As can be understood, the whole setup is expected to work in real time, since we expect to obtain the presence / absence information of the concealed object while the target 90 is still nearby. However, processing by the trained model is generally a bottleneck. Since the input images to the trained model are inherently 3D, it is expected that the processing time will be longer due to the increased computational complexity of 3D. Existing methods to reduce the processing time, as suggested in the background art, resize the 3D image to a smaller size by scaling or downsampling the 3D image, or by only projecting the 3D image to 2D. However, this is not a good method because it may also affect the performance of the trained model due to information loss.

[0020] As mentioned above, the detection unit using the trained model is required to function in real time. This is because our target is moving as a preferred mode, and we aim to detect dangerous objects while the target 90 is still near the detection system 900. As mentioned above, the input image to the trained model is inherently 3D, and it is difficult to obtain predictions in real time due to the complexity of 3D calculations. According to the detection device 100 of this embodiment, the processing time of the trained model can be reduced without compromising performance for radar images. This is achieved by using a 3D radar image (3D sub-image) of the extracted subject as input to the trained model instead of the original 3D image. The 3D image of the extracted subject is reduced in size compared to the original 3D radar image, but the information is the same. The processing time of the trained model is sensitive to the input size, especially in the case of 3D, so the processing time is reduced. In particular, the present disclosure relates to a subject extraction system for 3D radar images that reduces the processing of the trained model by reducing the input image size without affecting performance. The object detection system 900 for radar imagery is preferably capable of functioning in real time.

[0021] Note that a subject refers to a target, and an object refers to an object about which you want to know the presence or absence. The object may be part of the subject, but they do not have to be the same.

[0022] <Example of feature-based configuration> FIG. 2 is a diagram showing an example of a function-based configuration of the detection device 100 according to the first embodiment. The detection device 100 according to the first embodiment includes a position identification unit 12, an extraction unit 14, a model selection unit 16, and a detection unit 18. The position identification unit 12 identifies the position of a subject in a 3D radar image. The extraction unit 14 extracts a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of the extraction sizes determined for each type of subject. The model selection unit 16 selects at least a trained model based on at least one of the size of the 3D sub-image and the type of subject included in the 3D radar image. The detection unit 18 detects an object in the 3D sub-image using the selected trained model. This will be described in detail below.

[0023] The detection device 100 of this embodiment may be included in an object detection system 900. An example of the object is a dangerous object (such as a knife or a gun) carried by a target 90.

[0024] First, the acquisition of a radar image will be described with reference to FIG. 1. A person (target 90) is assumed to be walking in a screening area 96 in front of a fixed radar antenna (radar 92) installed on a side panel 94. When in the screening area, the target may also be in the field of view of a camera 98, which captures an image synchronized with the radar sensor. The measured scattered radar signal is transmitted for imaging. The generated radar image is 3D in nature. The 3D radar image is generated from the measured scattered radar signal. The 3D radar image may be generated within the detection device 100 or may be generated by another device. The generated 3D radar image is stored in a radar image database (DB). The radar image DB is realized by one or more storage devices. The radar image DB may or may not be included in the detection device 100.

[0025] The generated radar image is used to detect whether the target is carrying a dangerous object. The detection unit 18 detects objects from the 3D image using a trained model. The trained model may include a deep learning network. To obtain a prediction from the deep learning network in a short time, preferably in real time, the extraction unit extracts smaller sized images given as input to find the presence of objects. The detection device 100 is also called a subject extraction device.

[0026] Here, the subject refers to the whole or part of the object from which the image is extracted, and the object refers to, for example, a dangerous object. The subject above refers to the subject of the image, i.e., the thing that occupies the largest area / volume in the image, such as a living body such as a human being, or a moving object such as a car. Without loss of generality, there may be multiple subjects.

[0027] In this embodiment, it is assumed that the identity of the subject is known to the detection device 100 as prior information. For example, a 3D radar image is stored in the radar image DB in association with a subject ID. The subject ID indicates the type of subject, such as adult, child, elderly, etc. The subject ID may also indicate the type or make of vehicle. The subject ID of each image is identified by other means, such as a sensor that detects the characteristics of the target 90 or a camera that captures the target 90, and is stored in the radar image DB.

[0028] <Hardware configuration example> In some embodiments, each functional unit included in the detection device 100 may be implemented by at least one hardware component, and each hardware component may implement one or more functional units. In some embodiments, each functional unit may be implemented by at least one software component. In some embodiments, each functional unit may be implemented by a combination of hardware and software components.

[0029] The detection device 100 may be realized by a dedicated computer manufactured for the purpose of implementing the detection device 100, or may be realized by a general-purpose computer such as a personal computer (PC), a server machine, or a mobile device.

[0030] FIG. 3 is a block diagram showing an example of a hardware configuration of a computer 1000 that realizes the detection device 100 of the first embodiment. In FIG. 3, a computer 1000 includes a bus 1020 , a processor 1040 , a memory 1060 , a storage device 1080 , an input / output (I / O) interface 1100 , and a network interface 1120 .

[0031] The bus 1020 is a data transmission path for the processor 1040, the memory 1060, and the storage device 1080 to transmit and receive data to and from each other. The processor 1040 is a processor such as a central processing unit (CPU), a graphics processing unit (GPU), or a field-programmable gate array (FPGA). The memory 1060 is a primary storage device such as a random access memory (RAM). The storage medium 1080 is a secondary storage device such as a hard disk drive, a solid state drive (SSD), or a read only memory (ROM).

[0032] The I / O interface is an interface between the computer 1000 and peripheral devices such as a keyboard, a mouse, a display device, etc. The network interface is an interface between the computer 1000 and a communication line that enables the computer 1000 to communicate with other computers.

[0033] The storage device 1080 may store a program module that realizes each functional unit of the detection device 100. Each functional unit of the detection device 100 is realized by the CPU 1040 executing each program module.

[0034] <Processing flow> FIG. 4 is a flowchart illustrating a processing procedure of the detection device 100 of the first embodiment.

[0035] The detection method according to the first embodiment is executed by a computer. This detection method includes a position identification step (S12), an extraction step (S14), a model selection step (S16), and a detection step (S18). In the position identification step, the position identification unit 12 identifies the position of a subject in a 3D radar image. In the extraction step, the extraction unit 14 extracts a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of the extraction sizes determined for each type of subject. In the model selection step, the model selection unit 16 selects at least a trained model based on at least one of the size of the 3D sub-image and the type of subject included in the 3D radar image. In the detection step, the detection unit 18 detects an object in the 3D sub-image using the selected trained model.

[0036] <Configuration Example of Detection Device 100> An example of the configuration of the radar image detection device 100 according to the first embodiment will be described in detail with reference to the block diagram of FIG.

[0037] As shown in FIG. 5, an example configuration of the detection device 100 according to the first embodiment includes a radar image DB storage unit 101, a subject finder 102, an image extraction unit 103, a detection unit 104, a subject DB storage unit 105, an approach selection unit 106, a network selection unit 107, and a network architecture DB storage unit 117.

[0038] The subject finder 102 functions as the location identification unit 12. The image extraction unit 103 functions as the extraction unit 14. The network selection unit 107 functions as the model selection unit 16. The detection unit 104 functions as the detection unit 18.

[0039] The radar image DB storage unit 101 may or may not be included in the detection device 100. The subject DB storage unit 105 may or may not be included in the detection device 100. The network architecture DB storage unit 117 may or may not be included in the detection device 100.

[0040] In the example of this figure, the detection device 100 further includes an extraction size specification unit. The extraction size specification unit specifies an extraction size to be used for extracting a 3D sub-image based on at least the subject information in which an extraction size is associated with each of a plurality of subject IDs and the subject ID of the subject included in the 3D radar image. The approach selection unit 106 in FIG. 5 functions as the extraction size specification unit.

[0041] Next, the operation of the detection device 100 will be described. It should be noted that this operation describes the prediction phase of the deep learning network. However, the same can be extended to the learning phase without loss of generality. During operation, 3D radar images are measured as described above and stored in the radar image DB storage unit 101, from which they are read one by one. The 3D radar images are then sent to the subject finder 102. The subject finder 102 locates the subject (single or multiple) and passes the subject's location along with the 3D radar image to the image extraction unit 103. The subject finder 102 receives a means for finding the subject from the approach selection unit 106. The approach selection unit 106 also outputs the extracted image size (extraction size) to the image extraction unit 103. The image extraction unit 103 uses the subject position information to extract 3D sub-images of the size output from the approach selection unit 106 and passes them to the detection unit 104. The detection unit 104 can use a deep learning module for object detection. The approach selection unit 106 also passes the extracted image size to the network selection unit 107, which selects an appropriate deep learning network architecture from the DB based on the extracted image size and / or the subject ID. The selected network architecture is given as an input to the detection unit 104, which also receives a smaller sized extracted image (3D sub-image) to detect the object. The detection unit 104 inputs the 3D sub-image to the network architecture (trained model) and obtains a prediction result as an output of the network architecture (trained model). The output of the detection unit 104 informs the presence / absence information (and possibly the position) of the object.

[0042] The detector 18 may output at least one of information indicating the presence or absence of an object, a class of the detected object, and object location information. In this configuration, the expected output from the detector 104 is presence or absence information of an object of interest (hereinafter referred to as "object"), which may be part of the subject, but need not be the same as the subject. The object presence or absence information can take the form of image level classes (classifiers) or pixel level classes (segmentation). Additionally, the object location can also be provided as an output.

[0043] The radar image DB storage unit 101 provides radar images. Measured and generated 3D radar images are stored in the radar image DB storage unit 101. It functions as a data source by providing the 3D radar images as input to the subject finder 102.

[0044] A plurality of subject sizes are stored in advance in the subject DB storage unit 105. Each size is associated with a subject ID. In this configuration example, the subject DB storage unit 105 stores various information of the subject (size, etc.) in a table format in association with the subject ID. The subject DB storage unit 105 can look up information of the subject using the ID as a primary key. An example of the subject information is the size of the subject, as shown in the example table of the subject DB in FIG. 6. Here, the subject size in the subject DB is the size of the subject assumed in the 3D radar image.

[0045] In addition, the subject DB storage unit 105 stores a plurality of means for finding the subject position in advance. Each means for finding the subject position is associated with a subject ID. In this configuration example, the approach selection unit 106 determines an approach for finding the subject position based on the subject information acquired from the subject DB storage unit 105. The approach selection unit 106 outputs the means for finding the subject position to the subject finder 102. In addition, the approach selection unit 106 outputs the extracted image size and subject ID to the image extraction unit 103 and the network selection unit 107 based on the subject information. The means for finding the subject position depends on the design of the subject finder 102, and some examples of the means may include a projection axis, a position axis (the axis along which the position can be found), and the like. Some configuration examples of the approach selection unit 106 will be described together with the configuration example of the subject finder 102.

[0046] When a 3D radar image is associated with a subject ID and stored in the radar image DB storage unit 101, the approach selection unit 106 acquires the subject ID of the 3D radar image to be processed. The approach selection unit 106 reads out, based on the acquired subject ID, from the subject DB storage unit 105, a means for determining the size and subject position of the subject to which the 3D radar image is applied.

[0047] The network architecture DB storage unit 117 stores a plurality of trained models in advance. Each trained model is associated with at least one of the size of the 3D sub-image and the subject ID. In this configuration example, the network architecture DB storage unit 117 provides architectures (for various image sizes, types of subjects, etc.). The network architecture DB storage unit 117 includes various trained network architectures for various image sizes, types of subjects, etc. The network architectures are pre-trained for classification, object detection, and / or segmentation tasks, and can be distinguished by input image size and / or subject ID. In the network architecture DB storage unit 117, the network architecture can be examined, for example, by using metadata such as extracted image size and / or subject ID. An example of the configuration of the network architecture DB stored in the network architecture DB storage unit 117 is shown in FIG. 7, in which different network architectures can be distinguished by metadata information.

[0048] The network selection unit 107 selects a network architecture for the detection unit 104 based on the extracted image size and / or subject ID received from the approach selection unit 106. The network architecture is selected from the network architecture DB storage unit 117 using the extracted image size and / or subject ID as a search key. The network selection unit 107 outputs the network architecture to the detection unit 104.

[0049] It is understood that using a network architecture for image sizes different from the one it was trained on will lead to a degradation in performance. Here, it is assumed that images are given as input to the network architecture without resizing due to the processing time and performance related reasons mentioned above. Thus, as the extracted image size changes, the input image size to the network architecture will change and the network architecture will subsequently need to be modified. Furthermore, if there are different types of subjects, a dedicated network architecture will be needed to detect objects for each subject type. The given reasons indicate the need for the network selection unit 107.

[0050] The detection unit 104 obtains object presence / absence information in the radar image based on the selected network architecture from the network selection unit 107 and the extracted 3D sub-images from the image extraction unit 103. The detection unit 104 outputs the object presence / absence information. As mentioned above, the object may be part of the aforementioned subject, but it does not have to be the same. Also, the deep learning module can be a classifier (outputting single / multiple classes per image), an object detector (outputting object class and location per image), or a segmentation network (outputting pixel-level class map of the same size as the input image). The detection unit 104 can output the object presence / absence information in the form of image-level classes (classification) or pixel-level classes (segmentation). In addition to the presence / absence information, the object location can also be output.

[0051] The image extraction unit 103 extracts a 3D radar image of a subject based on the subject position received from the subject finder 102 and the extracted image size received from the approach selection unit 106. The image extraction unit 103 receives the original 3D radar image from the subject finder 102 and performs extraction. The image extraction unit 103 extracts an image from the original 3D radar image using the received subject position. The image extraction unit 103 outputs the extracted image (i.e., a 3D sub-image) to the detection unit 104. The subject position may indicate, for example, the center position of the subject or any of the corners. The extracted 3D sub-image may be an image that is simply a cut-out of a part of the 3D radar image. Each 3D sub-image includes only a part of the subject or the entire subject. The image extraction unit 103 may generate multiple 3D sub-images from one 3D radar image. The extracted 3D sub-image and the 3D radar image have the same image quality, resolution, etc. The crop location in the 3D radar image is determined based on the subject location received from the subject finder 102. The size of the 3D sub-image is determined based on the crop size received from the approach selector 106.

[0052] The subject finder 102 determines the position of the subject in the 3D radar image based on the subject finding means received from the approach selection unit 106 and the original 3D radar image read from the radar image DB storage unit 101. The subject finder 102 outputs the subject position to the image extraction unit 103. The subject finder 102 also outputs the original radar image to the image extraction unit 103.

[0053] <Configuration of Subject Finder 102> Next, the subject finder 102 and the approach selector 106 will be described in detail using some example configurations.

[0054] <<First Example of Subject Finder 102>> In this example, the extraction size used for extracting the 3D sub-images is the subject extraction size for extracting the entire subject. The reference position indicates the position of the subject.

[0055] A first example of the configuration of the subject finder 102 is described with reference to FIG. 8, where the subject finder 102 consists of two sub-blocks: a 2D projector 102a and a subject location finder 102b. Although the method is described assuming a single subject, the same can be extended to multiple subjects. In this example, a 2D image is first generated by projection, which is then processed to identify the subject's location. The 2D projector 102a outputs the projected 2D image to the subject location finder 102b based on the projection axis (the means for finding the subject) from the approach selector 106. The 2D image is generated by projecting the 3D image along the projection axis. The projection can be a maximum projection, an energy projection, or other. The subject location finder 102b outputs the subject's location in the 2D image based on the 2D projected image from the 2D projector 102a and the location axis (the means for finding the subject) from the approach selector 106. The location can be found in various ways, for example by finding the point of maximum intensity. The subject location is provided as output from the subject finder 102 to the image extractor 103. An example of the overall operation for obtaining an extracted radar image from the entire radar image is illustrated in FIG.

[0056] <<Second Example of Subject Finder 102>> The subject finder 102 may consist of an image processor 102e which outputs a processed 2D image based on several image processing algorithms, e.g. filtering, sharpening, etc. The purpose of the image processor 102e is to support the operations of the subject location finder block, e.g. clustering, peak detection, etc. It is shown in Fig. 10.

[0057] <<Third Example of Subject Finder 102>> In this example, the extraction size used for extracting the 3D sub-image is a partial extraction size that extracts only a part of the subject, and the reference position indicates the position of the part of the subject.

[0058] This configuration example of the subject finder 102, which is a special case of the first example where the subject finder 102 is supposed to find a (pre-known) part of the subject, will be described with reference to FIG. 11. The configurations of the 2D projector 102a and the subject position finder 102b are similar to those described in the first example above, so their description will be omitted. The additional block here is that the body part finder 102c outputs the position of the subject body part to the image extraction unit 103 based on the subject position received from the subject position finder 102b and the relative position of the subject body part to the subject received from the approach selection unit 106. The body part finder 102c can output the (global) position of the subject body part when it has the (global) position information of the subject and the relative position of the body part to the subject using simple coordinate mathematics. In this configuration example, the subject DB storage unit 105 stores a plurality of combinations of the relative position of the subject body part and the size of the subject body part in advance. Each combination is associated with a subject ID. Furthermore, the approach selection unit 106 uses the subject ID to read the relative position of the subject part with respect to the subject and the size of the subject part from the subject DB storage unit 105. An example of the table configuration used in the subject DB storage unit 105 is shown in FIG.

[0059] In this configuration example, the image extraction unit outputs extracted images (i.e., 3D sub-images) of the subject's parts to the detection unit 104 based on the subject part's position received from part finder 102c and the extracted image size of each part received from approach selection unit 106. Approach selection unit 106 reads the relative position of the subject part and the size of the subject part from subject DB storage unit 105 based on the subject ID. Detection unit 104 further receives a network (which may be the same or different) for each subject part selected by network selection unit 107. Then, detection unit 104 outputs information on the presence or absence of an object in the extracted image by individually analyzing the image of each subject part.

[0060] Next, an example of the operation in the operation mode of the radar image detection device 100 according to the first embodiment will be described with reference to the flowchart shown in Fig. 13. Subject information (sizes associated with subject IDs) is stored in the subject DB storage unit 105. Furthermore, various network architectures are pre-learned for different image sizes and stored in the network architecture DB storage unit 117.

[0061] When the detection device 100 is started, in step S101, a 3D radar image is read from the radar image DB storage unit 101. In step S105, subject information is read from the subject DB storage unit 105. Next, the approach selection unit 106 identifies the approach, i.e., what means are required to identify the subject's position, and provides it as an output to the image extraction unit 103 (step S106). The network selection unit 107 selects a network architecture from the network architecture DB storage unit 117 and outputs it to the detection unit 104 (step S107). The subject finder 102 finds the subject position and outputs it to the image extraction unit 103 (step S102). The image extraction unit 103 extracts an image using the subject position and the extracted image size, and outputs the extracted image to the detection unit 104 (step S103). The detection unit 104 predicts the presence or absence information of an object in the extracted 3D sub-image using classification or segmentation (step S104).

[0062] As described above, the detection device 100 according to the first embodiment of the present disclosure extracts images of smaller size using subject position information. This reduces the processing time of the trained model, i.e., the prediction of object presence / absence information in the detection unit, and enables desired real-time operation. It can be seen that the reduction in processing time of the trained model due to the reduction in image size is much greater than the slight increase in processing time by the subject finder 102 (which performs image processing operations).

[0063] Second embodiment

[0064] FIG. 14 is a diagram showing an example of a function-based configuration of the model generating device 200 according to the second embodiment. The model generating device 200 includes a position identifying unit 22, an extraction unit 24, a model selecting unit 26, and a learning unit 28. The position identifying unit 22 identifies the position of a subject in a 3D radar image. The extraction unit 24 extracts a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of the extraction sizes determined for each type of subject. The model selecting unit 26 selects at least a model based on at least one of the size of the 3D sub-image and the type of subject included in the 3D radar image. The learning unit 28 performs machine learning on the selected model using a combination of the 3D sub-image and information indicating the position of an object in the 3D sub-image as learning data. This will be described in detail below.

[0065] The position identifying unit 22, the extraction unit 24, and the model selecting unit 26 are similar to the position identifying unit 12, the extraction unit 14, and the model selecting unit 16 according to the first embodiment, respectively.

[0066] The model generation device 200 can generate or update a trained model used in the detection device 100 according to the first embodiment. The trained model generated or updated by the model generation device 200 may be stored in the network architecture DB storage unit 117 of the detection device 100.

[0067] The model generating device 200 may also function as the detection device 100 described in the first embodiment. That is, the detection device 100 may include an annotation adjustment unit 209, and the detection unit 104 may also function as the learning unit 204. In this case, the performance of the detection device 100 may be evaluated by comparing the output of the trained model with information indicating the position of an object in a 3D sub-image as ground truth data.

[0068] <Hardware configuration example> In some embodiments, each functional unit included in model generator 200 may be implemented by at least one hardware component, and each hardware component may implement one or more functional units. In some embodiments, each functional unit may be implemented by at least one software component. In some embodiments, each functional unit may be implemented by a combination of hardware and software components.

[0069] The model generating device 200 may be realized by a dedicated computer manufactured for the purpose of realizing the model generating device 200, or may be realized by a general-purpose computer such as a personal computer (PC), a server machine, or a mobile device.

[0070] The model generating device 200 may be realized by a computer 1000 shown in Fig. 3. The storage device 1080 may store program modules that realize each functional unit of the model generating device 200. Each functional unit of the model generating device 200 is realized by the CPU 1040 executing each program module.

[0071] <Processing flow> FIG. 15 is a flowchart showing the processing procedure of the model generating device 200 of the second embodiment.

[0072] The model generation method according to the second embodiment is executed by a computer. The model generation method includes a position identification step (S22), an extraction step (S24), a model selection step (S26), and a learning step (S28). In the position identification step, the position identification unit 22 identifies the position of the subject in the 3D radar image. In the extraction step, the extraction unit 24 extracts a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and an extraction size determined for each type of subject. In the model selection step, the model selection unit 26 selects at least a model based on at least one of the size of the 3D sub-image and the type of subject included in the 3D radar image. In the learning step, the learning unit 28 performs machine learning on the selected model using a combination of the 3D sub-image and information indicating the position of the object in the 3D sub-image as learning data.

[0073] <Configuration Example of Model Generator 200> Next, an example of the configuration of the model generating device 200 according to the second embodiment will be described in detail with reference to the block diagram of FIG.

[0074] As shown in FIG. 16, the configuration of the model generating device 200 according to the second embodiment can include a radar image DB storage unit 201, a subject finder 202, an image extraction unit 203, an annotation DB storage unit 208, an annotation adjustment unit 209, a learning unit 204, a subject DB storage unit 205, an approach selection unit 206, a network selection unit 207, and a network architecture DB storage unit 217. The annotation DB storage unit 208 may or may not be included in the model generating device 200. The configurations and functions of the radar image DB storage unit 201, the subject finder 202, the subject DB storage unit 205, the approach selection unit 206, and the network selection unit 207 are the same as the configurations and functions of the radar image DB storage unit 101, the subject finder 102, the subject DB storage unit 105, the approach selection unit 106, and the network selection unit 107 according to the first embodiment, respectively. Therefore, the description will not be repeated. Moreover, the image extraction unit 203 is similar to the image extraction unit 103 according to the first embodiment, except for the points described below.

[0075] The subject finder 202 functions as the location identification unit 22. The image extraction unit 203 functions as the extraction unit 24. The network selection unit 207 functions as the model selection unit 26. The training unit 204 functions as the training unit 28.

[0076] The configuration of a model generating device 200 according to the second embodiment will be described with reference to the block diagram of FIG. 16. In this configuration, it is also assumed that the identity of the subject is known to the subject finder 202 as prior information. The expected output may be the location of the object in addition to the presence / absence information. This object may be part of the subject, but need not be the same as the subject. The presence / absence and location information of the object may be expressed in the form of image-level classes (classification) or bounding boxes (object detection).

[0077] In this particular embodiment, ground truth object positions are known in advance and the goal is to evaluate the performance of the detection apparatus 100 (prediction phase) or to improve the performance of the detection apparatus 100 (training phase), or both (online prediction and training). Information indicative of the position of the object in the 3D sub-images (ground truth data) is obtained using the ground truth object positions as described below.

[0078] The ground truth position and presence / absence information of an object are collectively called annotation. Since the position of the subject in the extracted image and the extracted image size also change, the object position information also needs to be adjusted. The purpose of the second embodiment is to adjust the position of the object according to the position of the subject in the extracted image (hereinafter referred to as annotation adjustment).

[0079] The annotation DB storage unit 208 stores annotation information for all radar images included in the radar image storage unit 201 so that the information can be distinguished by the name of the radar image. The subject's position is identified by another method, and annotation information is prepared in advance.

[0080] The annotation adjustment unit 209 outputs adjusted annotations to the learning unit 204 based on the subject position and the extracted image size received from the image extraction unit 203. The ground truth annotation information is read from the annotation DB storage unit 208. The annotation adjustment unit 209 also receives the original radar image size as part of the annotation information to assist in annotation adjustment. In this example, the ground truth position is specified by a rectangular bounding box for the original radar image. Adjustment means moving the center of the bounding box based on the subject position and adjusting the size based on the size of the extracted image and the original image.

[0081] The network architecture DB storage unit 217 includes various trained or untrained network architectures distinguishable by different input image sizes and / or subject IDs for classification and / or object detection. The architectures may or may not be pre-trained depending on the task at hand, e.g., training, performance evaluation, etc. The network selector 207 utilizes the network architecture DB storage unit 217 to select a network architecture, as described for the network selector 107 in the first embodiment of the present disclosure.

[0082] The learning unit 204 obtains a selected network architecture including a model from the network selection unit 207. The model includes a neural network. The learning unit 204 inputs the 3D sub-image obtained from the image extraction unit 203 into the network architecture (model). The learning unit 204 may output the position of the object in addition to the presence / absence information based on the network architecture selected from the network selection unit 207 and the extracted image from the image extraction unit 203. Furthermore, the learning unit 204 also receives adjusted annotation information from the annotation adjustment unit 209, which can be used to update the architecture parameters (learning) or evaluate the performance of the architecture, or both. In learning, a combination of the 3D sub-image and the adjusted annotation information is used as training data. The learning unit 204 can output the presence / absence and position information of the object in the form of image-level classes (classification) and bounding boxes (object detection).

[0083] Next, an operation example in the operation mode of the model generating device 200 according to the second embodiment will be described with reference to the flowchart shown in Fig. 17. Subject information (size associated with ID) is stored in the subject DB storage unit 205. In addition, various network architectures corresponding to different image sizes are stored in the network architecture DB storage unit 217. Note that steps S201, S202, S205, S206, and S207 are similar to steps S101, S102, S105, S106, and S107 described in the first embodiment of the present disclosure, and therefore description thereof will be omitted.

[0084] The image extraction unit 203 extracts an image using the subject position and the extracted image size, and outputs it to the learning unit 204. Furthermore, the image extraction unit 203 outputs the subject position and the extracted image size to the annotation adjustment unit 209 (step S203). The annotation adjustment unit 209 reads out an annotation prepared for processing the 3D radar image from the annotation DB storage unit 208, adjusts the annotation using the input subject position and extracted image size, and then the annotation adjustment unit 209 outputs the adjusted annotation to the learning unit 204 (step S209). The learning unit 204 receives the extracted image from the image extraction unit 203, predicts (may use classification / object detection) and outputs information on the presence or absence of an object and its position (step S204).

[0085] As described above, the model generating device 200 according to the second embodiment of the present disclosure extracts smaller sized images using subject position information. This reduces the processing time for acquiring the presence / absence and position information of objects, enabling desired real-time operation. Furthermore, the annotation adjustment function allows for evaluation and / or updating of learner performance, preferably in real time.

[0086] Third embodiment

[0087] The detection device 100 of the third embodiment is similar to the detection device 100 of the first embodiment, except for the points described below.

[0088] The detection device 100 of the third embodiment further includes a subject ID determination unit that identifies the type of subject included in the 3D radar image and determines a subject ID for determining one extraction size. This will be described in detail below.

[0089] With reference to the block diagram of FIG. 18, a configuration example of the radar image detection device 100 according to the third embodiment of the present disclosure will be described. In contrast to the previous configuration, in this configuration, it is not assumed that the identity of the subject is known to the subject finder as prior information. However, the identity of the subject can be obtained as real-time information. In a real-time operation setup, it is not always possible to know the identity of the subject in advance, so it is more realistic to find the identity of the subject at run-time. The expected output is the presence / absence information of the object, as in the first embodiment. This object may be part of the subject, but does not have to be the same as the subject. The presence / absence of the object can be expressed in the form of an image-level class (classification) or a pixel-level class (segmentation). In this embodiment, the 3D radar image stored in the radar image DB 301 may not be associated with a subject ID.

[0090] 18, one of the configurations of the detection device 100 according to the third embodiment can include a radar image DB storage unit 301, a subject finder 302, an image extraction unit 303, a detection unit 304, a subject DB storage unit 305, an approach selection unit 306, a subject identifier (subject ID identification unit) 310, a network selection unit 307, and a network architecture DB storage unit 317. The configurations and functions of the radar image DB storage unit 301, the subject finder 302, the image extraction unit 303, the detection unit 304, the subject DB storage unit 305, the approach selection unit 306, the network selection unit 307, and the network architecture DB storage unit 317 are the same as the configurations and functions of the radar image DB storage unit 101, the subject finder 102, the image extraction unit 103, the detection unit 104, the subject DB storage unit 105, the approach selection unit 106, the network selection unit 107, and the network architecture DB storage unit 117 according to the first embodiment, respectively. Therefore, the description will not be repeated.

[0091] The subject finder 302 may obtain the measured scattered radar signal from the radar 92. The 3D radar image may be generated within the detection device 100 instead of being obtained from the radar image DB store 301.

[0092] The subject identifier 310 outputs subject identification information, e.g., subject ID, to the approach selector 306. The subject identifier 310 may be connected to an external sensor 40 to obtain additional information used to identify the subject. The subject is a subject contained in the 3D radar image retrieved from the radar image DB storage 301 and processed. In one example configuration, the subject identifier 310 receives captured optical images from an optical camera. The external sensor 40 may be the camera 98. The subject identifier 310 may use object detection means (among other means) to identify the subject.

[0093] Next, an example of the operation in the operation mode of the radar image detection device 100 according to the third embodiment will be described with reference to the flowchart shown in FIG. 19. Subject information (size associated with ID) is stored in the subject DB storage unit 305. In addition, various network architectures corresponding to different image sizes are stored in the network architecture DB storage unit 317. Note that steps S301, S302, S303, S304, S305, S306, and S307 are similar to steps S101, S102, S103, S104, S105, S106, and S107 described in the first embodiment of the present disclosure. Therefore, the description thereof will not be repeated.

[0094] The subject identifier 310 identifies the subject and outputs the identification information to the approach selector 306. This step may or may not use external sensor information (step S310).

[0095] The model generating device 200 described in the second embodiment may include the above-mentioned subject classifier 310. In this case, the 3D radar images stored in the radar image DB 201 do not need to be associated with subject IDs.

[0096] As described above, the detection device 100 according to the third embodiment of the present disclosure extracts a smaller size image using the subject's location information. This reduces the processing time required to obtain the presence or absence and location information of an object, enabling desired real-time operation. Furthermore, it has the flexibility to identify subjects, preferably in real time, without the premise that the subject's identity is known in advance.

[0097] All or part of the embodiments disclosed above can be described as follows, but are not limited to these. 1-1. A position identification unit for identifying a subject's position in a 3D radar image; an extraction unit that extracts a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of the extraction sizes designated for each type of the subject; A model selection unit that selects at least one trained model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image; a detector that detects an object in the 3D sub-image using the selected trained model; A detection device comprising: 1-2. An extraction size determination unit that determines the one extraction size used to extract the 3D sub-image based on subject information in which an extraction size is associated with each of a plurality of subject IDs and the subject ID of the subject included in the 3D radar image. The detection device according to 1-1. further comprising: 1-3. A subject ID determination unit that determines the subject ID used to determine the one extraction size by identifying the type of the subject included in the 3D radar image. The detection device according to 1-2., further comprising: 1-4. The one extraction size used to extract the 3D sub-image is a subject extraction size for extracting the entire subject. The detection device according to any one of 1-1 to 1-3. 1-5. The one extraction size used for extracting the 3D sub-image is a partial extraction size that extracts only a part of the subject; The reference position indicates a position of a part of the subject. The detection device according to any one of 1-1 to 1-3. 1-6. The detection unit outputs at least one of information indicating the presence or absence of the object, a class of the detected object, and position information of the object. The detection device according to any one of 1-1 to 1-5.

[0098] 2-1. A position identification unit for identifying a subject's position in a 3D radar image; an extraction unit that extracts a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of the extraction sizes designated for each type of the subject; a model selector for selecting at least one model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image; a learning unit that performs machine learning on the selected model using a combination of the 3D sub-image and information indicating the position of the object within the 3D sub-image as training data; A model generating device comprising: 2-2. An extraction size determination unit that determines the one extraction size used to extract the 3D sub-image based on subject information in which an extraction size is associated with each of a plurality of subject IDs and the subject ID of the subject included in the 3D radar image. The model generating device according to 2-1. further comprises: 2-3. A subject ID determination unit that determines the subject ID used to determine the one extraction size by identifying the type of the subject included in the 3D radar image. The model generating device according to 2-2. further comprises: 2-4. The one extraction size used to extract the 3D sub-image is the subject extraction size for extracting the entire subject. The model generating device according to any one of 2-1 to 2-3. 2-5. The one extraction size used for extracting the 3D sub-image is a partial extraction size that extracts only a part of the subject; The reference position indicates a position of a part of the subject. The detection device according to any one of 2-1 to 2-3.

[0099] 3-1. Identify the subject's location in a 3D radar image; extracting a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of the extraction sizes designated for each type of subject; Selecting at least one trained model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image. Detecting an object in the 3D sub-image using the selected trained model. A computer implemented detection method comprising: 3-2. Determine the one extraction size used to extract the 3D sub-image based on subject information in which at least an extraction size is associated with each of a plurality of subject IDs and the subject ID of the subject included in the 3D radar image. The detection method according to 3-1., further comprising: 3-3. Determine the subject ID used to determine the one extraction size by identifying the type of the subject included in the 3D radar image. The detection method according to 3-2., further comprising: 3-4. The one extraction size used to extract the 3D sub-image is the subject extraction size for extracting the entire subject. The detection method according to any one of 3-1. to 3-3. 3-5. The one extraction size used for extracting the 3D sub-image is a partial extraction size that extracts only a part of the subject; The reference position indicates a position of a part of the subject. The detection method according to any one of 3-1. to 3-3. 3-6. Output at least one of information indicating the presence or absence of the object, a class of the detected object, and position information of the object. The detection method according to any one of 3-1. to 3-5., further comprising:

[0100] 4-1. Identify the subject's location in a 3D radar image; extracting a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of the extraction sizes designated for each type of subject; selecting at least one trained model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image; Detecting an object in the 3D sub-image using the selected trained model. A program for causing a computer to execute a detection method including the steps of: 4-2. The detection method further includes determining the one extraction size used to extract the 3D sub-image based on subject information in which an extraction size is associated with each of a plurality of subject IDs and the subject ID of the subject included in the 3D radar image. The program described in 4-1. 4-3. The detection method further includes determining the subject ID used to determine the one extraction size by identifying the type of the subject included in the 3D radar image. The program described in 4-2. 4-4. The one extraction size used to extract the 3D sub-image is the subject extraction size for extracting the entire subject. A program according to any one of 4-1. to 4-3. 4-5. The one extraction size used for extracting the 3D sub-image is a partial extraction size that extracts only a part of the subject; The reference position indicates a position of a part of the subject. A program according to any one of 4-1. to 4-3. 4-6. The detection method further includes outputting at least one of information indicating the presence or absence of the object, a class of the detected object, and position information of the object. A program according to any one of 4-1. to 4-5. [Explanation of symbols]

[0101] 100 Detection device 200 Model Generator 12 Location identification part 14 Extraction part 16 Model Selection Section 18 Detection section 101, 201, 301 Radar image DB storage section 102,202,302 Subject Finder 103,203,303 Image Extraction Section 104,304 Detector 22 Location identification part 24 Extraction part 26 Model Selection Section 28 Learning Department 204 Learning Department 105,205,305 Subject DB storage section 106,206,306 Approach Selection Section 107,207,307 Network selection section 117,217,317 Network architecture DB memory section 209 Annotation Adjustment Department 208 Annotation DB storage section 310 Subject Identifier

Claims

1. a position identification unit for identifying a position of a subject in a 3D radar image; an extraction unit that extracts a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of extraction sizes designated for each type of the subject; A model selection unit that selects at least one trained model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image; a detector for detecting an object in the 3D sub-image using the selected trained model; A detection device comprising:

2. an extraction size determination unit that determines the one extraction size used to extract the 3D sub-image based on subject information in which at least an extraction size is associated with each of a plurality of subject IDs and the subject ID of the subject included in the 3D radar image; The detection device of claim 1 further comprising:

3. a subject ID determination unit for determining the subject ID used to determine the one extraction size by identifying the type of the subject included in the 3D radar image; The detection device of claim 2 further comprising:

4. The one extraction size used to extract the 3D sub-image is a subject extraction size for extracting the entire subject. The detection device according to any one of claims 1 to 3.

5. the one extraction size used to extract the 3D sub-image is a partial extraction size that extracts only a portion of the subject; The reference position indicates a position of a part of the subject. The detection device according to any one of claims 1 to 3.

6. The detection unit outputs at least one of information indicating the presence or absence of the object, a class of the detected object, and position information of the object. The detection device according to any one of claims 1 to 5.

7. a position identification unit for identifying a position of a subject in a 3D radar image; an extraction unit that extracts a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of extraction sizes designated for each type of the subject; a model selection unit for selecting at least one model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image; a learning unit that performs machine learning on the selected model using a combination of the 3D sub-image and information indicating the position of the object within the 3D sub-image as training data; A model generating device comprising:

8. an extraction size determination unit that determines the one extraction size used to extract the 3D sub-image based on subject information in which at least an extraction size is associated with each of a plurality of subject IDs and the subject ID of the subject included in the 3D radar image; The model generating device of claim 7 further comprising:

9. a subject ID determination unit for determining the subject ID used to determine the one extraction size by identifying the type of the subject included in the 3D radar image; The model generating device of claim 8 further comprising:

10. The one extraction size used to extract the 3D sub-image is a subject extraction size for extracting the entire subject. The model generating device according to any one of claims 7 to 9.

11. the one extraction size used to extract the 3D sub-image is a partial extraction size that extracts only a portion of the subject; The reference position indicates a position of a part of the subject. The detection device according to any one of claims 7 to 9.

12. Locating a subject in a 3D radar image; extracting a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of the extraction sizes designated for each type of subject; Selecting at least one trained model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image. Detecting an object in the 3D sub-image using the selected trained model A computer implemented detection method comprising:

13. determining the one extraction size used to extract the 3D sub-image based on subject information in which at least an extraction size is associated with each of a plurality of subject IDs and the subject ID of the subject included in the 3D radar image; The method of claim 12, further comprising:

14. Determining the subject ID used to determine the one sample size by identifying the type of subject included in the 3D radar image. The method of claim 13, further comprising:

15. The one extraction size used to extract the 3D sub-image is a subject extraction size for extracting the entire subject. The detection method according to any one of claims 12 to 14.

16. the one extraction size used to extract the 3D sub-image is a partial extraction size that extracts only a portion of the subject; The reference position indicates a position of a part of the subject. The detection method according to any one of claims 12 to 14.

17. Outputting at least one of information indicating the presence or absence of the object, a class of the detected object, and position information of the object. The detection method according to any one of claims 12 to 16, further comprising:

18. Locating a subject in a 3D radar image; extracting a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of the extraction sizes designated for each type of subject; selecting at least one trained model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image; Detecting an object in the 3D sub-image using the selected trained model A program for causing a computer to execute a detection method including the steps of:

19. The detection method further includes determining the one extraction size used to extract the 3D sub-image based on subject information, in which an extraction size is associated with each of a plurality of subject IDs, and the subject ID of the subject included in the 3D radar image.

20. The program of claim 18.

20. The method further includes determining the subject ID, which is used to determine the sample size, by identifying the type of the subject included in the 3D radar image.

20. The program of claim 19.

21. The one extraction size used to extract the 3D sub-image is a subject extraction size for extracting the entire subject. The program according to any one of claims 18 to 20.

22. the one extraction size used to extract the 3D sub-image is a partial extraction size that extracts only a portion of the subject; The reference position indicates a position of a part of the subject. The program according to any one of claims 18 to 20.

23. The detection method further includes outputting at least one of information indicating the presence or absence of the object, a class of the detected object, and location information of the object. The program according to any one of claims 18 to 22.

Images