Detection device, model generation device, detection method and program

Abstract

The detection device (100) of the first embodiment includes a position identification unit (12), an extraction unit (14), a model selection unit (16), and a detection unit (18). The position identification unit (12) identifies a position of a subject in a 3D radar image. The extraction unit (14) extracts a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of the extraction sizes specified for each type of the subject. The model selection unit (16) selects at least one trained model based on at least one of the size of the 3D sub-image and the type of the subject included in the 3D radar image. The detection unit (18) detects an object in the 3D sub-image using the selected trained model.

Description

[Technical Field]

[0001] The present invention relates to a radar image object detection system, where imaging is performed to measure radio waves scattered by a moving target object and generate a 3D radar image of the target being scanned, which is then used by a deep learning module to detect, i.e. check for, the presence of hidden threats. [Background technology]

[0002] An example of a conventional radar image object detection system is described in Non-Patent Document 1. This conventional object detection system includes a radar signal measurement means, an image generation means, and an object detection means. Specifically, the measurement means includes a radar antenna that transmits radar waves and receives reflected and scattered waves. The generated 3D radar image is projected into 2D and used by a deep learning module to detect the presence or absence of a target object in the radar image. In Non-Patent Document 1, the radar image object detection system is used to detect concealed weapons.

[0003] Patent Document 1 describes setting an image processing area for extracting an image portion of a surveillance object according to the type of surveillance object.

[0004] Patent Document 2 discloses that a part of an image based on the results of a preliminary inspection is used to determine whether a target person is carrying a prohibited item. Patent Document 2 also discloses that machine learning can be used as an example of a determination method based on the shape of an object reflected in a transparent image. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2008 / 139529 [Patent Document 2] International Publication No. 2021 / 166150 [Non-patent literature]

[0006] [Non-Patent Document 1] L. Carrer, "Concealed Weapon Detection: A microwave imaging approach", Master of Science Thesis, Delft University of Technology, 2012 Summary of the Invention [Problem to be solved by the invention]

[0007] However, the technology of Non-Patent Document 1 takes a long time to detect objects. The technology of Patent Document 1 cannot be applied to the use of a trained model based on machine learning. Furthermore, Patent Document 2 does not disclose a technology for shortening processing time without degrading the performance of the determination unit by using a trained model.

[0008] One example of the objective of the present invention is to reduce the processing time for object detection without compromising detection accuracy when a trained model is used for detection. [Means for solving the problem]

[0009] The present invention provides a location determination unit for determining a location of a subject in a 3D radar image; an extraction unit that extracts a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of extraction sizes designated for each type of the subject; a model selection unit that selects at least one trained model based on at least one of the size of the 3D sub-image and the type of the subject included in the 3D radar image; a detector that detects objects in the 3D sub-images using the selected trained model; A detection device comprising: to provide.

[0010] The present invention provides a location determination unit for determining a location of a subject in a 3D radar image; an extraction unit that extracts a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of extraction sizes designated for each type of the subject; a model selector that selects at least one model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image; a learning unit that performs machine learning on the selected model using a combination of the 3D sub-image and information indicating the position of the object within the 3D sub-image as training data; and A model generating device comprising: to provide.

[0011] The present invention provides Identifying the subject's location in a 3D radar image; extracting a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of extraction sizes specified for each type of subject; selecting at least one trained model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image; Detecting an object in the 3D sub-image using the selected trained model. A computer-implemented detection method comprising: to provide.

[0012] The present invention provides Identifying the subject's location in a 3D radar image; extracting a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of extraction sizes designated for each type of subject; Selecting at least one trained model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image. Detecting an object in the 3D sub-image using the selected trained model. A program for causing a computer to execute a detection method including to provide. [Effects of the Invention]

[0013] An object of the present invention is to reduce the processing time for object detection without compromising detection accuracy when a trained model is used for detection. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram illustrating the setup of a radar image measurement system and the relative position of a target with respect to the radar. [Figure 2] FIG. 2 is a block diagram illustrating a functional configuration of the detection device according to the first embodiment. [Figure 3] FIG. 3 is a block diagram illustrating the hardware configuration of a computer that realizes the detection device according to the first embodiment. [Figure 4] FIG. 4 is a flowchart illustrating the flow of processing by the detection device of the first embodiment. [Figure 5] FIG. 5 is a block diagram showing an example of the functional configuration of the detection device according to the first embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of a table in the subject DB of the detection device according to the first embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of the configuration of the network architecture DB of the detection device according to the first embodiment. [Figure 8]FIG. 8 is a block diagram illustrating a function-based configuration of a detection device with a first example subject finder. [Figure 9] FIG. 9 is a diagrammatic representation showing an example of the operation of the detection device according to the first embodiment. [Figure 10] FIG. 10 is a block diagram illustrating a function-based configuration of a detection device including a subject finder according to the second embodiment. [Figure 11] FIG. 11 is a block diagram illustrating a function-based configuration of a detection device including a subject finder according to the third embodiment. [Figure 12] FIG. 12 is a diagram showing another example of a table of the subject DB of the detection device according to the first embodiment. [Figure 13] FIG. 13 is a flowchart illustrating the operation of the detection device according to the first embodiment. [Figure 14] FIG. 14 is a block diagram illustrating a function-based configuration of a model generating device according to the second embodiment. [Figure 15] FIG. 15 is a flowchart illustrating the flow of processing performed by the model generating device according to the second embodiment. [Figure 16] FIG. 16 is a block diagram showing an example of the configuration of the function base of the model generating device according to the second embodiment. [Figure 17] FIG. 17 is a flowchart illustrating the operation of the model generating device according to the second embodiment. [Figure 18] FIG. 18 is a block diagram showing an example of the functional configuration of a detection device according to the third embodiment. [Figure 19] FIG. 19 is a flowchart illustrating the operation of the detection device according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. For clarity of explanation, the same elements in the drawings will be denoted by the same reference numerals, and duplicate explanations will be omitted.

[0016] First embodiment

[0017] <Summary> First, the configuration of an object detection system 900 for radar images will be described with reference to FIG.

[0018] The object detection system 900 for radar images operates as follows: First, radar signals are measured. In the measurement step, the radar antenna transmits radar signals one by one in a specific order, and the reflected waves are received by the antenna receiver. The measured radar signals are used by the image generation means to create a 3D radar image using radar antenna information.

[0019] The specific purpose of this system is to determine whether a person (target) 90 is carrying a concealed dangerous object. The system 900 uses a fixed antenna (radar 92) installed on a side panel 94 to detect the target 90 as he or she walks through a screening area (area) 96. Transmitters in the antenna transmit signals one by one, and the received scattered signals are acquired. The system 900 also acquires camera images using a camera (camera 98) simultaneously with the radar signals. However, the system 900 according to this embodiment does not necessarily include a camera. The radar signals are processed using the antenna information to generate an inherently 3D radar image. If the person (target 90) is carrying a concealed dangerous object, it will appear in the radar image. In this way, the radar image is used to detect the concealed dangerous object. To detect the presence of a dangerous object from the radar image, a trained model based on machine learning is utilized. The trained model may be included in a deep learning module. As can be seen, the entire setup is expected to function in real time, since the presence or absence of a concealed object is expected to be acquired while the target 90 is still nearby. However, processing by the trained model is generally a bottleneck. Because the input images to the trained model are inherently 3D, the processing time is expected to be longer due to the increased computational complexity of 3D. Existing methods to reduce processing time, as suggested in the background art, involve scaling or downsampling 3D images, or simply projecting 3D images into 2D, thereby resizing the 3D images to a smaller size. However, this is not a good approach because it may also affect the performance of the trained model due to information loss.

[0020] As mentioned above, the detection unit using the trained model is required to function in real time. This is because our target is preferably moving, and we aim to detect dangerous objects while the target 90 is still near the detection system 900. As mentioned above, the input image to the trained model is inherently 3D, and the complexity of 3D calculations makes it difficult to obtain predictions in real time. According to the detection device 100 of this embodiment, the processing time of the trained model can be reduced without compromising performance for radar images. This is achieved by using a 3D radar image (3D subimage) of the extracted subject as input to the trained model instead of the original 3D image. The 3D image of the extracted subject is reduced in size compared to the original 3D radar image, but contains the same information. The processing time of the trained model is sensitive to the input size, especially in the case of 3D, so the processing time is reduced. In particular, the present disclosure relates to a subject extraction system for 3D radar images that reduces the processing time of the trained model by reducing the input image size without affecting performance. Preferably, the object detection system 900 for radar images is capable of functioning in real time.

[0021] Note that a subject refers to a target, and an object refers to the object about which you want to know the presence or absence information. The object may be part of the subject, but they do not have to be the same.

[0022] <Example of function-based configuration> FIG. 2 is a diagram illustrating an example of a function-based configuration of the detection device 100 according to the first embodiment. The detection device 100 according to the first embodiment includes a position identification unit 12, an extraction unit 14, a model selection unit 16, and a detection unit 18. The position identification unit 12 identifies the position of a subject in a 3D radar image. The extraction unit 14 extracts a 3D subimage from the 3D radar image using a reference position based on the identified position of the subject and one of extraction sizes determined for each type of subject. The model selection unit 16 selects at least a trained model based on at least one of the size of the 3D subimage and the type of subject included in the 3D radar image. The detection unit 18 detects an object in the 3D subimage using the selected trained model. This will be described in detail below.

[0023] The detection device 100 of this embodiment may be included in an object detection system 900. An example of the object is a dangerous object (such as a knife or a gun) carried by a target 90.

[0024] First, the acquisition of a radar image will be described with reference to FIG. 1. A person (target 90) is assumed to be walking through a screening area 96 in front of a fixed radar antenna (radar 92) installed on a side panel 94. When in the screening area, the target may also be within the field of view of a camera 98, which captures an image synchronized with the radar sensor. The measured scattered radar signal is transmitted for imaging. The generated radar image is inherently 3D. The 3D radar image is generated from the measured scattered radar signal. The 3D radar image may be generated within the detection device 100 or by another device. The generated 3D radar image is stored in a radar image database (DB). The radar image DB is realized by one or more storage devices. The radar image DB may or may not be included in the detection device 100.

[0025] The generated radar image is used to detect whether the target is carrying a dangerous substance. The detection unit 18 detects objects from the 3D image using a trained model. The trained model may include a deep learning network. To obtain predictions from the deep learning network in a short time, preferably in real time, the extraction unit extracts smaller images provided as input to find the presence of objects. The detection device 100 is also called a subject extraction device.

[0026] Here, the term "subject" refers to the whole or part of the object from which the image is extracted, and the term "object" refers to, for example, a dangerous object. The subject refers to the subject of the image, i.e., the object that occupies the largest area / volume in the image, such as a living body like a human or a moving object like a car. Without loss of generality, there may be multiple subjects.

[0027] In this embodiment, it is assumed that the identity of the subject is known to the detection device 100 as prior information. For example, a 3D radar image is associated with a subject ID and stored in the radar image DB. The subject ID indicates the type of subject, such as adult, child, or elderly. The subject ID may also indicate the type or make of vehicle. The subject ID of each image is identified by other means, such as a sensor that detects the characteristics of the target 90 or a camera that captures the target 90, and is stored in the radar image DB.

[0028] <Hardware configuration example> In some embodiments, each functional unit included in detection device 100 may be implemented by at least one hardware component, and each hardware component may implement one or more functional units. In some embodiments, each functional unit may be implemented by at least one software component. In some embodiments, each functional unit may be implemented by a combination of hardware and software components.

[0029] The detection device 100 may be realized by a dedicated computer manufactured for implementing the detection device 100, or may be realized by a general-purpose computer such as a personal computer (PC), a server machine, or a mobile device.

[0030] FIG. 3 is a block diagram showing an example of the hardware configuration of a computer 1000 that realizes the detection device 100 of the first embodiment. In FIG. 3, a computer 1000 includes a bus 1020 , a processor 1040 , a memory 1060 , a storage device 1080 , an input / output (I / O) interface 1100 , and a network interface 1120 .

[0031] The bus 1020 is a data transmission path for the processor 1040, the memory 1060, and the storage device 1080 to transmit and receive data to and from each other. The processor 1040 is a processor such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an FPGA (Field-Programmable Gate Array). The memory 1060 is a primary storage device such as a RAM (Random Access Memory). The storage medium 1080 is a secondary storage device such as a hard disk drive, an SSD (Solid State Drive), or a ROM (Read Only Memory).

[0032] The I / O interface is an interface between the computer 1000 and peripheral devices such as a keyboard, mouse, display device, etc. The network interface is an interface between the computer 1000 and a communication line that enables the computer 1000 to communicate with other computers.

[0033] The storage device 1080 may store program modules that realize the respective functional units of the detection device 100. The CPU 1040 executes the respective program modules, thereby realizing the respective functional units of the detection device 100.

[0034] <Processing flow> FIG. 4 is a flowchart illustrating a processing procedure of the detection device 100 according to the first embodiment.

[0035] The detection method according to the first embodiment is executed by a computer. This detection method includes a position identification step (S12), an extraction step (S14), a model selection step (S16), and a detection step (S18). In the position identification step, the position identification unit 12 identifies the position of a subject in a 3D radar image. In the extraction step, the extraction unit 14 extracts a 3D subimage from the 3D radar image using a reference position based on the identified position of the subject and one of extraction sizes defined for each type of subject. In the model selection step, the model selection unit 16 selects at least a trained model based on at least one of the size of the 3D subimage and the type of subject included in the 3D radar image. In the detection step, the detection unit 18 detects an object in the 3D subimage using the selected trained model.

[0036] <Configuration Example of Detection Device 100> An example of the configuration of the radar image detection device 100 according to the first embodiment will be described in detail with reference to the block diagram of FIG.

[0037] As shown in FIG. 5, an example configuration of the detection device 100 according to the first embodiment includes a radar image DB storage unit 101, a subject finder 102, an image extraction unit 103, a detection unit 104, a subject DB storage unit 105, an approach selection unit 106, a network selection unit 107, and a network architecture DB storage unit 117.

[0038] The subject finder 102 functions as the location identification unit 12. The image extraction unit 103 functions as the extraction unit 14. The network selection unit 107 functions as the model selection unit 16. The detection unit 104 functions as the detection unit 18.

[0039] The radar image DB storage unit 101 may or may not be included in the detection device 100. The subject DB storage unit 105 may or may not be included in the detection device 100. The network architecture DB storage unit 117 may or may not be included in the detection device 100.

[0040] In the example shown in this figure, the detection device 100 further includes an extraction size specifying unit. The extraction size specifying unit specifies an extraction size to be used for extracting a 3D sub-image based on subject information in which at least an extraction size is associated with each of a plurality of subject IDs and the subject IDs of the subjects included in the 3D radar image. The approach selection unit 106 in FIG. 5 functions as the extraction size specifying unit.

[0041] Next, the operation of the detection device 100 will be described. Note that this operation describes the prediction phase of the deep learning network. However, without loss of generality, the same can be extended to the training phase. During operation, 3D radar images are measured as described above, stored in the radar image DB storage unit 101, and then retrieved one by one from there. The 3D radar images are then sent to the subject finder 102. The subject finder 102 identifies the location of the subject(s) and passes the subject locations along with the 3D radar images to the image extraction unit 103. The subject finder 102 receives a means for finding the subject from the approach selection unit 106. The approach selection unit 106 also outputs the extracted image size (extraction size) to the image extraction unit 103. The image extraction unit 103 uses the subject location information to extract 3D sub-images of the size output from the approach selection unit 106 and passes them to the detection unit 104. The detection unit 104 can use a deep learning module for object detection. The approach selection unit 106 also passes the extracted image size to the network selection unit 107, which selects an appropriate deep learning network architecture from the DB based on the extracted image size and / or subject ID. The selected network architecture is provided as input to the detection unit 104, which also receives smaller-sized extracted images (3D sub-images) to detect objects. The detection unit 104 inputs the 3D sub-images into the network architecture (trained model) and obtains a prediction result as the output of the network architecture (trained model). The output of the detection unit 104 indicates the presence or absence information of the object (and possibly its location).

[0042] The detection unit 18 may output at least one of information indicating the presence or absence of an object, a class of the detected object, and location information of the object. In this configuration, the expected output from the detection unit 104 is presence or absence information of an object of interest (hereinafter referred to as "object"), which may be part of the subject, but need not be the same as the subject. The object presence or absence information can take the form of image-level classes (classifiers) or pixel-level classes (segmentations). Additionally, the location of the object can also be provided as an output.

[0043] The radar image DB storage unit 101 provides radar images. The radar image DB storage unit 101 stores measured and generated 3D radar images. It functions as a data source by providing the 3D radar images as input to the subject finder 102.

[0044] The subject DB storage unit 105 stores a plurality of subject sizes in advance. Each size is associated with a subject ID. In this configuration example, the subject DB storage unit 105 stores various information about the subject (size, etc.) in a table format in association with the subject ID. The subject DB storage unit 105 can look up subject information using the ID as a primary key. An example of subject information is the size of the subject, as shown in the example table of the subject DB in FIG. 6. Here, the subject size in the subject DB is the size of the subject expected in the 3D radar image.

[0045] Furthermore, the subject DB storage unit 105 pre-stores multiple means for locating a subject. Each means for locating a subject is associated with a subject ID. In this configuration example, the approach selection unit 106 determines an approach for locating a subject based on subject information acquired from the subject DB storage unit 105. The approach selection unit 106 outputs the means for locating a subject to the subject finder 102. Furthermore, the approach selection unit 106 outputs the extracted image size and subject ID to the image extraction unit 103 and the network selection unit 107 based on the subject information. The means for locating a subject depends on the design of the subject finder 102, and some examples of the means may include a projection axis, a position axis (the axis along which the position can be found), etc. Some configuration examples of the approach selection unit 106 will be described together with the configuration example of the subject finder 102.

[0046] When a 3D radar image is associated with a subject ID and stored in the radar image DB storage unit 101, the approach selection unit 106 acquires the subject ID of the 3D radar image to be processed. Based on the acquired subject ID, the approach selection unit 106 reads from the subject DB storage unit 105 a means for determining the size and subject position of the subject to which the 3D radar image is applied.

[0047] The network architecture DB storage unit 117 pre-stores multiple trained models. Each trained model is associated with at least one of the size of a 3D subimage and a subject ID. In this configuration example, the network architecture DB storage unit 117 provides architectures (for various image sizes, subject types, etc.). The network architecture DB storage unit 117 includes various trained network architectures for various image sizes, subject types, etc. The network architectures are pre-trained for classification, object detection, and / or segmentation tasks and can be distinguished by input image size and / or subject ID. The network architecture DB storage unit 117 can examine the network architectures, for example, by using metadata such as extracted image size and / or subject ID. An example configuration of the network architecture DB stored in the network architecture DB storage unit 117 is shown in FIG. 7, where different network architectures can be distinguished by metadata information.

[0048] The network selector 107 selects a network architecture for the detector 104 based on the extracted image size and / or subject ID received from the approach selector 106. The network architecture is selected from the network architecture DB storage 117 using the extracted image size and / or subject ID as a search key. The network selector 107 outputs the network architecture to the detector 104.

[0049] It is understood that using a network architecture for an image size different from the one it was trained on will lead to performance degradation. Here, for the aforementioned processing time and performance-related reasons, it is assumed that images are provided as input to the network architecture without resizing. Therefore, as the extracted image size changes, the input image size to the network architecture needs to change, and subsequently the network architecture needs to be modified. Furthermore, if different types of subjects exist, a dedicated network architecture is required to detect objects of each subject type. The given reasons demonstrate the need for the network selection unit 107.

[0050] The detection unit 104 obtains object presence / absence information in the radar image based on the selected network architecture from the network selection unit 107 and the extracted 3D sub-images from the image extraction unit 103. The detection unit 104 outputs the object presence / absence information. As mentioned above, the object may be part of the aforementioned subject, but it does not have to be the same. Furthermore, the deep learning module can be a classifier (outputting single / multiple classes per image), an object detector (outputting object class and location per image), or a segmentation network (outputting a pixel-level class map with the same size as the input image). The detection unit 104 can output the object presence / absence information in the form of image-level classes (classification) or pixel-level classes (segmentation). In addition to the presence / absence information, the object location can also be output.

[0051] The image extraction unit 103 extracts a 3D radar image of a subject based on the subject position received from the subject finder 102 and the extracted image size received from the approach selection unit 106. The image extraction unit 103 receives the original 3D radar image from the subject finder 102 and performs extraction. The image extraction unit 103 extracts an image from the original 3D radar image using the received subject position. The image extraction unit 103 outputs the extracted image (i.e., a 3D subimage) to the detection unit 104. The subject position may indicate, for example, the center position of the subject or one of its corners. The extracted 3D subimage may be an image obtained by simply cropping a portion of the 3D radar image. Each 3D subimage includes only a portion of the subject or the entire subject. The image extraction unit 103 may generate multiple 3D subimages from a single 3D radar image. The extracted 3D subimage and the 3D radar image have the same image quality, resolution, etc. The crop position in the 3D radar image is determined based on the subject position received from the subject finder 102. The size of the 3D sub-image is determined based on the crop size received from the approach selector 106.

[0052] The subject finder 102 determines the position of the subject in the 3D radar image based on the subject finding means received from the approach selection unit 106 and the original 3D radar image read from the radar image DB storage unit 101. The subject finder 102 outputs the subject position to the image extraction unit 103. The subject finder 102 also outputs the original radar image to the image extraction unit 103.

[0053] <Configuration of Subject Finder 102> Next, the subject finder 102 and the approach selector 106 will be described in detail using some example configurations.

[0054] <<First Example of Subject Finder 102>> In this example, the extraction size used to extract the 3D sub-images is the subject extraction size for extracting the entire subject. The reference position indicates the position of the subject.

[0055] A first example configuration of the subject finder 102 is described with reference to FIG. 8, in which the subject finder 102 consists of two sub-blocks: a 2D projector 102a and a subject location finder 102b. While this technique is described assuming a single subject, the same can be extended to multiple subjects. In this example, a 2D image is first generated by projection, and then processed to identify the subject's location. The 2D projector 102a outputs the projected 2D image to the subject location finder 102b based on a projection axis (the means for finding the subject) from the approach selector 106. The 2D image is generated by projecting a 3D image along the projection axis. The projection can be a maximum projection, an energy projection, or other projection. The subject location finder 102b outputs the subject's location in the 2D image based on the 2D projected image from the 2D projector 102a and the location axis (the means for finding the subject) from the approach selector 106. The location can be found in various ways, for example by finding the point of maximum intensity. The subject location is provided as an output from the subject finder 102 to the image extractor 103. An example of the overall operation for obtaining an extracted radar image from the entire radar image is illustrated in Figure 9.

[0056] <<Second Example of Subject Finder 102>> The subject finder 102 may consist of an image processor 102e that outputs a processed 2D image based on several image processing algorithms, e.g., filtering, sharpening, etc. The purpose of the image processor 102e is to support the operations of the subject location finder block, e.g., clustering, peak detection, etc., as shown in Figure 10.

[0057] <<Third Example of Subject Finder 102>> In this example, the extraction size used to extract the 3D sub-image is a partial extraction size that extracts only a portion of the subject, and the reference position indicates the position of the portion of the subject.

[0058] This configuration example of subject finder 102, which is a special case of the first example in which subject finder 102 is supposed to find (pre-known) parts of a subject, will be described with reference to FIG. 11. The configurations of 2D projector 102a and subject position finder 102b are similar to those described in the first example above, and therefore will not be described again. The additional block here is body part finder 102c, which outputs the position of a subject body part to image extraction unit 103 based on the subject position received from subject position finder 102b and the relative position of the subject body part with respect to the subject received from approach selection unit 106. Body part finder 102c can output the (global) position of a subject body part when it has the (global) position information of the subject and the relative position of the body part with respect to the subject using simple coordinate mathematics. In this configuration example, subject DB storage unit 105 pre-stores multiple combinations of the relative position of the subject body part and the size of the subject body part. Each combination is associated with a subject ID. Furthermore, the approach selection unit 106 uses the subject ID to read the relative position of the subject's body part with respect to the subject and the size of the subject's body part from the subject DB storage unit 105. An example of the structure of a table used in the subject DB storage unit 105 is shown in FIG.

[0059] In this configuration example, the image extraction unit outputs extracted images (i.e., 3D sub-images) of the subject's body parts to the detection unit 104 based on the position of the subject body part received from the body part finder 102c and the extracted image size of each body part received from the approach selection unit 106. The approach selection unit 106 reads the relative position and size of the subject body part from the subject DB storage unit 105 based on the subject ID. The detection unit 104 further receives a network (which may be the same or different) for each subject body part selected by the network selection unit 107. The detection unit 104 then analyzes the image of each subject body part individually and outputs information on the presence or absence of an object in the extracted image.

[0060] Next, an example of the operation in the operation mode of the radar image detection device 100 according to the first embodiment will be described with reference to the flowchart shown in Fig. 13. Subject information (sizes associated with subject IDs) is stored in the subject DB storage unit 105. Furthermore, various network architectures are pre-trained for different image sizes and stored in the network architecture DB storage unit 117.

[0061] When the detection device 100 is started, in step S101, a 3D radar image is read from the radar image DB storage unit 101. In step S105, subject information is read from the subject DB storage unit 105. Next, the approach selection unit 106 identifies an approach, i.e., what means is needed to identify the subject's location, and provides this as output to the image extraction unit 103 (step S106). The network selection unit 107 selects a network architecture from the network architecture DB storage unit 117 and outputs it to the detection unit 104 (step S107). The subject finder 102 finds the subject location and outputs it to the image extraction unit 103 (step S102). The image extraction unit 103 extracts an image using the subject location and the extracted image size, and outputs the extracted image to the detection unit 104 (step S103). The detection unit 104 predicts the presence or absence of an object in the extracted 3D sub-image using classification or segmentation (step S104).

[0062] As described above, the detection device 100 according to the first embodiment of the present disclosure extracts images of smaller size using subject location information. This reduces the processing time of the trained model, i.e., the detection unit, for predicting object presence / absence information, enabling desired real-time operation. It can be seen that the reduction in processing time of the trained model due to image size reduction is far greater than the slight increase in processing time of the subject finder 102 (which performs image processing operations).

[0063] Second embodiment

[0064] FIG. 14 is a diagram illustrating an example of a function-based configuration of a model generating device 200 according to the second embodiment. The model generating device 200 includes a position identifying unit 22, an extraction unit 24, a model selecting unit 26, and a learning unit 28. The position identifying unit 22 identifies the position of a subject in a 3D radar image. The extraction unit 24 extracts a 3D subimage from the 3D radar image using a reference position based on the identified subject's position and one of the extraction sizes defined for each type of subject. The model selecting unit 26 selects at least a model based on at least one of the size of the 3D subimage and the type of subject included in the 3D radar image. The learning unit 28 performs machine learning on the selected model using a combination of the 3D subimage and information indicating the position of an object in the 3D subimage as learning data. This will be described in detail below.

[0065] The position specifying unit 22, the extraction unit 24, and the model selecting unit 26 are similar to the position specifying unit 12, the extraction unit 14, and the model selecting unit 16 according to the first embodiment, respectively.

[0066] The model generation device 200 can generate or update a trained model used in the detection device 100 according to the first embodiment. The trained model generated or updated by the model generation device 200 may be stored in the network architecture DB storage unit 117 of the detection device 100.

[0067] The model generation device 200 may also function as the detection device 100 described in the first embodiment. That is, the detection device 100 may include an annotation adjustment unit 209, and the detection unit 104 may also function as the learning unit 204. In this case, the performance of the detection device 100 may be evaluated by comparing the output of the trained model with information indicating the position of an object in a 3D subimage as ground truth data.

[0068] <Hardware configuration example> In some embodiments, each functional unit included in model generator 200 may be implemented by at least one hardware component, and each hardware component may implement one or more functional units. In some embodiments, each functional unit may be implemented by at least one software component. In some embodiments, each functional unit may be implemented by a combination of hardware and software components.

[0069] The model generating device 200 may be realized by a dedicated computer manufactured for the purpose of realizing the model generating device 200, or may be realized by a general-purpose computer such as a personal computer (PC), a server machine, or a mobile device.

[0070] The model generating device 200 may be realized by a computer 1000 shown in Fig. 3. The storage device 1080 may store program modules that realize each functional unit of the model generating device 200. Each functional unit of the model generating device 200 is realized by the CPU 1040 executing each program module.

[0071] <Processing flow> FIG. 15 is a flowchart showing the processing procedure of the model generating device 200 of the second embodiment.

[0072] The model generation method according to the second embodiment is executed by a computer. The model generation method includes a position identification step (S22), an extraction step (S24), a model selection step (S26), and a learning step (S28). In the position identification step, the position identification unit 22 identifies the position of a subject in a 3D radar image. In the extraction step, the extraction unit 24 extracts a 3D subimage from the 3D radar image using a reference position based on the identified position of the subject and an extraction size determined for each type of subject. In the model selection step, the model selection unit 26 selects at least a model based on at least one of the size of the 3D subimage and the type of subject included in the 3D radar image. In the learning step, the learning unit 28 performs machine learning on the selected model using a combination of the 3D subimage and information indicating the position of an object in the 3D subimage as learning data.

[0073] <Configuration example of model generation device 200> Next, with reference to the block diagram of FIG. 16, an example of the configuration of the model generating device 200 according to the second embodiment will be described in detail.

[0074] 16 , the configuration of the model generation device 200 according to the second embodiment can include a radar image DB storage unit 201, a subject finder 202, an image extraction unit 203, an annotation DB storage unit 208, an annotation adjustment unit 209, a learning unit 204, a subject DB storage unit 205, an approach selection unit 206, a network selection unit 207, and a network architecture DB storage unit 217. The annotation DB storage unit 208 may or may not be included in the model generation device 200. The configurations and functions of the radar image DB storage unit 201, the subject finder 202, the subject DB storage unit 205, the approach selection unit 206, and the network selection unit 207 are the same as the configurations and functions of the radar image DB storage unit 101, the subject finder 102, the subject DB storage unit 105, the approach selection unit 106, and the network selection unit 107 according to the first embodiment, respectively. Therefore, description thereof will not be repeated. Moreover, the image extraction unit 203 is the same as the image extraction unit 103 according to the first embodiment, except for the points described below.

[0075] The subject finder 202 functions as a location identification unit 22. The image extraction unit 203 functions as an extraction unit 24. The network selection unit 207 functions as a model selection unit 26. The learning unit 204 functions as a learning unit 28.

[0076] The configuration of a model generation device 200 according to the second embodiment will be described with reference to the block diagram of FIG. 16. In this configuration, too, it is assumed that the identity of the subject is known to the subject finder 202 as a priori information. The expected output may be the location of the object in addition to presence / absence information. This object may be part of the subject, but need not be the same as the subject. The presence / absence and location information of the object can be expressed in the form of image-level classes (classification) or bounding boxes (object detection).

[0077] In this particular embodiment, the ground truth object positions are known in advance and the goal is to evaluate the performance of the detection apparatus 100 (prediction phase), or to improve the performance of the detection apparatus 100 (training phase), or both (online prediction and training). Information indicating the position of the object in the 3D sub-image (ground truth data) is obtained using the ground truth object positions, as described below.

[0078] The ground truth position and presence information of an object are collectively called annotation. Because the position of the subject in the extracted image and the size of the extracted image also change, the object position information also needs to be adjusted. The purpose of the second embodiment is to adjust the position of the object according to the position of the subject in the extracted image (hereinafter referred to as annotation adjustment).

[0079] The annotation DB storage unit 208 stores annotation information for all radar images contained in the radar image storage unit 201 so that the information can be distinguished by the name of the radar image. The subject's position is identified by another method, and annotation information is prepared in advance.

[0080] The annotation adjustment unit 209 outputs adjusted annotations to the learning unit 204 based on the subject position and the extracted image size received from the image extraction unit 203. The ground truth annotation information is read from the annotation DB storage unit 208. The annotation adjustment unit 209 also receives the original radar image size as part of the annotation information to assist in annotation adjustment. In this example, the ground truth position is specified by a rectangular bounding box for the original radar image. Adjustment means moving the center of the bounding box based on the subject position and adjusting the size based on the sizes of the extracted image and the original image.

[0081] The network architecture DB storage unit 217 includes various trained or untrained network architectures distinguishable by different input image sizes and / or subject IDs for classification and / or object detection. The architectures may or may not be pre-trained depending on the task at hand, such as training, performance evaluation, etc. The network selector 207 selects a network architecture using the network architecture DB storage unit 217, as described for the network selector 107 in the first embodiment of the present disclosure.

[0082] The training unit 204 obtains a selected network architecture including a model from the network selection unit 207. The model includes a neural network. The training unit 204 inputs the 3D sub-images obtained from the image extraction unit 203 into the network architecture (model). The training unit 204 may output object location in addition to presence / absence information based on the network architecture selected by the network selection unit 207 and the extracted images from the image extraction unit 203. Furthermore, the training unit 204 also receives adjusted annotation information from the annotation adjustment unit 209, which can be used to update architecture parameters (training) or evaluate the performance of the architecture, or both. In training, a combination of the 3D sub-images and the adjusted annotation information is used as training data. The training unit 204 can output object presence / absence and location information in the form of image-level classes (classification) and bounding boxes (object detection).

[0083] Next, an example of operation in operation mode of the model generation device 200 according to the second embodiment will be described with reference to the flowchart shown in Fig. 17. Subject information (sizes associated with IDs) is stored in the subject DB storage unit 205. Also, various network architectures corresponding to different image sizes are stored in the network architecture DB storage unit 217. Note that steps S201, S202, S205, S206, and S207 are similar to steps S101, S102, S105, S106, and S107 described in the first embodiment of the present disclosure, and therefore description thereof will be omitted.

[0084] The image extraction unit 203 extracts an image using the subject position and the extracted image size and outputs it to the learning unit 204. The image extraction unit 203 further outputs the subject position and the extracted image size to the annotation adjustment unit 209 (step S203). The annotation adjustment unit 209 reads out annotations prepared for processing 3D radar images from the annotation DB storage unit 208 and adjusts the annotations using the input subject position and extracted image size. The annotation adjustment unit 209 then outputs the adjusted annotations to the learning unit 204 (step S209). The learning unit 204 receives the extracted image from the image extraction unit 203 and predicts (may use classification / object detection) and outputs information on the presence or absence of an object and its position (step S204).

[0085] As described above, the model generation device 200 according to the second embodiment of the present disclosure extracts smaller images using subject position information. This reduces the processing time required to acquire object presence and position information, enabling desired real-time operation. Furthermore, the annotation adjustment function allows for evaluation and / or updating of learner performance, preferably in real time.

[0086] Third embodiment

[0087] The detection device 100 of the third embodiment is similar to the detection device 100 of the first embodiment, except for the points described below.

[0088] The detection device 100 of the third embodiment further includes a subject ID determination unit that identifies the type of subject included in the 3D radar image and determines a subject ID for determining one extraction size, as will be described in detail below.

[0089] A configuration example of a radar image detection device 100 according to a third embodiment of the present disclosure will be described with reference to the block diagram of FIG. 18. In contrast to the previous configuration, this configuration does not assume that the subject's identity is known to the subject finder as prior information. However, the subject's identity can be obtained as real-time information. In a real-time operation setup, it is not always possible to know the subject's identity in advance, so it is more realistic to find the subject's identity at runtime. Similar to the first embodiment, the expected output is object presence / absence information. This object may be part of the subject, but does not have to be the same as the subject. The presence / absence of an object can be expressed in the form of an image-level class (classification) or a pixel-level class (segmentation). In this embodiment, the 3D radar images stored in the radar image DB 301 do not need to be associated with a subject ID.

[0090] 18, one of the components of the detection device 100 according to the third embodiment can include a radar image DB storage unit 301, a subject finder 302, an image extraction unit 303, a detection unit 304, a subject DB storage unit 305, an approach selection unit 306, a subject classifier (subject ID identification unit) 310, a network selection unit 307, and a network architecture DB storage unit 317. The configurations and functions of the radar image DB storage unit 301, the subject finder 302, the image extraction unit 303, the detection unit 304, the subject DB storage unit 305, the approach selection unit 306, the network selection unit 307, and the network architecture DB storage unit 317 are the same as the configurations and functions of the radar image DB storage unit 101, the subject finder 102, the image extraction unit 103, the detection unit 104, the subject DB storage unit 105, the approach selection unit 106, the network selection unit 107, and the network architecture DB storage unit 117 according to the first embodiment, respectively. Therefore, the description will not be repeated.

[0091] The subject finder 302 may acquire the measured scattered radar signal from the radar 92. The 3D radar image may be generated within the detection device 100 instead of being acquired from the radar image DB store 301.

[0092] The subject identifier 310 outputs subject identification information, e.g., a subject ID, to the approach selector 306. The subject identifier 310 may be connected to an external sensor 40 to obtain additional information used to identify the subject. The subject is a subject contained in the 3D radar image retrieved from the radar image DB storage 301 and processed. In one example configuration, the subject identifier 310 receives captured optical images from an optical camera. The external sensor 40 may be the camera 98. The subject identifier 310 may use object detection means (among other means) to identify the subject.

[0093] Next, an example of operation in an operation mode of the radar image detection device 100 according to the third embodiment will be described with reference to the flowchart shown in Fig. 19. Subject information (size associated with ID) is stored in the subject DB storage unit 305. Also, various network architectures corresponding to different image sizes are stored in the network architecture DB storage unit 317. Note that steps S301, S302, S303, S304, S305, S306, and S307 are similar to steps S101, S102, S103, S104, S105, S106, and S107 described in the first embodiment of the present disclosure. Therefore, their description will not be repeated.

[0094] The subject identifier 310 identifies the subject and outputs the identification information to the approach selector 306. In this step, external sensor information may or may not be used (step S310).

[0095] The model generating device 200 described in the second embodiment may include the above-mentioned subject classifier 310. In this case, the 3D radar images stored in the radar image DB 201 do not need to be associated with subject IDs.

[0096] As described above, the detection device 100 according to the third embodiment of the present disclosure extracts a smaller image size using subject location information. This reduces the processing time required to acquire object presence and location information, enabling desired real-time operation. Furthermore, the device has the flexibility to identify subjects, preferably in real time, without requiring prior knowledge of the subject's identity.

[0097] All or part of the embodiments disclosed above can be described as follows, but are not limited to these. 1-1. A location determination unit for determining a subject's location in a 3D radar image; an extraction unit that extracts a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of extraction sizes designated for each type of the subject; a model selection unit that selects at least one trained model based on at least one of the size of the 3D sub-image and the type of the subject included in the 3D radar image; a detector that detects objects in the 3D sub-images using the selected trained model; A detection device comprising: 1-2. An extraction size determination unit that determines the one extraction size used to extract the 3D sub-image based on subject information in which at least an extraction size is associated with each of a plurality of subject IDs and the subject ID of the subject included in the 3D radar image. The detection device according to 1-1. further comprises: 1-3. A subject ID determination unit that determines the subject ID used to determine the one extraction size by identifying the type of the subject included in the 3D radar image. The detection device according to 1-2. further comprises: 1-4. The one extraction size used to extract the 3D sub-image is the subject extraction size for extracting the entire subject. The detection device according to any one of 1-1 to 1-3. 1-5. The one extraction size used to extract the 3D sub-image is a partial extraction size that extracts only a part of the subject; The reference position indicates a position of a part of the subject. The detection device according to any one of 1-1 to 1-3. 1-6. The detection unit outputs at least one of information indicating the presence or absence of the object, a class of the detected object, and position information of the object. The detection device according to any one of 1-1 to 1-5.

[0098] 2-1. A location determination unit for determining the location of a subject in a 3D radar image; an extraction unit that extracts a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of extraction sizes designated for each type of the subject; a model selector that selects at least one model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image; a learning unit that performs machine learning on the selected model using a combination of the 3D sub-image and information indicating the position of the object within the 3D sub-image as training data; and A model generation device comprising: 2-2. An extraction size determination unit that determines the one extraction size used to extract the 3D sub-image based on subject information in which at least an extraction size is associated with each of a plurality of subject IDs and the subject ID of the subject included in the 3D radar image. The model generating device according to 2-1. further comprises: 2-3. A subject ID determination unit that determines the subject ID used to determine the one extraction size by identifying the type of the subject included in the 3D radar image. The model generating device according to 2-2. further comprises: 2-4. The single extraction size used to extract the 3D sub-image is the subject extraction size for extracting the entire subject. The model generating device according to any one of 2-1 to 2-3. 2-5. The one extraction size used to extract the 3D sub-image is a partial extraction size that extracts only a part of the subject; The reference position indicates a position of a part of the subject. The detection device according to any one of 2-1 to 2-3.

[0099] 3-1. Identify the subject's location in a 3D radar image; extracting a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of extraction sizes designated for each type of subject; Selecting at least one trained model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image. Detecting an object in the 3D sub-image using the selected trained model. A computer-implemented detection method comprising: 3-2. Determine the one extraction size used to extract the 3D sub-image based on subject information in which at least an extraction size is associated with each of a plurality of subject IDs and the subject ID of the subject included in the 3D radar image. The detection method according to 3-1., further comprising: 3-3. Determine the subject ID used to determine the extraction size by identifying the type of subject included in the 3D radar image. The detection method according to 3-2., further comprising: 3-4. The one extraction size used to extract the 3D sub-image is the subject extraction size for extracting the entire subject. The detection method according to any one of 3-1 to 3-3. 3-5. The one extraction size used to extract the 3D sub-image is a partial extraction size that extracts only a part of the subject; The reference position indicates a position of a part of the subject. The detection method according to any one of 3-1 to 3-3. 3-6. Output at least one of information indicating the presence or absence of the object, the class of the detected object, and the position information of the object. The detection method according to any one of 3-1 to 3-5, further comprising:

[0100] 4-1. Identify the subject's location in a 3D radar image; extracting a 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and one of extraction sizes specified for each type of subject; selecting at least one trained model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image; Detecting an object in the 3D sub-image using the selected trained model. A program for causing a computer to execute a detection method including the steps of: 4-2. The detection method further includes determining the one extraction size used to extract the 3D sub-image based on subject information in which an extraction size is associated with each of a plurality of subject IDs and the subject ID of the subject included in the 3D radar image. The program described in 4-1. 4-3. The detection method further includes determining the subject ID used to determine the one sampling size by identifying the type of the subject included in the 3D radar image. The program described in 4-2. 4-4. The one extraction size used to extract the 3D sub-image is the subject extraction size for extracting the entire subject. A program according to any one of 4-1. to 4-3. 4-5. The one extraction size used to extract the 3D sub-image is a partial extraction size that extracts only a part of the subject; The reference position indicates a position of a part of the subject. A program according to any one of 4-1. to 4-3. 4-6. The detection method further includes outputting at least one of information indicating the presence or absence of the object, a class of the detected object, and position information of the object. A program described in any one of 4-1. to 4-5. [Explanation of symbols]

[0101] 100 Detection device 200 Model Generation Device 12 Location identification part 14 Extraction part 16 Model selection section 18 Detector 101, 201, 301 Radar image DB storage section 102,202,302 Subject Finder 103,203,303 Image extraction section 104,304 Detector 22 Location identification part 24 Extraction part 26 Model Selection Section 28 Learning Department 204 Learning Department 105,205,305 Subject DB storage section 106,206,306 Approach Selection Section 107,207,307 Network selection section 117,217,317 Network Architecture DB Storage 209 Annotation Adjustment Department 208 Annotation DB storage unit 310 Subject Identifier

Claims

1. A position identification unit that identifies the position of a subject in a 3D radar image, the position identification unit being associated with a subject ID that indicates a type of subject and stored in a radar image database; an extraction size determination unit that determines one extraction size to be used to extract a 3D sub-image based on subject information in which at least an extraction size is associated with each of a plurality of subject IDs, and the subject IDs of the subjects included in the 3D radar image, the subject IDs being associated with the 3D radar image in the radar image database; an extractor that extracts the 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and the one extraction size; a model selection unit that selects at least one trained model based on at least one of the size of the 3D sub-image and the type of the subject included in the 3D radar image; a detector that detects an object in the 3D sub-image using the selected trained model; A detection device comprising:

2. A position identification unit that identifies the position of a subject in a 3D radar image; a subject ID determination unit that determines a subject ID used to determine one extraction size by identifying the type of the subject included in the 3D radar image using an image including the subject that is captured in synchronization with the 3D radar image; and an extraction size determination unit that determines the one extraction size used to extract a 3D sub-image based on subject information in which at least an extraction size is associated with each of a plurality of subject IDs and the determined subject ID of the subject included in the 3D radar image; an extractor that extracts the 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and the one extraction size; a model selection unit that selects at least one trained model based on at least one of the size of the 3D sub-image and the type of the subject included in the 3D radar image; a detector that detects an object in the 3D sub-image using the selected trained model; A detection device comprising:

3. The one extraction size used to extract the 3D sub-image is a subject extraction size for extracting the entire subject.

3. The detection device according to claim 1 or 2.

4. the one extraction size used to extract the 3D sub-image is a partial extraction size that extracts only a part of the subject; The reference position indicates a position of a part of the subject.

3. The detection device according to claim 1 or 2.

5. The detection unit outputs at least one of information indicating the presence or absence of the object, a class of the detected object, and position information of the object. The detection device according to any one of claims 1 to 4.

6. A position identification unit that identifies a position of a subject in a 3D radar image, the position identification unit being associated with a subject ID that indicates a type of subject and stored in a radar image database; an extraction size determination unit that determines one extraction size to be used to extract a 3D sub-image based on subject information in which at least an extraction size is associated with each of a plurality of subject IDs, and the subject IDs of the subjects included in the 3D radar image, the subject IDs being associated with the 3D radar image in the radar image database; an extractor that extracts the 3D sub-image from the 3D radar image using a reference position based on the identified position of the subject and the one extraction size; a model selector that selects at least one model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image; a learning unit that performs machine learning on the selected model using a combination of the 3D sub-image and information indicating the position of an object within the 3D sub-image as training data; and A model generation device comprising:

7. Identifying a position of a subject in a 3D radar image, which is stored in a radar image database in association with a subject ID indicating a type of subject; determining an extraction size to be used to extract a 3D sub-image based on subject information in which at least an extraction size is associated with each of a plurality of subject IDs and the subject IDs of the subjects included in the 3D radar image, the subject IDs being associated with the 3D radar image in the radar image database; extracting the 3D sub-image from the 3D radar image using a reference position based on the identified subject's position and the one extraction size; Selecting at least one trained model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image. Detecting an object in the 3D sub-image using the selected trained model A computer-implemented detection method comprising:

8. Identifying a subject's location in a 3D radar image; determining a subject ID to be used to determine one sampling size by identifying the type of the subject included in the 3D radar image using an image including the subject that is captured synchronously with the 3D radar image; determining the one extraction size used to extract a 3D sub-image based on subject information in which at least an extraction size is associated with each of a plurality of subject IDs and the determined subject ID of the subject included in the 3D radar image; extracting the 3D sub-image from the 3D radar image using a reference position based on the identified subject's position and the one extraction size; selecting at least one trained model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image; Detecting an object in the 3D sub-image using the selected trained model A computer-implemented detection method comprising:

9. Identifying a position of a subject in a 3D radar image, which is stored in a radar image database in association with a subject ID indicating a type of subject; determining an extraction size to be used to extract a 3D sub-image based on subject information in which at least an extraction size is associated with each of a plurality of subject IDs and the subject IDs of the subjects included in the 3D radar image, the subject IDs being associated with the 3D radar image in the radar image database; extracting the 3D sub-image from the 3D radar image using a reference position based on the identified subject's position and the one extraction size; Selecting at least one trained model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image. Detecting an object in the 3D sub-image using the selected trained model A program for causing a computer to execute a detection method including the steps of:

10. Identifying a subject's location in a 3D radar image; determining a subject ID to be used to determine one sampling size by identifying the type of the subject included in the 3D radar image using an image including the subject that is captured synchronously with the 3D radar image; determining the one extraction size used to extract a 3D sub-image based on subject information in which at least an extraction size is associated with each of a plurality of subject IDs and the determined subject ID of the subject included in the 3D radar image; extracting the 3D sub-image from the 3D radar image using a reference position based on the identified subject's position and the one extraction size; selecting at least one trained model based on at least one of a size of the 3D sub-image and a type of the subject included in the 3D radar image; Detecting an object in the 3D sub-image using the selected trained model A program for causing a computer to execute a detection method including the steps of:

Images