Apparatus and method for object detection based on infrared images

Abstract

The present invention relates to an IR image-based object detection device and method. The object detection device may include a feature extraction network that receives an IR image of an object captured by a thermal imaging camera and generates first feature maps, a multi-scale network that generates second feature maps with different scales from the first feature maps, and a header network that receives the second feature maps and estimates the location and class of the object.

Description

Technology Field

[0001] The present invention relates to an IR image-based object detection method and apparatus. Background Technology

[0002] Infrared sensor-based thermal imaging (IR) domain imaging is widely utilized in various fields, including military, security, surveillance, and aviation. In particular, because IR imaging is robust against changes in illumination, it offers the advantage of being able to identify objects even at night or in situations where it is difficult to identify them using visible light imaging.

[0003] Conventional image recognition technology has primarily used traditional image processing techniques such as edge detection and histogram analysis. While these techniques were effective in simple environments based on temperature differences, they have limitations in that image accuracy is reduced in complex backgrounds or low-contrast environments because they fail to clearly distinguish the boundaries of objects.

[0004] For example, the complex shape and structure of an object cause non-uniform surface temperatures, and changes in surface temperature occur depending on the actual moving speed of the object. Although machine learning-based technologies such as SIFT and HOG have improved some of these problems, there are still limitations in that object features must be extracted manually.

[0005] Compared to RGB images, IR images contain less image information, and due to hardware limitations, the low resolution of IR images results in the generation of feature maps with poor object discrimination capabilities. Due to the aforementioned issues, it is difficult to acquire semantic information about objects, raising concerns that the ability to distinguish small objects or objects in complex backgrounds is insufficient.

[0006] The background technology described above should be understood as merely for explaining the background in which the present invention was derived, and does not imply that it is technology known prior to the filing of the present invention. The problem to be solved

[0007] The objective of the present invention is to provide an IR image-based object detection device and method that can solve the problem of object detection in IR images and achieve high accuracy by using a deep learning-based network model.

[0008] However, the technical problem to be solved by the embodiments of the present invention is not limited to the technical problem described above, and other technical problems can be inferred from the following embodiments. means of solving the problem

[0009] An IR image-based object detection device according to one embodiment of the present invention may include a feature extraction network that receives an IR image of an object captured by a thermal imaging camera and generates first feature maps, a multi-scale network that generates second feature maps having different scales from the first feature maps, and a header network that receives the second feature maps and estimates the location of the object and the class of the object.

[0010] In addition, the feature extraction network may be a deep neural network including a convolutional neural network.

[0011] In addition, the above feature extraction network can be combined with a Cross Stage Partial (CSP) network technique to a Darknet-based backbone model.

[0012] In addition, the multi-scale network can generate the second feature maps having different sizes by using a top-down and a bottom-up approach.

[0013] Additionally, the header network may include a classification head that classifies the class of the object and a regression head that predicts the location of the object.

[0014] In addition, the classification head classifies the class of the object based on a Binary Cross-Entropy (BCE) loss function, and the regression head can estimate the location of the object using an Intersection over Union (IoU) based loss function.

[0015] An IR image-based object detection method according to one embodiment of the present invention may include the steps of: receiving an IR image of an object captured by a thermal imaging camera and generating first feature maps; generating second feature maps having different scales from the first feature maps; and receiving the second feature maps and estimating the location of the object and the class of the object.

[0016] In addition, the step of generating the first feature maps may include the step of generating the first feature maps using a deep neural network including a convolutional neural network.

[0017] In addition, it may further include a step of using a model that combines a Darknet-based backbone model with a Cross-Stage Partial (CSP) network technique.

[0018] In addition, the step of generating the second feature maps may include the step of generating second feature maps having different sizes using a top-down method and a bottom-up method.

[0019] In addition, the step of estimating the location and class of the object may include a step of classifying the class of the object using a Binary Cross-Entropy (BCE) loss function and a step of estimating the location of the object using an Intersection over Union (IoU) based loss function.

[0020] In addition, according to one embodiment of the present invention, a computer-readable recording medium may be provided on which a computer program for executing an IR image-based object detection method is recorded. Effects of the invention

[0021] The IR image-based object detection device and method according to an embodiment of the present invention can solve the problem of object detection in IR images and achieve high accuracy by using a deep learning-based network model.

[0022] In addition, according to an embodiment of the present invention, various characteristics of an image can be analyzed in depth through a deep learning-based feature extraction network.

[0023] In addition, according to an embodiment of the present invention, by considering objects of various scales through a multi-scale based network, small objects can also be effectively detected.

[0024] In addition, according to an embodiment of the present invention, both the accuracy and speed of task processing can be improved by performing object classification and location estimation tasks separately.

[0025] The effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art to which the present invention pertains (referred to as "person skilled in the art") from the description in the claims. Brief explanation of the drawing

[0026] FIG. 1 is a block diagram illustrating the functional configuration and operation of an object detection device according to an embodiment of the present invention. FIG. 2 is a diagram illustrating the operation of a multi-scale network according to an embodiment of the present invention. FIG. 3 is a diagram illustrating the operation of a header network according to an embodiment of the present invention. FIG. 4 is a flowchart showing the sequence of an object detection method according to an embodiment of the present invention. FIG. 5 is a block diagram showing the detailed configuration of an object detection device according to an embodiment of the present invention. Specific details for implementing the invention

[0027] Hereinafter, specific details for implementing the present invention will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions regarding widely known functions or configurations will be omitted if there is a risk of unnecessarily obscuring the essence of the present invention.

[0028] In the attached drawings, identical or corresponding components are assigned the same reference numerals. Additionally, in the description of the following embodiments, the description of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0029] In the present invention, terms such as "first," "second," etc. are used to distinguish a specific component from another component, and the components described above are not limited by these terms. For example, the "first" component may be an element of the same or similar form as the "second" component.

[0030] In this specification, singular expressions include plural expressions unless the context clearly specifies them as singular. Additionally, plural expressions include singular expressions unless the context clearly specifies them as plural. Throughout the specification, when a part is described as including a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0031] In the present invention, terms such as “comprising,” “comprising,” etc. may indicate the presence of features, steps, operations, elements, and / or components, but do not exclude the addition of one or more other functions, steps, operations, elements, components, and / or combinations thereof.

[0032] In this specification, an IR (Infrared-Red) image may be a thermal image generated by infrared radiation emitted or reflected from an object. Additionally, the IR image may include a thermal image generated through a digital or analog image output function using infrared energy data detected by an infrared sensor.

[0033] The IR image-based object detection device and method according to an embodiment of the present invention may utilize various techniques based on the YOLO model. Furthermore, a method is proposed to improve object detection accuracy and speed in IR images where object detection is difficult due to a lack of image information. While the YOLO model may be a YOLOX model, it is not necessarily limited thereto, and various YOLO models may be used.

[0034] Hereinafter, an IR image-based object detection device and method according to an embodiment of the present invention will be described with reference to the attached drawings.

[0035] FIG. 1 is a block diagram schematically showing an object detection device (10) according to an embodiment of the present invention.

[0036] An object detection device (10) according to an embodiment of the present invention may include a feature extraction network (100), a multi-scale network (110), and a header network (120). Additionally, the object detection device (10) may receive IR image data from a thermal imaging camera (5).

[0037] A thermal imaging camera (5) may be a non-contact measuring device that detects infrared energy or heat signals and converts the signals into a visible image. The thermal imaging camera (5) may include an infrared sensor that detects heat on the surface of an object. Infrared energy emitted or reflected from an object can be captured using the infrared sensor, and this can be output as an image or data through digital or analog video output.

[0038] The feature extraction network (100) receives IR images and IR images captured from a thermal imaging camera (5) and can extract various features of the images (e.g., heat emission patterns, small temperature changes). In addition, it can generate a discriminative feature map based on the extracted features.

[0039] At this time, the input IR image may have a data augmentation technique applied by an object detection device (10) according to an embodiment of the present invention. The data augmentation technique may be either Mosaic Augmentation or Mixup Augmentation.

[0040] Mosaic Augmentation may be a technique that generates a new image by arranging four different images in a grid pattern and including parts of each image in a single final image, and Mixup Augmentation may be a technique that generates a new image by mixing two images pixel by pixel. According to the above techniques, the diversity of training data input to the feature extraction network (100) can be improved.

[0041] The feature extraction network (100) may be a deep neural network (DNN) including a convolutional neural network (CNN). By using a deep learning-based model, iterative image data learning can be performed, and key features of an object can be extracted even from IR images with low resolution and little image information. Additionally, the feature map may include multiple sub-feature maps generated through input image data and multiple filters.

[0042] The feature extraction network (100) may be a network that combines the DarkNet53 backbone model of YOLO with CSPNet (Cross-Stage Partial Network), but is not necessarily limited to this, and various versions of the YOLO backbone model may be used.

[0043] The DarkNet53 backbone model is a backbone network consisting of 53 layers, and it can learn high-level features through its deep network structure.

[0044] The CSP network technique (CSPNet) can be a network design that minimizes redundant computations occurring in existing neural networks and implements an efficient data flow. By combining CSPNet with the DarkNet53 backbone model, it is possible to provide a data flow suitable for removing minute differences in temperature changes and background noise in IR images, while simultaneously ensuring performance improvements such as preventing overfitting and training stability.

[0045] CSPNet can reduce computational load by using a structure that splits data within layers, processes it in parallel, and then merges it. Additionally, it can extract features of objects of various sizes within images by refining and recombining feature maps from each layer.

[0046] Accordingly, the feature extraction network (100) can extract a discriminative feature map from an IR image lacking image information and can improve the data processing speed.

[0047] The multi-scale network (110) can generate feature maps of various scales by taking the feature map output from a layer of the feature extraction network (100) as input.

[0048] The multi-scale network (110) can utilize a PAFPN (Path Aggregation Feature Pyramid Network) structure in which a bottom-up method is added to an FPN (Feature Pyramid Network) structure that uses a top-down method.

[0049] Specifically, the FPN structure may be a structure that generates multi-scale features to detect objects of various sizes, and can generate a multi-scale feature map by combining features generated at multiple layers. The PAFPN structure may be a structure that adds a bottom-up approach to the FPN structure and can prevent information loss between multi-scale features.

[0050] A multi-scale network (110) can generate multiple major feature maps by combining feature maps of various layers through a top-down and bottom-up method.

[0051] Accordingly, information of different resolutions can be obtained depending on the size and location of each object within the IR image, and objects of various scales can be detected.

[0052] Specifically, the top-down approach transfers information from high-level features to low-level features and can integrate a wide range of contextual information. During the integration process, skip connections can be utilized to merge information between high-level and low-level features. This allows for the simultaneous preservation of object details and contextual information, enabling the clear recognition of large objects and the analysis of their relationships with the surrounding environment.

[0053] The bottom-up structure can transfer data from low-level features to high-level features. This enhances the feature content of IR images and enables better detection of small-scale objects. Furthermore, it can improve detection performance by compensating for details of small objects even in low-resolution IR images, and allows for more accurate detection of small objects or objects with complex structures.

[0054] In this case, high-level features can refer to characteristics that clearly distinguish between objects by reflecting their global shape and location, while low-level features can refer to performance that improves detection performance for complex backgrounds or small objects by preserving detailed temperature variations and the boundaries of small objects.

[0055] The header network (120) can receive a multi-scale feature map from the multi-scale network (110) as input and infer the location and class (label) information of the object.

[0056] Unlike conventional single-header structures (Coupled Head), the header network (120) can utilize a Decoupled Head structure composed of multiple heads. Accordingly, object classification and location estimation tasks can be separated and processed in parallel. In addition, accuracy and efficiency can be simultaneously secured through a design that takes into account the special characteristics of IR images (e.g., lack of texture, temperature difference-based information).

[0057] The header network (120) can process object classification and regression tasks separately through a Decoupled Head structure, thereby minimizing interference between tasks and improving the performance of each task.

[0058] Specifically, the header network (120) may include a classification head that determines whether an object belongs to a specific class and a regression head that predicts the exact location and size of the object.

[0059] Each head structure shares feature map information and can be independently optimized by applying a design specialized for each task.

[0060] In the classification head, high-level abstract features and low-level detailed features are combined to achieve high classification performance even for small objects and low-contrast regions. Accordingly, even in IR images, which lack texture information compared to RGB images, objects can be classified by accurately learning minute temperature changes and heat dissipation patterns.

[0061] In the regression head, by considering the characteristics of IR images where boundaries between objects are unclear, object boundary information based on temperature differences can be learned, thereby enhancing the detection of small objects commonly found in IR images and the processing of complex backgrounds. Accordingly, minute temperature differences and blurry boundaries of objects within IR images can be accurately learned.

[0062] FIG. 2 is a diagram illustrating the operation of a multi-scale network (110) according to an embodiment of the present invention.

[0063] Referring to FIG. 2, the multi-scale network (110) can receive feature maps from multiple layers of the feature extraction network (100). By combining the received feature maps, a multi-scale feature map that considers objects of various scales can be generated.

[0064] As illustrated in FIG. 2(a), the output of the feature extraction network (100) is It can be represented as. may be a feature map with a size reduced by 1 / 4, 1 / 8, 1 / 16, or 1 / 32 of the input image, but is not necessarily limited to this.

[0065] A feature extraction network (100) having high-level features The output is one of the feature maps of the multi-scale network (110). It can be input as such. The multi-scale network (110) can utilize a skip connection method that maintains the information of the high-level feature map and merges it with the low-level feature map.

[0066] A multi-scale network (110) can perform upsampling by a factor of 2 (e.g., Bilinear Interpolation or Deconvolution) to make the resolution of the high-level feature map equal to that of the low-level feature map. Additionally, the upsampled high-level feature map and the low-level feature map of the corresponding resolution can be combined by adding them element by element. Through this, a new feature map can be created that simultaneously contains the details of the low-level information and the abstract context of the high-level information.

[0067] After merging feature maps, 1x1 convolution can be used to optimize information interaction between channels and remove unnecessary noise. The above-described operation can be utilized in a top-down approach, while down-sampling can be utilized in a bottom-up approach.

[0068] For example, input It can be up-sampled by a factor of 2, and the up-sampled feature map is an output with the same size. Combined by element, a new feature map It can generate. Through the aforementioned top-down structure, four feature maps of different sizes It can generate.

[0069] As shown in Fig. 2(b), the feature map Is It is input as, and through a bottom-up structure, four feature maps of different sizes It can generate. In this case, down-sampling can be performed instead of up-sampling.

[0070] In a preferred embodiment of the present invention, four different multi-scale feature maps are output, but this is not necessarily limited to, and multiple multi-scale feature maps can be generated and output.

[0071] Next, as illustrated in FIG. 2(c), multi-scale feature maps of different sizes can be combined and input into a header network (120).

[0072] FIG. 3 is a diagram illustrating the operation of a header network (120) according to an embodiment of the present invention.

[0073] Referring to FIG. 3, the header network (120) can receive a multi-scale feature map from the multi-scale network (110) and infer the location and class of an object.

[0074] The header network (120) may include a classification head (121) and a regression head (122), and each head may perform a prediction task in parallel.

[0075] The classification head (121) can receive a multi-scale feature map as input, compress information between channels using 1x1 convolution, and filter to retain only important features. It can output the presence or absence of a class as a probability using a sigmoid function, and can optimize the classification task by using a Binary Cross-Entropy (BCE) loss function to account for the unbalanced distribution of objects in the IR image (e.g., cases where the background is overwhelmingly dominant).

[0076] In addition, SimOTA can mitigate class imbalance issues by dynamically matching positive anchors during training.

[0077] Accordingly, objects with textures similar to the background in IR images can be accurately classified, and small temperature differences or weak thermal signals can be effectively learned. The combination of SimOTA and BCE can prevent the problem of overfitting on small amounts of data, and by utilizing the feature map through the PAFPN structure of the multi-scale network (110), objects of various sizes can be classified simultaneously.

[0078] The regression head (122) receives the same multi-scale feature map as the classification head (121) as input and emphasizes local temperature changes and fine boundary information through 3x3 convolution, thereby effectively extracting features of small objects and objects with weak boundaries. Additionally, after channel compression with 1x1 convolution, bounding box coordinates (e.g., center point x, y, width, height) can be predicted.

[0079] The regression head (122) can output continuous coordinate values ​​by utilizing a linear activation function. Additionally, by using an Intersection over Union (IoU) based loss function to comprehensively consider the center distance and size ratio between the bounding box and the actual object, accurate boundaries can be learned even in complex areas of small objects and backgrounds.

[0080] The regression head (122) can generate bounding boxes with high accuracy even with small objects and complex backgrounds, and can precisely learn the center distance and size difference between the object and the prediction box using an IoU-based loss function. In addition, it can generate bounding boxes that reflect temperature differences in IR images through a multi-scale feature map.

[0081] Therefore, the header network (120) can independently optimize object classification and location estimation through a Decoupled Head structure, and by separating different learning objectives in the IR image, it can optimize temperature-based detailed learning (classification) and object boundary learning (location estimation) respectively.

[0082] Furthermore, rapid learning and high accuracy can be achieved, and learning efficiency can be maximized by using loss functions (BCE, IoU) tailored to each task. Accordingly, high accuracy can be achieved even in complex situations, such as small heat-emitting objects within IR images.

[0083] FIG. 4 is a flowchart showing the sequence of an object detection method according to an embodiment of the present invention.

[0084] Referring to FIG. 4, the feature extraction network (100) can generate a feature map by receiving an image in an IR image obtained from a thermal imaging camera (5) (S210).

[0085] The feature extraction network (100) may be a deep neural network (DNN) including a convolutional neural network (CNN). By using a deep learning-based model, iterative image data learning can be performed, and key features of an object can be extracted even from IR images with low resolution and little image information.

[0086] The feature extraction network (100) may be a network in which CSPNet (Cross-Stage Partial Network) is combined with YOLO's DarkNet53 backbone model.

[0087] The DarkNet53 backbone model is a backbone network composed of 53 layers, and can learn high-level features through a deep network structure. The CSP network technique (CSPNet) is a network design that minimizes redundant operations occurring in existing neural networks and implements efficient data flow.

[0088] Accordingly, the feature extraction network (100) can extract a distinguishable feature map from an IR image lacking image information and can improve the data processing speed.

[0089] Next, the multi-scale network (100) can generate multi-scale feature maps of different sizes by taking the feature map output corresponding to each layer of the feature extraction network (100) as input (S220). At this time, the multi-scale feature map may be composed of a plurality of feature maps, and preferably, four different feature maps.

[0090] A multi-scale network (110) can generate different major feature maps by combining feature maps of various layers through a top-down and bottom-up method. At this time, the multi-scale network (110) may be a PAFPN structure in which a bottom-up method is added to the FPN method.

[0091] Accordingly, information of different resolutions can be obtained depending on the size and location of each object within the IR image, and objects of various scales can be detected.

[0092] Next, the header network (120) can receive a multi-scale feature map from the multi-scale network (110) as input and infer the location and class (label) information of the object (S230).

[0093] Unlike conventional single-header structures (Coupled Head), the header network (120) can utilize a Decoupled Head structure composed of multiple heads. Accordingly, object classification and location estimation tasks can be separated and processed in parallel.

[0094] The header network (120) can process object classification and regression tasks separately through a Decoupled Head structure, thereby minimizing interference between tasks and maximizing the performance of each task.

[0095] Specifically, the header network (120) may include a classification head that determines whether an object belongs to a specific class and a regression head that predicts the exact location and size of the object.

[0096] Each head structure shares feature map information and can be independently optimized by applying a design specialized for each task.

[0097] Finally, objects within an IR image can be detected by classifying the location and label of the object according to the estimation result of the header network (120) (S240).

[0098] Accordingly, the object detection device and method according to an embodiment of the present invention can perform real-time object recognition through a one-stage detector structure capable of simultaneously determining object location inference and object class.

[0099] In addition, deep learning-based object recognition technology enables the classification and localization of objects with much higher accuracy. Furthermore, by using the latest data augmentation techniques and extensive training data to significantly improve the model's object recognition generalization capabilities, robust object recognition can be achieved even in real-world environments.

[0100] To verify the performance of the object detection device and method according to an embodiment of the present invention, the performance was compared in an On-GPU (NVIDIA RTX A6000 48GB) environment and an On-Board (Hailo-8) environment, and the object detection performance was also verified in a thermal image-based dataset.

[0101] The model's mAP (detection accuracy), power consumption, and execution speed (FPS) were used as object detection performance metrics.

[0102] [Table 1]

[0103]

[0104] Tests using YOLOX-M, YOLOX-S, and YOLOX-Tiny-based models on the COCO 2017 dataset composed of IR images showed that in an on-board environment, mAP decreased by up to 2.8% compared to on-GPU, power consumption decreased by approximately 98%, and FPS improved from 45.5 to 82.9.

[0105] In addition, the results of testing on the On-GPU using the YOLOX-Tiny model on a thermal image-based dataset to account for actual battlefield conditions showed that when the input image size was 256×256, it recorded an mAP of 19.3% and an FPS of 64.6, and when it was 416×416, it recorded an mAP of 42.8% and an FPS of 61.5.

[0106] As such, the object detection device and method according to the embodiment of the present invention can achieve low power consumption and real-time inference even in an on-board environment utilizing the Hailo-8 NPU. Accordingly, it can be utilized in battlefield situations where power supply conditions are limited, and can improve object detection performance in battlefield scenarios based on thermal imaging data.

[0107] FIG. 5 is a block diagram showing the detailed configuration of an object detection device (10) according to an embodiment of the present invention.

[0108] The object detection device (10) may include a memory (320), a processor (310), a communication module (330), and an input / output interface (340), and as shown in FIG. 5, the object detection device (10) may be configured to communicate information and / or data through a network using the communication module (330). The feature extraction network (100), multi-scale network (110), and header network (120) of the object detection device (10) shown in FIG. 1 may be provided by the processor (310) as components of an IR image-based object detection method in stages or in parallel.

[0109] The memory (320) may include any non-transient computer-readable recording medium. According to one embodiment, the memory (320) may include a permanent mass storage device such as random access memory (RAM), read-only memory (ROM), disk drive, solid state drive (SSD), flash memory, etc. As another example, a permanent mass storage device such as ROM, SSD, flash memory, disk drive, etc. may be included in the object detection device (10) as a separate permanent storage device distinct from the memory. Additionally, an operating system and at least one program code may be stored in the memory (320).

[0110] The memory (320) may store instructions, data structures, and program code that can be read by the processor (310). In the disclosed embodiments, operations performed by the processor (310) may be implemented by executing the instructions or code of the program stored in the memory (320).

[0111] These software components may be loaded from a computer-readable recording medium separate from the memory (320). This separate computer-readable recording medium may include a recording medium that can be directly connected to the object detection device (10), for example, a computer-readable recording medium such as a floppy drive, disk, tape, DVD / CD-ROM drive, memory card.

[0112] The processor (310) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to another user terminal (not shown) or another external system by memory (320) or a communication module (330).

[0113] The communication module (330) may provide a configuration or function for the user terminal (not shown) and the object detection device (10) to communicate with each other via a network, and may provide a configuration or function for the object detection device (10) to communicate with an external system (e.g., a separate cloud system). For example, control signals, commands, data, etc. provided under the control of the processor (310) of the object detection device (10) may be transmitted to the user terminal and / or the external system through the communication module (330) and the network, and through the communication module of the user terminal and / or the external system.

[0114] Additionally, the input / output interface (340) of the object detection device (10) may be a means for interfacing with a device (not shown) for input or output that is connected to the object detection device (10) or that the object detection device (10) may include. In FIG. 4, the input / output interface (340) is shown as an element configured separately from the processor (310), but is not limited thereto, and the input / output interface (340) may be configured to be included in the processor (310). The object detection device (10) may include more components than those in FIG. 5. However, there is no need to clearly illustrate most of the prior art components.

[0115] The methods and / or various embodiments described above may be realized in digital electronic circuits, computer hardware, firmware, software, and / or combinations thereof. For example, the methods and / or various embodiments of the present invention may be performed by special-purpose logic circuits such as Field Programmable Gate Arrays (FPGAs) or Application Specific Integrated Circuits (ASICs), and devices and / or systems for performing the methods and / or embodiments of the present invention may be implemented as special-purpose logic circuits such as FPGAs or ASICs.

[0116] In the present invention, various embodiments may be implemented in a computing system comprising backend components (e.g., data servers), middleware components (e.g., application servers), and / or frontend components. In this case, the components may be interconnected by any form or medium of digital data communication, such as a communication network. For example, the communication network may include a Local Area Network (LAN), a Wide Area Network (WAN), etc.

[0117] Although the present invention has been described in relation to some embodiments, various modifications and changes may be made without departing from the scope of the invention as understood by a person skilled in the art to which the invention pertains. Furthermore, such modifications and changes should be considered to fall within the scope of the claims appended to this specification. Explanation of the symbols

[0118] 5: Thermal imaging camera 10: Object detection device 100: Feature extraction network 110: Multiscale Network 120: Header Network 121: Classification Head 122: Regression Head 310: Processor 320: Memory 330: Communication module 340: Input / Output Interface

Claims

Claim 1 An IR image-based object detection device comprising: a feature extraction network that receives an IR image of an object captured by a thermal imaging camera and generates first feature maps; a multi-scale network that generates second feature maps with different scales from the first feature maps; and a header network that receives the second feature maps and estimates the location and class of the object. Claim 2 An IR image-based object detection device, wherein the feature extraction network is a deep neural network including a convolutional neural network, in claim 1. Claim 3 In paragraph 2, the feature extraction network is an IR image-based object detection device in which a Cross Stage Partial (CSP) network technique is combined with a Darknet-based backbone model. Claim 4 In paragraph 2, the IR image-based object detection device, wherein the multi-scale network generates the second feature maps having different sizes using a top-down method and a bottom-up method. Claim 5 An IR image-based object detection device according to claim 1, wherein the header network comprises: a classification head for classifying the class of the object; and a regression head for predicting the location of the object. Claim 6 An IR image-based object detection device according to claim 5, wherein the classification head classifies the class of the object based on a BCE (Binary Cross-Entropy) loss function, and the regression head estimates the position of the object using an IoU (Intersection over Union) based loss function. Claim 7 An IR image-based object detection method performed by an object detection device, comprising: a step of receiving an IR image of an object captured by a thermal imaging camera and generating first feature maps; a step of generating second feature maps having different scales from the first feature maps; and a step of receiving the second feature maps and estimating the location of the object and the class of the object. Claim 8 In claim 7, the step of generating the first feature maps comprises the step of generating the first feature maps using a deep neural network including a convolutional neural network, an IR image-based object detection method. Claim 9 An IR image-based object detection method according to claim 8, further comprising the step of using a model in which a Cross Stage Partial (CSP) network technique is combined with a Darknet-based backbone model. Claim 10 In claim 8, the step of generating the second feature maps comprises generating the second feature maps having different sizes using a top-down method and a bottom-up method, in an IR image-based object detection method. Claim 11 In claim 7, the step of estimating the location and class of the object comprises: a step of classifying the class of the object using a Binary Cross-Entropy (BCE) loss function; and a step of estimating the location of the object using an Intersection over Union (IoU) based loss function; an IR image-based object detection method. Claim 12 A computer-readable recording medium having a computer program for executing an IR image-based object detection method according to any one of claims 7 through 11.

Images