Method for enhancing target detection performance and electronic apparatus therefor

Abstract

A method for improving target detection performance, performed by an electronic device, may include: acquiring an infrared image containing a target; acquiring a multi-channel target image by merging a plurality of single-channel target images, each including a single-channel target image generated from the infrared image based on each parameter set included in a plurality of parameter sets; providing a target detection result acquired in correspondence with the multi-channel target image through a target detection model; and training the target detection model based on training data configured to include the multi-channel target image and the target detection result.

Description

Technology Field

[0001] The present disclosure relates to a method for improving target detection performance, and more specifically, to a method for improving target detection performance that acquires infrared images, merges a plurality of single-channel target images to acquire multi-channel target images, provides target detection results through a target detection model, and trains a target detection model. Background Technology

[0002] Unlike visible light, infrared radiation can detect thermal energy emitted from objects themselves, offering the advantage of enhanced object detection performance even in environments where visible light is blocked, such as at night, in fog, and / or in smoke. Due to these characteristics, infrared image processing technology has played a significant role in various fields, including security, surveillance, military, medical, and industrial sectors.

[0003] Meanwhile, deep learning technology has recently made rapid advancements across the entire field of image recognition, particularly in the field of object detection. Deep learning-based object detection models can automatically recognize objects within images, identify their locations, and classify their types, providing higher accuracy and efficiency than existing manual methods.

[0004] However, existing infrared image processing techniques have the following limitations. For example, while histogram equalization can improve image contrast by flattening the histogram, excessive application can amplify noise or damage the natural features of the image. Adaptive Histogram Equalization (AHE), which performs histogram equalization by reflecting local characteristics of the image, or variations such as Contrast Limited Adaptive Histogram Equalization (CLAHE), can mitigate the noise amplification problem to some extent; however, the quality of the results still varies significantly depending on the parameter settings. Filtering techniques, such as Gaussian filters, median filters, and sharpening filters, are effective for suppressing noise and enhancing features in images, but they can cause problems such as resolution degradation or the loss of important features. Frequency Domain Processing (FDM) techniques can improve image quality by analyzing frequency components using Fourier transforms and emphasizing or removing specific frequency bands, but they involve high computational complexity and can cause image distortion.

[0005] Deep learning-based object detection technologies also have the following limitations. R-CNN (Region-based Convolutional Neural Network), a model that classifies objects by extracting candidate object regions within an image and features for each region, has the disadvantage of requiring a significant amount of time for candidate region extraction and having a slow training speed. Fast R-CNN is a model that processes the entire image only once to classify objects by extracting features from candidate regions, aiming to improve upon the shortcomings of R-CNN; while it has faster training speeds than R-CNN, it still has the limitation of requiring the candidate region extraction process. Faster R-CNN is a model that processes candidate region extraction using a neural network to improve upon the shortcomings of Fast R-CNN; it offers fast training speeds and high accuracy, but still possesses a complex structure. YOLO (You Only Look Once) is a model that detects objects by processing an image only once; while it offers fast training speeds and enables real-time processing, it has the disadvantage of low performance in detecting small objects. DETR (Detection Transformer) is a transformer-based object detection model that offers the advantage of high performance in detecting small objects but suffers from the disadvantage of low processing speed. Since most of these models are optimized for standard RGB images, problems arise where detection performance degrades due to the low contrast, resolution, and / or noise of infrared images.

[0006] Therefore, to overcome the limitations of existing technologies, there is a need for a technology that can improve infrared image-based object detection performance by using image processing techniques that consider the characteristics of infrared images. The problem to be solved

[0007] One objective of the embodiments of the present disclosure is to provide a method and apparatus for improving target detection performance by improving the quality of infrared images acquired from an infrared camera and utilizing them as input to a target detection model, by securing high-quality infrared images even in various environments to accurately detect target objects.

[0008] One objective of the embodiments of the present disclosure is to provide a method and apparatus for improving target detection performance that can be usefully utilized in applications requiring real-time processing, such as security, surveillance, and / or autonomous driving, by providing an algorithm that diversifies images by effectively controlling the contrast, brightness, and noise of infrared images, designing a deep learning model optimized for infrared images to configure training data, and simultaneously applying a data augmentation technique that considers the characteristics of infrared images to maximize the detection and generalization performance of the deep learning model, and by controlling the input images to process infrared images in real time and detect target objects without imposing additional computational burden on the deep learning itself.

[0009] One objective of the embodiments of the present disclosure is to provide a method and apparatus for improving target detection performance that enhances surveillance and security performance in nighttime or adverse weather environments, ensures the safety of autonomous driving systems, and enables efficient maintenance of industrial facilities.

[0010] The technical problems to be solved by the present disclosure are not limited to the technical problems described above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which the present disclosure belongs from the description below. means of solving the problem

[0011] A method for improving target detection performance performed by an electronic device according to one embodiment may include: acquiring an infrared image containing a target; acquiring a multi-channel target image by merging a plurality of single-channel target images, each including a single-channel target image generated from an infrared image based on each parameter set included in a plurality of parameter sets; providing a target detection result acquired in correspondence with the multi-channel target image through a target detection model; and training a target detection model based on training data configured to include the multi-channel target image and the target detection result.

[0012] In a method for improving target detection performance performed by an electronic device according to one embodiment, each parameter set includes each parameter setting value for each parameter included in one or more parameters pre-set for image conversion, and the step of acquiring a multi-channel target image may include the step of determining the parameter setting values ​​included in each parameter set based on the environment for detecting the target and the characteristics of the target.

[0013] In a method for improving target detection performance performed by an electronic device according to one embodiment, one or more parameters may include contrast, brightness, gamma, and CLAHE (Contrast Limited Adaptive Histogram Equalization) parameters.

[0014] In a method for improving target detection performance performed by an electronic device according to one embodiment, the step of acquiring a multi-channel target image includes the step of acquiring a multi-channel target image by merging each single-channel target image into each channel included in a plurality of channels, and the plurality of channels may include R, G, and B channels.

[0015] In a method for improving target detection performance performed by an electronic device according to one embodiment, the target detection model may be set based on a deep learning-based object detection model.

[0016] In a method for improving target detection performance performed by an electronic device according to one embodiment, the target detection result may include the localization, class, and confidence of the target.

[0017] In a method for improving target detection performance performed by an electronic device according to one embodiment, the training data may be configured to further include infrared images.

[0018] In a method for improving target detection performance performed by an electronic device according to one embodiment, the step of training a target detection model may include: a step of verifying augmented data obtained by applying data augmentation to training data; and a step of training a target detection model based on the augmented data.

[0019] In a computer-readable, non-transient computer-readable storage medium storing a program for executing a target detection performance enhancement method according to one embodiment on a computer, the target detection performance enhancement method may include: a step of acquiring an infrared image containing a target; a step of acquiring a multi-channel target image by merging a plurality of single-channel target images, each comprising a single-channel target image generated from an infrared image based on each parameter set included in a plurality of parameter sets; a step of providing a target detection result acquired in correspondence with the multi-channel target image through a target detection model; and a step of training a target detection model based on training data configured to include the multi-channel target image and the target detection result.

[0020] An electronic device for performing a method to improve distance target detection performance according to one embodiment comprises: a processor; and one or more memories for storing one or more instructions, wherein, when the one or more instructions are executed, the processor may control the processor to perform the steps of: acquiring an infrared image containing a target; acquiring a multi-channel target image by merging a plurality of single-channel target images, each comprising a single-channel target image generated from an infrared image based on each parameter set included in a plurality of parameter sets; providing a target detection result acquired in correspondence with the multi-channel target image through a target detection model; and training a target detection model based on training data configured to include the multi-channel target image and the target detection result.

[0021] The various embodiments of the present disclosure described above are merely some of the preferred embodiments of the present disclosure, and various embodiments reflecting the technical features of the various embodiments of the present disclosure can be derived and understood by those skilled in the art based on the detailed description to be described below. Effects of the invention

[0022] The target detection performance enhancement method of the present disclosure has a technical effect in that it provides robust target detection performance against various environmental changes by obtaining an AGC effect with different parameters applied using a single infrared image.

[0023] The target detection performance enhancement method of the present disclosure has a technical effect in terms of improving domain generalization performance by securing diversity of training data through the utilization of various channel information generated with different parameters.

[0024] The target detection performance enhancement method of the present disclosure has technical effects in terms of improving the generalization performance of a target detection model and increasing learning efficiency by utilizing a 16-bit 1-channel infrared image and an 8-bit 3-channel image together as training data.

[0025] The target detection performance enhancement method of the present disclosure has a technical effect in that it enables high-accuracy target detection even in complex environments by securing optimal detection performance through the application of various parameter combinations.

[0026] The effects obtainable from the present disclosure are not limited to those described above, and other unmentioned effects will be clearly understood by those skilled in the art from the description below. Brief explanation of the drawing

[0027] FIG. 1 is a diagram showing the configuration of an electronic device according to one embodiment. FIG. 2 is a flowchart of a method for improving target detection performance using an electronic device according to one embodiment. FIG. 3 is a flowchart of a method for an electronic device to train a target detection model according to one embodiment. FIG. 4 is a diagram showing the configuration of an electronic device that performs a method to improve target detection performance according to one embodiment. FIG. 5 is a diagram showing the configuration of an electronic device that performs a target detection performance enhancement method according to one embodiment. Specific details for implementing the invention

[0028] Hereinafter, various embodiments of the present disclosure are described in detail with reference to the accompanying drawings so that those skilled in the art can easily practice them. The detailed description disclosed below is intended to explain exemplary embodiments of various examples and is not intended to limit the scope to specific embodiments.

[0029] The terms used in the embodiments have been selected to be as widely used as possible, taking into account their functions in the present disclosure; however, these may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant explanatory section. Therefore, terms used in the present disclosure should be defined not merely by their names, but based on their meanings and the overall content of the present disclosure.

[0030] The following embodiments are combinations of the components and features of various embodiments in a predetermined form. Each component or feature may be considered optional unless otherwise explicitly stated. Each component or feature may be implemented in a form not combined with other components or features. Additionally, various embodiments may be constructed by combining some components and features. The order of operations described in various embodiments may be changed. Some components or features of one embodiment may be included in another embodiment, or may be replaced with corresponding components or features of another embodiment.

[0031] In the description of the drawings, procedures or steps that could obscure the essence of the various embodiments were not described, nor were procedures or steps that could be understood by a person with ordinary knowledge in the relevant technical field described.

[0032] Throughout the specification, when a part is described as "comprising" or "including" a component, it means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. In this document, the singular form of a noun corresponding to an item may be used to include both singular and plural forms, unless otherwise indicated in the specification or clearly contradicted by the context. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or any combination thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish a component from other corresponding components and do not limit the components in any other aspect (e.g., importance or order).

[0033] Each component (e.g., module or program) of the components described in this document may include a singular or multiple entities. According to various embodiments, one or more of the components or operations may be omitted, or one or more other components or operations may be added. Additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the components of the multiple components in the same or similar manner as they were performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by a module, program, or other component may be executed sequentially, in parallel, or repeatedly; one or more of the operations may be executed in a different order; may be omitted; or one or more other operations may be added.

[0034] As used in this document, the terms "module" or "...part" refer to a unit that processes at least one function or operation, and may include a unit implemented by hardware, software, firmware, or a combination thereof.

[0035] Various embodiments of this document may be implemented as software (e.g., a program or application) comprising one or more instructions stored in a storage medium (e.g., memory) readable by a machine. For example, the processor of the machine may call at least one of the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to at least one called instruction. One or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by a machine may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain a signal, and does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium.

[0036] In addition, specific terms used in various embodiments are provided to aid in understanding the various embodiments, and the use of such specific terms may be modified in other forms within the scope of not departing from the technical concept of the various embodiments.

[0037] In the present disclosure, the method for improving target detection performance and the electronic device for the same may comprise a method and device for acquiring an infrared image containing a target, acquiring a multi-channel target image by merging a plurality of single-channel target images including each single-channel target image generated from the infrared image based on each parameter set included in a plurality of parameter sets, providing a target detection result acquired in correspondence with the multi-channel target image through a target detection model, and training a target detection model based on training data configured to include the multi-channel target image and the target detection result.

[0038] In particular, the target detection performance enhancement method of the present disclosure can be effectively applied in defense operational environments where visible light-based detection is limited, such as at night, in fog, and / or in smoke. For example, when combat vehicles, unmanned aerial vehicles (UAVs), unmanned ground vehicles (UGVs), and / or surveillance and reconnaissance equipment detect and track concealed and / or camouflaged targets in a battlefield environment, accurate target detection must be possible even in infrared images with low contrast and strong noise; therefore, technology capable of robust target detection under various environmental conditions is required. In response to such characteristics of the defense sector, the distance target detection performance enhancement method according to the present disclosure below can be understood as a technical concept that is easily applicable to the defense sector by merging multiple single-channel target images with multiple parameters applied to infrared images to generate a multi-channel target image, inputting the multi-channel target image into a deep learning model to detect targets, and generating training data to enhance the detection performance of the target detection model.

[0039] FIG. 1 is a diagram showing the configuration of an electronic device (100) according to one embodiment.

[0040] Referring to FIG. 1, the electronic device (100) may include a processor (110) and a memory (120) according to one embodiment. The electronic device (100) illustrated in FIG. 1 is illustrated only with components related to the present embodiment, and it will be understood by those skilled in the art related to the present embodiment that other general-purpose components may be included in addition to the components illustrated in FIG. 1.

[0041] For example, an electronic device (100) may include a communication device comprising one or more transceivers, an input unit, and an output unit. The communication device is a device for performing wired / wireless communication and can communicate with an external electronic device. The external electronic device may be a terminal or a server. In addition, communication technologies used by the communication device may include GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), LTE (Long Term Evolution), 5G, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Bluetooth, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), ZigBee, NFC (Near Field Communication), etc. The input unit may be, for example, a traditional type of keypad or keyboard, a mouse, a microphone for receiving voice signals, a camera, and various other types of input means for detecting or receiving various forms of user input. The output unit may be, for example, a display that outputs images, a speaker that outputs sound, a haptic device that generates vibrations, and various other forms of output means.

[0042] Additionally, at least some of the components within the electronic device (100) may be implemented as an integrated unit or as a single or multiple unit. At least some of the components within the electronic device (100) may be connected to each other via a bus, GPIO (general purpose input / output), SPI (serial peripheral interface) or MIPI (mobile industry processor interface), etc., to exchange data and / or signals.

[0043] The electronic device (100) of FIG. 1 can improve target detection performance. Specifically, the electronic device (100) can acquire an infrared image containing a target. Additionally, the electronic device (100) can acquire a multi-channel target image by merging multiple single-channel target images, each including a single-channel target image generated from an infrared image based on each parameter set included in multiple parameter sets. Furthermore, the electronic device (100) can provide target detection results acquired in correspondence with the multi-channel target image through a target detection model. The electronic device (100) can train the target detection model based on training data configured to include the multi-channel target image and the target detection results.

[0044] A processor (110) is a component capable of performing operations or data processing regarding the control and / or communication of each component of an electronic device (100). For example, the processor (110) can control the electronic device (100) overall by executing programs stored in memory (120) within the electronic device (100). The processor (110) may be implemented as a CPU (central processing unit), GPU (graphics processing unit), AP (application processor), etc., provided within the electronic device (100), but is not limited thereto. Unless there are special circumstances, in this disclosure, the processor (110) may refer to a set of one or more processors (110).

[0045] The processor (110) may be implemented as a computer or a similar device according to hardware, software, or a combination thereof. In terms of hardware, the processor (110) may be implemented in the form of an electronic circuit that processes electrical signals to perform control functions, and in terms of software, it may be implemented in the form of a program that drives the hardware processor (110). Meanwhile, unless otherwise specifically mentioned in the description below, the operation of the electronic device (100) may be interpreted as being performed by the control of the processor (110). That is, when modules implemented in the electronic device (100) to improve target detection performance are executed, the modules may be interpreted as controlling the processor (110) to perform the following operations of the electronic device (100).

[0046] Memory (120) is hardware that stores various data processed within an electronic device (100), and memory (120) can temporarily or semi-permanently store data processed and data to be processed in the electronic device (100). For example, data regarding an operating system (OS) for operating the electronic device (100) can be stored in the memory (120) of the electronic device (100). Memory (120) may include random access memory (RAM) such as dynamic random access memory (DRAM) and static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), CD-ROM, Blu-ray or other optical disc storage, hard disk drive (HDD), solid state drive (SSD), or flash memory. Such memory (120) may be provided as an embedded type or a removable type. Additionally, memory (120) may store instructions for the operation of the processor (110). Unless there are special circumstances, in the present disclosure, memory (120) may mean a set of one or more memories (120).

[0047] The target detection performance enhancement method of the present disclosure, performed by the electronic device (100) of FIG. 1, may also be implemented by a computer-readable non-transient computer-readable storage medium (or non-transient recording medium) for the operation. The target detection performance enhancement method may be implemented as a software module or algorithm and may be stored on a computer-readable recording medium as computer-readable code or program instructions executable on a processor (110). Here, computer-readable recording media include magnetic storage media (e.g., ROM (read-only memory), RAM (random-access memory), floppy disk, hard disk, etc.) and optical reading media (e.g., CD-ROM, DVD (Digital Versatile Disc)). Computer-readable recording media may be distributed across networked computer systems, so that computer-readable code can be stored and executed in a distributed manner. The medium is readable by a computer, stored in memory (120), and can be executed on a processor (110).

[0048] An electronic device (100) according to one embodiment may further include a display (not shown). The display may visually provide various information to the outside under the control of a processor (110). The display may include a touch circuit configured to detect a user's touch or a sensor circuit configured to measure the intensity of the force generated by the touch.

[0049] FIG. 2 is a flowchart of a method for improving target detection performance using an electronic device (100) according to one embodiment.

[0050] In step S210, the electronic device (100) can acquire an infrared image containing a target.

[0051] In the present disclosure, a target may refer to an object that is the subject of detection, surveillance, reconnaissance, and / or tracking by an electronic device (100). For example, a person, vehicle, ship, aircraft, etc., may be understood as a target. The target may be a single object or a set of multiple objects, and all or part of the target may be included in the infrared image.

[0052] Infrared images can be acquired through an infrared camera. An infrared camera can detect targets by selecting one or more of the frequency bands of Long Wavelength Infrared (LWIR), Medium Wavelength Infrared (MWIR), and Short Wavelength Infrared (SWIR). The focal length and / or field of view (FOV) of the lens included in the infrared camera can be appropriately selected depending on the target detection environment.

[0053] Infrared images containing a target can be acquired in a 16-bit, 1-channel image format. Capture parameters such as frame resolution, frame rate, and / or exposure time of the infrared image may be set differently based on the characteristics of the target and detection environment conditions. In some embodiments, the electronic device (100) acquires a continuous frame stream when acquiring an infrared image and, if necessary, selects only a single frame from the continuous frame stream to use for improving target detection performance. Additionally, a time stamp, capture parameter metadata, and / or capture location information may be stored in each frame of the infrared image.

[0054] In some embodiments, the electronic device (100) can perform a pre-correction procedure to minimize noise in the infrared image and thereby obtain an infrared image of higher reliability.

[0055] In step S220, the electronic device (100) can acquire a multi-channel target image by merging multiple single-channel target images, each including a single-channel target image generated from an infrared image based on each parameter set included in a plurality of parameter sets.

[0056] In some embodiments, parameters may refer to control variables that control the characteristics of an infrared image and are adjusted to transform the image, such as contrast, brightness, gamma, and CLAHE (Contrast Limited Adaptive Histogram Equalization) parameters. One or more parameters to be used as control variables for infrared image transformation may be pre-set, and each parameter set may include a parameter setting value for each parameter included in the one or more parameters pre-set for image transformation. When various parameters are applied, target detection performance may be improved by obtaining an AGC (Auto Gain Control) effect.

[0057] The parameter setting values ​​may represent specific numerical values ​​or conditions of the parameters. In some embodiments, the electronic device (100) may determine the parameter setting values ​​included in each parameter set based on the environment in which the target is detected and the characteristics of the target. For example, in a night surveillance environment, the entire infrared image is captured darkly, so the electronic device (100) may set the brightness and contrast of the infrared image relatively high to highlight targets included in dark areas. Conversely, in a daytime environment where strong sunlight is reflected, an excessively bright image may be captured, so the electronic device (100) may lower the brightness and adjust the gamma to suppress excessive image saturation. Additionally, in a low-contrast environment such as fog and / or smoke, the electronic device (100) may adjust the CLAHE parameter to enhance local contrast, thereby enabling effective identification of targets that are difficult to distinguish from the background. Furthermore, when detecting targets with low heat emission, such as concealed or camouflaged vehicles, the electronic device (100) may set the contrast high and adjust the gamma to enable effective identification of the targets. [Table 1] shows an example of parameter sets being set, and the electronic device (100) can obtain multiple single-channel target images, each including a single-channel target image generated corresponding to each parameter set, by applying multiple parameter sets of [Table 1] to an infrared image.

[0058] [Table 1]

[0059]

[0060] In the present disclosure, each single-channel target image may correspond to an 8-bit 1-channel image format, and the multi-channel target image may correspond to an 8-bit 3-channel image format in which three 8-bit 1-channel target images are merged. In some embodiments, the electronic device (100) may obtain a merged multi-channel target image by assigning each single-channel target image to each channel included in a plurality of channels. For example, the merging process may be performed by obtaining an 8-bit 3-channel image by assigning three 8-bit 1-channel images with different parameter sets to the R channel, G channel, and B channel, respectively.

[0061] In step S230, the electronic device (100) can provide a target detection result obtained in response to a multi-channel target image through a target detection model.

[0062] In some embodiments, the target detection model may refer to a deep learning-based object detection model trained to determine the presence of a target from an input image and to produce target detection results such as the locality, class, and / or confidence of the target. In some embodiments, the target detection model is configured based on a Convolutional Neural Network (CNN) structure to extract a feature map of an input multi-channel target image, classify the target based on the extracted features, and produce target detection results. For example, the target detection model may be designed based on an object detection model such as R-CNN (Region-based CNN), Fast R-CNN, Faster R-CNN, YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), or DETR (Detection Transformer).

[0063] In step S240, the electronic device (100) can train a target detection model based on training data configured to include multi-channel target images and target detection results.

[0064] In the present disclosure, training data is a dataset configured to train a target detection model to reliably detect targets in various environments. An electronic device (100) can generate and utilize multi-channel target images as training input data and target detection results as training output data. In some embodiments, a data augmentation technique may be applied to the training data to ensure diversity of the training data. A method of training a target detection model by applying a data augmentation technique to the training data will be described in detail through FIG. 3, which will be described later. In some embodiments, the training data may be configured to include infrared images. By utilizing infrared images and converted multi-channel target images together as training data, the target detection model can simultaneously learn the characteristics of single-channel infrared images and multi-channel target images, thereby improving the generalization performance of the model.

[0065] FIG. 3 is a flowchart of a method for an electronic device (100) according to one embodiment to train a target detection model.

[0066] In step S310, the electronic device (100) can verify the augmented data obtained by applying data augmentation to the training data.

[0067] In the present disclosure, data augmentation may include transformations to artificially expand the diversity of training data to prevent overfitting and improve the generalization performance of a target detection model, and transformations such as rotation, scaling / resize and / or translation, etc. may be applied to the training data for data augmentation.

[0068] In step S320, the electronic device (100) can train a target detection model based on augmented data. It can be understood that the training of the target detection model is performed based on the various embodiments of FIG. 2 described above.

[0069] FIG. 4 is a diagram showing the configuration of an electronic device (400) that performs a target detection performance enhancement method according to one embodiment. Referring to FIG. 4, the electronic device (400) may include an infrared camera (410), a multiple AGC module (420), a channel merging module (430), an automatic target detection module (440), a learning data generation module (450), and a control unit (not shown).

[0070] In some embodiments, the infrared camera (410) can detect heat emission from a target and acquire a 16-bit 1-channel infrared image. The acquired infrared image may contain non-uniformity or noise due to the characteristics of the infrared sensor. The infrared camera (410) can transmit the 16-bit 1-channel infrared image to a multiple AGC module (420).

[0071] In some embodiments, the multi-AGC module (420) can generate three 8-bit 1-channel images by applying three different sets of parameters to a 16-bit 1-channel infrared image. The parameters may correspond to contrast, brightness, gamma, CLAHE parameters, etc., and parameter values ​​may be set according to the detection environment and the characteristics of the target. The multi-AGC module (420) can compress and convert image information to facilitate distinction between the target and the background by emphasizing the signal-concentrated area during the process of converting the 16-bit 1-channel image into an 8-bit 1-channel image. Specifically, by setting the information of the signal-concentrated area within the 16-bit 1-channel image to be utilized primarily, the subtle difference between the target and the background can be effectively expressed, thereby strengthening the contrast between the target and the background and improving target detection performance. The multi-AGC module (420) can transmit the generated three 8-bit 1-channel images to the channel merging module (430).

[0072] In some embodiments, the channel merging module (430) can generate an 8-bit 3-channel image by assigning three 8-bit 1-channel images generated by the multiple AGC module (420) to the R, G, and B channels, respectively. This allows the infrared image to be used as an input to an automatic target detection module (440) designed for RGB image input. The channel merging module (430) can transmit the generated 8-bit 3-channel image to the automatic target detection module (440).

[0073] In some embodiments, the automatic target detection module (440) can take an 8-bit 3-channel image provided from the channel merging module (430) as input and detect a target based on a target detection model to obtain a target detection result. The target detection result may include the location, type, and / or confidence of the detected target. The target detection model may be set based on a deep learning-based object detection model such as R-CNN, Faster R-CNN, YOLO, SSD, or DETR. The automatic target detection module (440) can transmit the obtained target detection result to the training data generation module (450).

[0074] In some embodiments, the training data generation module (450) may generate training data to include an 8-bit 3-channel image generated by the channel merging module (430) and a target detection result acquired by the automatic target detection module (440). The training data may be provided to the training process of the automatic target detection module (440) and utilized to improve target detection performance. The training data generation module (450) may apply data augmentation techniques to the training data to improve the generalization performance of the automatic target detection module (440). If necessary, the training data generation module (450) may utilize a 16-bit 1-channel infrared image acquired by the infrared camera (410) as training data.

[0075] In FIG. 4, it can be understood that each operation of the infrared camera (410), the multi-AGC module (420), the channel merging module (430), the automatic target detection module (440), and the learning data generation module (450) is controlled by a control unit (not shown).

[0076] FIG. 4 illustrates only the components related to the present embodiment, and it will be understood by those skilled in the art related to the present embodiment that other general components may be included in addition to the components illustrated in FIG. 4.

[0077] FIG. 5 is a diagram showing the configuration of an electronic device (500) that performs a target detection performance enhancement method according to one embodiment. Referring to FIG. 5, the electronic device (500) may include an infrared image preprocessing module (510), a model learning module (520), an object detection module (530), a result output module (540), and a control unit (not shown).

[0078] The infrared image preprocessing module (510) can perform preprocessing operations to improve the quality of the input infrared image. The infrared image preprocessing module (510) can improve the accuracy of target detection performance by removing noise included in the infrared image and improving the contrast between the target and the background. The infrared image preprocessing module (510) can be understood as a configuration corresponding to an example of a module that integrates an infrared camera (410), a multiple AGC module (420), and a channel merging module (430) included in the electronic device (400) of FIG. 4 described above.

[0079] In some embodiments, the infrared image preprocessing module (510) may remove noise by applying various filtering techniques to improve target detection performance. A filtering technique may refer to an image processing method that removes noise from an image by correcting each pixel value of the image through operations with surrounding pixels, while simultaneously preserving the image features required for target detection. For example, a median filter, a Gaussian filter, or a non-local means filter may be applied. A median filter replaces a pixel with the median value of surrounding pixels and can be effective in removing salt-and-pepper noise or discontinuous outliers caused by sensor characteristics. A Gaussian filter replaces a pixel value with the weighted average of surrounding pixels by applying a weighted mask following a Gaussian distribution, and can suppress overall high-frequency components of the image to smooth it. The non-local means filter removes noise by considering pixel blocks with similar patterns across the entire image, rather than simply referencing adjacent pixel values. This method can effectively suppress noise while preserving the original features of the image and boundary information between the target and the background, especially in cases where minute differences in temperature distribution are important, such as in infrared images.

[0080] In some embodiments, the infrared image preprocessing module (510) may apply a histogram equalization technique to enhance the contrast of the infrared image. The histogram equalization technique may refer to an image processing method that improves overall contrast by uniformly redistributing the distribution of pixel values ​​within the image to make dark areas brighter and bright areas darker. Since histogram equalization applies a global transformation and performs the same correction across the entire image, excessive application may emphasize noise or distort features in small areas. To mitigate the problems that occur when applying histogram equalization, the Contrast Limited Adaptive Histogram Equalization (CLAHE) technique may be used in the infrared image preprocessing module (510). The CLAHE technique may refer to a method that enhances local contrast by performing histogram equalization on small area units, such as tiles or blocks, rather than on the entire image. Additionally, the CLAHE technique may limit the degree of contrast enhancement to prevent excessive amplification of noise.

[0081] In some embodiments, the infrared image preprocessing module (510) can generate multiple AGC images by applying multiple parameter sets during the noise removal and contrast enhancement process. For example, three different parameter sets can be applied to a 16-bit 1-channel infrared image to generate three 8-bit 1-channel images, and an 8-bit 3-channel image can be generated by assigning them to the R, G, and B channels, respectively. The 8-bit 3-channel image can be provided as an input to an object detection module (530) and utilized to improve target detection performance.

[0082] The model learning module (520) can train the object detection module (530) by extracting features from an 8-bit 3-channel image. For example, the weights and / or biases of the object detection module (530) can be iteratively optimized by utilizing the correct labels corresponding to the 8-bit 3-channel image. At this time, the model learning module (520) can use the 8-bit 3-channel image as training data for the target detection model. In addition, the model learning module (520) can ensure diversity of the training data and prevent overfitting by applying a data augmentation technique to the training data. The model learning module (520) can be understood as a configuration corresponding to an example of the training data generation module (450) of FIG. 4 described above.

[0083] The object detection module (530) can receive an 8-bit 3-channel image as input and detect targets included in the image. The object detection module (530) is configured based on a deep learning-based object detection model to extract features from the 8-bit 3-channel image and detect targets based on the extracted features. The object detection module (530) can be understood as a configuration corresponding to an example of the target detection model of FIGS. 1 to 3 described above or the automatic target detection module (440) of FIG. 4 described above.

[0084] The result output module (540) can provide the target detection result calculated by the object detection module (530) to the user. The result output module (540) can visually display the detection result, including the location, type, and / or reliability of the detected target, or record it on an external storage medium or database. The result output module (540) can be understood as a configuration corresponding to an example of a module that outputs the target detection result in the target detection model of FIGS. 1 to 3 described above or the automatic target detection module (440) of FIG. 4 described above.

[0085] In some embodiments, the result output module (540) may display the location of the detected target in the form of a bounding box or mask on the infrared image and display the type and confidence value of each target together so that the user can intuitively check the target detection results.

[0086] In some embodiments, the result output module (540) can store the target detection results in the form of a text file, a CSV file, or a database, so that they can be used for subsequent target detection or for generating training data.

[0087] In some embodiments, the result output module (540) may support the user in directly viewing target detection results, changing system operation settings, and saving and managing target detection results as needed through a user interface (UI).

[0088] In some embodiments, the target detection result output by the result output module (540) is evaluated through indicators such as precision, recall, F1-score, and / or mAP (mean Average Precision), and the performance of the electronic device (500) can be determined.

[0089] In FIG. 5, it can be understood that each operation of the image preprocessing module (510), model learning module (520), object detection module (530), and result output module (540) is controlled by a control unit (not shown).

[0090] FIG. 5 illustrates only the components related to the present embodiment, and it will be understood by those skilled in the art related to the present embodiment that other general components may be included in addition to the components illustrated in FIG. 5.

[0091] Meanwhile, an example of an actual device implementation and experimental environment for carrying out the target detection performance enhancement method according to the present disclosure may be as follows.

[0092] - Hardware: CPU: Intel i7-11700, GPU: NVIDIA GeForce RTX 3070, RAM: 32GB

[0093] - Operating System: Ubuntu 20.04

[0094] - Programming Language: Python

[0095] - Deep learning framework: TensorFlow or PyTorch

[0096] - Image processing library: OpenCV

[0097] - Dataset: Utilizing infrared imaging datasets such as the FLIR ADAS dataset

[0098] The above device implementation and experimental environment are merely examples, and the target detection performance enhancement method of the present disclosure may be performed according to various device implementations and experimental environments.

[0099] An infrared image processing system including the method of the present invention described above can be widely utilized in various industrial fields such as night surveillance, autonomous driving, robot vision, and industrial automation.

[0100] Specifically, in the field of security and surveillance, surveillance performance can be improved by applying the system of the present invention to existing CCTV systems to provide high-quality infrared images even at night or in adverse weather conditions. In addition, it can be applied to intrusion detection systems using infrared cameras to effectively detect disguised or concealed intruders, and furthermore, it can be utilized in military and police operations for night surveillance, target identification, search and rescue operations, etc., thereby improving operational capabilities.

[0101] In the automotive and autonomous driving sectors, it can be applied to Advanced Driver Assistance Systems (ADAS) to detect pedestrians, animals, and vehicles even at night or in adverse weather conditions, thereby preventing collision accidents. Furthermore, when integrated into the sensor systems of autonomous vehicles, it can support safe driving even in poor visual environments.

[0102] In the field of firefighting and disaster response, it can detect smoke or heat when a fire occurs to provide early warnings and minimize casualties, and can be utilized to support rescue operations at fire or disaster sites and rapidly assess the situation on site.

[0103] The infrared image processing system of the present invention can be utilized in various fields in addition to the aforementioned industrial fields and can be understood to contribute to the development of related industries. In particular, it can be understood that by combining it with deep learning-based object detection technology, it can provide higher target detection performance and reliability, thereby securing market competitiveness.

[0104] In the foregoing, although all components constituting the embodiments disclosed in this document have been described as being combined or operating in combination, the embodiments disclosed in this document are not necessarily limited to such embodiments. That is, within the scope of the purposes of the embodiments disclosed in this document, all components may be selectively combined in one or more ways to operate.

[0105] Furthermore, terms such as "include," "compose," or "have" as described above, unless specifically stated otherwise, mean that the relevant component may be inherent; thus, they should be interpreted as allowing for the inclusion of additional components rather than excluding them. All terms, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the embodiments disclosed in this document pertain, unless otherwise defined. Commonly used terms, such as those defined in advance, should be interpreted in accordance with their meaning in the context of the relevant technology and, unless explicitly defined in this document, should not be interpreted in an ideal or overly formal sense.

[0106] The foregoing description is merely an illustrative explanation of the technical concept disclosed in this document, and a person skilled in the art to which the embodiments disclosed in this document pertain can make various modifications and variations within the scope of the essential characteristics of the embodiments disclosed in this document. Accordingly, the embodiments disclosed in this document are intended to explain, not limit, the technical concept of the embodiments disclosed in this document, and the scope of the technical concept disclosed in this document is not limited by these embodiments. The scope of protection of the technical concept disclosed in this document shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of this document.

Claims

Claim 1 A method for improving target detection performance, performed by an electronic device, comprising: acquiring a single-channel infrared image containing a target; acquiring a plurality of single-channel target images by merging the plurality of single-channel target images, each comprising a single-channel target image generated from the infrared image based on each parameter set included in a plurality of parameter sets; and providing a target detection result acquired in correspondence with the multi-channel target image through a target detection model. A method for improving target detection performance, comprising the step of training a target detection model based on training data configured to include the multi-channel target image and the target detection result, wherein the step of acquiring the multi-channel target image further comprises the step of generating each single-channel target image so as to facilitate distinction between the target and the background excluding the target in the infrared image by emphasizing a region where the signal is concentrated in the infrared image, wherein the infrared image, each single-channel target image, and the multi-channel target image correspond to a 16-bit 1-channel, an 8-bit 1-channel, and an 8-bit 3-channel, respectively, and wherein the target detection model is managed to learn single-channel image characteristics and multi-channel image characteristics simultaneously based on the fact that the training data is configured to further include the infrared image in order to improve the generalization performance of the target detection model. Claim 2 A method for improving target detection performance according to claim 1, wherein each parameter set includes each parameter setting value for each parameter included in one or more parameters pre-set for image conversion, and the step of acquiring the multi-channel target image includes the step of determining the parameter setting values ​​included in each parameter set based on the environment for detecting the target and the characteristics of the target. Claim 3 A method for improving target detection performance according to claim 2, wherein one or more parameters include contrast, brightness, gamma, and CLAHE (Contrast Limited Adaptive Histogram Equalization) parameters. Claim 4 A method for improving target detection performance according to claim 1, wherein the step of acquiring the multi-channel target image comprises the step of acquiring the multi-channel target image by merging the single-channel target image by assigning it to each channel included in the plurality of channels, and wherein the plurality of channels include R, G, and B channels. Claim 5 A method for improving target detection performance according to claim 1, wherein the target detection model is configured based on a deep learning-based object detection model. Claim 6 A method for improving target detection performance according to claim 1, wherein the target detection result includes the localization, class, and confidence of the target. Claim 7 delete Claim 8 A method for improving target detection performance according to claim 1, wherein the step of training the target detection model comprises: a step of verifying augmented data obtained by applying data augmentation to the training data; and a step of training the target detection model based on the augmented data. Claim 9 In a computer-readable, non-transient computer-readable storage medium storing a program for executing a method for improving target detection performance on a computer, the method for improving target detection performance comprises: a step of acquiring a single-channel infrared image containing a target; a step of acquiring a multi-channel target image by merging a plurality of single-channel target images, each comprising a single-channel target image generated from the infrared image based on each parameter set included in a plurality of parameter sets; and a step of providing a target detection result acquired in correspondence with the multi-channel target image through a target detection model. A non-transient computer-readable storage medium comprising the step of training the target detection model based on training data configured to include the multi-channel target image and the target detection result, wherein the step of acquiring the multi-channel target image further comprises the step of generating each single-channel target image so as to facilitate distinction between the target and the background excluding the target in the infrared image by emphasizing a region where the signal is concentrated in the infrared image, wherein the infrared image, each single-channel target image, and the multi-channel target image correspond to a 16-bit 1-channel, an 8-bit 1-channel, and an 8-bit 3-channel, respectively, and wherein the target detection model is managed to learn single-channel image characteristics and multi-channel image characteristics simultaneously based on the fact that the training data is configured to further include the infrared image in order to improve the generalization performance of the target detection model. Claim 10 An electronic device for performing a method to improve target detection performance comprises: a processor; and one or more memories for storing one or more instructions, wherein, when the one or more instructions are executed, the processor comprises: a step of acquiring a single-channel infrared image containing a target; a step of acquiring a multi-channel target image by merging a plurality of single-channel target images, each comprising a single-channel target image generated from the infrared image based on each parameter set included in a plurality of parameter sets; and a step of providing a target detection result acquired in correspondence with the multi-channel target image through a target detection model. An electronic device that controls the processor to perform the step of training the target detection model based on training data configured to include the multi-channel target image and the target detection result, wherein the step of acquiring the multi-channel target image further includes the step of generating each single-channel target image so as to facilitate distinction between the target and the background excluding the target in the infrared image by emphasizing a region where the signal is concentrated in the infrared image, wherein the infrared image, each single-channel target image, and the multi-channel target image correspond to a 16-bit 1-channel, an 8-bit 1-channel, and an 8-bit 3-channel, respectively, and wherein the target detection model is managed to learn single-channel image characteristics and multi-channel image characteristics simultaneously based on the fact that the training data is configured to further include the infrared image in order to improve the generalization performance of the target detection model.

Images