Apparatus for detecting object using compound image and method thereof

Abstract

The present invention discloses an object detection device and a method utilizing a synthetic image. The object detection device utilizing a synthetic image according to the present invention comprises: an optical imaging unit for capturing an optical image; a thermal imaging unit for capturing a thermal image; a VL encoder for extracting a first VL feature vector from an optical image; an IR encoder for extracting a first IR feature vector from a thermal image; a feature fusion module for receiving the first VL feature vector and the first IR feature vector as inputs and synthesizing them based on deep learning to update the feature vector; a decoder for upsampling the feature vector updated from the feature fusion module; and an object position prediction unit for predicting an object position based on the feature vector upsampled by the decoder.

Description

Technology Field

[0001] The present invention relates to an object detection device and method utilizing synthetic images, and more specifically, to an object detection device and method utilizing synthetic images that detects power lines through an artificial neural network model by extracting feature vectors from optical images and thermal images, respectively, and synthesizing them. Background Technology

[0002] Recently, deep learning models have been changing the research paradigm by demonstrating remarkable performance improvements in various fields of computer vision. While conventional computer vision research focused on designing useful features based on human knowledge, such as SIFT (Scale Invariant Feature Transform) and HOG (Histogram of Oriented Gradient), deep learning adopts an approach of directly learning useful features from data using multilayer artificial neural networks.

[0003] These learning-based methods have a greater advantage in performance improvement because they can directly discover useful features through data that humans are unaware of.

[0004] In the electric power industry, the periodic monitoring of power facilities is considered a critical task for ensuring a stable power supply to households. Previously, monitoring was performed by physically accessing power facilities located in remote areas, such as mountainous regions; however, this method was time-consuming due to the large number of facilities and required a significant amount of labor because accessibility to individual equipment was poor.

[0005] Therefore, in order to reduce the labor and time required for monitoring power facilities, there have recently been attempts to attach vision sensors to unmanned aerial vehicles (e.g., drones) to acquire images and then perform monitoring using image analysis algorithms. In this process, power lines are treated not only as obstacles to be avoided during flight but also as a critical monitoring task for detecting defects in the power lines themselves. Consequently, detecting the location of power lines within images is a very important factor in power facility monitoring.

[0006] Meanwhile, in the field of image recognition, there is a trend of using algorithms based on data-driven deep learning, which demonstrate improved performance compared to existing rule-based machine vision methods, for detecting power lines in images.

[0007] The background technology of the present invention is disclosed in Korean Patent Publication No. 10-1897923 (published on October 31, 2018, system for providing tracking object extraction service based on neural network learning model, method thereof, and computer-readable storage medium storing the method thereof). The problem to be solved

[0008] As such, the algorithm utilizing deep learning receives images via a camera attached to a drone for efficient power line detection, and applies a deep learning model with the received images as input to finally recognize the location of power lines within the images.

[0009] However, this method acquires images by attaching only an optical camera to the drone, and thus has performance limitations due to the disadvantages of optical imaging.

[0010] Optical camera images have the disadvantage of being sensitive to changes in the external environment. That is, depending on external conditions such as the presence or absence of sunlight, shadows, or backlighting at the time of shooting, power lines may not appear clearly in the captured images. Additionally, environmental factors such as foggy weather, rain, or snow may make it difficult to detect power lines in the images.

[0011] As shown in Figure 1 (a), the background appears clearly in optical camera images, and objects in the background, such as trees, grass, and structures, often obscure power lines. Due to these disadvantages, artificial neural networks that use only optical camera images as input are sensitive to the environment at the time of shooting, and there is a problem in that the power line detection performance fluctuates significantly depending on the state of the image.

[0012] Furthermore, power lines can be detected by utilizing thermal imaging camera footage that detects temperature, based on the observation that heat generated by power facilities (transmission towers, insulators, etc.) and power lines has a specific temperature range distinct from surrounding background objects. However, relying solely on thermal imaging cameras limits performance due to the inherent drawbacks of thermal images.

[0013] As shown in Figure 1 (b), thermal imaging cameras have lower resolution compared to optical camera images, so there is a problem in that it is difficult to locate the exact position of power lines. That is, although they are relatively more robust than optical cameras in the presence or absence of sunlight and changes in weather, they have low contrast, making it difficult to identify the position of power lines with the naked eye.

[0014] The present invention has been devised to improve upon the aforementioned problems, and the objective of the present invention according to one aspect is to provide an object detection device and method utilizing a composite image that detects power lines through an artificial neural network model by extracting feature vectors from an optical image and a thermal image, respectively, and synthesizing them. means of solving the problem

[0015] An object detection device utilizing a synthetic image according to one aspect of the present invention comprises: an optical imaging unit for capturing an optical image; a thermal imaging unit for capturing a thermal image; a VL encoder for extracting a first VL feature vector from an optical image; an IR encoder for extracting a first IR feature vector from a thermal image; a feature fusion module for receiving the first VL feature vector and the first IR feature vector as inputs and synthesizing them based on deep learning to update the feature vector; a decoder for upsampling the feature vector updated from the feature fusion module; and an object position prediction unit for predicting the object position based on the feature vector upsampled by the decoder.

[0016] In the present invention, the feature fusion module comprises: a VL feature vector extraction unit that extracts a new second VL feature vector by performing a convolution operation on a first VL feature vector; an IR feature vector extraction unit that extracts a new second IR feature vector by performing a convolution operation on a first IR feature vector; an attention vector extraction unit that extracts an attention vector having a probability value between 0 and 1 after continuously stacking the second VL feature vector and the second IR feature vector in the channel direction and performing a convolution operation; a VL channel adjustment unit that extracts a third VL feature vector by applying the attention vector extracted from the attention vector extraction unit to the first VL feature vector to update the feature vector; and an IR channel adjustment unit that extracts a third IR feature vector by applying the attention vector extracted from the attention vector extraction unit to the first IR feature vector to update the feature vector.

[0017] In the present invention, the VL feature vector extraction unit and the IR feature vector extraction unit are each characterized by performing calculations by applying a plurality of convolution filters of different sizes.

[0018] In the present invention, the attention vector extraction unit is characterized by extracting an attention vector by normalizing the value of each element by mapping it to a value between 0 and 1 through a sigmoid function.

[0019] In the present invention, the VL channel adjustment unit is characterized by extracting a third VL feature vector by multiplying a first VL feature vector and an attention vector by channel.

[0020] In the present invention, the IR channel adjustment unit is characterized by extracting a third IR feature vector by multiplying a first IR feature vector and an attention vector by channel.

[0021] An object detection method using a synthetic image according to another aspect of the present invention comprises: a step in which a VL encoder and an IR encoder each receive an optical image and a thermal image; a step in which the VL encoder and the IR encoder each extract a first VL feature vector and a first IR feature vector from the optical image and the thermal image, respectively; a step in which a feature fusion module synthesizes the first VL feature vector and the first IR feature vector based on deep learning to update the feature vector; a step in which a decoder upsamples and decodes the feature vector updated from the feature fusion module; and a step in which an object position prediction unit predicts the object position based on the upsampled feature vector.

[0022] The step of updating the feature vector in the present invention comprises: a step in which a feature fusion module extracts a new second VL feature vector and a second IR feature vector by performing a convolution operation on the first VL feature vector and the first IR feature vector, respectively; a step in which a feature fusion module extracts an attention vector having a probability value between 0 and 1 after performing a convolution operation by continuously stacking the second VL feature vector and the second IR feature vector in the channel direction; and a step in which a feature fusion module extracts a third VL feature vector and a third IR feature vector by applying the attention vector to the first VL feature vector and the first IR feature vector, respectively.

[0023] In the present invention, the step of extracting the second VL feature vector and the second IR feature vector is characterized by the feature fusion module performing calculations by applying a plurality of convolution filters of different sizes.

[0024] In the present invention, the step of extracting an attention vector is characterized by the feature fusion module normalizing the value of each element by mapping it to a value between 0 and 1 through a sigmoid function to extract the attention vector.

[0025] In the present invention, the step of extracting the third VL feature vector and the third IR feature vector is characterized by the feature fusion module extracting the third VL feature vector and the third IR feature vector by multiplying the first VL feature vector and the first IR feature vector by an attention vector for each channel. Effects of the invention

[0026] An object detection device and method utilizing a synthetic image according to one aspect of the present invention can improve performance compared to a method using a single image by extracting feature vectors from an optical image and a thermal image, respectively, synthesizing them, and detecting power lines through an artificial neural network model. Furthermore, it can inspect power lines more stably than existing methods not only in clear and sunny weather but also in deteriorated weather conditions such as snow and rain, and situations where visibility is obscured such as fine dust and fog, and can expand the scope to improve the success rate of recognizing objects identifiable by thermal imaging cameras throughout the power facilities.

[0027] In addition, according to the present invention, since the number of parameters of the artificial neural network model is not significantly increased through the feature fusion module, the processing speed is not increased, so it can be used as a real-time inspection module, and can also be used efficiently even in situations where hardware with low memory capacity, such as drones, is used. Brief explanation of the drawing

[0028] Figure 1 is an image showing power lines captured through a general thermal imaging camera and an optical camera. FIG. 2 is a block diagram showing an object detection device using a synthetic image according to an embodiment of the present invention. FIG. 3 is an exemplary diagram illustrating the processing process of an object detection device utilizing a synthetic image according to an embodiment of the present invention. FIG. 4 is a block diagram showing a feature fusion module in an object detection device using a synthetic image according to an embodiment of the present invention. FIG. 5 is a graph showing the output value by a sigmoid function in an object detection device using a synthetic image according to one embodiment of the present invention. FIG. 6 is an example diagram illustrating the channel adjustment process in an object detection device utilizing a synthetic image according to one embodiment of the present invention. FIG. 7 is an example diagram showing the use of a convolution filter in a feature fusion module of an object detection device utilizing a synthetic image according to one embodiment of the present invention. FIG. 8 is a flowchart illustrating an object detection method using a synthetic image according to an embodiment of the present invention. Specific details for implementing the invention

[0029] Hereinafter, an object detection device and method utilizing a composite image according to the present invention will be described with reference to the attached drawings. In this process, the thickness of lines or the size of components depicted in the drawings may be exaggerated for clarity and convenience of explanation. Furthermore, the terms described below are defined considering their functions in the present invention, and these may vary depending on the intention or convention of the user or operator. Therefore, the definitions of these terms should be based on the content throughout this specification.

[0030] FIG. 2 is a block diagram showing an object detection device using a synthetic image according to an embodiment of the present invention, FIG. 3 is an example diagram illustrating the processing process of an object detection device using a synthetic image according to an embodiment of the present invention, FIG. 4 is a block diagram showing a feature fusion module in an object detection device using a synthetic image according to an embodiment of the present invention, FIG. 5 is a graph showing the output value by a sigmoid function in an object detection device using a synthetic image according to an embodiment of the present invention, FIG. 6 is an example diagram illustrating the channel adjustment process in an object detection device using a synthetic image according to an embodiment of the present invention, and FIG. 7 is an example diagram showing the use of a convolution filter in the feature fusion module of an object detection device using a synthetic image according to an embodiment of the present invention.

[0031] As illustrated in FIG. 2, an object detection device utilizing a synthetic image according to one embodiment of the present invention may include an optical imaging unit (10), a thermal imaging unit (20), a VL encoder (30), an IR encoder (40), a feature fusion module (50), a decoder (60), and an object position prediction unit (70).

[0032] The optical imaging unit (10) captures an optical image of visible light (VL) to detect an object.

[0033] The thermal imaging unit (20) captures a thermal image of infrared (IR; Infra Red) to detect an object.

[0034] The VL encoder (30) can extract a first VL feature vector from an optical image.

[0035] The IR encoder (40) can extract a first IR feature vector from the thermal image.

[0036] Here, the VL encoder (30) and the IR encoder (40) can each receive an optical image and a thermal image and extract a feature vector, and the extracted feature vector has a smaller size than the input image.

[0037] In addition, the operation process in the VL encoder (30) and IR encoder (40) can be configured to extract feature vectors by including a convolution layer that performs convolution operations, an activation function such as ReLU and leaky ReLU that adds non-linearity to the operation, and a batch normalization operation that performs normalization in the feature vector extraction process.

[0038] The Feature Fusion Module (50) can update the feature vector by receiving the first VL feature vector and the first IR feature vector as input and synthesizing them based on deep learning.

[0039] The decoder (60) can upsample the updated feature vector from the feature fusion module (50) to create a feature vector with a reduced size through computation in the VL encoder (30) and IR encoder (40), making it the same size as the input image.

[0040] Here, the decoder (60) may include bilinear up-sampling, bicubic up-sampling, etc., which do not use learning parameters, and may also include an upsampling technique using a transposed convolutional layer that uses learning parameters.

[0041] As shown in FIG. 3, an optical image and a thermal image are input, and a first VL feature vector and a first IR feature vector are extracted through the computation process of a VL encoder (30) and an IR encoder (40). Then, the feature vectors synthesized through a feature fusion module are stacked continuously (concatenated) in the channel direction with the feature vectors extracted by the decoder (60) to perform an upsampling process.

[0042] The object location prediction unit (70) can predict the object location based on the feature vector upsampled in the decoder (60).

[0043] To explain this more specifically, as shown in FIG. 4, the feature fusion module (50) may include a VL feature vector extraction unit (510), an IR feature vector extraction unit (520), an attention vector extraction unit (530), a VL channel adjustment unit (540), and an IR channel adjustment unit (550).

[0044] The VL feature vector extraction unit (510) can extract a new second VL feature vector by performing a convolution operation on the first VL feature vector.

[0045] The IR feature vector extraction unit (520) can extract a new second IR feature vector by performing a convolution operation on the first IR feature vector.

[0046] Here, the VL feature vector extraction unit (510) and the IR feature vector extraction unit (520) can extract the second VL feature vector and the second IR feature vector, respectively, using convolution filters of sizes 1X1, 3X3, and 5X5 as shown in FIG. 7.

[0047] The attention vector extraction unit (530) can extract an attention vector having a probability value between 0 and 1 by concatenating the second VL feature vector and the second IR feature vector in the channel direction and then performing a convolution operation with the result as input.

[0048] Here, the attention vector extraction unit (530) can extract an attention vector by normalizing the value of each element to a value between 0 and 1 through the sigmoid function of Equation 1. Thus, as shown in FIG. 5, the attention vector has a value between 0 and 1 and has the characteristic that the importance of the value and the magnitude of the value are directly proportional.

[0049]

[0050] The VL channel adjustment unit (540) can extract a third VL feature vector by updating the feature vector through channel-by-channel multiplication of the attention vector extracted from the attention vector extraction unit (530).

[0051] The IR channel adjustment unit (550) can extract a third IR feature vector by updating the feature vector through channel-by-channel multiplication of the attention vector extracted from the attention vector extraction unit (530).

[0052] That is, as shown in Fig. 6, the first VL feature vector and the attention vector (c) can be updated to the third VL feature vector through an operation of multiplying them by channel.

[0053] Here, the attention vector serves to emphasize the channel values ​​that are helpful for detecting power line regions in the first VL feature vector and the first IR feature vector, while reducing the influence of unimportant channel values. In other words, each element of the attention vector can be trained to have an importance value for the corresponding channel of the feature vector.

[0054] As described above, according to the object detection device utilizing a synthetic image according to an embodiment of the present invention, power lines can be detected through an artificial neural network model by extracting feature vectors from optical images and thermal images, respectively, and synthesizing them, thereby improving performance compared to a method using a single image. Furthermore, power lines can be inspected more stably than conventional methods not only in clear and sunny weather but also in deteriorated weather conditions such as snow and rain, or in situations where visibility is obscured such as fine dust and fog. Additionally, the scope can be expanded to improve the success rate of recognizing objects identifiable by thermal cameras throughout the power facility. Moreover, since the number of parameters of the artificial neural network model is not significantly increased through the feature fusion module, processing speed is not increased, allowing it to be used as a real-time inspection module. It can also be used efficiently even in situations where hardware with limited memory capacity, such as drones, is used.

[0055] FIG. 8 is a flowchart illustrating an object detection method using a synthetic image according to an embodiment of the present invention.

[0056] As illustrated in FIG. 8, in an object detection method using a composite image according to an embodiment of the present invention, first, a VL encoder (30) and an IR encoder (40) each receive an optical image and a thermal image from an optical imaging unit (10) and a thermal imaging unit (20), respectively (S10).

[0057] In step S10, the VL encoder (30) and the IR encoder (40) receive an optical image and a thermal image, and then extract a first VL feature vector and a first IR feature vector from the optical image and the thermal image, respectively (S20).

[0058] Here, the VL encoder (30) and the IR encoder (40) can each receive an optical image and a thermal image and extract a feature vector, and the extracted feature vector has a smaller size than the input image.

[0059] In addition, the operation process for extracting the first VL feature vector and the first IR feature vector in the VL encoder (30) and IR encoder (40) can extract the feature vector by including a convolution layer that performs a convolution operation, an activation function such as ReLU and leaky ReLU that adds non-linearity to the operation, and a batch normalization operation that performs normalization in the feature vector extraction process.

[0060] After extracting the first VL feature vector and the first IR feature vector in step S20, the feature fusion module (50) can update the feature vector by synthesizing the first VL feature vector and the first IR feature vector based on deep learning.

[0061] That is, the feature fusion module (50) extracts a new second VL feature vector and a second IR feature vector by performing a convolution operation on the first VL feature vector and the first IR feature vector, respectively (S30).

[0062] Here, the feature fusion module (50) can extract the second VL feature vector and the second IR feature vector, respectively, using convolution filters of sizes 1X1, 3X3, and 5X5 as shown in FIG. 7.

[0063] After extracting the second VL feature vector and the second IR feature vector in step S30, the feature fusion module (50) continuously stacks the second VL feature vector and the second IR feature vector in the channel direction and performs a convolution operation to extract an attention vector having a probability value between 0 and 1 (S40).

[0064] Here, the feature fusion module (50) can extract an attention vector by normalizing the value of each element to a value between 0 and 1 through the sigmoid function of Equation 2. Thus, as shown in FIG. 5, the attention vector has a value between 0 and 1 and has the characteristic that the importance of the value and the magnitude of the value are directly proportional.

[0065]

[0066] After extracting the attention vector in step S40, the feature fusion module (50) multiplies the attention vector by the first VL feature vector and the first IR feature vector by channel to extract the third VL feature vector and the third IR feature vector (S50).

[0067] That is, as shown in Fig. 6, the first VL feature vector and the attention vector (c) can be updated to the third VL feature vector through a multiplication operation in which they are multiplied by channel.

[0068] Here, the attention vector serves to emphasize the values ​​of channels that are helpful for detecting power line regions in the first VL feature vector and the first IR feature vector, while reducing the influence of unimportant channel values. In other words, each element of the attention vector can be trained to have an importance value for the corresponding channel of the feature vector.

[0069] In step S50, the decoder (60) upsamples and decodes the feature vector updated with the third VL feature vector and the third IR feature vector (S60).

[0070] The decoder (60) can upsample the updated feature vector from the feature fusion module (50) to create a feature vector with a reduced size through computation in the VL encoder (30) and IR encoder (40), making it the same size as the input image.

[0071] Here, the decoder (60) may include bilinear up-sampling, bicubic up-sampling, etc., which do not use learning parameters, and may also include an upsampling technique using a transposed convolutional layer that uses learning parameters.

[0072] In step S60, the object location prediction unit (70) predicts the object location based on the upsampled feature vector (S70).

[0073] As described above, according to the object detection method using a synthetic image according to an embodiment of the present invention, power lines can be detected through an artificial neural network model by extracting feature vectors from optical images and thermal images, respectively, and synthesizing them, thereby improving performance compared to a method using a single image. Furthermore, power lines can be inspected more stably than conventional methods not only in clear and sunny weather but also in deteriorated weather conditions such as snow and rain, or in situations where visibility is obscured such as fine dust and fog. Additionally, the scope can be expanded to improve the success rate of recognizing objects identifiable by thermal cameras throughout the power facility. Moreover, since the number of parameters of the artificial neural network model is not significantly increased through the feature fusion module, processing speed is not increased, allowing it to be used as a real-time inspection module. It can also be used efficiently even in situations where hardware with limited memory capacity, such as drones, is used.

[0074] The implementations described herein may be implemented, for example, as methods or processes, devices, software programs, data streams, or signals. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), the implementation of the discussed features may also be implemented in other forms (e.g., devices or programs). Devices may be implemented in appropriate hardware, software, and firmware, etc. Methods may be implemented in devices such as processors, which generally refer to processing devices including, for example, computers, microprocessors, integrated circuits, or programmable logic devices. Processors also include communication devices such as computers, cell phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate the communication of information between end-users.

[0075] Although the present invention has been described with reference to the embodiments illustrated in the drawings, this is merely illustrative, and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom.

[0076] Therefore, the true technical scope of protection of the present invention should be determined by the claims below. Explanation of the symbols

[0077] 10: Optical imaging unit 20: Thermal imaging unit 30: VL encoder 40: IR encoder 50: Feature Fusion Module 60: Decoder 70 : Object position prediction unit 510 : VL feature vector extraction unit 520 : IR feature vector extraction unit 530: Attention Vector Extraction Unit 540: VL Channel Adjustment Unit 550 : IR Channel Adjustment Unit

Claims

Claim 1 An optical imaging unit for capturing an optical image; a thermal imaging unit for capturing a thermal image; a VL encoder for extracting a first VL feature vector from the optical image; an IR encoder for extracting a first IR feature vector from the thermal image; a feature fusion module for receiving the first VL feature vector and the first IR feature vector as inputs and synthesizing them based on deep learning to update the feature vector; a decoder for upsampling the feature vector updated from the feature fusion module; and an object position prediction unit for predicting an object position based on the feature vector upsampled by the decoder; wherein the feature fusion module comprises: a VL feature vector extraction unit for extracting a new second VL feature vector by performing a convolution operation on the first VL feature vector; and an IR feature vector extraction unit for extracting a new second IR feature vector by performing a convolution operation on the first IR feature vector. An object detection device utilizing a synthetic image, characterized by comprising: an attention vector extraction unit that extracts an attention vector having a probability value between 0 and 1 after continuously stacking the second VL feature vector and the second IR feature vector in the channel direction and performing a convolution operation; a VL channel adjustment unit that extracts a third VL feature vector by applying the attention vector extracted by the attention vector extraction unit to the first VL feature vector to update the feature vector; and an IR channel adjustment unit that extracts a third IR feature vector by applying the attention vector extracted by the attention vector extraction unit to the first IR feature vector to update the feature vector. Claim 2 delete Claim 3 An object detection device utilizing a synthetic image according to claim 1, wherein the VL feature vector extraction unit and the IR feature vector extraction unit each apply a plurality of convolution filters of different sizes to perform calculations. Claim 4 An object detection device utilizing a synthetic image, wherein, in claim 1, the VL channel adjustment unit extracts the third VL feature vector by multiplying the first VL feature vector and the attention vector by channel. Claim 5 An object detection device utilizing a synthetic image, wherein, in claim 1, the IR channel adjustment unit extracts the third IR feature vector by multiplying the first IR feature vector and the attention vector by channel. Claim 6 An object detection device utilizing a synthetic image according to claim 1, wherein the attention vector extraction unit extracts the attention vector by normalizing the value of each element to a value between 0 and 1 through a sigmoid function. Claim 7 A step in which a VL encoder and an IR encoder each receive an optical image and a thermal image, respectively; a step in which the VL encoder and the IR encoder each extract a first VL feature vector and a first IR feature vector from the optical image and the thermal image, respectively; a step in which a feature fusion module synthesizes the first VL feature vector and the first IR feature vector based on deep learning to update the feature vector; a step in which a decoder upsamples and decodes the feature vector updated from the feature fusion module; and a step in which an object location prediction unit predicts the object location based on the upsampled feature vector, wherein the step of updating the feature vector comprises: a step in which the feature fusion module extracts a new second VL feature vector and a second IR feature vector by performing a convolution operation on the first VL feature vector and the first IR feature vector, respectively. A method for object detection using a synthetic image, characterized by comprising: a step in which the feature fusion module continuously stacks the second VL feature vector and the second IR feature vector in the channel direction and performs a convolution operation to extract an attention vector having a probability value between 0 and 1; and a step in which the feature fusion module applies the attention vector to the first VL feature vector and the first IR feature vector, respectively, to extract a third VL feature vector and a third IR feature vector. Claim 8 delete Claim 9 In claim 7, the step of extracting the second VL feature vector and the second IR feature vector is characterized by the feature fusion module performing calculations by applying a plurality of convolution filters of different sizes to each, thereby an object detection method using a synthetic image. Claim 10 In claim 7, the step of extracting the attention vector is characterized by the feature fusion module normalizing the value of each element to a value between 0 and 1 through a sigmoid function to extract the attention vector, thereby providing an object detection method using a synthetic image. Claim 11 In claim 7, the step of extracting the third VL feature vector and the third IR feature vector is characterized in that the feature fusion module extracts the third VL feature vector and the third IR feature vector by multiplying the first VL feature vector and the first IR feature vector by the attention vector on a channel-by-channel basis, thereby providing an object detection method using a synthetic image.

Images