Apparatus and method for compositing multiple images

Abstract

The present invention relates to an apparatus and a method for compositing multiple images and, more particularly, to an apparatus and a method for compositing multiple images, which are capable of visualizing a hidden object by means of an image obtained using terahertz waves. The present invention discloses a multi-image compositing apparatus (100) comprising: an image acquisition unit (110) for acquiring a first image in which the shape and background of a human body (B) are captured, and a second image in which a human body hidden object (C) is captured; an image processing unit (120) for performing image processing on the first image and the second image; and an image compositing unit (130) for compositing the image-processed first image and second image, wherein the second image is a terahertz wave image.

Description

Multi-image synthesis device and method

[0001] The present invention relates to a multi-image synthesis device and method, and more particularly, to a multi-image synthesis device and method capable of visualizing a hidden object through an image using terahertz waves.

[0002] Terahertz waves are penetrating electromagnetic waves. The word is a combination of "tera," meaning 10 to the 12th power, and "hertz" (Hz), the unit of frequency. Terahertz waves can be utilized in diverse fields, including spectroscopy, medicine, national defense, and security. Therefore, active research is being conducted on technologies utilizing terahertz waves as a promising technology.

[0003] Terahertz waves have longer wavelengths than visible light or infrared, so they not only possess strong penetrative power, but also have lower energy than X-rays, making them harmless to the human body. In other words, the low energy of terahertz waves, which ensures human safety, and the sub-millimeter wavelength, which provides high spatial resolution, are attracting attention as a next-generation security screening technology.

[0004] However, despite these advantages, there are limitations in accurately detecting objects hidden within the human body or clothing using terahertz waves. While terahertz waves exhibit high reflectivity against hidden objects such as metal and plastic, they exhibit low reflectivity against human skin, and absorption and scattering by clothing make it difficult to accurately determine the location of hidden objects.

[0005] The purpose of the present invention is to recognize the above problems and provide a multi-image synthesis device and method capable of synthesizing an image of a human body shape extracted and an image of a hidden object detected using terahertz waves to determine the exact location of a hidden object within the human body.

[0006] In particular, the purpose of the present invention is to provide a multi-image synthesis device and method capable of extracting a human body shape in real time and synthesizing it with a hidden object image without being affected by surrounding conditions such as lighting, background color, and shadows, and thereby determining the presence and location of a hidden object in real time.

[0007] In addition, the purpose of the present invention is to provide a multi-image synthesis device and method capable of more clearly identifying the human body shape, enabling high-speed image processing, and thereby enabling more accurate synthesis with hidden object images, thereby effectively detecting hidden objects in the human body in real time.

[0008] The present invention discloses a multi-image synthesis device (100) including an image acquisition unit (110) for acquiring a first image in which the shape and background of a human body (B) are captured and a second image in which a human body concealment (C) is captured, an image processing unit (120) for performing image processing on the first image and the second image, and an image synthesis unit (130) for synthesizing the processed first image and the second image.

[0009] The above second image may be a terahertz wave image.

[0010] The above first image may be a thermal image or an infrared image.

[0011] The image processing unit (120) may include a first image processing unit (122) that performs image processing on the first image and a second image processing unit (124) that performs image processing on the second image.

[0012] The first image processing unit (122) may include a background synchronization unit (122a) that synchronizes the brightness of the first image and a background image in which only the background is captured without the shape of the human body (B), a shape extraction unit (122b) that extracts the shape of the human body (B) from the first image synchronized with the brightness of the background image, and a first binarized image generation unit (122d) that generates a first binarized image by binarizing the background and the shape of the human body (B) extracted from the first image.

[0013] The above first image processing unit (122) may further include a shape correction unit (122c) that corrects the shape of the human body (B) extracted from the shape extraction unit (122b).

[0014] The second image processing unit (124) can generate a second binary image by binarizing the background and the hidden object (C) in the second image.

[0015] In the first binarized image, the pixel value of the background may be 255 and the pixel value of the shape of the human body (B) may be 1, and in the second binarized image, the pixel value of the background may be 1 and the pixel value of the hidden object (C) may be 255.

[0016] The first image processing unit (122) can generate the first binary image by adjusting the pixel values ​​of the background of the first image and the pixel values ​​of the shape of the human body (B) to specific values.

[0017] The second image processing unit (124) can generate the second binary image by adjusting the pixel values ​​of the background of the second image and the pixel values ​​of the shape of the human body (B) to specific values.

[0018] The above image synthesis unit (130) can generate a synthetic image by multiplying pixel values ​​of pixels at the same location of the first binary image and the second binary image.

[0019] In another aspect, the present invention discloses a multi-image synthesis method including an image acquisition step of acquiring a first image in which the shape and background of a human body (B) are captured and a second image in which a human body concealment (C) is captured; an image processing step of performing image processing on the first image and the second image; and an image synthesis step of synthesizing the processed first image and the second image.

[0020] The above image processing step may include a background synchronization step of synchronizing the brightness of the first image and a background image in which only the background is captured without the shape of the human body (B), a shape extraction step of extracting the shape of the human body (B) from the first image synchronized with the brightness of the background image, and a first image binarization step of generating a first binarized image by binarizing the background and the shape of the human body (B) extracted from the first image.

[0021] The above image processing step may include a second image binarization step of generating a second binarized image by binarizing the background and the hidden object (C) in the second image.

[0022] The above image synthesis step may be a step of generating a synthetic image by synthesizing the first binary image and the second binary image.

[0023] The multi-image synthesis device and method according to the present invention have the advantage of being able to determine the exact location of a hidden object within the human body by synthesizing an image of a human body shape extracted and an image of a hidden object detected using terahertz waves.

[0024] In particular, the present invention can extract a human body shape in real time and synthesize it with a hidden object image without being affected by surrounding conditions such as lighting, background color, and shadows, and accordingly, can determine the presence or absence of a hidden object and its location in real time.

[0025] In addition, the present invention has the advantage of being able to more clearly identify the human body shape, enabling high-speed image processing, and thereby enabling more accurate synthesis with hidden object images, thereby effectively detecting hidden objects within the human body in real time.

[0026] Figure 1 is a plan view illustrating a multi-image synthesis device according to the present invention.

[0027] Figure 2 is a conceptual diagram illustrating the scan area (SA) of Figure 1.

[0028] Figure 3a is an example of a first image in which the shape of a human body and the background are captured.

[0029] Figure 3b is an example of background images in which only the background is captured without the shape of a human body.

[0030] Figure 4a illustrates the shape of a human body extracted from the first image of Figure 3a.

[0031] Figure 4b is a corrected version of the human body shape of Figure 4a.

[0032] Figure 5 is an example of a second image in which a human body is hidden.

[0033] Figure 6 is an example of a composite image that synthesizes the first image and the second image.

[0034] Figure 7 is a block diagram of a multi-image synthesis device according to the present invention.

[0035] Figure 8 is a block diagram of the first image acquisition unit of Figure 7.

[0036] Figure 9 is a flowchart illustrating a multi-image synthesis method according to the present invention.

[0037] Figure 10 is a flowchart illustrating the image processing steps of Figure 9.

[0038] The description of the present invention is merely an example for structural and functional explanation, and therefore, the scope of the present invention should not be construed as being limited by the embodiments described in the text. That is, since the embodiments can be modified in various ways and can take various forms, the scope of the present invention should be understood to include equivalents that can realize the technical idea. In addition, the purposes or effects presented in the present invention do not mean that a specific embodiment must include all of them or only such effects, and therefore, the scope of the present invention should not be construed as being limited thereby.

[0039] Meanwhile, the meaning of the terms described in this application should be understood as follows.

[0040] Terms such as "first" and "second" are intended to distinguish one component from another, and the scope of the rights should not be limited by these terms. For example, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.

[0041] When a component is said to be "connected" to another component, it should be understood that while it may be directly connected to that other component, there may also be other components intervening. Conversely, when a component is said to be "directly connected" to another component, it should be understood that there are no other intervening components. Similarly, other expressions describing relationships between components, such as "between" and "directly between," or "adjacent to" and "directly adjacent to," should be interpreted similarly.

[0042] Singular expressions should be understood to include plural expressions unless the context clearly indicates otherwise, and terms such as "comprises" or "have" should be understood to specify the presence of a feature, number, step, operation, component, part or combination thereof, but not to exclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0043] For each step, the identifiers (e.g., a, b, c, etc.) are used for convenience of explanation and do not describe the order of the steps. The steps may occur in a different order than stated unless the context clearly dictates a specific order. That is, the steps may occur in the same order as stated, may be performed substantially simultaneously, or may be performed in the opposite order.

[0044] The present invention can be implemented as computer-readable code on a computer-readable recording medium, and the computer-readable recording medium includes all types of recording devices that store data that can be read by a computer system. Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. In addition, the computer-readable recording medium can be distributed across network-connected computer systems, so that the computer-readable code can be stored and executed in a distributed manner.

[0045] Unless otherwise defined, all terms used herein have the same meaning as commonly understood by those of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted to be consistent with their meaning within the context of the relevant technology, and should not be interpreted as having an idealized or overly formal meaning unless explicitly defined herein.

[0046] Hereinafter, a multi-image synthesis device and method according to one embodiment of the present invention will be described with reference to the drawings.

[0047] A multi-image synthesis device (100) according to the present invention includes an image acquisition unit (110) for acquiring a first image in which the shape and background of a human body (B) are captured and a second image in which a human body concealment object (C) is captured, an image processing unit (120) for performing image processing on the first image and the second image, and an image synthesis unit (130) for synthesizing the processed first image and the second image.

[0048] Referring to FIG. 1, the image acquisition unit (110) may be an imaging device that captures the shape of a human body (B) and a background positioned in a scan area (SA) in front. The scan area (SA) is spaced apart from the image acquisition unit (110) by a certain distance (L1) and may cover a certain horizontal section (L2) area. As illustrated in FIG. 2, the upper and lower heights L3 of the scan area (SA) may be set in various ways, and the human body (B) and the background positioned within the scan area (SA) may be captured.

[0049] The above image acquisition unit (110) is configured to acquire a first image in which the shape and background of a human body (B) are captured and a second image in which a human body concealment (C) is captured, and may include a first image acquisition unit (112) for capturing the first image and a second image acquisition unit (114) for capturing the second image.

[0050] The above first image acquisition unit (112) is an imaging device capable of acquiring a first image capable of extracting the shape of a human body (B) including the human body (B) and the background, such as visible light, thermal imaging, infrared, laser, low-light, or TOF (time of flight), and can be configured in various ways.

[0051] That is, the first image may be a variety of images, such as a visible light image, a thermal image, an infrared image, a laser image, a low-light image, or a TOF image, and is not limited to a specific image.

[0052] For example, the first image acquisition unit (112) may be a thermal imaging camera for acquiring a thermal image. Since a thermal imaging camera is a camera that constructs an image by measuring the distribution of thermal energy emitted by a subject, thermal images have the advantage of not being affected by surrounding conditions such as lighting and background color. In particular, since the temperature difference of a human body (B) is utilized, the shape of the human body (B) can be more clearly identified, and there is also the advantage of being able to acquire images at high speed.

[0053] As another example, the first image acquisition unit (122) may be an infrared camera for acquiring infrared images. Since an infrared camera visualizes infrared (heat rays) to form an image, it has the advantage of being unaffected by ambient conditions such as lighting and background color. In particular, since it utilizes the temperature difference of a human body (B), it can more clearly identify the shape of the human body (B) and has the advantage of enabling high-speed image acquisition.

[0054] The second image acquisition unit (114) may be a terahertz wave imaging device for acquiring a terahertz wave image. That is, the second image may be a terahertz wave image, which may be an image of a hidden object (C) hidden within a human body (B). Terahertz waves can penetrate the human body (B) and be reflected by plastic, metal, etc., thereby detecting the hidden object (C) hidden within the human body (B).

[0055] The image processing unit (120) can be configured in various ways to perform image processing on the first and second images. For example, the image processing unit (120) may include a first image processing unit (122) that performs image processing on the first image and a second image processing unit (124) that performs image processing on the second image.

[0056] The above first image processing unit (122) is configured to perform image processing on the first image, and as an example, can perform image processing to extract the shape of a human body (B) within the first image.

[0057] Specifically, the first image processing unit (122) may include a background synchronization unit (122a) that synchronizes the brightness of the first image and a background image in which only the background is captured without the shape of the human body (B), a shape extraction unit (122b) that extracts the shape of the human body (B) from the first image synchronized with the brightness of the background image, and a first binarized image generation unit (122d) that generates a first binarized image by binarizing the background and the shape of the human body (B) extracted from the first image.

[0058] Fig. 3a is an example of an image in which the shape of a human body (B) and a background are captured together when the first image is a thermal image. However, since the thermal image quantifies the temperature of the subject into pixel values, the brightness changes in real time. Therefore, it is necessary to synchronize the brightness to remove the background under consistent conditions and extract the shape of the human body (B). Fig. 3b is an example of a background image. It can be seen that the brightness of the two images is different, and thus it is necessary to synchronize the brightness under consistent conditions.

[0059] Accordingly, the background synchronization unit (122a) can synchronize the brightness of the first image and the background image in which only the background is captured without the shape of the human body (B). By synchronizing the brightness of the first image and the background image in which only the background is captured without the shape of the human body (B), consistent processing of the background is possible regardless of the presence or absence of the human body (B), and the shape of the human body (B) can be extracted by removing the background under consistent conditions.

[0060] The above shape extraction unit (122b) can be configured in various ways to extract the shape of a human body (B) from the first image synchronized with the brightness of the background image. Extracting the shape of the human body (B) can be implemented through image recognition of the first image and various shape extraction algorithms. Fig. 4a is an example of the shape of the human body (B) extracted through the shape extraction unit (122b).

[0061] However, when looking at the shape of the extracted human body (B) in Fig. 4a, the shape may appear incomplete, such as the outer line not being smooth and the pixel value being significantly different from other areas and appearing bright even though it is an area inside the shape of the human body (B). Accordingly, the first image processing unit (122) may further include a shape correction unit (122c) that corrects the shape of the human body (B) extracted by the shape extraction unit (122b).

[0062] The above shape correction unit (122c) can be configured in various ways to correct (or interpolate) the shape of the human body (B) extracted by the shape extraction unit (122b). For example, the shape correction unit (122c) can apply various shape correction algorithms, such as an algorithm that interpolates the outline of the shape of the human body (B) to smoothly correct the outline (for example, replaces a pixel value with the average of adjacent pixel values), or an algorithm that fills in a closed area within the shape of the human body (B). Fig. 4b is an example of an image in which the shape of the human body (B) of Fig. 4a is corrected through the shape correction unit (122c).

[0063] The above first binary image generation unit (122d) is configured to generate a first binary image by binarizing the background and the shape of the extracted human body (B) from the first image, and can be configured in various ways.

[0064] The first binary image generation unit (122d) can generate the first binary image by adjusting the pixel values ​​of the background of the first image and the pixel values ​​of the shape of the human body (B) to different specific values.

[0065] For example, the first binarization image generation unit (122d) can binarize the shape of the human body (B) into a pixel value corresponding to a dark color and the background into a pixel value corresponding to a bright color by making the pixel value of the shape of the human body (B) different from the pixel value of the background.

[0066] Specifically, the first binarized image generation unit (122d) can binarize the first image so that the pixel value of the background is 255 and the pixel value of the shape of the human body (B) is 1. That is, in the first binarized image, the pixel value of the background can be set to 255 and the pixel value of the shape of the human body (B) can be set to 1.

[0067] As a result, in the first binary image, the background can be expressed in a bright color and the shape of the human body (B) can be expressed in black. Here, the pixel values ​​of the background and the pixel values ​​of the shape of the human body (B) are only examples, and the first binary image generation unit (122d) can of course generate the first binary image by variously adjusting the pixel values ​​of the background and the pixel values ​​of the shape of the human body (B).

[0068] The second image processing unit (124) can be configured in various ways to perform image processing on the second image. The second image processing unit (124) can generate a second binary image by binarizing the background and the hidden object (C) in the second image.

[0069] The second image processing unit (124) can generate the second binary image by adjusting the pixel values ​​of the background of the second image and the pixel values ​​of the shape of the human body (B) to different specific values.

[0070] For example, the second image processing unit (124) may binarize the shape of the hidden object (C) into a pixel value corresponding to a bright color and the background into a pixel value corresponding to a dark color, in order to make the pixel value of the hidden object (C) different from the pixel value of the background.

[0071] Specifically, the second image processing unit (124) can binarize the second image so that the pixel value of the background is 1 and the pixel value of the hidden object (C) is 255. That is, in the second binarized image, the pixel value of the background is 0 and the pixel value of the hidden object (C) can be set to 255.

[0072] As a result, in the second binary image, the background can be expressed in a dark color and the shape of the human body (B) can be expressed in black. Here, the pixel values ​​of the background and the pixel values ​​of the shape of the human body (B) are only examples, and the second image processing unit (124) can of course generate the second binary image by variously adjusting the pixel values ​​of the background and the pixel values ​​of the shape of the human body (B).

[0073] The image synthesis unit (130) described above can be configured in various ways to synthesize the processed first and second images. Specifically, the image synthesis unit (130) can synthesize the first and second binarized images described above. Various image synthesis algorithms can be applied for image synthesis, and the present invention is not limited to a specific method. Figure 6 is an example of a composite image synthesized from the first and second binarized images.

[0074] For example, the image synthesis unit (130) can generate a synthetic image by multiplying the pixel values ​​of pixels in the same position of the first binarized image and the second binarized image. Since the pixel value of the shape of the human body (B) in the first binarized image and the pixel value of the background in the second binarized image are not 0 but 1, by multiplying the pixel values ​​of the corresponding pixels of the first binarized image and the second binarized image, the shape of the human body (B) and the hidden object (C) inside can be clearly identified in a bright color in the synthetic image.

[0075] Meanwhile, if the first image processing unit (122) determines that there is no shape of a human body (B) in the first image, it can omit unnecessary work and improve image processing speed by not performing the work of extracting the shape of the human body (B) and the work of correction.

[0076] The above multi-image synthesis device (100) may further include an output unit (140) for outputting the composite image. The image processing unit (120), the image synthesis unit (130), and the output unit (140) may be configured as a computing device. In this case, the computing device may further include a display unit for displaying the composite image. In addition, the computing device may include a user input interface for user input and a communication module for communicating with an external server.

[0077] Additionally, the computing device may be equipped with and run a dedicated program or application for the aforementioned multi-image synthesis. Furthermore, the computing device may transmit or receive predetermined data for the aforementioned multi-image synthesis and access a database (DB) established for this purpose. This series of processes may be accomplished through an interface provided by the dedicated program or application.

[0078] Meanwhile, the present invention discloses a multi-image synthesis method performed in the above-described multi-image synthesis device (100).

[0079] The above multi-image synthesis method may include an image acquisition step (S902) of acquiring a first image in which the shape and background of a human body (B) are captured and a second image in which a human body concealment (C) is captured, an image processing step (S904) of performing image processing on the first image and the second image, and an image synthesis step (S906) of synthesizing the image-processed first image and the second image.

[0080] The above image acquisition step (S902) can be performed through the first image acquisition unit (122) and the second image acquisition unit (124). The first image can be a thermal image, and the second image can be a terahertz wave image.

[0081] The above image processing step (S904) may include a first image processing step for performing image processing on a first image, and a second image processing step for performing image processing on a second image. The first and second image processing steps may be performed sequentially or simultaneously. It is also possible for the second image processing step to be performed after the second image processing.

[0082] The first image processing step may include a background synchronization step (S1010) of synchronizing the brightness of the first image and a background image in which only the background is captured without the shape of the human body (B), a shape extraction step (S1020) of extracting the shape of the human body (B) from the first image synchronized with the brightness of the background image, and a first image binarization step (S1040) of generating a first binarized image by binarizing the background and the shape of the human body (B) extracted from the first image. The first image processing step may further include a shape correction step (S1030) of correcting the shape of the extracted human body (B) prior to the first image binarization step (S1040).

[0083] The second image processing step may include a second image binarization step of generating a second binarized image by binarizing the background and the hidden object (C) in the second image.

[0084] The above image synthesis step may be a step of generating a synthetic image by synthesizing the first binary image and the second binary image.

[0085] In addition, the above multi-image synthesis method may further include a synthesis image output step (S908) for outputting the synthesis image.

[0086] Meanwhile, the above-described multi-image synthesis method can automate the process of detecting hidden objects (C) using an artificial intelligence algorithm and minimize errors in detecting hidden objects (C). Furthermore, based on information about the shape and location of hidden objects (C) extracted through the artificial intelligence algorithm, information can be extracted, such as whether a hidden object (C) actually exists within a human body (B) and what type of hidden object (C) it is.

[0087] Furthermore, the present invention discloses a multi-image synthesis computer program stored in a computer-readable recording medium for performing the above-described multi-image synthesis method.

[0088]

[0089] The above is only a description of some of the preferred embodiments that can be implemented by the present invention, and as is well known, the scope of the present invention should not be construed as being limited to the above embodiments, and the technical ideas of the present invention described above and the technical ideas underlying them are all included in the scope of the present invention.

Claims

1. It includes an image acquisition unit (110) for acquiring a first image in which the shape and background of a human body (B) are captured and a second image in which a human body concealment (C) is captured; an image processing unit (120) for performing image processing on the first image and the second image; and an image synthesis unit (130) for synthesizing the processed first image and the second image. A multi-image synthesis device (100) characterized in that the above second image is a terahertz wave image.

2. In claim 1, A multi-image synthesis device (100), characterized in that the first image is a thermal image or an infrared image.

3. In claim 1, A multi-image synthesis device (100), characterized in that the image processing unit (120) includes a first image processing unit (122) that performs image processing on the first image and a second image processing unit (124) that performs image processing on the second image.

4. In claim 3, A multi-image synthesis device (100) characterized in that the first image processing unit (122) includes a background synchronization unit (122a) that synchronizes the brightness of the first image and a background image in which only the background is captured without the shape of the human body (B), a shape extraction unit (122b) that extracts the shape of the human body (B) from the first image synchronized with the brightness of the background image, and a first binarized image generation unit (122d) that generates a first binarized image by binarizing the background and the shape of the human body (B) extracted from the first image.

5. In claim 4, A multi-image synthesis device (100), characterized in that the first image processing unit (122) further includes a shape correction unit (122c) that corrects the shape of the human body (B) extracted from the shape extraction unit (122b).

6. In claim 4, A multi-image synthesis device (100), characterized in that the second image processing unit (124) generates a second binary image by binarizing the background and the hidden object (C) in the second image.

7. In claim 6, In the first binary image above, the pixel value of the background is 255 and the pixel value of the shape of the human body (B) is 1. A multi-image synthesis device (100), characterized in that the pixel value of the background in the second binary image is 1 and the pixel value of the hidden object (C) is 255.

8. In claim 6, The first image processing unit (122) generates the first binary image by adjusting the pixel values ​​of the background of the first image and the pixel values ​​of the shape of the human body (B) to specific values. The second image processing unit (124) is a multi-image synthesis device (100) that generates the second binary image by adjusting the pixel values ​​of the background of the second image and the pixel values ​​of the shape of the human body (B) to specific values.

9. In claim 8, A multi-image synthesis device (100), characterized in that the image synthesis unit (130) generates a composite image by multiplying pixel values ​​of pixels in the same position of the first binary image and the second binary image.

10. An image acquisition step for acquiring a first image in which the shape and background of a human body (B) are captured and a second image in which a human body concealment (C) is captured; an image processing step for performing image processing on the first image and the second image; and an image synthesis step for synthesizing the processed first image and the second image. A multi-image synthesis method characterized in that the second image is a terahertz wave image.

11. In claim 10, A multi-image synthesis method characterized in that the image processing step includes a background synchronization step for synchronizing the brightness of the first image and a background image in which only the background is captured without the shape of the human body (B), a shape extraction step for extracting the shape of the human body (B) from the first image synchronized with the brightness of the background image, and a first image binarization step for generating a first binarized image by binarizing the background and the shape of the human body (B) extracted from the first image.

12. In claim 11, A method for generating multiple images, characterized in that the image processing step includes a second image binarization step of generating a second binary image by binarizing the background and the hidden object (C) in the second image.

13. In claim 12, A method for generating multiple images, characterized in that the above image synthesis step is a step of generating a composite image by synthesizing the first binary image and the second binary image.

Images