System and method for monitoring vital signs of vehicle occupants using multi-wavelength-band camera images

Abstract

The present invention relates to a system and method for monitoring vital signs of a vehicle occupant using multi-wavelength band camera images. The vital sign monitoring method according to the present invention comprises the steps of: generating a multi-wavelength band image by photographing a target area according to a set wavelength band using a multi-wavelength band camera; extracting feature points of an infrared image and a visible light image included in the multi-wavelength band image using a feature point analysis technique, and generating an infrared-visible light fused image by fusing the infrared image and the visible light image based on the feature points; and determining a respiration area in the infrared-visible light fused image based on the brightness of the infrared-visible light fused image.

Description

Technology Field

[0001] The present invention relates to a system and method for monitoring the vital signs of a vehicle occupant based on images of multiple wavelength bands. Background Technology

[0002] Conventional technology provides methods to detect the presence or absence of a passenger using radar or thermal imaging cameras, or to capture passenger movement using visible light cameras (standard RGB cameras). However, in the case of radar, due to the characteristics of the sensor, there is a problem in that it is difficult to detect the presence or absence of a passenger when there is no movement or the movement is minimal. Additionally, in the case of thermal imaging cameras, the presence or movement of a passenger is detected by using a method that detects high-temperature objects; however, there is a problem in that it is difficult to distinguish between these objects and other high-temperature objects (e.g., hot soup, heating appliances, etc.) when they are present. Furthermore, in the case of visible light cameras, since they are heavily affected by ambient light, there is a problem in that object detection errors occur when placed in an environment where the surroundings are dark or reflected light is incident. Prior art literature

[0003] Republic of Korea Published Patent Application 10-2021-0097920 A (Date of Publication: August 10, 2021) Republic of Korea Published Patent Application 10-2018-0120901 A (Date of Publication: November 7, 2018) The problem to be solved

[0004] The present invention aims to provide a vehicle occupant vital sign monitoring system and method capable of detecting high-temperature objects robust to changes in illumination by utilizing a thermal imaging band, and generating occupant vital sign information through the visualization of carbon dioxide (CO2).

[0005] The objectives of the present invention are not limited to those mentioned above, and other unmentioned objectives will be clearly understood by those skilled in the art from the description below. means of solving the problem

[0006] A vital sign monitoring method according to one embodiment of the present invention comprises: a vital sign monitoring system using a multi-wavelength band camera to capture a target area according to a set wavelength band to generate a multi-wavelength band image; a step in which the system extracts feature points of an infrared image and a visible light image included in the multi-wavelength band image using a feature point analysis technique, and fuses the infrared image and the visible light image based on the feature points to generate an infrared-visible light fused image; and a step in which the system determines a respiration region in the infrared-visible light fused image based on the brightness of the infrared-visible light fused image.

[0007] In one embodiment of the present invention, the step of generating the infrared-visible light fusion image may include the system searching for a common feature point region of the infrared image and the visible light image based on the feature points, and when fusing the infrared image and the visible light image, setting the importance of the brightness of the infrared image higher than the importance of the brightness of the visible light image for a region other than the common feature point region.

[0008] In one embodiment of the present invention, the step of determining the breathing region may include the system detecting a person's head portion using an object recognition algorithm in the infrared-visible light fusion image and determining the breathing region by performing a search based on the head portion.

[0009] In one embodiment of the present invention, the vital signs monitoring method may further include the step of the system determining whether the subject is breathing by analyzing whether the breathing region is included in the infrared-visible light fusion image in chronological order.

[0010] In one embodiment of the present invention, the set wavelength band may include a mid-wavelength infrared band and a long-wavelength infrared band.

[0011] In one embodiment of the present invention, the set wavelength band may include a wavelength band of 14 μm to 16 μm.

[0012] In one embodiment of the present invention, the step of determining the presence or absence of breathing may further include the system estimating the breathing depth of the subject based on the width of the infrared-visible light fusion image and the width of the breathing area.

[0013] A vital sign monitoring system according to one embodiment of the present invention comprises: a multi-wavelength camera; a memory for storing computer-readable commands; and at least one processor implemented to execute said commands.

[0014] The above-mentioned multi-wavelength camera captures a target area according to a set wavelength band to generate a multi-wavelength image.

[0015] The above at least one processor is configured to, by executing the above commands, extract feature points of an infrared image and a visible light image included in the multi-wavelength band image using a feature point analysis technique, generate an infrared-visible fused image by fusing the infrared image and the visible light image based on the feature points, and determine a breathing region in the infrared-visible fused image based on the brightness of the infrared-visible fused image.

[0016] In one embodiment of the present invention, the at least one processor may be configured to search for a common feature point region of the infrared image and the visible light image based on the feature points during the process of generating the infrared-visible light fusion image, and to set the importance of the brightness of the infrared image higher than the importance of the brightness of the visible light image for a region other than the common feature point region when fusing the infrared image and the visible light image.

[0017] In one embodiment of the present invention, the at least one processor may be configured to determine the breathing region by using an object recognition algorithm in the infrared-visible light fusion image to detect a person's head portion and performing a search based on the head portion during the process of determining the breathing region.

[0018] In one embodiment of the present invention, the at least one processor may be configured to determine whether the subject is breathing by analyzing whether the breathing region is included in the infrared-visible light fusion image in chronological order.

[0019] In one embodiment of the present invention, the set wavelength band may include a mid-wavelength infrared band and a long-wavelength infrared band.

[0020] In one embodiment of the present invention, the set wavelength band may include a wavelength band of 14 μm to 16 μm.

[0021] In one embodiment of the present invention, the at least one processor may be configured to estimate the breathing depth of a subject based on the width of the infrared-visible light fusion image and the width of the breathing area during the process of determining the presence or absence of breathing. Effects of the invention

[0022] According to one embodiment of the present invention, there is an effect of monitoring the presence and vital signs of passengers in a vehicle and taking necessary measures even when the primary guardian is not present in the vehicle while passengers such as pets, children, or the elderly are on board.

[0023] For example, rescue activities can be initiated through an alarm regarding the presence of vulnerable road users, such as children or pets left unattended in specialized vehicles like school buses or general vehicles. Additionally, upon discovery of a passenger, the system provides the guardian with the passenger's vital signs, enabling an efficient emergency treatment process to be carried out in the event of an emergency.

[0024] The effects obtainable from the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description below. Brief explanation of the drawing

[0025] FIG. 1 is a block diagram showing the configuration of a vital sign monitoring system according to one embodiment of the present invention. FIG. 2 is a flowchart illustrating a vital sign monitoring method according to one embodiment of the present invention. Figures 3a to 3d are diagrams regarding the process of exploring the breathing area. Specific details for implementing the invention

[0026] The present invention aims to visualize the respiration and body temperature of an occupant by utilizing the Mid-Wave InfraRed (MWIR) band and / or the Long-Wave InfraRed (LWIR) band.

[0027] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Meanwhile, the terms used in this specification are for describing the embodiments and are not intended to limit the present invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. The terms "comprises" and / or "comprising" as used in this specification do not exclude the presence or addition of one or more other components, steps, actions, and / or elements in addition to the mentioned components, steps, actions, and / or elements.

[0028] Terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. These terms may be used for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component.

[0029] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. Conversely, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between. Other expressions describing the relationship between components, such as "between" and "exactly between," or "adjacent to" and "directly adjacent to," should be interpreted in the same way.

[0030] In describing the present invention, detailed descriptions of related prior art are omitted if it is determined that such detailed descriptions may unnecessarily obscure the essence of the invention.

[0031] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In order to facilitate overall understanding in describing the present invention, the same reference numerals will be used for the same means regardless of the drawing number.

[0032] Referring to the absorption spectra of atmospheric gases, it can be confirmed that carbon dioxide (CO2) is visualizable in the 3–5 µm and 14–16 µm wavelength bands. Specifically, the 3–5 µm and 14–16 µm wavelength bands, which exhibit high variability and absorption in common for carbon dioxide (CO2) and water vapor (H2O) in the atmospheric gas absorption spectra, can be designated as the wavelength bands for the infrared imaging used in this invention, respectively. Through experimental results for this invention, it was confirmed that 14–16 µm is the most suitable band for infrared imaging. Furthermore, the carbon dioxide absorption wavelength band data released by allora Factory BV ( https: / / controlair.cloud / en / hardware / co2-meten / As can be seen from the reference, carbon dioxide (CO2) has a higher absorption rate for infrared radiation in the 4.26 µm and 14-16 µm bands compared to other bands, so it is preferable that the wavelength band of the infrared image used in the present invention includes 4.26 µm and 14-16 µm.

[0033] In this specification, we propose a vehicle occupant vital signs monitoring system (hereinafter abbreviated as 'vital signs monitoring system') that can notify a primary caregiver (driver) of the presence and status of a remaining occupant by using a multi-wavelength camera sensor capable of detecting a medium-wavelength infrared (MWIR) band including 3 to 5 µm (particularly including 4.26 µm) and / or a long-wavelength infrared (LWIR) band including 14 to 16 µm.

[0034] FIG. 1 is a block diagram showing the configuration of a vital signs monitoring system according to one embodiment of the present invention, and FIG. 2 is a flowchart for explaining a vital signs monitoring method according to one embodiment of the present invention.

[0035] Referring to FIG. 1, a vital signs monitoring system (100) according to one embodiment of the present invention comprises a multi-wavelength band camera (110), a memory (120), a processor (130), and a communication device (140). The vital signs monitoring system (100) illustrated in FIG. 1 is according to one embodiment, and the components of the vital signs monitoring system (100) according to the present invention are not limited to the embodiment illustrated in FIG. 1 and may be added, changed, or deleted as needed.

[0036] The vital signs monitoring system (100) visualizes the level of respiration based on multi-wavelength band images, searches for and determines a respiration area, and performs the operation of detecting the presence or absence of a vehicle occupant and the respiration rate based on the determined respiration area.

[0037] In the present invention, 'respiration level visualization' refers to the visualization of carbon dioxide (CO2) and water vapor (H2O), which are major components generated during respiration. To visualize respiration levels, a wavelength band must be selected in which the differences between major atmospheric components and carbon dioxide (CO2) and water vapor (H2O) are clearly evident. In the present invention, considering the absorption spectra of each gas in the atmosphere, the mid-wave infrared band and the long-wave infrared band were selected as the target wavelength bands when collecting infrared images using a multi-wavelength band camera. That is, in the present invention, the wavelength band of the infrared image collected using a multi-wavelength band camera is the mid-wave infrared band and / or the long-wave infrared band. As previously explained, it is preferable that the mid-wave infrared band includes 3 to 5 μm (particularly 4.26 μm), and the long-wave infrared band includes 14 to 16 μm. However, it goes without saying that the wavelength band of the infrared image to be collected using a multi-wavelength band camera in the present invention is not limited to the wavelength bands described above (3~5 µm, 14~16 µm).

[0038] The multi-wavelength band camera (110) generates a multi-wavelength band image. Specifically, the multi-wavelength band camera (110) can generate an infrared image and a visible light image. In the present invention, the infrared image may include an infrared image in the mid-wavelength infrared band (e.g., 3 to 5 μm) and / or an infrared image in the long-wavelength infrared band (e.g., 14 to 16 μm). That is, the multi-wavelength band image may include either an infrared image in the mid-wavelength infrared band and / or the long-wavelength infrared band, and a visible light image, or a combination thereof.

[0039] The memory (120) may include various forms of volatile or non-volatile storage media. For example, the memory (120) may include ROM (read-only memory) and RAM (random access memory). In the embodiments of this description, the memory (120) may be located inside or outside the processor (130), and the memory (120) may be connected to the processor (130) through various known means. The memory (120) is various forms of volatile or non-volatile storage media, and for example, the memory (120) may include read-only memory (ROM) or random access memory (RAM).

[0040] The processor (130) may be a central processing unit (CPU) or a semiconductor device that executes computer-readable instructions stored in memory (120). The processor (130) may generate information regarding the presence of a passenger, the passenger's breathing pattern, breathing rate, breathing depth, and the passenger's body temperature based on multi-wavelength band images.

[0041] The communication device (140) can transmit or receive wired or wireless signals. For example, the communication device (140) can transmit a message to the vehicle driver's smartphone, either directly or via another device, indicating the presence of a passenger, the passenger's breathing pattern, breathing depth, breathing rate, and the passenger's body temperature.

[0042] FIG. 2 is a flowchart illustrating a vital sign monitoring method according to an embodiment of the present invention. With reference to FIG. 2, the operation of the vital sign monitoring system (100) and each component illustrated in FIG. 1 will be explained.

[0043] Referring to FIG. 2, a vital signs monitoring method according to one embodiment of the present invention consists of steps S210 to S260. The vital signs monitoring method illustrated in FIG. 2 is according to one embodiment, and the steps of the vital signs monitoring method according to the present invention are not limited to the embodiment illustrated in FIG. 2 and may be added, changed, or deleted as necessary.

[0044] Step S210 is the multi-wavelength band image generation step.

[0045] A multi-wavelength band camera (110) captures a target area (e.g., inside a vehicle) according to a set wavelength band to generate a multi-wavelength band image. The multi-wavelength band image includes an infrared image and a visible light image. The infrared image may include an infrared image in a mid-wavelength infrared band and / or an infrared image in a long-wavelength infrared band. The infrared image and the visible light image may be still images.

[0046] When the multi-wavelength band camera (110) generates a multi-wavelength band image, the processor (130) can perform preprocessing of the multi-wavelength band image. For example, the infrared image can be represented in a grayscale manner, which is a general representation method for thermal images. However, the bitmap range of the infrared image in the present invention is not limited. For example, when the infrared image is represented in grayscale in the present invention, the brightness value of each pixel may be represented as 8 bits (0 to 255), or as 12 bits or 16 bits.

[0047] Meanwhile, the processor (130) can perform filtering on wavelength bands capable of detecting gases other than water vapor (H2O) and carbon dioxide (CO2) in an infrared image. For example, the processor (130) can perform filtering on wavelength bands in the infrared image where the variation or absorption of gases other than water vapor and carbon dioxide is high in the absorption spectrum of each gas in the atmosphere.

[0048] Step S220 is the infrared-visible light fusion image generation step.

[0049] The processor (130) fuses the infrared image and the visible light image included in the multi-wavelength band image to generate an infrared-visible fused image. Since the infrared-visible fused image is intended to determine the breathing region, during the process of fusing the infrared image and the visible light image, candidate regions where the breathing region can be well detected are identified based on feature point analysis, and the infrared image and the visible light image are fused so that the importance of the infrared image is reflected highly for the said candidate regions.

[0050] First, the processor (130) extracts feature points included in the visible light image and the infrared image using a known feature point analysis technique. Areas corresponding to the feature points extracted in each image are marked as 1, and parts that are not are marked as 0. As the feature point analysis technique, various techniques such as boundary detection, histogram technique, SIFT (Scale-Invariant Feature Transform), shape feature analysis, Fourier transform, wavelet transform, and CNN may be applied.

[0051] The processor (130) sets the inverse of the majority voting result for feature points extracted from the visible light image and the infrared image as the candidate region. Accordingly, the part that was marked as 1 in the majority voting result is changed to 0, and the part that was marked as 0 becomes 1 and is set as the candidate region. For reference, the visible light image and the infrared image may each include multiple images according to wavelength band and time.

[0052] The reason for setting the inverse result of the majority vote as the candidate region is that areas of common feature points appearing identically across multiple images are highly likely to be fixed objects or fixed parts of the human body, while areas not appearing as common feature points are likely to be regions occupied by the movement of bodily organs related to respiration or air expelled during breathing—in other words, areas where carbon dioxide or water vapor can be detected. In short, parts where carbon dioxide or water vapor appears in infrared images are highly unlikely to be the aforementioned areas of common feature points. Therefore, the candidate region is determined through the inverse of the majority vote regarding the feature points.

[0053] When the processor (130) fuses the visible light image and the infrared image, the previously determined Candidate area Regarding this, the importance (weight) of the infrared image can be set higher than the importance (weight) of the visible light image to fuse. That is, when fusing a candidate region, the processor (130) can set the importance of the intensity of the infrared image higher than the importance of the brightness of the visible light image. The importance has a value between 0 and 1. For example, when fusing one visible light image and one infrared image, the sum of the importance (n) of the visible light image and the importance (m) of the infrared image becomes 1. The processor (130) can fuse the parts that are not candidate regions by setting n to be greater than or equal to m, and fuse the candidate regions by setting m to be greater than n.

[0054] The processor (130) can generate an infrared-visible fused image by fusing each image by multiplying the importance of each image (e.g., infrared image, visible light image) by the brightness of each image, and then adding the brightness multiplied by the importance.

[0055] Step S230 is the step for determining the breathing zone.

[0056] The processor (130) detects a person's head (including the face) using an object recognition algorithm in an infrared-visible light fusion image and determines a breathing area by searching the surrounding area of ​​the head.

[0057] For example, since there are various objects and structures inside a vehicle, it is difficult to determine the breathing area solely through the visualization of water vapor (H2O) and carbon dioxide (CO2). Therefore, based on the human being who is the subject of respiration, the breathing area can be accurately identified by exploring the area surrounding the human head (or face).

[0058] In the present invention, the 'breathing region' refers to a region where carbon dioxide (CO2) and water vapor (H2O) are mixed or a carbon dioxide (CO2) region. The processor (130) can determine the breathing region using a simple binarization technique. For example, the processor (130) can determine the breathing region using a simple binarization technique in which the pixel brightness is marked as 1 if it is above a threshold value and as 0 if it is below the threshold value. Depending on the wavelength band, the processor (130) may determine a pixel marked as 1 as a breathing region or a pixel marked as 0 as a breathing region. The processor (130) can determine the breathing region by utilizing an artificial intelligence-based algorithm.

[0059] The processor (130) detects a person's head portion using a known object recognition algorithm in an infrared-visible light fusion image. Then, the processor (130) defines a super pixel based on the person's head portion and then grids the infrared-visible light fusion image based on the super pixel. The present invention refers to this process as 'super grid generation'.

[0060] Specifically, the processor (130) defines the area occupied by a person's head (or face) as a super pixel (see FIG. 3a), and sets the size of the super grid by dividing the super pixel by a set value. For example, the processor (130) 2 super pixels n A super grid can be created by dividing it. Here, n is a setting value.

[0061] That is, the processor (130) applies a super grid to the infrared-visible fusion image. In other words, the infrared-visible fusion image is divided into regions by the super grid. In the infrared-visible fusion image divided into regions by the super grid, the processor (130) determines a priority search region based on the human head (or face) region. For example, the processor (130) may designate the region of the 8-direction connected component (8-connected component) of the human head region as the priority search region (see FIG. 3b).

[0062] The processor (130) searches for a breathing area included in the priority search area (orange area in FIG. 3c). That is, the processor (130) finds and marks a breathing area in the priority search area. Then, based on the discovered breathing area, the processor (130) additionally designates an area of ​​an 8-direction connected component (8-connected component) as the priority search area (expansion of the search area, FIG. 3d), and searches for a breathing area again in the additionally designated priority search area (re-search). During re-search, the search is performed in super pixel units, and the search is performed in the same manner as during the initial search. The processor (130) repeats this process until no more breathing areas are found, and when no more breathing areas are found, it determines (confirms) the breathing area.

[0063] Step S240 is the breathing depth estimation step.

[0064] The processor (130) arranges infrared-visible light fusion images in a time series and determines whether a breathing area is included in the infrared-visible light fusion images and / or the degree of change of the breathing area according to the time series to determine whether the subject (passenger) is breathing and the breathing pattern. Here, it is assumed that all images are arranged in a time series. Then, the processor (130) estimates the breathing depth of the subject (passenger) based on whether breathing is present and the breathing pattern.

[0065] The processor (130) checks whether a breathing region is included (presence or absence of breathing, a) among the infrared-visible light fusion images arranged in chronological order. a can be expressed as a value of 0 or 1. In this case, 0 means a state without breathing, and 1 means a state with breathing.

[0066] Additionally, the processor (130) extracts the area (b) of the breathing region from an infrared-visible light fusion image including the breathing region and calculates the average temperature (c) of the main object (person) generating the breathing.

[0067] The processor (130) generates a breathing pattern based on the time axis t. This breathing pattern may consist of any one of a, b (or b'), c, or a combination thereof.

[0068] The processor (130) calculates the area (x*y) of the infrared-visible fusion image by multiplying the number of horizontal pixels (x) and the number of vertical pixels (y) of the infrared-visible fusion image, and calculates the ratio of the breathing area (b' = b / xy) in the infrared-visible fusion image by dividing the area (b) of the breathing area by the area (x*y) of the infrared-visible fusion image. The processor (130) can calculate the ratio of the breathing area (b') over time.

[0069] The processor (130) calculates a function of time g(t) = a*b'. g(t) represents the relative depth of respiration (hereinafter 'respiration depth'). That is, g(t) is not a function representing the absolute value of the depth, but a function for determining the relative trend of the respiration depth on a time series graph.

[0071] Step S250 is the step for determining the presence (survival) of occupants.

[0072] The processor (130) determines the presence (survival) of a passenger (subject) based on the presence or absence of breathing, breathing pattern, and breathing depth, or a combination thereof. For example, the processor (130) can determine the presence (survival) of a passenger (subject) based on the presence or absence of breathing confirmed in step S240. That is, the processor (130) can determine that a passenger is present if the number of breaths within a set time is greater than or equal to a set value. Alternatively, the processor (130) can determine that a passenger is present if the time at which the breathing depth is greater than or equal to a first threshold value is greater than or equal to a set value.

[0073] The first threshold value can be set differently depending on the vehicle environment. For example, the first threshold value may vary depending on the vehicle interior temperature and the concentration / distribution of each gas.

[0074] The processor (130) can calculate the number of breaths based on the breathing pattern or the pattern of breathing depth when a passenger is present, and determine whether the passenger is breathing normally based on the number of breaths.

[0075] For example, the processor (130) determines whether the ratio of the breathing area (b') or the breathing depth (g(t)) at each time point is in a range greater than or equal to a second threshold and less than a third threshold, and counts 1 breath if it is in said range. In this way, the processor (130) can calculate the passenger's breath rate per minute and determine that it is normal breathing if it is within a set range (e.g., 15 breaths or more and 20 breaths or less). The processor (130) determines that it is abnormal breathing if the passenger's breath rate per minute is outside the set range.

[0076] And, the processor (130) can subdivide abnormal breathing based on the ratio of the breathing area (b'), the breathing depth and / or the breathing depth (g(t)).

[0077] The subdivision of abnormal breathing can be set as shown in Table 1. However, this may be changed depending on the user and medical judgment. A feature of the present invention is that it can predict the frequency of breathing through images and determine the depth of breathing based on the area of ​​the breathing region.

[0078] Types of abnormalities Respiratory rate Ratio of breathing area (b') Slow breathing 10 times or less / minute - tachypnea 24 times or more / minute - Hyperventilation Increased compared to normal breathing b' > third threshold hypoventilation Irregular respiration / respiratory rate decrease along the time axis b' < second threshold etc Apnea / variable / irregular respiratory rate Inconsistent breathing depth

[0079] That is, the processor (130) can determine whether there is a passenger and, if there is a passenger, whether there is hyperventilation or hypoventilation, based on the result of performing step S250.

[0080] As a result of executing step S250, if there is a passenger, step S260 is executed; otherwise, the process is terminated.

[0081] Step S260 is the notification step.

[0082] The processor (130) can transmit the presence of a passenger and, if a passenger is present, the passenger's condition (normal breathing, hyperventilation / hypoventilation in case of abnormal breathing) to the vehicle driver's smartphone via a communication device (140).

[0083] Additionally, the processor (130) can transmit information such as the presence of a passenger, the condition of the passenger if a passenger is present, breathing patterns, and breathing depth to an external server via a communication device (140). This information can be used as a user notification / warning service and as an auxiliary indicator for medical measures when a remaining passenger is discovered.

[0084] The above-described vital signs monitoring method has been explained with reference to the flowchart presented in the drawings. For simplicity of explanation, the method has been illustrated and described in a series of blocks; however, the present invention is not limited to the order of said blocks, and some blocks may occur in a different order or simultaneously with other blocks as illustrated and described herein, and various other branches, flow paths, and sequences of blocks may be implemented to achieve the same or similar results. Furthermore, not all illustrated blocks may be required for the implementation of the method described herein.

[0085] Meanwhile, in the description with reference to FIG. 2, each step may be further divided into additional steps or combined into fewer steps according to an embodiment of the present invention. Also, some steps may be omitted as necessary, and the order between steps may be changed. Furthermore, even if other omitted details are included, the contents of FIG. 1 may be applied to the contents of FIG. 2. Also, the contents of FIG. 2 may be applied to the contents of FIG. 1.

[0086] Although the present invention has been described above with reference to preferred embodiments, those skilled in the art will understand that various modifications and changes can be made to the invention without departing from the spirit and scope of the invention as described in the following claims. Explanation of the symbols

[0087] 100: Vital Signs Monitoring System 110: Multi-wavelength camera 120: Memory 130: Processor 140: Communication device

Claims

Claim 1 A vital signs monitoring method comprising: a step of generating a multi-wavelength image by photographing a target area according to a set wavelength band using a multi-wavelength band camera; a step of generating an infrared-visible fused image by fusing the infrared image and the visible light image based on the feature points using a feature point analysis technique; and a step of determining a respiration area in the infrared-visible fused image based on the brightness of the infrared-visible fused image, wherein the set wavelength band includes a wavelength band of 14 μm to 16 μm, and the step of generating the infrared-visible fused image comprises: a step of searching for a common feature point area of ​​the infrared image and the visible light image based on the feature points; and a step of setting the importance of the brightness of the infrared image higher than the importance of the brightness of the visible light image for an area excluding the common feature point area when fusing the infrared image and the visible light image. Claim 2 delete Claim 3 A vital sign monitoring method according to claim 1, wherein the step of determining the breathing area comprises the system detecting a person's head portion using an object recognition algorithm in the infrared-visible light fusion image and determining the breathing area by performing a search based on the head portion. Claim 4 A vital sign monitoring method according to claim 1, wherein the system further includes the step of determining whether the subject is breathing by analyzing whether the breathing region is included in the infrared-visible light fusion image in chronological order. Claim 5 A vital sign monitoring method according to claim 1, wherein the set wavelength band includes a mid-wave infrared band and a long-wave infrared band. Claim 6 A vital sign monitoring method according to claim 4, wherein the step of determining the presence or absence of respiration further comprises the system estimating the respiration depth of the subject based on the width of the infrared-visible light fusion image and the width of the respiration area. Claim 7 A vital signs monitoring method comprising: a step of generating a multi-wavelength image by photographing a target area according to a set wavelength band using a multi-wavelength band camera; a step of generating an infrared-visible fused image by fusing the infrared image and the visible light image based on the feature points using a feature point analysis technique; and a step of determining a respiration area in the infrared-visible fused image based on the brightness of the infrared-visible fused image, wherein the set wavelength band includes a wavelength band of 3 μm to 5 μm, and the step of generating the infrared-visible fused image comprises: a step of searching for a common feature point area of ​​the infrared image and the visible light image based on the feature points; and a step of setting the importance of the brightness of the infrared image higher than the importance of the brightness of the visible light image for an area excluding the common feature point area when fusing the infrared image and the visible light image. Claim 8 Multi-waveband camera; memory that stores computer-readable commands; The system includes at least one processor implemented to execute the above commands, wherein the multi-wavelength band camera captures a target area according to a set wavelength band to generate a multi-wavelength band image, and the at least one processor, by executing the above commands, extracts feature points of an infrared image and a visible light image included in the multi-wavelength band image using a feature point analysis technique, fuses the infrared image and the visible light image based on the feature points to generate an infrared-visible fused image, and is configured to determine a breathing area in the infrared-visible fused image based on the brightness of the infrared-visible fused image, wherein the set wavelength band includes a wavelength band of 14 μm to 16 μm, and the at least one processor, in the process of generating the infrared-visible fused image, searches for a common feature point region of the infrared image and the visible light image based on the feature points, and when fusing the infrared image and the visible light image, is configured to set the importance of the brightness of the infrared image higher than the importance of the brightness of the visible light image for the region excluding the common feature point region. Vital Signs Monitoring System. Claim 9 delete Claim 10 A vital sign monitoring system according to claim 8, wherein at least one processor is configured to determine the breathing area by using an object recognition algorithm in the infrared-visible light fusion image to detect a person's head portion and performing a search based on the head portion during the process of determining the breathing area. Claim 11 A vital sign monitoring system according to claim 8, wherein at least one processor is configured to determine whether the subject is breathing by analyzing whether the breathing region is included in the infrared-visible light fusion image in chronological order. Claim 12 A vital sign monitoring system according to claim 8, wherein the above-described wavelength band includes a mid-wave infrared band and a long-wave infrared band. Claim 13 A vital sign monitoring system according to claim 11, wherein at least one processor is configured to estimate the depth of respiration of a subject based on the width of the infrared-visible light fusion image and the width of the respiration area in the process of determining the presence or absence of respiration. Claim 14 Multi-waveband camera; memory that stores computer-readable commands; A vital sign comprising at least one processor implemented to execute the above commands, wherein the multi-wavelength band camera captures a target area according to a set wavelength band to generate a multi-wavelength band image, and the at least one processor, by executing the above commands, extracts feature points of an infrared image and a visible light image included in the multi-wavelength band image using a feature point analysis technique, fuses the infrared image and the visible light image based on the feature points to generate an infrared-visible fused image, and is configured to determine a respiration region in the infrared-visible fused image based on the brightness of the infrared-visible fused image, wherein the set wavelength band includes a wavelength band of 3 µm to 5 µm, and the at least one processor, in the process of generating the infrared-visible fused image, searches for a common feature point region of the infrared image and the visible light image based on the feature points, and when fusing the infrared image and the visible light image, sets the importance of the brightness of the infrared image higher than the importance of the brightness of the visible light image for the region excluding the common feature point region. Monitoring system.

Images