Respiratory monitoring system using structured light

Abstract

Monitoring system for measuring and monitoring a subject's respiratory rate, wherein the monitoring system comprises the following: an optical projector configured to project an optical pattern, and an image capture module configured to capture images of the optical pattern reflected from a surface; an alarm device; and a processor and a memory containing instructions which, when executed by the processor, cause the processor to: Monitoring a subject's respiratory rate based on changes in the optical pattern reflected from the surface; Determine whether the monitored respiratory rate is within a threshold variance; and After determining that the monitored respiratory rate is outside the threshold variance, a notification is generated using the alarm device.

Description

AREA OF REVELATION

[0001] The present disclosure relates generally to structured light monitoring systems and, more particularly, to the use of structured light to monitor respiration. SUMMARY OF REVELATION

[0002] According to one aspect of the present disclosure, a monitoring system for measuring and monitoring a subject's respiratory rate includes an optical projector configured to project an optical pattern and an image capture module configured to capture images of the optical pattern reflected from a surface. The monitoring system further includes an alarm device, a processor, and a memory. The memory includes instructions that, when executed by the processor, cause the processor to monitor the subject's respiratory rate based on changes in the optical pattern reflected from the surface.The processor is further caused to determine whether the monitored respiratory rate is within a threshold variance and, upon a determination that the monitored respiratory rate is outside the threshold variance, to generate a notification using the alarm device.

[0003] According to another aspect of the present invention, a monitoring system for measuring and monitoring a respiratory rate of a subject in a vehicle includes a structured light array. The structured light array includes an optical projector configured to project an optical pattern and an image acquisition module configured to acquire images of the optical pattern reflected from a surface. The monitoring system further includes an alarm device, a processor, and a memory. The memory includes instructions that, when executed by the processor, cause the processor to monitor the respiratory rate of the subject based on changes in a speckle content of the optical pattern reflected from the surface.The processor is further caused to determine whether the monitored respiratory rate is within a threshold variance and, upon a determination that the monitored respiratory rate is outside the threshold variance, to generate a notification using the alarm device.

[0004] According to yet another aspect of the present disclosure, a monitoring system for measuring and monitoring a respiratory rate of a subject includes an optical projector including a laser and configured to project an optical pattern including at least one spot of light, and an image capture module configured to capture images of the optical pattern reflected from a surface. The monitoring system further includes an alarm device, a processor, and a memory. The memory includes instructions that, when executed by the processor, cause the processor to monitor the respiratory rate of the subject based on changes in the optical pattern reflected from the surface.The processor is further caused to determine whether the monitored respiratory rate is within a threshold variance and, upon a determination that the monitored respiratory rate is outside the threshold variance, to generate a notification using the alarm device.

[0005] These and other features, advantages, and purposes of the present disclosure will be further understood and appreciated by those skilled in the art with reference to the following description, claims, and accompanying drawings. SHORT DESCRIPTION OF THE DRAWING

[0006] They show: Fig. 1A is a perspective view of an automobile incorporating a structured light monitoring system according to one aspect of the present disclosure; Fig. 1B is a perspective view of an aircraft incorporating a structured light surveillance system according to one aspect of the present disclosure; Fig. 1C is a perspective view of a building incorporating a structured light surveillance system according to one aspect of the present disclosure; Fig. 2 is an interior view of a vehicle incorporating a structured light monitoring system according to one aspect of the present disclosure; Fig. 3 schematically illustrates the structured light surveillance system according to one aspect of the present disclosure; Fig. 4 illustrates a captured image including a spot of light reflected from a surface according to one aspect of the present disclosure; Fig. 5 illustrates a captured image including a spot of light reflected from a surface after movement of the surface on a microscale, according to one aspect of the present disclosure; Fig. 6 illustrates a monitoring system that monitors a respiratory rate vv of an individual over a period of time, in accordance with one aspect of the present disclosure; Fig. 7 illustrates an image of an individual captured by an image capture module in accordance with one aspect of the present disclosure; Fig. 8 graphically illustrates an unfiltered respiratory signal of the individual according to one aspect of the present disclosure; Fig. 9 graphically illustrates a frequency range of the respiratory signal according to one aspect of the present disclosure; Fig. 10 graphically illustrates a filtered respiratory signal of the individual according to one aspect of the present disclosure; Fig. 11 schematically illustrates a control system of the structured light surveillance system according to one aspect of the present disclosure; and Fig. 12 is a flowchart for monitoring a subject's respiratory rate using the structured light monitoring system according to one aspect of the present disclosure. DETAILED DESCRIPTION

[0007] The presently illustrated embodiments are essentially combinations of device components relating to a structured light surveillance system. Accordingly, where appropriate, the device components have been represented by conventional symbols in the drawings, showing only the specific details relevant to an understanding of the embodiments of the present disclosure, in order not to obscure the disclosure with details readily apparent to one of ordinary skill in the art having the benefit of the present description. Furthermore, like reference numerals in the description and the drawings represent like elements.

[0008] For the purpose of description, the terms “upper,” “lower,” “right,” “left,” “rear,” “front,” “vertical,” “horizontal,” and derivatives thereof herein shall refer to the disclosure as set forth in Fig. 1. Unless otherwise indicated, the term "front" shall refer to the surface of the device that is closer to an intended viewer of the device, and the term "rear" shall refer to the surface of the device that is farther away from the intended viewer of the device. However, it is to be understood that the disclosure is capable of assuming various alternative orientations unless expressly stated otherwise. It is also to be understood that the specific devices illustrated in the accompanying drawings and described in the following description are merely exemplary embodiments of the inventive terms defined in the appended claims.Therefore, specific dimensions and other physical characteristics associated with the embodiments disclosed herein should not be considered limiting unless the claims expressly state otherwise.

[0009] The terms "including," "comprises," "comprising," or any other variations thereof, are intended to be non-exclusive in that an article or device comprising a list of elements is not necessarily limited to those elements, but may also include other elements not expressly listed or that are inherent in such article or device. An element preceded by "comprises a..." does not preclude, without further limitation, the existence of additional identical elements in the article or device comprising the element.

[0010] With reference to Fig. 1A-1C, reference numeral 10 generally designates a monitoring system (e.g., a structured light monitoring system). The monitoring system 10 is configured to monitor the respiration of one or more individuals (i.e., people or animals) in a variety of environments. In particular, the monitoring system 10 may be integrated into one or more structures 12A-12C. For example, Fig. 1A illustrates a vehicle 12A utilizing the surveillance system 10. All or some components of the surveillance system 10 may be located within an interior rearview mirror 13 and / or other locations within a vehicle that orient the surveillance system 10 toward an interior cabin of the vehicle. The vehicle 12A may include a commercial vehicle, an emergency vehicle, a residential vehicle, a train, or the like. Fig. 1B illustrates an aircraft 12B employing the monitoring system 10. The monitoring system 10 may be located within one or more areas to monitor occupants of the aircraft 12B. For example, the monitoring system 10 may be located in a cockpit to monitor the status of a pilot. Fig. 1C illustrates a building 12C using the monitoring system 10 to monitor one or more individuals within the building 12C. The building 12C may be a residential building, a commercial building, a medical facility, a retirement home, a daycare center, a veterinary clinic, an animal facility, and / or the like. Generally speaking, the monitoring system 10 may be integrated into any environment where respiratory and / or other micro-motion events of an individual can be monitored (e.g., a heartbeat, a pulse, and / or the like).

[0011] With reference to Fig. 2, the monitoring system 10 includes a vision system, such as a structured light array 14. The structured light array 14 includes at least one optical projector 16 and at least one image acquisition module 18. In some embodiments, the structured light array 14 (e.g., the optical projector 16 and / or the image acquisition module 18) may be disposed within and / or otherwise coupled to the rearview mirror 13. The monitoring system 10 may further include an alarm device 20, such as a display device located within and / or otherwise coupled to the interior rearview mirror 13, other locations in the vehicle, monitors, portable electronic devices, in-vehicle audio systems, localized audio devices, haptic feedback devices, and / or the like. The rearview mirror 13 may include a partially reflective, partially transmissive element 22, such asan electro-optical device that defines a viewing area of ​​the rearview mirror 13. The rearview mirror 13 includes a housing 24 that can support the alarm device 20 and the partially reflective, partially transmissive element 22. The rearview mirror 13 can further include a mounting member 26 that couples the housing 24 to the interior cabin of the vehicle 12A. In some embodiments, the image acquisition module 18 and the optical projector 16 are coupled to other areas of the vehicle 12A, such as an instrument panel, an overhead console, or other parts, such as the interior cabin. As described in more detail below, the image acquisition module 18 can be packaged with the optical projector 16 (i.e., in a static relationship). The image acquisition module 18 is positioned to generate images 28 (. Fig. 7) the individuals within the vehicle 12A are recorded.

[0012] With continued reference to Fig. 2, the optical projector 16 may include one or more laser projectors. In particular, the optical projector 16 may be configured as one or more single-mode or multi-mode lasers operating within a wavelength spectrum, such as the infrared (IR) spectrum. Accordingly, in some examples, the image acquisition module 18 is a camera operable within the IR spectrum (or other spectra corresponding to the optical projector 16). The image acquisition module 18 may have a field of view covering a front compartment and / or an aft compartment of the interior cabin. In this manner, the image acquisition module 18 may be oriented to capture images 28 of one of several or specific locations where an individual would be located.

[0013] With further reference to Fig. 2, the present disclosure may be used with a rearview mirror 13, such as that described in U.S. Pat. No. 9,505,349 and U.S. Pat. No. 10,739,591, which are hereby incorporated by reference in their entirety into this disclosure. The alert device 20 may generate a notification that includes at least one of an audible, visual, or haptic stimulus. For example, a visual notification may be generated on the rearview mirror 13 (e.g., the display) that includes a respiratory rate; an audible notification may be generated that provides a beep or voice output recommending checking a subject (e.g., in a medical setting); and a haptic notification may include shaking a vehicle seat, a steering wheel, a personal electronic device, and / or the like.

[0014] Now with reference to Fig. 3, the optical projector 16 is configured to display a structured optical pattern 30 on an object 32 (e.g., an individual or something associated with an individual). The structured optical pattern 30 may include an arrangement of shapes, such as dots 34, a series of alternating light bands, alternating intensity, and / or the like, that can be detected by the image capture module 18. Regardless of the shape and distribution of the shapes (e.g., dots 34), when the surface of the object 32 reflecting the dots 34 moves, the dots 34 also move, and this movement is detected by the image capture module 18.In a first mode of operation, a control system 100 may process the images 28 acquired by the image acquisition module 18 and extrapolate the movement of the points 34 into a depth of the surface based on the principles of triangulation and known geometries between the image acquisition module 18, the optical projector 16, and the distribution of the structured optical pattern 30. For example, the control system 100 may be configured to determine movement based on an outer perimeter or centroid of each point 34. In the first mode of operation, the image acquisition module 18 and the optical projector 16 may be closely and rigidly mounted on a common optical bench structure (e.g.,within the rearview mirror 13 or at another common location), and based on the known distance between the image acquisition module 18 and the optical projector 16 and the distribution of the structured optical pattern 30, the location of the reflected point 34 along an epipolar line can be detected, which in turn can be triangulated to extract a depth of the surface. The depth of the surface at each point 34 can then be used to extrapolate a contour of the surface. Likewise, depth changes can be used to extrapolate the current location of the surface and the movement of the surface as a function of time.

[0015] Now with reference to Fig. 4 and Fig. 5, the monitoring system 10 is also configured to operate in a second mode of operation. In the second mode of operation, a speckle content of each reflected spot 34 can be monitored to detect changes in the internal intensity distribution of the surface according to the principles of speckle interferometry, such as tilts or other movements on a microscale (e.g., a micrometer or a microradian scale). For example, if a surface has roughness, microradial tilt changes affect the reflectance of the reflected spot 34 and, thus, the speckle content. The image acquisition module 18 and the optical projector 16 can be spaced apart from each other in accordance with the second mode of operation. For example, the optical projector 16 and the image acquisition module 18 can be located at different locations within the interior of the vehicle 12A.However, it should be noted that a single image acquisition module 18 can be packaged with the optical projector 16 and used for both the first and second modes of operation.

[0016] With continued reference to Fig. 4 and Fig. 5, microscale detection can be used to monitor a physiological state of an occupant, which can be difficult in the first mode of operation. For example, various vital signs of the occupant can be monitored by detecting micro-movement of the occupant (e.g., body surface), a surface of a covering (e.g., clothing or blanket), a car seat surface, and a child seat surface. In this way, even if the points 34 are not directly reflected from the body surface, microscale movements of the occupant that are transmitted to surfaces surrounding the occupant are detected, enabling occupant monitoring without a direct line of sight to the occupant's body. This can be particularly advantageous in scenarios where a child is in a rear-facing child seat and / or covered by a blanket.In particular, the speckle content of each reflected point 34 is captured by the image acquisition module 18 in the image 28.

[0017] Fig. Figure 4 illustrates one of the images 28 of one of the reflected shapes projected onto a surface, and Fig. Figure 5 illustrates one of the images 28 of the reflected point 34 after the surface has changed its position. It is intended that a time interval between the Fig. 4 and Fig. 5 captured images 28 is within one second, one millisecond, one microsecond, or any time interval that allows the detection of small position changes (e.g., on a microscale) of the surface. The speckle content in the image 28 of each point 34 includes a pixel array 36 with corresponding values ​​associated with each pixel. The pixel array 36 may, for example, have pixel data that includes at least one value corresponding to a grayscale intensity value that corresponds to the level of light reflected toward the image acquisition module 18. Generally, under constant illumination conditions and without changes in the position of the surface, the intensity values ​​or the pixel values ​​for each of the pixels in the pixel array 36 remain relatively constant or within a threshold range / profile of grayscale intensity. For example, since the structured light arrangement 14 (e.g.,via the control system 100) may be configured to detect changes in position on a microscale, some degree of "noise" may be present in the image 28, thereby affecting the pixel values. As described in more detail below, the control system 100 may be configured to distinguish between the noise and the actual movement of the object 32 inside the vehicle 12A or other structure. For example, in some embodiments, the control system 100 may be configured to determine a baseline noise amount (e.g., engine vibration, road conditions, or other external factors affecting the relative movement between the optical projector 16, the image acquisition module 18, and the surface). This baseline may be determined by comparing and / or profiling changes in the position of shapes (i.e.,in the first operating mode) or changes in the speckle content of a reflected point 34 (ie in the second operating mode).

[0018] With continued reference to Fig. 4 and Fig. 5, the pixel array 36 may include a first portion 38 corresponding to an area surrounding the reflected spot 34, and a second portion 40 corresponding to the reflected spot 34. The second portion 40 may approximate the shape of the reflected spot 34, which in this case has a circular Gaussian shape. Small changes other than noise are detected by the control system 100. For example, the slight change or modification of pixel values ​​due to position changes (e.g., on the micrometer scale) of the surface (e.g., Fig. 3 and Fig. 4) correspond to a redistribution of the black or dark pixels.

[0019] As previously explained, the slight changes in the position of the surface may correspond to the vital signs of an occupant. These vital signs may include the presence, rate, and magnitude of respiration (i.e., respiratory rate), heart rate, pulse, and / or other vital signs or physiological conditions of the occupant within the vehicle 12A. In general, the control system 100 may be configured to execute respiration, pulse, or other sensing algorithms to determine vital signs or physiological conditions of the occupant (i.e., a person or animal). Although not illustrated in detail, the control system 100 may evaluate some or all of the plurality of shapes projected by the optical projector 16 and composite the image 28 corresponding to each of the plurality of shapes to further refine the determination of vital signs or physiological conditions of an occupant.For example, the control system 100 may use one or more statistical modeling techniques to composite or otherwise average the change in pixel values ​​of each reflected point 34 to enable discrimination against noise and / or vibrations from the movement of the vehicle 12A (or other structure 12B, 12C) due to environmental or operational factors (e.g., gear shifting, braking, engine vibration, road conditions, and / or the like).

[0020] It is also envisaged that the Fig. 4 and Fig. 5 may have a lower or higher resolution than the illustrated resolution. For example, each reflected point 34 may include any number of pixels corresponding to the particular resolution of the image acquisition module 18. In general, the control system 100 may utilize the principles of speckle interferometry in the second mode of operation and thus evaluate the structured optical pattern 30 based on changes in the speckle content of each of the plurality of shapes (e.g., points 34). The second mode of operation is performed on a per-point basis based on the individual reflections of the shapes.

[0021] The image acquisition module 18 captures the reflected points 34, and each point 34 can be associated with a pixel count and a perceived intensity value. The perceived intensity value of the respective pixel depends on the interference of the microstructure of the illuminated object 32 and the projected light, with the structured optical pattern 30 diffusely projected as a reflected point 34 ( Fig. 5). The illuminated and projected light from the object 32 can therefore be used to determine characteristics of the object 32 (e.g., shape, position, depth, location, etc.) and any changes to the object 32 (e.g., shape, position, micro-movements, etc.). For example, if the object 32 (i.e., the individual or occupant) moves (e.g., tilts relative to the at least one image acquisition module 18 and / or the at least one optical projector 16), a change in the reflected spot 34 results. This change is very sensitive, such that a tilt on the microradian scale causes a detectable change in the reflected spot 34 that can be detected by the at least one image acquisition module 18.The reflected light in the reflected spot 34 depends on properties of the light projected onto the structured optical pattern 30 (wavelength and intensity) and the relative locations of at least one optical projector 16, the object 32 (e.g., material reflectivity and textures), and the image acquisition module 18. In operation, all relative positions are essentially constant except for the location of the object 32, whose movement can be determined on a micrometer scale. Changes in the location of the object 32 on the micrometer scale (i.e., microvibrations) do not typically result in changes in the shape of the structured optical pattern 30, but rather in a perceived intensity value of the pixels of each spot in the reflected spot 34.

[0022] How best in Fig. As illustrated in Figure 6, these changes to object 32 can be modeled for specific activities, such as breathing. For example, a healthy adult typically has a normal respiratory rate of 12 to 20 breaths per minute, while a healthy toddler typically has a normal respiratory rate of 40 to 60 breaths per minute. A semi-linear progression occurs in children between childhood and adulthood, with normal respiratory rates typically decreasing as children age. An important consideration is that these healthy respiratory rates are known in the medical community and can be used as accurate data.Likewise, other vital signs or the progression of physiological conditions are known to the medical community and can be used by control system 100 as a variable or baseline to monitor the individual based on the individual's age, size, gender, and species. Accordingly, subject 32 can be a person (e.g., an infant), and monitoring system 10 can be configured to detect and monitor respiratory rates for normality within expected ranges. In particular, in situations where monitoring an infant's respiratory rate is useful, small respiratory movements of the infant, such as of the chest, head, and other body parts, can be monitored by filtering out other types of movement.For example, if the monitoring system 10 is implemented in a building 12C, the non-respiratory movement may include rolling, posture adjustments, arm and leg movements, clothing movements, etc. Additionally, if the monitoring system 10 is implemented in a vehicle 12A or aircraft 12B, other types of non-respiratory movement may be further filtered, such as turbulence, road conditions, acceleration of the vehicle 12A or aircraft 12B, etc. Fig. 7 illustrates the reflected spot 34 on an object 32 including an infant and a child seat, where the perceived intensity value of the pixels (and changes therein) may be determined with image data received from the at least one image acquisition module 18 (e.g., images 28).

[0023] With reference to Fig. 8-10, the term "filtering" as used herein may refer to one or more respiratory models that accurately reflect the frequency and / or amplitude of movement caused by respiration, and may also refer to discarding other types of detected movements that are not related to respiration (i.e., non-respiratory movement). In some embodiments, the respiratory models are pre-stored in the control system 100 prior to monitoring and categorized by age, weight, gender, health status, etc. In some embodiments, the monitoring system 10 develops a baseline frequency / amplitude for an individual as a first step in monitoring, and the baseline frequency / amplitude may then be stored in the control system 100.In other embodiments, the monitoring system 10 includes both pre-stored models and baseline frequencies / amplitudes during monitoring that can be compared by the control system 100. Fig. Figure 8 illustrates a respiratory signal, wherein changes in the perceived intensity value in a single pixel in one of the points in the reflected point 34 are represented over a period of time, wherein the object 32 is an abdominal region of a child, wherein the Y value is equal to the pixel value of a point and the X value is equal to a frame number. Fig. 9 illustrates the frequency range from Fig. 8 before filtering “BF” and after filtering “AF”, where the Y value equals the amplitude and the X value equals the frequency. Fig. 10 shows a filtered representation of the Fig. 7, where the Y value equals a pixel value of a point 34 and the X value equals a frame number. In some embodiments, filtering may result in changes in the perceived intensity value. These changes in perceived intensity may be scaled for accurate representation during monitoring.

[0024] With further reference to Fig. 6, the base frequency / amplitudes can be measured over a period T1-T N be determined, for example, 5 minutes or less, 1 minute or less, 45 seconds or less, 30 seconds or less, or 15 seconds or more. During an individual's breathing, movements D1-D3 (e.g., micro-movements) between inhalation D2 and exhalation D1 and D3 are measured to determine an amplitude. The amplitude can vary based on clothing, subject position, etc. During the entire time period T1-T N(e.g. T1-T3) a number of breaths can be determined based on the movement D between inhalation D2 and exhalation D1 and D3. The total number of breaths over the period T1-T N can be used to determine a respiratory rate. The measured respiratory rate can then be compared to pre-stored models to ensure accuracy. In some embodiments, a hybrid model can be stored that includes features of the pre-stored model and the base frequency / amplitudes, for example, an average between the two.

[0025] Fig. 11 schematically illustrates the control system 100 of the monitoring system 10. The control system 100 may include an electronic control unit (ECU) 102. The ECU 102 may include a processor 104 and a memory 106. The processor 104 may include any suitable processor 104. Additionally or alternatively, the ECU 102 may include any number of processors in addition to or instead of the processor 104. The memory 106 may comprise a single hard disk or a plurality of hard disks (e.g., hard disk drives) and includes a memory management module that manages one or more partitions within the memory 106. In some embodiments, the memory 106 may include flash memory, semiconductor memory (read-only memory), or the like.The memory 106 may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), or a combination thereof. The memory 106 may include instructions that, when executed by the processor 104, cause the processor 104 to perform at least the functions associated with the components of the monitoring system 10. The at least one optical projector 16, the at least one image acquisition module 18, and the alarm device 20 may therefore be controlled by the ECU 102. The memory 106 may therefore include software 108, pre-stored respiratory models 110, parameter data 112, base frequency / amplitude data 114 acquired by the structured light array 14, user preference data 116, and image data 118 (e.g., a series of images 28).

[0026] With continued reference to Fig. 11, the software 108 may be updated to incorporate additional functionality or override the functionality as described herein. The pre-stored respiratory models 110 may include respiratory rates based on age, gender, weight, health status, and / or other medical information associated with a subject. The parameter data 112 may include predetermined thresholds, wherein deviations between the amplitude and / or frequency of the measured respiratory rate generate a notification using the alarm device 20. The predetermined threshold of the parameter data 112 may vary based on the selection of the pre-stored respiratory models 110; for example, variances in respiratory rate may be more or less common for medical conditions, age, health, weight, gender, etc. The data related to the base frequency / amplitudes 114 may include a detected respiratory rate.The predetermined threshold of the parameter data 112 may also vary based on the data related to the base rate / amplitude 114. For example, individual respiratory rates may vary within expected ranges. A healthy individual may have a slower respiratory rate than the mean. Accordingly, the predetermined threshold may measure variances from the data related to the base rate / amplitude 114 or an average of the data related to the base rate / amplitude 114 and the pre-stored respiratory models 110. The user preference data 116 may include user selections of the functions and / or the order of functions. For example, a user interface on a full-view mirror or other electronic device may allow a user to turn the monitoring system on and off, change the type of notification generated by the alarm device 20, etc.The image data 118 may include real-time images 28 in which measured respiratory rates of the subject may then be compared with the pre-stored respiratory models 110 and / or the data relating to the base frequency / amplitude 114 for variances outside a threshold provided by the parameter data 112.

[0027] Fig.12 illustrates a flowchart 200 for monitoring a respiratory rate of a subject. At 202, the flowchart 200 includes pre-storing respiratory models based on age, gender, weight, health status, and / or other medical information. At 204, the flowchart 200 includes monitoring the subject using a structured light array. Step 204 may further include, at 206, developing a base frequency / amplitude of the subject's respiratory rate. Step 204 may further include, at 208, filtering for non-respiratory-related motion. At 210, the flowchart 200 includes monitoring the subject's respiratory rate using a structured light array.At 212, flowchart 200 may include comparing the monitored respiratory rate to at least one of the pre-stored respiratory models, the base rate / amplitude of the subject's respiratory rate, or an average of the two. At 214, flowchart 200 includes determining whether the compared monitored respiratory rate is within a threshold variance. At 216, after determining that the monitored respiratory rate is outside the threshold variance, the flowchart includes generating a notification. If the monitored respiratory rate is within the threshold variance, the flowchart repeats steps 210 through 214. Steps of flowchart 200 may be stored in memory 106 and executed by processor 104.

[0028] The invention disclosed herein is further summarized in the following sections and is further characterized by combinations of any and all of the various aspects described therein.

[0029] According to one aspect of the present disclosure, a monitoring system for measuring and monitoring a subject's respiratory rate includes an optical projector configured to project an optical pattern and an image capture module configured to capture images of the optical pattern reflected from a surface. The monitoring system further includes an alarm device, a processor, and a memory. The memory includes instructions that, when executed by the processor, cause the processor to monitor the subject's respiratory rate based on changes in the optical pattern reflected from the surface.The processor is further caused to determine whether the monitored respiratory rate is within a threshold variance and, upon a determination that the monitored respiratory rate is outside the threshold variance, to generate a notification using the alarm device.

[0030] In another aspect, the alarm device is configured to generate at least one of an audible notification, a haptic notification, or a visual notification.

[0031] According to yet another aspect, the alarm device comprises a display device.

[0032] According to yet another aspect, the display device is arranged in a rearview mirror of a vehicle.

[0033] According to another aspect, an image capture module is located in a rearview mirror.

[0034] According to yet another aspect, a rearview mirror includes a partially reflective, partially transmissive element.

[0035] According to yet another aspect, an optical projector is configured to project an optical pattern within an infrared wavelength.

[0036] According to another aspect, an optical pattern includes at least one dot.

[0037] In yet another aspect, a memory includes instructions that cause a processor to detect changes in a speckle content of at least one dot.

[0038] In yet another aspect, a memory includes instructions that cause a processor to extrapolate micromovements of a subject related to respiration based on a detected change in speckle content.

[0039] In another aspect, a memory includes instructions that cause a processor to develop a baseline characteristic of a subject's respiratory rate and to compare a monitored respiratory rate to at least one of a plurality of pre-stored respiratory models, the baseline characteristic of the subject's respiratory rate, or an average of the plurality of pre-stored respiratory models and the baseline characteristic of the subject's respiratory rate.

[0040] According to yet another aspect, a baseline characteristic of a subject's respiratory rate includes at least one of a frequency or an amplitude of the respiration.

[0041] In yet another aspect, a memory includes instructions that cause a processor to filter non-respiratory related movements when developing a baseline characteristic of a subject's respiratory rate.

[0042] According to another aspect of the present invention, a monitoring system for measuring and monitoring a respiratory rate of a subject in a vehicle includes a structured light array. The structured light array includes an optical projector configured to project an optical pattern and an image acquisition module configured to acquire images of the optical pattern reflected from a surface. The monitoring system further includes an alarm device, a processor, and a memory. The memory includes instructions that, when executed by the processor, cause the processor to monitor the respiratory rate of the subject based on changes in a speckle content of the optical pattern reflected from the surface.The processor is further caused to determine whether the monitored respiratory rate is within a threshold variance and, upon a determination that the monitored respiratory rate is outside the threshold variance, to generate a notification using the alarm device.

[0043] According to another aspect, changes in speckle content are related to movements on a micrometer scale.

[0044] According to another aspect, the surface is a backrest of a child car seat.

[0045] According to yet another aspect, an alarm device includes a display disposed in a rearview mirror assembly,

[0046] According to yet another aspect of the present disclosure, a monitoring system for measuring and monitoring a respiratory rate of a subject includes an optical projector including a laser and configured to project an optical pattern including at least one spot of light, and an image capture module configured to capture images of the optical pattern reflected from a surface. The monitoring system further includes an alarm device, a processor, and a memory. The memory includes instructions that, when executed by the processor, cause the processor to monitor the respiratory rate of the subject based on changes in the optical pattern reflected from the surface.The processor is further caused to determine whether the monitored respiratory rate is within a threshold variance and, upon a determination that the monitored respiratory rate is outside the threshold variance, to generate a notification using the alarm device.

[0047] In another aspect, a laser is configured to project an optical pattern in an infrared light wavelength, and an image capture module is configured to capture images within the infrared light wavelength.

[0048] In yet another aspect, a memory includes instructions that cause a processor to filter non-respiratory related movements when developing a baseline characteristic of a subject's respiratory rate.

[0049] A person skilled in the art will understand that the construction of the described disclosure, as well as other components, is not limited to any particular material. Other embodiments of the disclosure disclosed herein may be formed from a wide variety of materials, unless otherwise noted herein.

[0050] For the purposes of this disclosure, the term "coupled" (in all its forms, coupling, coupled, etc.) generally means the connecting of two components together (electrically or mechanically) directly or indirectly. Such connecting may be stationary or movable in nature. Such connecting may be achieved with the two components (electrically or mechanically) and any additional intervening elements integrally formed as a single unitary body with each other or with the two components. Such connecting may be permanent or removable or detachable in nature, unless otherwise specified.

[0051] It is also important to note that the structure and arrangement of the elements of the disclosure, as shown in the embodiments, is illustrative only. Although only a few embodiments of the present inventions have been described in detail, those skilled in the art who study this disclosure will readily understand that numerous modifications (e.g., in sizes, dimensions, structures, shapes, and proportions of the various elements, parameter values, assembly arrangements, use of materials, colors, orientations, etc.) are possible without materially departing from the novel teachings and advantages of the recited subject matter.For example, elements shown as being integrally formed may be constructed from multiple parts, or elements shown as being integrally formed may be integrally formed, the function of the interfaces may be reversed or otherwise varied, the length or width of the structures and / or elements or connectors or other elements of the system may be varied, the type or number of adjustment positions provided between the elements may be varied. It should be noted that the elements and / or assemblies of the system may be constructed from any of a wide variety of materials that provide sufficient strength or durability in any of a wide variety of colors, textures, and combinations. Accordingly, all such modifications are intended to be included within the scope of the present inventions.Further additions, modifications, changes and omissions may be made to the design, operating conditions and arrangement of the desired and other exemplary embodiments without departing from the spirit of the present inventions.

[0052] It will be appreciated that embodiments of the disclosure described herein may include one or more conventional processors and unique stored program instructions controlling one or more processors to implement, in conjunction with certain non-processor circuits, some, most, or all of the functions of a monitoring system as described herein. The non-processor circuits may include, but are not limited to, signal drivers, clock circuits, power source circuits, and / or user input devices. In a more concrete sense, these functions may be interpreted as steps of a flowchart used in using or creating a classification system.Alternatively, some or all of the functions could be implemented by a state machine that does not have stored program instructions, or in one or more application-specific integrated circuits (ASICs), with each function or some combination of certain of the functions implemented as custom logic. Of course, a combination of the two approaches could be used. Therefore, the means for these functions have been described herein. Furthermore, it is expected that, notwithstanding the potentially considerable effort and many design possibilities motivated by, for example, available time, current technology, and economic considerations, one of ordinary skill in the art will readily be able to generate such software instructions and programs, as well as ICs, with minimal experimentation when guided by the concepts and principles disclosed herein.

[0053] It is understood that any described processes or steps within the described processes may be combined with other disclosed processes or steps to form structures within the scope of the present disclosure. The exemplary structures and processes disclosed herein are for illustrative purposes and are not to be construed as limiting.

[0054] It is also to be understood that variations and modifications may be made to the foregoing structures without departing from the concepts of the present disclosure, and it is further understood that such concepts are intended to be covered by the following claims unless the language of these claims expressly states otherwise. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] US 9,505,349

[0013] US 10,739,591

[0013]

Claims

[1] A monitoring system for measuring and monitoring a respiratory rate of a subject, the monitoring system comprising: an optical projector configured to project an optical pattern and an image capture module configured to capture images of the optical pattern reflected from a surface; an alarm device; and a processor and a memory including instructions that, when executed by the processor, cause the processor to: Monitoring a respiratory rate of the subject based on changes in the optical pattern reflected from the surface; Determining whether the monitored respiratory rate is within a threshold variance; and after determining that the monitored respiratory rate is outside the threshold variance, generating a notification using the alarm device. [2] The monitoring system of claim 1, wherein the alarm device is configured to generate at least one of an audible notification, a haptic notification, or a visual notification. [3] A monitoring system according to claim 1 or 2, wherein the alarm device includes a display device. [4] A monitoring system according to claim 3, wherein the display device is arranged in a rearview mirror of a vehicle. [5] The monitoring system of claim 4, wherein the image acquisition module is arranged in the rearview mirror. [6] A surveillance system according to claim 4 or 5, wherein the rearview mirror includes a partially reflective, partially transmissive element. [7] A surveillance system according to any one of the preceding claims 1-5, wherein the optical projector is configured to project the optical pattern within an infrared wavelength. [8] Surveillance system according to one of the preceding claims 1-5, wherein the optical pattern includes at least one point. [9] The monitoring system of claim 8, wherein the memory further includes instructions causing the processor to: Detecting changes in a speckle content of at least one point. [10] The monitoring system of claim 9, wherein the memory further includes instructions causing the processor to: based on the detected changes in speckle content, extrapolating micromovements of the subject associated with breathing. [11] A monitoring system according to any one of the preceding claims 1-5, wherein the memory further includes instructions causing the processor to: Developing a baseline characteristic of the subject’s respiratory rate; and Comparing the monitored respiratory rate to at least one of a plurality of pre-stored respiratory models, the baseline characteristic of the respiratory rate of the subject, or an average of the plurality of pre-stored respiratory models and the baseline characteristic of the respiratory rate of the subject. [12] The monitoring system of claim 11, wherein the baseline characteristic of the subject's respiratory rate includes at least one of a frequency and an amplitude of the respiration. [13] The monitoring system of claim 11, wherein the memory further includes instructions causing the processor to: Filtering non-respiratory movements when developing the baseline characteristic of the subject's respiratory rate. [14] A monitoring system for measuring and monitoring a respiratory rate of a subject in a vehicle, the monitoring system comprising: a structured light assembly including an optical projector configured to project an optical pattern and an image capture module configured to capture images of the optical pattern reflected from a surface; an alarm device; and a processor and a memory including instructions that, when executed by the processor, cause the processor to: Monitoring a respiratory rate of the subject based on changes in a speckle content of at least a portion of the optical pattern reflected from the surface; Determining whether the monitored respiration rate is within a threshold variance; and after determining that the monitored respiratory rate is outside the threshold variance, generating a notification using the alarm device. [15] A monitoring system according to claim 14, wherein the changes in speckle content are associated with movements on a micrometer scale. [16] The monitoring system of claim 15, wherein the surface is a backrest of a child car seat. [17] A monitoring system according to any one of claims 14-16, wherein the alarm device comprises a display arranged in a rearview mirror assembly. [18] A monitoring system for measuring and monitoring a respiratory rate of a subject, the monitoring system comprising: an optical projector having a laser configured to project an optical pattern including at least one dot, and an image capture module configured to capture images of the optical pattern reflected from a surface; an alarm device; and a processor and a memory including instructions that, when executed by the processor, cause the processor to: Monitoring a respiratory rate of the subject based on changes in the optical pattern reflected from the surface; Determining whether the monitored respiration rate is within a threshold variance; and after determining that the monitored respiratory rate is outside the threshold variance, generating a notification using the alarm device. [19] The surveillance system of claim 18, wherein the laser is configured to project the optical pattern in an infrared wavelength of light, and the image capture module is configured to capture images within the infrared wavelength of light. [20] A monitoring system according to any one of claims 18 and 19, wherein the memory further includes instructions causing the processor to: Filtering out non-respiratory movements when developing a baseline characteristic of the subject's respiratory rate.

Images