Non-contact infant monitoring using artificial intelligence

Abstract

Use of a non-contact monitoring system to monitor a subject (e.g., a child, infant, child) in a sleeping or resting environment. The system utilizes artificial intelligence (AI) to identify a potential hazard to an object and alert a caregiver. The system may be trained to identify a location of an object as a potential hazard, a location of an object as a non-potential hazard, and / or a type of object as a non-potential hazard. The caregiver can accept that the potential danger is indeed danger and can also clear the potential danger and remind the potential danger, and the system can remember the information for use when similar situations occur later.

Description

Background Art

[0001] Baby monitors are well known. Such monitors typically have either or both a camera and an audio microphone placed very close to the child, which send signals to a remote monitor (video and / or audio speaker) to provide visual and / or audio signals to a caregiver. Based on the signal, the caregiver can determine whether the child is uncomfortable or in difficulty. Unfortunately, the caregiver is informed of whether the child is uncomfortable or in pain by the sounds made by the child. Typical monitors do not provide any indication, for example, whether the child has stopped breathing (which occurs in sudden infant death syndrome (SIDS)), or whether the child is in a dangerous position.

[0002] Therefore, a better child monitoring system is needed. Summary of the Invention

[0003] This disclosure relates to using a non-contact monitoring system to monitor an object (e.g., a toddler, an infant, a child) in a sleeping or resting environment. The system utilizes artificial intelligence (AI) to identify potential dangers to the object and alert the caregiver. The caregiver can acknowledge that the potential danger is indeed a danger, or can clear the potential danger and the alert, and the system will remember this information for use in similar situations in the future.

[0004] One specific embodiment described herein is a method of monitoring an object. The method includes: detecting the position of the object with a non-contact monitoring system that utilizes depth measurement; detecting objects near the object in an area with a non-contact monitoring system that utilizes depth measurement; determining with the non-contact monitoring system whether the detected objects meet one or more criteria for designating the detected objects as potentially dangerous to the object; and initiating an alert when determining whether the detected objects meet one or more criteria for designating the detected objects as potentially dangerous to the object.

[0005] Another specific embodiment described herein is another method of monitoring an object with a non-contact monitoring system. The method includes: detecting the position of the object in a region of interest (ROI); detecting objects near the object; determining with the non-contact monitoring system whether the detected objects meet one or more criteria for designating the detected objects as potentially dangerous to the object, wherein the non-contact monitoring system utilizes artificial intelligence (AI) to determine whether the detected objects meet one or more of the criteria; and initiating an alert when determining that the detected objects meet one or more of the criteria and are thus potentially dangerous to the object.

[0006] Other embodiments are also described and recited herein.

[0007] The present invention content is provided to introduce a series of concepts further described below in the detailed implementation in a simplified form. The present invention content is not intended to identify the key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0008] After considering the specific implementation and the accompanying drawings herein, these and other aspects of the technology described herein will be apparent. However, it should be understood that the scope of the claimed subject matter should be determined by the issued claims, rather than by whether the given subject matter solves any or all of the problems mentioned in the background art or whether it includes any features or aspects listed in the invention content. Brief Description of the Drawings

[0009] Figure 1A is a simulated image of the monitored object in an acceptable situation; Figure 1B is a simulated image of the monitored object, showing a potential danger to the object; Figure 1C is a simulated image of the monitored object, showing another potential danger to the object.

[0010] Figure 2 is a schematic diagram of an example non-contact monitoring system.

[0011] Figure 3 is a schematic diagram of another example non-contact monitoring system.

[0012] Figure 4 is a block diagram of a non-contact monitoring system including a computing device, a server, and an image capture device according to various embodiments described herein.

[0013] Figure 5 is a step-by-step method for monitoring an object. Detailed Description

[0014] As described above, the present disclosure aims to monitor an object (e.g., a young child, an infant, a toddler) who is resting or sleeping. Using the non-contact monitoring system described herein, the risk factors of suffocation, sudden infant death syndrome (SIDS), physical entanglement, and other threats are reduced by detecting the posture of the object and the proximity of physical objects present nearby (such as clutter (e.g., blankets, stuffed animal toys, or toys, etc.) to the object (especially the head and face of the object)). Additionally, the non-contact monitoring system can determine the posture of the object (e.g., prone (lying face down) or supine (lying on the back)) and the position of the object (e.g., against one side of the crib, e.g., too close to the crib mattress). These systems can be used in a residential environment or in a medical or commercial environment (such as a hospital).

[0015] A non-contact monitoring system uses a video signal of an object, identifies physiologically relevant regions within the video image (such as the head, face, neck, arms, legs, or torso of the object), and uses vision-based artificial intelligence (AI) methods to learn to identify potential hazards present in the relevant regions. A potential hazard can be a hazard determined to meet one or more criteria, where the one or more criteria are provided to determine the likelihood that a detected object poses a hazard or potential hazard to the object. For example, criteria for determining whether a detected object is a hazard or potential hazard can include: whether the distance between the object and the object is less than 12 inches, whether the distance between the object and the object is less than 6 inches, whether the object covers some or all of a portion of certain parts of the object's body (e.g., the head or face), whether the object has a particular size and / or shape, etc. Using the video image, the system extracts distance or depth signals from the relevant regions, correlates the depth signals with the presence of an object, and uses this indication to determine whether the object is under a potential threat (such as evaluating the object according to the one or more criteria). In some embodiments, the system correlates changes in the depth signal over time with the movement or introduction of an object.

[0016] Using a non-contact monitoring system, signals representative of the three-dimensional structure and optionally the movement of an object are detected by a camera or camera system that observes but does not touch the object. The camera or camera system can utilize any one or all of depth signals, color signals (e.g., RGB signals), and IR signals. By appropriate selection and filtering of the signals detected by the camera, the physiological effects of each detected signal can be separated and measured.

[0017] Generally, remotely sensing an object using a video-based monitoring system tends to encounter some challenges. One challenge is ambient light. In this context, "ambient light" means surrounding light that is not emitted by the components of the camera or monitoring system. In some embodiments of the non-contact monitoring system, the desired physiological signals are generated or carried by a light source. Therefore, for this reason, ambient light cannot be completely filtered, removed, or avoided as noise. Indoor light (including ceiling lights, sunlight, TV screens, night lights, changes in reflected light, and shadows created by moving objects) can all affect the light signal reaching the camera. Even minor movements outside the camera's field of view can reflect light onto the object being monitored.

[0018] This disclosure describes methods for non-contact monitoring of an object to determine potential hazards due to the object's posture and undesired objects very close to the object (e.g., at a distance less than 12 inches from the object) and issue alerts. These methods are particularly applicable for alerting a caregiver (e.g., a parent) about the posture of an object (e.g., a child, e.g., an infant) in bed, or whether there is a potential risk of suffocation or SIDS, e.g., whether an object is too close to the object's head or face, or whether an object has fallen onto the bed.

[0019] Non-contact monitoring systems developed for contactless monitoring of subjects are designed to recognize features of the subject and identify objects whose position and / or location may pose a threat to the subject. Non-contact monitoring systems utilize AI to learn objects and postures / positions that may pose a threat. Additionally, non-contact monitoring systems can also identify the presence of new objects. Upon identifying a potential threat, these systems provide alerts to the caregiver.

[0020] The contactless systems receive video signals from the object and the environment and extract distance or depth signals from the relevant areas from the video signals to provide a three-dimensional structure map based on the depth signals; these systems can also determine any movement or motion based on the depth signals. These systems can also receive a second signal, namely a light intensity signal reflected from the object and the environment, and calculate the depth or distance and movement or motion based on the reflected light intensity signal. In some embodiments, the light intensity signal is a reflection of a pattern or feature (e.g., using visible color or infrared) projected onto the object by a projector.

[0021] The depth sensing feature of the system provides a measurement of the distance or depth between the detection system and the object. One or two cameras may be used to determine the depth and depth change from the system to the object. When two cameras are used that are set at a fixed distance apart, stereoscopic vision is provided because the perspective of the scene from which the distance information is extracted is slightly different. When different features are present in the scene, the stereo image algorithm can find the location of the same feature in both image streams. However, if the object is featureless (e.g., a smooth surface with a single color), it may be difficult for the depth camera system to distinguish the perspective difference. By including an image projector for projecting features (e.g., in the form of dots, pixels, etc., visual or IR) onto the scene, the projected features can be monitored over time to make an estimate of the object's position and any position changes.

[0022] In the following description, reference is made to the accompanying drawings forming part thereof, and at least one specific embodiment is shown by way of demonstration in the accompanying drawings. The following description provides additional specific embodiments. It should be understood that other embodiments can be conceived and made without departing from the scope or spirit of the present disclosure. Therefore, the following detailed description will not be considered restrictive. Although the present disclosure is not limited to this, an understanding of various aspects of the present disclosure will be obtained by discussing the examples provided below, including the accompanying drawings. In some cases, a figure mark may have an associated sub-label consisting of lowercase letters to represent one of multiple similar parts. When a reference is made to a figure mark without a sub-label description, the reference is intended to refer to all such multiple similar parts.

[0023] Figure 1A , Figure 1B and Figure 1CShows example simulated images from the non-contact monitoring system of the present disclosure, which illustrate various situations detectable by the system. The images are simulated images on a device (such as a mobile phone or tablet), showing an object I (specifically, a baby in these images) in a crib.

[0024] In Figure 1A , the baby is seen in the center of the crib. Since the system does not detect any potential hazards near the baby, the "Status Good" is indicated by the tick icon 101 in the image.

[0025] In Figure 1B , the baby is seen covered with a blanket, there is a pillow near the child's head, and another pillow near the child's feet. The "Warning Status" is indicated by the X icon 102 in the image. The pillow near the head is within the area identified as a potential hazard (e.g., within 12 inches of the object, e.g., within 6 inches of the object), and is depicted by a square 104 because the system has determined that the object is a potential hazard that requires a reminder (e.g., since the detected object meets one or more criteria for designating a detected object as a potential hazard; in this case, the criterion is that the object is within 12 inches or 6 inches of the object). The depicted feature can be a warning color, such as red or orange. The pillow near the feet is also depicted by a rectangle 106 because the system has monitored the presence of an object within the area identified as a potential hazard, but has determined that the pillow is not a potential hazard because it is at the feet of the object (e.g., since the detected object does not meet one or more criteria for designating a detected object as a potential hazard; in this case, the criterion is that the object is not within 12 inches or 6 inches of the object, or not within 12 inches or 6 inches of the head or face of the object). The depicted feature can be, for example, green or blue. For both pillows, the system has been programmed, for example, using vision-based artificial intelligence (AI) methods, to identify potential hazards present in relevant areas (e.g., within a predetermined area near at least the head and face) based on meeting one or more hazard criteria in some cases. Examples of items that may pose a potential hazard due to being near the object include pillows, stuffed animal toys, plush toys, padding or railings on the side of the crib.

[0026] The nearby area used as part of identifying whether an object poses a potential hazard to the object can exist in the head / face / neck / shoulder area of the object, and can include the torso or even the entire body of the object. The area can be, for example, within 12 inches, 6 inches, or 3 inches of the object, including contacting the object. The area is programmed into the monitoring system and can be adjusted manually by the caregiver, for example, as the object grows older.

[0027] In Figure 1CIn it, it is seen that the baby is covered by a blanket and to the extent that the head is covered by the blanket. Since, for example, this condition meets one or more danger criteria, the system determines that the object is potentially dangerous and thus it is necessary to issue a reminder. The warning status is indicated by the X icon 102 in the image. The system has been programmed to determine that the head and / or face of the object is no longer recognizable, for example, by using color or IR images or by using the three-dimensional structure determined by a depth camera.

[0028] In both of these cases ( Figure 1B and Figure 1C ), the system issues a reminder to the caregiver. In this embodiment, the reminder is indicated by the bell icon 114 in Figure 1B and Figure 1C ; the reminder can be visual (e.g., the bell icon 114) and / or can be auditory.

[0029] The system can be programmed not to issue an alarm or reminder for certain recognized objects such as pacifiers, teething rings, and the hands of the object, even if these objects are very close to the head / face of the object.

[0030] Figure 2 A non-contact object monitoring system 200 and an object I (in this particular example, a baby in a crib) are shown. It should be noted that the systems and methods described herein are not limited to cribs and can also be used with cradles, enclosures, or any other place where the object is left alone. The system 200 includes a non-contact detector system 210 placed away from the object I. In this embodiment, the detector system 210 includes a camera system 214, particularly a camera including infrared (IR) detection features. The camera 214 can be a depth-sensing camera, such as the Kinect camera from Microsoft Corp. (Redmond, Washington) or the RealSense TM D415, D435, or D455 camera from Intel Corp. (Santa Clara, California). The camera system 214 is away from the object I because it is separated from the object I and does not physically contact the object. The camera system 214 can be positioned very close to the crib or on the crib. The camera system 214 includes a detector exposed to a field of view F that encompasses at least a portion of the object I.

[0031] The camera system 214 includes a depth-sensing camera that can detect the distance between the camera system 214 and an object in its field of view F. As disclosed herein, such information can be used to determine that an object is within the field of view of the camera system 214 and to determine a region of interest (ROI) on the object to be monitored. Once the ROI is identified, the ROI can be monitored over time, and depth data can be used to locate the presence of an object (e.g., object I), and changes in the depth of points can indicate movement of object I or an object within the ROI. The field of view F is selected to be at least the upper torso of the object. However, since young children and infants typically move within the confines of their cribs, beds, or other sleeping areas, the entire area that object I may occupy (e.g., the crib) can be the field of view F. The ROI can be the entire field of view F or less than the entire field of view F.

[0032] The camera system 214 can operate at a set frame rate, which is the number of image frames taken per second (or other time period). Example frame rates include 20, 30, 40, 50, or 60 frames per second, greater than 60 frames per second, or other values in between. A frame rate of 20 to 30 frames per second produces a useful signal, although frame rates above 100 or 120 frames per second help avoid aliasing due to light flicker (for artificial light at a frequency of about 50 or 60 Hz).

[0033] The distance from the ROI on object I to the camera system 214 is measured by the system 200. Generally, the camera system 214 detects the distance between the camera system 214 and a surface within the ROI; changes in the depth or distance of the ROI can indicate movement of the object or the presence of an object within the ROI (e.g., an animal-shaped stuffed toy that has fallen on object I).

[0034] In some embodiments, the system 200 determines the skeletal profile of object I to identify one or more points from which the ROI is inferred. For example, a skeleton can be used to find the center point of the chest, shoulder points, waist points, hands, head, and / or any other points on the body. These points can be used to determine the ROI. In other embodiments, instead of using a skeleton, other points are used to determine the ROI. For example, the face can be identified, and the torso and waist regions proportional and spatially related to the face can be inferred.

[0035] In another example, the object I may wear a specially constructed garment that identifies various points on the body, such as the torso or arm. The system 200 may identify those points by recognizing the indicative features of the garment. Such indicative features may be visual encoded messages (e.g., barcodes, QR codes, etc.), or brightly colored shapes that contrast with the rest of the object's garment. In some embodiments, a garment worn by the object may have a grid or other recognizable pattern to assist in identifying the object and / or their movement. In some embodiments, the indicative features may be adhered to the garment using fastening mechanisms (such as adhesives, pins, etc.), or directly adhered to the object's skin, such as by an adhesive. For example, small stickers or other indicators may be placed on the object's hand, which can be easily recognized from the images captured by the camera.

[0036] In some embodiments, the system 200 may receive user input to identify a starting point for defining the ROI. For example, an image may be reproduced on an interface such that a user of the interface can select a point on the object (such as a point on the head) that can be used to determine the ROI. Other methods for identifying the object and the points on the object that define the ROI may also be used.

[0037] However, if the ROI is substantially featureless (e.g., has a smooth, monochromatic surface, such as a blanket or sheet covering the object I), the camera system 214 may have difficulty discerning perspective differences. To address this issue, the system 200 may include a projector 216 to project various features (e.g., points, crosses or X's, lines, individual pixels, etc.) onto the object within the ROI; these features may be visible light, UV, infrared (IR) light, etc. The projector may be part of the detector system 210 or the entire system 200.

[0038] The projector 216 generates a series of features over time on the ROI, from which the intensity of the reflected light is monitored and measured. The measure of the amount, color, or brightness of the light within all or a portion of the reflected features over time is referred to as the light intensity signal. The camera system 214 detects the features and determines the light intensity signal based on these features. In an embodiment, each visible image projected by the projector 216 includes a two-dimensional pixel array or grid, and each pixel may include three color components - for example, red, green, and blue (RGB). The measure of one or more color components of one or more pixels over time is referred to as a "pixel signal", which is a type of light intensity signal. In another embodiment, when the projector 216 projects IR features that are invisible to the human eye, the camera system 214 includes infrared (IR) sensing features. In another embodiment, the projector 216 projects UV features. In other embodiments, other modalities including millimeter wave, hyperspectral, etc. may be used.

[0039] The projector 216 can alternatively or additionally project a featureless intensity pattern (e.g., a uniform pattern, a gradient pattern, or any other pattern that does not necessarily have distinct features, or a pattern of random intensity). In some embodiments, the projector 216 or more than one projector can project a combination of a feature-rich pattern and a featureless pattern onto the ROI.

[0040] The detector system 210 detects the light intensity of the image reflected by the object surface.

[0041] Measurements (e.g., depth signal, RGB reflection, light intensity) are sent to the computing device 220 via a wired or wireless connection 221. The computing device 220 includes a display 222, a processor 224, and a hardware memory 226 for storing software and computer instructions. Sequential image frames of the object I are recorded by the camera system 214 and sent to the computing device 220 for analysis by the processor 224. The display 222 can be remote from the computing device 220, such as a video screen located separately from the processor and memory. Other embodiments of the computing device 220 can have different, fewer, or additional components compared to Figure 2 those shown. In some embodiments, the computing device can be a server. In other embodiments, Figure 2 the computing device can be connected to a server. The captured images (e.g., still images or videos) can be processed or analyzed at the computing device and / or at the server to create a three-dimensional structure diagram or image to identify the object I having the ROI and any other objects.

[0042] In some embodiments, the computing device 220 is operatively connected (e.g., wirelessly, via a WiFi connection, a cellular signal, a Bluetooth TM connection, etc.) to a remote device 230, such as a smartphone, a tablet, or just a screen. The remote device 230 can be remote from the computing device 220 and the object I, e.g., in an adjacent or nearby room. The computing device 220 can send a video data stream to the remote device 230 that shows, for example, the object I and / or the field of view F. Additionally or alternatively, the computing device 220 can send instructions to the remote device 230 to trigger an alarm, such as when the system 200 detects that an object is in a problematic location or position that poses a potential danger to the object I.

[0043] Figure 3Shows another non-contact object monitoring system 300 and an object I (in this example, a baby in a crib). System 300 includes a non-contact detector 310 placed away from object I. In this embodiment, detector 310 includes a first camera 314 and a second camera 315, and at least one of the first camera and the second camera includes infrared (IR) camera features. Cameras 314, 315 are positioned such that their ROIs at least intersect and, in some embodiments, completely overlap. Detector 310 also includes an IR projector 316 that projects various features (e.g., dots, crosses or Xs, lines, or featureless patterns, or combinations thereof, etc.) onto object I in the ROI. Projector 316 can be separate from or integrated with detector 310, as Figure 3 shown. In some embodiments, more than one projector 316 can be used. Both cameras 314, 315 are intended to have the features projected by projector 316 within their ROIs. Cameras 314, 315 and projector 316 are away from object I as they are separated from object I and do not contact object I. In this embodiment, projector 316 is physically positioned between cameras 314, 315, while in other embodiments this may not be the case.

[0044] The distances from the ROI to cameras 314, 315 are measured by system 300. Generally, cameras 314, 315 detect the distances between cameras 314, 315 and the projected features on the inner surface of the ROI. The light emitted by projector 316 scatters / diffuses in all directions after hitting the surface; the diffused pattern depends on the reflection and scattering properties of the surface. Cameras 314, 315 also detect the light intensities of the various features projected in their ROIs. Based on the distances and light intensities, the presence of monitored object I and any objects, as well as any movement of object I or the objects, are monitored.

[0045] The detected images, diffusion measurements, and / or reflection patterns are sent to a computing device 320 via a wired or wireless connection 321. Computing device 320 includes a display 322, a processor 324, and a hardware memory 326 for storing software and computer instructions. Display 322 can be away from computing device 320, such as a video screen being positioned separately from the processor and the memory. In other embodiments, Figure 3 the computing device can be connected to a server. The captured images (e.g., still images or videos) can be processed or analyzed at the computing device and / or at the server to create a three-dimensional structure diagram or image to identify object I having the ROI and any other objects.

[0046] In some embodiments, computing device 320 is operably connected (e.g., wirelessly, via a WiFi connection, a cellular signal, Bluetooth TMconnect (such as via Wi-Fi, Bluetooth, or other wireless communication means) to a remote device 330, such as a smartphone, a tablet, or just a screen. The remote device 330 can be away from the computing device 320 and the object I. For example, it can be in an adjacent or nearby room. The computing device 320 can send a video data stream to the remote device 330, which shows, for example, the object I and / or the field of view F. Additionally or alternatively, the computing device 320 can send instructions to the remote device 230 to trigger an alarm, such as when the system 300 detects that an object is in a problematic location or position that poses a potential danger to the object I.

[0047] For both systems 200 and 300 and their variants, the computing devices 220, 320 identify whether any object within the ROI is close enough to the head and / or face of the object I.

[0048] The computing devices 220, 320 determine the position and pose of the head and / or face of the object based on the image of the ROI (formed by, for example, depth signals, RGB reflections, light intensity measurements). Then, the computing devices 220, 320 determine based on the image whether any object is close enough to the head and / or face of the object to warrant an alarm. If a danger is recognized (see, for example, Figure 1B which indicates two potential dangers), then the computing devices 220, 320 initiate an alarm on the remote devices 230, 330.

[0049] The computing devices 220, 320 can be trained using vision-based artificial intelligence (AI) methods to learn to identify objects in the image (including the face and / or head of the object) as well as other objects (such as pillows, stuffed animal toys, etc.). The computing devices 220, 320 can also be trained using AI to determine whether there are potential dangers in the relevant area. Any standard AI models and standard methods can be used to train the computing devices 220, 320. For example, a large number of data points can be used to create a dataset of images. The systems 200, 300 can be configured to learn whether the objects identified by the computing devices 220, 320 are indeed potential dangers.

[0050] For example, referring to Figure 1B , the computing devices 220, 320 have identified two potential dangers in the scene: a pillow near the head of the object I identified in the square 104, and a pillow near the feet of the object identified in the rectangle 106. Using repeated data inputs, the computing devices 220, 320 can be taught to identify pillows as objects that may pose a potential danger to the object. Then, the systems 200, 300 can alert the caregiver via an alarm (auditory and / or visual) on the remote devices 230, 330.

[0051] In some cases, an object may not be dangerous even if it is recognized as being near the subject. Using the same data input, the computing devices 220, 320 can be taught that although the object is within the field of view and near the subject, it does not pose a danger when it is far enough away from the subject's head / face, such as a pillow near the feet of the subject identified in rectangle 106. Additionally or alternatively, if the computing devices 220, 320 determine that it is necessary to issue a reminder about the object, the user can manually intervene with the computing devices 220, 320. The intervention can be a one-time intervention, or the computing devices 220, 320 can save the instruction and apply the intervention to subsequent similarly positioned objects.

[0052] The computing devices 220, 320 can be trained to recognize an object near the subject and consider the object as not dangerous. For example, the system can be trained not to trigger an alarm when a pacifier or teething ring is positioned near the subject's head / face.

[0053] The computing devices 220, 320 also determine based on an image of the ROI (formed by, for example, depth signals, RGB reflections, light intensity measurements) whether an image of the subject's head and / or face is not recognized; see, for example, the case where the subject's head is covered by a blanket. Figure 1C If the head and / or face is not recognized, for example, due to the lack of three-dimensional structural features representing a part of the face or head (e.g., ears), the computing devices 220, 230 initiate an alarm on the remote devices 230, 330.

[0054] The computing devices 220, 320 have appropriate memory, processors, and software or other programs to evaluate the ROI image, recognize objects, maintain a database of objects, and determine whether any object poses a potential danger. Figure 4 FIG. [FIGURE NUMBER] is a block diagram showing a system including a computing device 400, a server 425, and an image capture device 485 (e.g., a camera, such as camera system 214 or cameras 314, 315). In various embodiments, fewer, additional, and / or different components can be used in the system.

[0055] The computing device 400 includes a processor 415 coupled to a memory 405. The processor 415 can store and invoke data and applications in the memory 405, including applications that process information and send commands / signals according to any of the methods disclosed herein. The processor 415 can also display objects, applications, data, etc. on the interface / display 410 and / or provide an auditory reminder via the speaker 412. The processor 415 can also receive input via the interface / display 410 either additionally or alternatively. The processor 415 is also coupled to a transceiver 420. With this configuration, the processor 415 and subsequently the computing device 400 can communicate with other devices (such as the server 425) via a connection 470 and communicate with an image capture device 485 via a connection 480. For example, the computing device 400 can send information about an object determined from an image captured by the image capture device 485, such as depth information of an object or an object in the image, to the server 425.

[0056] The server 425 also includes a processor 435, which is coupled to a memory 430 and a transceiver 440. The processor 435 can store and invoke data and applications in the memory 430. With this configuration, the processor 435 and subsequently the server 425 can communicate with other devices (such as the computing device 400) via a connection 470.

[0057] The computing device 400 can be, for example Figure 2 the computing device 220 or Figure 3 the computing device 320. Thus, the computing device 400 can be located away from the image capture device 485, or it can be located locally and close to the image capture device 485 (e.g., in the same room). The processor 415 of the computing device 400 can perform any one or all of the various steps disclosed herein. In other embodiments, these steps can be performed on the processor 435 of the server 425. In some embodiments, the various steps and methods disclosed herein can be performed by both the processors 415 and 435. In some embodiments, certain steps can be performed by the processor 415 and other steps by the processor 435. In some embodiments, the information determined by the processor 415 can be sent to the server 425 for storage and / or further processing.

[0058] The devices shown in the illustrative embodiments can be used in various ways. For example, either or both of the connections 470, 480 can be changed. For example, either or both of the connections 470, 480 can be hardwired connections. A hardwired connection can involve connecting the devices through a USB (Universal Serial Bus) port, a serial port, a parallel port, or other types of wired connections for facilitating the transfer of data and information between the processor of the device and the second processor of the second device. In another example, one or both of the connections 470, 480 can be a docking piece into which one device can be inserted into another device. As another example, one or both of the connections 470, 480 can be a wireless connection. These connections can be any kind of wireless connection, including but not limited to Bluetooth connections, Wi-Fi connections, infrared, visible light, radio frequency (RF) signals, or other wireless protocols / methods. For example, other possible modes of wireless communication can include near field communication, such as passive radio frequency identification (RFID) and active RFID technologies. RFID and similar near field communication can enable various devices to communicate over a short distance when placed in close proximity to each other. In yet another example, various devices can be connected via an Internet (or other network) connection. That is, one or both of the connections 470, 480 can represent a number of different computing devices and network components that allow various devices to communicate over the Internet via either a hardwired connection or a wireless connection. One or both of the connections 470, 480 can also be a combination of several connection modes.

[0059] Figure 4 The configuration of the devices in [the figure] is only one physical system on which the disclosed embodiments can be implemented. There can be other configurations of the devices shown to practice the disclosed embodiments. Further, Figure 4 there can be more or fewer device configurations than those shown in [the figure] to practice the disclosed embodiments. Additionally, Figure 4 the devices shown in [the figure] can be combined together to have fewer devices than shown, or separated so that there are more than three devices in the system. It will be understood that various combinations of computing devices can execute the methods and systems disclosed herein. Examples of such computing devices can include other types of infrared cameras / detectors, night vision cameras / detectors, other types of cameras, radio frequency transmitters / receivers, smart phones, personal computers, servers, laptop computers, tablets, RFID-enabled devices, or any combination of such devices.

[0060] The disclosed non-contact surveillance system and method utilize the depth (distance) information between the (multiple) cameras and an object to determine the presence of the object, and then determine whether the object poses a threat to an object. The system is programmed or trained to identify different types of objects and their possible locations, so as to determine whether the location of the object poses a threat.

[0061] Figure 5 Method 500 for monitoring an object (e.g., an infant, a child) using a non-contact monitoring system as described herein is provided. In a first step 510, the non-contact monitoring system detects the position of the object using depth; the depth can be determined based on depth measurements, reflected signals, or reflected intensity signals (for light, IR, RGB, etc.). In a second step 520, the non-contact monitoring system detects objects near the object in the area around the object, and in step 530, the non-contact monitoring system determines whether the detected object poses a potential danger to the object (e.g., by determining whether the detected object meets one or more danger criteria). When it is determined that the detected object poses a potential danger to the object, in step 540, the non-contact monitoring system initiates an alarm.

[0062] As described above, in addition to the method of using depth (distance) information between the (multiple) cameras and the object in the present disclosure to determine the presence and position of an object, this method can also use the reflected light intensity from projected light features and / or IR features (e.g., points, grids, stripes, crosses, squares, etc., or featureless patterns, or combinations thereof) in the scene to estimate depth (distance). Based on the depth information, the presence of an object related to the object can be identified, and the system can determine whether it is necessary to alert the caregiver via AI learning.

[0063] The above specification and examples provide a complete description of the structure and use of exemplary embodiments of the present invention. The above description provides specific embodiments. It should be understood that other embodiments can be envisioned and made without departing from the scope or spirit of the present disclosure. Therefore, the above detailed description will not be considered restrictive. For example, the elements or features of one example, embodiment, or implementation can be applied to any other example, embodiment, or implementation described herein, as long as there is no conflict. Although the present disclosure is not limited thereto, an understanding of various aspects of the present disclosure will be obtained through the discussion of the provided examples.

[0064] Unless otherwise indicated, all numbers representing feature sizes, amounts, and physical properties will be understood to be modified by the term "about", whether or not the term "about" is directly present. Thus, unless there is a contrary indication, the numerical parameters set forth are approximations that can vary depending on the desired properties sought by those skilled in the art using the teachings disclosed herein.

[0065] As used herein, the singular forms "a", "an", and "the" encompass embodiments having plural referents unless the context clearly dictates otherwise. As used in this specification and the appended claims, the term "or" is generally meant to include "and / or" unless the context clearly dictates otherwise.

Claims

1. A method for monitoring an object, the method comprises: detecting the object with a non-contact monitoring system utilizing depth measurement; detecting an object in the vicinity of the object within a region with the non-contact monitoring system utilizing depth measurement; determining, with the non-contact monitoring system, whether the detected object meets one or more criteria for designating the detected object as a potential danger to the object; and initiating an alarm when it is determined that the detected object meets one or more criteria for designating the detected object as a potential danger to the object.

2. The method according to claim 1, wherein determining whether the detected object meets one or more criteria for designating the detected object as a potential danger to the object comprises: utilizing artificial intelligence (AI) to determine whether the detected object meets one or more criteria for designating the detected object as a potential danger to the object.

3. The method according to claim 2, wherein utilizing AI to determine whether the detected object meets one or more criteria for designating the detected object as a potential danger comprises: utilizing AI to determine whether the detected object is near the face of the object.

4. The method according to claim 2, wherein utilizing AI to determine whether the detected object is near the face of the object comprises: utilizing AI to determine whether the detected object covers part or all of the face of the object.

5. The method according to claim 1, wherein determining whether the detected object meets one or more criteria for designating the detected object as a potential danger to the object comprises: utilizing AI to determine that the detected object does not meet one or more criteria for designating the detected object as a potential danger, and thus does not pose a potential danger to the object.

6. The method according to claim 5, wherein utilizing AI to determine whether the detected object does not meet one or more criteria for designating the detected object as a potential danger comprises: utilizing AI to determine whether the detected object is in a position not near the face of the object.

7. The method according to claim 1, wherein initiating an alarm comprises: providing a visual alarm on a display.

8. The method according to claim 7, wherein providing a visual alarm on the display comprises: providing the visual alarm on a device remote from the patient.

9. The method according to claim 1, wherein initiating an alarm comprises: providing an audible alarm.

10. The method according to claim 9, wherein providing an audible alarm comprises: providing the audible alarm from a device remote from the patient.

11. A method for monitoring an object with a non-contact monitoring system, the method comprises: detecting an object in a region of interest (ROI); detecting an object near the object; Determine whether the detected object meets one or more criteria for designating the detected object as a potential danger to the subject using the non-contact monitoring system, wherein the non-contact monitoring system utilizes artificial intelligence (AI) to determine whether the detected object meets one or more criteria for designating the detected object as a potential danger; and Initiate an alarm when it is determined that the detected object meets one or more criteria for designating the detected object as a potential danger to the subject.

12. The method according to claim 13, wherein One of the one or more criteria for designating the detected object as a potential danger to the subject is whether the detected object is near one or more of the head, face, neck, and shoulders of the subject.

13. The method according to claim 11, wherein Initiating an alarm includes: Initiating a visual alarm.

14. The method according to claim 11, wherein Initiating an alarm includes: Initiating an auditory alarm.

15. The method according to claim 11, wherein Initiating an alarm includes: Initiating the alarm on a device remote from the non-contact monitoring system.

Images