Infant monitoring

Abstract

The invention relates to infant monitoring. The proposed concept relates to evaluating whether an image of a monitoring area captured by an image sensor is suitable for image-based monitoring of an infant. In particular, embodiments are directed to a method of assessing the suitability of an image to be used for infant monitoring based on measurements that determine the size and / or orientation of a region of the image. If it is determined that the image is not suitable, an instruction to adjust the image sensor is provided.

Description

Technical Field

[0001] The present invention relates to the field of infant monitoring, and in particular to image-based infant sleep monitoring. Background Art

[0002] Parents and caregivers often use image-based baby monitoring systems to monitor their infants while they sleep, allowing parents / caregivers to remotely view their sleeping infants. Baby monitoring systems are typically equipped with one or more sensors to capture visual and / or audio data from the monitored area, which can then be transmitted to parents via a separate device or via an app on the parent's smartphone or other personal smart device.

[0003] Modern infant monitoring systems may include additional features to track and monitor an infant's behavior while sleeping. These features may include image-based monitoring of the infant's breathing or vital signs or tracking of the infant's movements. If the images obtained by the monitoring system are of poor quality, the results of these sleep monitoring procedures may be unreliable.

[0004] SHARMA SHASHANK et al.: “Automated detection of newborn sleep apnea using video monitoring system”, 2015 EIGHTH INTERNATIONAL CONFERENCE ON ADVANCES INPATTERN RECOGNITION (ICAPR), IEEE, January 4, 2015, pp. 1-6 (XP032740575) discloses the use of video monitoring to detect neonatal sleep apnea.

[0005] FANG CHIUNG-YAO et al.: “A Vision-Based Infant Monitoring System Using PTIP Camera”, 2016 INTERNATIONAL SYMPOSIUM ON COMPUTER, CONSUMER AND CONTROL (IS3C), IEEE, July 4, 2016, pp. 279-282 (XP032947292) discloses video-based infant monitoring, the purpose of which is to identify infant movements and avoid accidents.

[0006] US2017 / 055877 discloses detecting the rise and fall of an infant's abdomen during sleep. 3D digital camera images are analyzed. Summary of the Invention

[0007] The present invention is defined by the claims.

[0008] According to one aspect of the present invention, there is provided a computer-implemented method for evaluating the suitability of an image of a monitoring area captured by an image sensor for use in monitoring an infant, the method comprising: determining boundaries of a target area within the image, or receiving an indication of boundaries of a target area within the image, wherein the target area is an area containing image data to be used for monitoring an infant; determining a measurement of at least one of: a size; and an orientation of the target area of ​​the image; determining the suitability of the image for use in monitoring an infant based on the determined measurements of the size and / or orientation of the target area; and providing an output based on the determined suitability of the image.

[0009] Thus, the concepts presented are directed to providing schemes, solutions, concepts, designs, methods, and systems relating to evaluating whether an image of a monitoring area captured by an image sensor is suitable for use in image-based monitoring of an infant. Specifically, embodiments are directed to providing a method for evaluating the suitability of an image based on measurements that determine the size and / or orientation of an area of ​​the image to be used for infant monitoring. The image sensor can be any image sensor suitable for imaging an infant, including sensors for optical imaging, ultrasound imaging, radar imaging, or lidar imaging. In some embodiments, if the image is determined to be unsuitable, instructions are provided for adjusting the image sensor.

[0010] Infant monitoring is preferably performed during the infant's sleep and can provide information about the infant's sleep (such as sleep cycles and wake cycles). However, the same method can be used more broadly for image-based vital sign monitoring or other image-based monitoring functions, such as whether the infant is awake or asleep, or in light or deep sleep.

[0011] The image sensor is mounted in a relatively static location, such as above a crib, and monitors a static field of view.

[0012] The invention will be described below with reference to sleep monitoring, but this is only a preferred embodiment of the invention.

[0013] It is recommended to use the size and / or orientation of the area of ​​the image to be used for monitoring the infant as a measure of whether the image is suitable for this purpose. Based on this assessment, it may be recommended to adjust the settings of the image sensor used to capture the image to ensure that the quality of future images is sufficient to allow accurate infant monitoring. For example, this method can be used as part of system installation / calibration before the system is used.

[0014] Modern image-based infant monitoring systems can provide various functions to track and monitor the behavior of infants (such as sleeping infants). These rely on processing image data of the infant or its bed / crib to extract information about the infant, such as the infant's presence, movement, and vital signs. This can be achieved by processing the image data using a variety of different models and algorithms, including motion detection models, pixel shift algorithms, object detection models, and edge detection models. In order for the output of these processing techniques to be accurate, the image must be of sufficient quality. For example, infant breathing monitoring relies on accurate detection of very small movements of the infant's chest or upper body, and therefore requires depicting the infant with sufficient resolution to detect these movements.

[0015] It has been recognized that one way to ensure accurate image-based monitoring of an infant (e.g., a sleeping infant) is to crop image data captured of the monitoring area so that it includes only the region of interest to be used for image data tracking the infant. For example, an image of a crib can be cropped so that it includes only the area of ​​the mattress where the infant is likely to be found. This can reduce noise generated by the background of the image and reduce processing requirements. The proposed method utilizes determining the boundaries of the cropped region and assessing the suitability of the image based on measurements of the size or orientation of the region. The inventors of the present application have discovered that the size of the target region (i.e., the region of the captured image that contains image data to be used for infant sleep monitoring) is a useful metric for assessing whether the image is of sufficient quality to allow reliable infant monitoring. Similarly, the orientation of the target region (e.g., whether it is landscape or portrait) can also be a good indicator of image quality because it can indicate the position of the image sensor relative to the sleeping infant.

[0016] Ultimately, the proposed concept(s) may enable improved computer-implemented methods for evaluating images of a monitoring area captured by an image sensor for use in assessing the suitability of an infant's sleep.

[0017] In some embodiments, the method determines a measurement of the size of the target area, and determining the suitability of the image for monitoring the infant may further include: determining that the image is suitable if the measurement of the size of the target area is greater than a predetermined threshold size; and determining that the image is unsuitable if the measurement of the size of the target area is less than the predetermined threshold size. Thus, the method may allow for a simple binary determination of whether an image is suitable based on the predetermined threshold size. This may allow known factors regarding the required quality of the image to be incorporated into the suitability assessment step.

[0018] In some embodiments, the measurement of the size of the target region may include the pixel area of ​​the target region, and the predetermined threshold size is between 550,000 pixels and 950,000 pixels. Target regions larger than this threshold size have been shown to allow accurate monitoring of sleeping infants, including respiratory monitoring and movement tracking.

[0019] In some embodiments, the measurement of the size of the target region may include at least one of: the area of ​​the target region; the size of the target region; and the proportion of the image covered by the target region. Thus, any suitable measurement of the target region may be used as a measurement of its size. Different measurements may be appropriate for different infant monitoring functions.

[0020] In some embodiments, the measurement of the orientation of the target area may include at least one of: longitudinal orientation; transverse orientation; and an angle between an axis of the target area and an axis of the image. The orientation of the target area may be a useful indicator of the horizontal distance between the image sensor and the monitored infant, and therefore may be a useful metric for determining whether the image is suitable for infant monitoring.

[0021] In some embodiments, determining the boundaries of the target region within the image can include processing the image using at least one of: an object detection model; an edge detection model; or a contrast enhancement model. This allows the process of determining the target region in the image to be fully automated. For example, if the monitoring region corresponds to a crib, a machine learning model can be trained to detect the image region depicting the crib's mattress and output a bounding box associated with that detection.

[0022] In some embodiments, receiving an indication of a boundary of a target region within an image includes receiving data provided by a user using a user interface. This allows the user to select an area corresponding to the target region. This can be more flexible and potentially more reliable than automatically selecting the target region.

[0023] In some embodiments, providing an output based on the determined suitability of the image includes providing instructions for adjusting the image sensor when the image is determined to be unsuitable. In this way, the suitability assessment can be used directly to provide recommendations for improving image sensor settings to ensure that future images are of sufficient quality for infant monitoring.

[0024] In some embodiments, when an image is determined to be unsuitable, providing instructions for adjusting the image sensor includes at least one of: providing instructions for adjusting the position of the image sensor; providing instructions for adjusting the orientation of the image sensor; providing instructions for adjusting operating parameters of the image sensor; and providing instructions for adjusting the optical zoom of the image sensor. Thus, various parameters related to the image sensor settings can be adjusted to improve the quality of the captured image and, thereby, its suitability. Different adjustment instructions can be provided depending on the specific configuration of the image sensor.

[0025] In some embodiments, monitoring an infant can include at least one of: monitoring the infant's breathing; monitoring the infant's vital signs; monitoring the infant's movement; and tracking the infant's sleep stages. Image-based monitoring of a sleeping infant can track various behaviors of the infant to gather information about the quantity and quality of the infant's sleep. These different monitoring functions may require different image quality requirements for the monitored area. Therefore, the suitability assessment method of the present invention can be tailored to the specific subsequent processing steps expected to be performed on the image.

[0026] According to another aspect of the present invention, there is provided a computer program comprising code means for implementing any one of the above methods when the program is run on a programming system.

[0027] According to yet another aspect of the present invention, there is provided a processing device comprising a computer program and configured to execute the computer program.

[0028] There is also provided a system for evaluating the suitability of an image of a monitoring area captured by an image sensor for monitoring an infant, the system comprising: a processing device configured to execute the above-mentioned computer program.

[0029] In some embodiments, the infant monitoring system may further include an image sensor configured to capture images of the monitoring area.

[0030] In some embodiments, the infant monitoring system may further include at least one of: a user input interface configured to allow a user to select boundaries of a target area within an image; and an output interface configured to: provide an indication to the user of the suitability of the image for monitoring an infant; and provide instructions to the user for adjusting the image sensor when the image is determined to be unsuitable.

[0031] In some embodiments, the processing device of the infant monitoring system can be configured to provide instructions to automatically adjust the operating parameters of the image sensor. Thus, the system can be equipped with the ability to automatically adjust the settings of the image sensor to ensure that suitable images are captured without user input and thereby reducing instances of user error in the image sensor device.

[0032] The embodiments can be used in conjunction with conventional / existing infant monitoring methods and systems. In this way, the embodiments can be integrated into legacy systems to improve and / or expand their functionality and capabilities. Thus, the proposed embodiments can provide an improved infant monitoring system.

[0033] Thus, there may be a concept of assessing the suitability of an image for monitoring an infant by determining a measure of the size and / or orientation of a target area within the image.These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] For a better understanding of the invention, and to show more clearly how it may be put into practice, reference will now be made, by way of example only, to the accompanying drawings, in which:

[0035] Figure 1A and Figure 1B shows a simplified flow chart of a method for evaluating the suitability of an image of a monitoring area captured by an image sensor for monitoring an infant according to two proposed embodiments;

[0036] Figure 2 shows an image containing a target area to be used for monitoring an infant according to the proposed embodiment;

[0037] Figure 3 shows a baby monitoring system setup according to the proposed embodiment;

[0038] Figure 4 shows a simplified block diagram of an infant monitoring system according to the proposed embodiment; and

[0039] Figure 5 A simplified block diagram illustrating an example of a computer in which one or more portions of the embodiments may be employed. DETAILED DESCRIPTION

[0040] The present invention will be described with reference to the accompanying drawings.

[0041] It should be understood that although the detailed description and specific examples illustrate exemplary embodiments of the devices, systems, and methods, these descriptions and specific examples are for illustrative purposes only and are not intended to limit the scope of the present invention. These and other features, aspects, and advantages of the devices, systems, and methods of the present invention will be better understood from the following description, the appended claims, and the accompanying drawings. It should be understood that the figures are schematic and not drawn to scale. It should also be understood that the same reference numerals are used throughout the figures to indicate the same or similar components.

[0042] Embodiments according to the present disclosure relate to various techniques, methods, schemes, and / or solutions for evaluating the suitability of images for use in monitoring an infant (e.g., for image-based monitoring of an infant's movements or vital signs (e.g., while sleeping), or for image-based tracking of an infant's sleep cycles). Based on the concepts presented, several possible solutions can be implemented individually or in combination. That is, although these possible solutions may be described separately below, two or more of these possible solutions may be implemented in some combination or in another combination.

[0043] Embodiments of the present invention provide a concept for automatically assessing the suitability of an image for monitoring an infant in a monitored area. Specifically, measurements of the size and / or orientation of a target area of ​​an image to be used for infant monitoring are used as indicators of image quality. The suitability of the image is then determined based on these measurements.

[0044] Modern infant monitoring systems are equipped with advanced infant sleep tracking capabilities. These capabilities may include processing image data of the monitoring area using models or algorithms to extract information about the infant's presence in the monitoring area, the infant's posture, the infant's activity level, and the infant's vital signs. For this process to be reliable, the infant and the monitoring area must be imaged with sufficient resolution. The size and / or orientation of the target area of ​​the image data to be used for infant sleep tracking is a useful indicator of the relative position of the image sensor and the infant's sleeping position, and can therefore be used to provide an accurate assessment of the image's suitability for advanced infant monitoring.

[0045] If an image is found to be unsuitable, the proposed system can also provide instructions for adjusting the image sensor to improve the suitability of future images captured by the sensor. Therefore, the proposed method can aid the process of setting up an infant monitoring system for accurate monitoring.

[0046] In summary, the exemplary embodiment includes the following modules:

[0047] A. Target Region Identification: Identify the region of image data to be used for monitoring the infant and determine its boundaries. This can be done automatically or through user input.

[0048] B. Size / Orientation Determination: Determine measurements of the size and / or orientation of the target region. This measurement may include the area or size of the target region, or the proportion of the entire image covered by the target region, and an indication of whether the target region is oriented portrait or landscape.

[0049] C. Suitability Assessment: Using the determined measurements of the size and / or orientation of the target area, a predetermined suitability logic is used to determine whether the captured image is suitable for use.

[0050] D. Output Indication: An indication as to whether the captured image is suitable for use, and any adjustments that may be made to the infant monitoring settings to improve such suitability, may be output to the user or an external device or system.

[0051] Now refer to Figure 1A , which depicts a simplified flow chart of a computer-implemented method 100 for evaluating the suitability of an image for use in monitoring an infant (eg, monitoring the infant's sleep).

[0052] The method will be explained with reference to a preferred embodiment of monitoring an infant during sleep.

[0053] The method begins at step 110 and includes receiving an indication of the boundaries of a target region within an image, where the target region is an area containing image data to be used for monitoring infant sleep. In this exemplary embodiment, step 110 further includes a sub-step 115, which includes receiving data provided by a user using a user interface. For example, the user may be presented with the image through the user interface and asked to select which region of the image corresponds to the target region. In this example, the image depicts an infant's crib, and the user is asked to select a region of the image depicting the crib's mattress area.

[0054] Once the boundaries of the target area are received, the method proceeds to step 120 where a measure of the size of the target area is determined. In the exemplary embodiment, this includes determining the total number of pixels of the image that are within the boundaries of the target area.

[0055] In step 125, a measure of the orientation of the target area is also determined. In this exemplary embodiment, the target area is rectangular, and determining the orientation of the target area includes determining whether the target area is oriented vertically or horizontally. This is achieved by comparing the width of the target area with the height of the target area.

[0056] Once the measurements of the size and orientation of the target area have been determined, the method proceeds to step 130, where the suitability of the image for monitoring the sleep of an infant is determined based on the determined measurements of the size and orientation of the target area. In this exemplary embodiment, step 130 includes a sub-step 135 of comparing the measurements of the size of the target area with a threshold. The determined number of pixels forming the target area is compared with a predetermined threshold known to be suitable for infant sleep monitoring. If the number of pixels is less than the threshold, the image is determined to be unsuitable; if the number of pixels is greater than the threshold, the image is determined to be suitable if the orientation of the image is also considered suitable. In this exemplary embodiment, the predetermined threshold size is between 550,000 pixels and 950,000 pixels. In this exemplary embodiment, the image is considered suitable only if the size of the target area is greater than the predetermined threshold size and the orientation of the target area is landscape.

[0057] If it is determined in step 130 that the image is not suitable, the method proceeds to step 140, where instructions for adjusting the image sensor are provided. In this exemplary embodiment, if the image is determined to be unsuitable for monitoring an infant's sleep, the instructions provided in step 140 include instructions on how to adjust the position of the image sensor to ensure accurate infant monitoring (for example, the instructions may include lowering the height of the image sensor above the bed / crib and moving the image sensor to the long side of the bed / crib). These instructions may be provided to the user via a user interface or to an automatic image sensor adjustment system.

[0058] Specifically, in this exemplary embodiment, if the target area is found to be smaller than a predetermined threshold size, the user is instructed to lower the vertical height of the image sensor so that the image sensor is closer to the position where the infant will lie. If the target area is found to be oriented in a portrait orientation, the user is instructed via the user interface to move the image sensor to the long side of the crib so that the target area is in a landscape orientation. If the target area is larger than the predetermined threshold size and the target area is oriented in a landscape orientation, the user is notified that the image sensor is in the correct position.

[0059] Now refer to Figure 1B , which shows a simplified flow chart of a computer-implemented method 150 for evaluating the suitability of an image for monitoring an infant according to another embodiment. Again, the method will be described with reference to a preferred embodiment of sleep monitoring.

[0060] Method 150 is similar to method 100 , but begins at step 150 , which is different from step 110 of method 100 . Figure 1B The same reference numerals are used to denote Figure 1A The same steps are followed in the previous step and the discussion of these steps will not be repeated.

[0061] Step 150 includes determining the boundaries of a target region within the image, where the target region is the area of ​​the image to be used for monitoring infant sleep. In this exemplary method, step 150 also includes a sub-step 155 of processing the image using an object detection model to identify the boundaries of the target region. For example, a trained machine learning model can be used to detect the mattress region of a crib within an image of an infant nursery. The trained model outputs a bounding box of the detected mattress, which serves as the boundary of the target region.

[0062] Although an object detection model is used in this exemplary embodiment, in other embodiments, different models can be used to determine the target area within the image data. Any suitable model can be used, including but not limited to object detection models, edge detection models, and contrast enhancement models.

[0063] Although the image size measurement in methods 100 and 150 is the number of pixels comprising the target region, other suitable image size measurements may be used in different embodiments. For example, the image size measurement may include at least one of: the area of ​​the target region; the length of the target region; and the proportion of the image covered by the target region. The target region may cover any proportion of the image, up to and including the entire image.

[0064] Similarly, other aspects of methods 100 and 150 can be implemented differently in alternative embodiments. For example, the target area need not be rectangular, and thus the measurement of the orientation of the target area may not simply be longitudinal or transverse. In some embodiments, the measurement of the orientation of the target area may be a value describing the angle between an axis of the target area and an axis of the image.

[0065] Suitability assessment need not rely on both the orientation and size of the target region, but may rely on only one of these values, i.e., in an alternative embodiment, only one of steps 120 and 125 may be included. In some cases, only one of the size and orientation of the target region may be available. For example, if the target region has a particularly irregular shape, the orientation may not be calculated, and therefore only one of these measurements may be used. However, using both the orientation and size of the target region may more accurately determine the suitability of the image.

[0066] In some embodiments, the determination of image suitability may also involve comparing the size of the target area to a maximum threshold, and determining that the image is suitable only if the size of the target area is less than the maximum threshold. The maximum suitable size of the target area may help reduce the processing power required to monitor the infant.

[0067] Methods 100 and 150 produce a binary determination of whether an image is suitable or unsuitable. In alternative embodiments, a more nuanced suitability assessment can be provided. For example, suitability can be categorized as a quantitative or qualitative value that ranks the suitability of the image on a predetermined scale (e.g., a 5-point scale). This can allow for more detailed feedback on how much the image sensor settings need to be adjusted to achieve accurate infant monitoring.

[0068] The predetermined threshold size value may be adjusted depending on the resolution of the image sensor used, the level of background noise in the image data, the actual size of the infant's bed / crib, or the age of the infant being monitored (larger infants may have larger features and movements, and thus image resolution requirements may be less stringent for accurate monitoring of larger infants). Thus, in some embodiments, the suitability assessment of the present invention may rely on additional information provided by the user or by the image sensor.

[0069] In methods 100 and 150, step 140 includes providing instructions to adjust the position of the image sensor used to capture the image. However, in alternative embodiments, the instructions may include at least one of: instructions for adjusting the orientation of the image sensor; instructions for adjusting operating parameters of the image sensor; and instructions for adjusting the optical zoom of the image sensor. In some contemplated embodiments, a more general output related to the suitability of the image for infant monitoring is provided. For example, a value or description of the suitability of the image may be output without providing specific instructions for adjusting parameters of the image sensor. This may facilitate flexible application of the proposed method to different image sensors and infant monitoring system settings that may require different degrees and types of adjustment.

[0070] Now refer to Figure 2 , according to the proposed embodiment, an image 200 containing a target area 230 to be used for monitoring an infant (eg, the infant's sleep) is shown.

[0071] Image 200 depicts an infant 210 in a crib 220. The boundaries of a target area 230 are indicated by a dashed box. This target area corresponds to the image data of the image to be used for monitoring the infant's sleep, in this example, the mattress area of ​​crib 220. The image data of the target area can be intended for sleep monitoring functions, including at least one of: monitoring the infant's breathing; monitoring the infant's movement; monitoring the infant's vital signs; monitoring the infant's presence in the monitoring area; and tracking the infant's sleep stages.

[0072] Although depicted as a rectangular area, in other images, the target area may have different shapes, including but not limited to: multiple rectangular areas; a circle; and an irregular shape. The boundaries of the target area 230 can be determined automatically (e.g., using a trained machine learning model) or based on data received from a user, who can select the boundaries of the target area using a touch-sensitive screen.

[0073] Now refer to Figure 3 , shows an infant monitoring system setup 300 according to the proposed embodiment.

[0074] Image sensor 310 is attached to crib 330 via mounting bracket 320. With this setup, image sensor 310 is configured to obtain a bird's-eye view image of the mattress area of ​​crib 330. For example, image sensor 310 can be used to capture image 200.

[0075] The mounting bracket 320 is configured so that the height of the image sensor above the base of the crib 300 can be adjusted. In addition, the position of the mounting bracket along the side of the crib is also adjustable.

[0076] Once an image is captured using the image sensor 310, the image is processed using any of the computer-implemented methods described above to assess the suitability of the image for use in monitoring an infant's sleep, which relies on the use of measurements of the size and / or orientation of a target area of ​​the image. The size of the target area in the image is a useful measure of the vertical height 360 of the image sensor above the infant's sleeping position, because as the height of the image sensor increases, the size of the target area will decrease. Thus, image analysis can provide (if not directly measure) an indication of the distance between the image sensor and the infant. For example, an object detection model can be used to provide an indication of distance.

[0077] The orientation of the target area can be a useful measure of the horizontal distance 350 between the image sensor and the sleeping position of the crib (the infant is more likely to be close to the image sensor if the image sensor is positioned along the long edge of the crib than if the image sensor is positioned along the short edge of the crib, which may be the opposite end of the crib from the sleeping infant). Using these two measurements (size and orientation) in conjunction with each other allows information to be gathered about the straight-line distance 370 between the image sensor and the infant.

[0078] All three distances 350, 360, and 370 affect the quality of the image captured by image sensor 310. In order to accurately monitor an infant's features and movements using image sensor 310, the image sensor must be positioned close enough to the infant to image the infant with sufficient resolution.

[0079] Once the image suitability has been assessed by any of the methods proposed herein, if the image is determined to be unsuitable, instructions are provided to the user via a separate user interface device (not shown) to adjust the image sensor 310. Possible adjustments include adjusting the height of the image sensor above the crib, adjusting the orientation of the image sensor 310, or adjusting the position of the image sensor along the side of the crib. If the image sensor 310 is equipped with an optical zoom function, the zoom of the image sensor can also be adjusted to ensure that the infant is depicted in a sufficiently large proportion of the field of view.

[0080] In image monitoring setup 300 , adjustments to image sensor 310 are manually made by the user. However, in other proposed embodiments, image sensor 310 and mounting bracket 320 may be adjustable to allow for automatic adjustment of the image sensor's position and orientation. This may involve continuously and automatically adjusting the image sensor's position and orientation during an infant's sleep to allow for dynamic monitoring of the infant. This can ensure that the images captured by the image sensor are appropriate even if the infant changes position.

[0081] Now refer to Figure 4 , shows a simplified block diagram of an infant monitoring system 400 for evaluating the suitability of an image of a monitoring area for monitoring an infant (eg, for monitoring the infant's sleep).

[0082] The infant monitoring system 400 includes an image sensor 410 , an input interface 420 , an output interface 430 , and an image processing unit 440 .

[0083] The system 400 is configured to assess the suitability of images of a monitoring area captured by the image sensor 410 for use in monitoring the sleep of an infant. This is achieved by processing the images from the image sensor 410 in an image processing unit. This can be implemented using the method 100.

[0084] Specifically, image sensor 410 generates image data of a monitoring area. Input interface 420 allows a user to select a target area of ​​the image data, which corresponds to the area of ​​the image to be used for monitoring the infant. Processing device 440 receives the image from image sensor 410 and an indication of the boundaries of the target area from input interface 420.

[0085] Image processing unit 440 is configured to execute a computer program comprising code means for implementing method 100. Thus, upon receiving an image from the image sensor and an indication of the boundaries of the image from input interface 420, the processor uses a computer-implemented suitability assessment method to determine the size of the target region and, thereby, the suitability of the image for monitoring an infant's sleep. The image processing unit is further configured to output a signal describing the suitability of the image to output interface 430 and, if the image is determined to be unsuitable, to output instructions to a user for adjusting the image sensor to improve the suitability of future images.

[0086] The output interface 430 outputs the suitability assessment and any instructions for adjusting the image sensor to the user. This may be achieved through electronic output signals and / or visual / audible outputs.

[0087] Although in system 400, image sensor 410, input interface 420, and output interface 430 are present in the same device, it is understood that these components can be distributed across various devices. For example, a baby monitor can be provided that includes only an image sensor. Image data captured by the image monitor's image sensor can then be forwarded to a parent / caregiver's personal smart device. The processing and interface capabilities of the personal smart device can implement the functionality of input interface 420, image processing unit 440, and output interface 430.

[0088] In an alternative embodiment, the image monitoring system may include more than one image sensor. Each image sensor may have different settings (e.g., different lens settings, sensors with different resolutions, or different locations). In this case, the instructions for adjusting the image sensors may include instructions for changing which image sensor in the image monitoring system is used to monitor infant sleep. This embodiment may allow for a greater variety of possible image settings and, therefore, may increase the likelihood of capturing suitable images of the infant.

[0089] Figure 5 An example of a computer 500 is shown in which one or more portions of the embodiments may be employed. The various operations discussed above may utilize the functionality of the computer 500. In this regard, it should be understood that the system functional blocks may be executed on a single computer or distributed across multiple computers and locations (e.g., connected via the Internet).

[0090] Computer 500 includes, but is not limited to, a PC, a workstation, a laptop, a PDA, a handheld device, a server, a memory, and the like. Typically, in terms of hardware architecture, computer 500 may include one or more processors 510, a memory 520, and one or more I / O devices 530, which are communicatively coupled via a local interface (not shown). The local interface may be, for example, but not limited to, one or more buses or other wired or wireless connections, as known in the art. The local interface may have additional components, such as a controller, a buffer (cache), a driver, a repeater, and a receiver, to enable communication. In addition, the local interface may include address, control, and / or data connections to enable appropriate communication between the above components.

[0091] Processor 510 is a hardware device for executing software that may be stored in memory 520. Processor 510 may be virtually any custom or commercially available processor, central processing unit (CPU), digital signal processor (DSP), or auxiliary processor among several processors associated with computer 500, and may be a semiconductor-based microprocessor (in the form of a microchip) or a microprocessor.

[0092] The memory 520 may include any one or a combination of the following: volatile memory elements (e.g., random access memory (RAM), such as dynamic random access memory (DRAM), static random access memory (SRAM), etc.) and non-volatile memory elements (e.g., ROM, erasable programmable read-only memory (EPROM), electronically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic tape, compact disk read-only memory (CD-ROM), magnetic disk, floppy disk, cassette, cartridge, etc.). In addition, the memory 520 may incorporate electronic, magnetic, optical, and / or other types of storage media. Note that the memory 520 may have a distributed architecture in which various components are remote from each other but can be accessed by the processor 510.

[0093] The software in the memory 520 may include one or more separate programs, each of which includes an ordered list of executable instructions for implementing logical functions. The software in the memory 520 includes a suitable operating system (O / S) 550, a compiler 560, source code 570, and one or more application programs 580 according to exemplary embodiments. As shown, the application programs 580 include numerous functional components for implementing the features and operations of the exemplary embodiments. The application programs 580 of the computer 500 may represent various applications, computing units, logic, functional units, processes, operations, virtual entities, and / or modules according to exemplary embodiments, but the application programs 580 are not intended to be limiting.

[0094] The operating system 550 controls the execution of other computer programs and provides scheduling, input and output control, file and data management, memory management, communication control and related services. The inventors contemplate that the application program 580 used to implement the exemplary embodiments can be adapted to all commercially available operating systems.

[0095] Application 580 can be a source program, an executable program (object code), a script, or any other entity that includes a set of instructions to be executed. If it is a source program, the program is typically translated by a compiler (such as compiler 560), an assembler, an interpreter, etc., which may or may not be included in memory 520 in order to run correctly with O / S 550. In addition, application 580 can be written in an object-oriented programming language that has classes of data and methods, or in a procedural programming language that has routines, subroutines, and / or functions, such as, but not limited to, C, C++, C#, Pascal, Python, BASIC, API calls, HTML, XHTML, XML, ASP scripts, JavaScript, FORTRAN, COBOL, Perl, Java, ADA, .NET, etc.

[0096] I / O devices 530 may include input devices such as, but not limited to, a mouse, keyboard, scanner, microphone, camera, etc. Furthermore, I / O devices 530 may also include output devices such as, but not limited to, a printer, a display, etc. Finally, I / O devices 530 may also include devices that perform both input and output communications, such as, but not limited to, a NIC or modulator / demodulator (for accessing remote devices, other files, devices, systems, or networks), a radio frequency (RF) or other transceiver, a telephone interface, a bridge, a router, etc. I / O devices 530 also include components for communicating over various networks, such as the Internet or an intranet.

[0097] If computer 500 is a PC, workstation, smart device, or the like, the software in memory 520 may also include a basic input / output system (BIOS) (omitted for simplicity). The BIOS is a set of basic software routines that initialize and test the hardware at startup, start the operating system 550, and support data transfer between hardware devices. The BIOS is stored in some type of read-only memory, such as ROM, PROM, EPROM, EEPROM, etc., so that it can be executed when computer 500 starts up.

[0098] When computer 500 is running, processor 510 is configured to execute software stored in memory 520, transfer data to and from memory 520, and generally control the operation of computer 500 according to the software. Applications 580 and O / S 550 are read in whole or in part by processor 510, possibly buffered within processor 510, and then executed.

[0099] When application 580 is implemented in software, it should be noted that application 580 can be stored on virtually any computer-readable medium for use by or in conjunction with any computer-related system or method. In the context of this document, a computer-readable medium can be an electronic, magnetic, optical, or other physical device or means that can contain or store a computer program for use by or in conjunction with a computer-related system or method.

[0100] Application 580 can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, a system containing a processor, or other system that can retrieve instructions from and execute the instructions of the instruction execution system, apparatus, or device. In the context of this document, a "computer-readable medium" can be any means that can store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium.

[0101] Figure 1A and Figure 1B Methods and Figure 5 The system can be implemented in hardware or software or a hybrid of the two (for example, as firmware running on a hardware device). If the embodiment is implemented in part or in whole in software, the functional steps shown in the flowchart can be performed by a suitably programmed physical computing device, such as one or more central processing units (CPUs) or graphics processing units (GPUs). Each process (and its individual component steps shown in the flowchart) can be performed by the same or different computing devices. According to an embodiment, a computer-readable storage medium stores a computer program, which includes computer program code that is configured to cause one or more physical computing devices to perform the method as described above when the program is run on one or more physical computing devices.

[0102] Storage media can include volatile and nonvolatile computer memory, such as RAM, PROM, EPROM, and EEPROM, optical disks (e.g., CDs, DVDs, and BDs), and magnetic storage media (e.g., hard disks and magnetic tapes). Various storage media can be fixed within a computing device or removable so that one or more programs stored thereon can be loaded into a processor.

[0103] If the embodiment is partially or completely implemented in hardware, then Figure 5 The blocks shown in the block diagrams may be separate physical components, or logical subdivisions of a single physical component, or may all be implemented in an integrated manner in one physical component. The functionality of a block shown in the diagram may be divided among multiple components in implementation, or the functionality of multiple blocks shown in the diagram may be combined in a single component in implementation. Hardware components suitable for embodiments of the present invention include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs). One or more blocks may be implemented as a combination of dedicated hardware to perform certain functions, and one or more programmed microprocessors and associated circuits to perform other functions.

[0104] Variations on the disclosed embodiments may be understood and implemented by those skilled in the art in practicing the claimed invention, by studying the drawings, the disclosure and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may perform the functions of several items recited in the claims. The fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. If a computer program is discussed above, it may be stored / distributed on a suitable medium, such as an optical storage medium or solid-state medium provided with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems. If the term "suitable for" is used in the claims or in the description, it should be noted that the term "suitable for" is intended to be equivalent to a vehicle "configured to". Any reference signs in the claims should not be construed as limiting the scope.

[0105] The flowcharts and block diagrams in the figures illustrate the architecture, functions, and operations that may be implemented according to various embodiments of the present invention, the system, method, and computer program product. In this regard, each block in the flowchart or block diagram may represent a portion of a module, segment, or instruction that includes one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions indicated in the blocks may be different from the order indicated in the figures. For example, two blocks shown in succession may actually be executed substantially simultaneously, or blocks may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart and the combination of blocks in the block diagram and / or flowchart may be implemented by a dedicated hardware system that performs the specified function or action or performs a combination of dedicated hardware and computer instructions, according to the architecture, functions, and operations that may be implemented according to various embodiments of the present invention, the system, method, and computer program product.

Claims

1. A computer-implemented method for evaluating the suitability of an image of a monitoring area captured by an image sensor for use in monitoring an infant, the method comprising: (150) determining boundaries of a target region within the image, or (110) receiving an indication of boundaries of a target region within the image, wherein the target region is a region containing image data to be used for monitoring the infant; (120, 125) determining a measurement of at least one of: a size of the target region of the image; and orientation; (130) determining suitability of the image for use in monitoring the infant based on the determined measurements of the size and / or the orientation of the target area; as well as (140) Providing an output based on the determined suitability of the image. 2 . The method of claim 1 , wherein the boundary is a rectangle, the method comprising determining measurements of a size and an orientation of the target area of ​​the image, and the orientation is a portrait orientation / landscape orientation.

3. The method of claim 2, wherein the suitable image comprises an image target area having a sufficient number of pixels and a landscape orientation.

4. The method of any one of claims 1 to 3, wherein determining the suitability of the image for monitoring the infant further comprises: determining that the image is suitable if the measurement of the size of the target area is greater than a predetermined threshold size; and If the measure of the size of the target area is less than the predetermined threshold size, then the image is determined to be unsuitable. 5 . The method of claim 4 , wherein the measure of the size of the target region comprises a pixel area of ​​the target region, and the predetermined threshold size is between 550,000 pixels and 950,000 pixels.

6. A method according to any one of the preceding claims, wherein the measure of the size of the target region comprises at least one of: the area of ​​the target region; the size of the target region; and the proportion of the image covered by the target area.

7. A method according to any one of the preceding claims, comprising: determining boundaries of a target region within the image by processing the image using at least one of: an object detection model; Edge detection model; and contrast enhancement mode; or An indication of a boundary of a target area within the image is received by receiving data provided by a user using a user interface.

8. The method of any preceding claim, wherein providing an output regarding the suitability of the image comprises: When the image is determined to be unsuitable, instructions for adjusting the image sensor are provided.

9. The method of claim 8, wherein providing instructions for adjusting the image sensor when determining that the image is unsuitable comprises at least one of: providing instructions for adjusting the position of the image sensor; providing instructions for adjusting the orientation of the image sensor; providing instructions for adjusting operating parameters of the image sensor; providing instructions for adjusting the optical zoom of the image sensor; and Provides instructions for using additional image sensors.

10. The method of any preceding claim, wherein monitoring the infant comprises at least one of: monitoring the infant's breathing; monitoring the infant's vital signs; monitoring the infant's movements; monitoring the position of the infant; and The infant's sleep stages are tracked.

11. A computer program comprising code means for implementing the method according to any one of the preceding claims when said program is run on a processing system.

12. A processing device (540) comprising a computer program according to claim 11 and configured to execute said computer program.

13. A system for evaluating the suitability of an image of an infant in a monitoring area captured by an image sensor for monitoring the infant's breathing, the system comprising: Processing means (240) configured to run a computer program according to claim 11.

14. An infant monitoring system comprising: The system according to claim 13; as well as An image sensor (310, 510) is configured to capture the image of the infant in the monitoring area, and optionally wherein the processing device is further configured to automatically adjust the image sensor.

15. The system according to any one of claims 13 and 14, further comprising at least one of the following: a user input interface configured to allow the user to select the boundaries of the target area within the image; and An output interface, wherein the output interface is configured as: providing an indication to the user of the suitability of the image for monitoring the infant's breathing; and When it is determined that the image is unsuitable, instructions for adjusting the image sensor are provided to the user.

Images