A machine vision-based infant sleep kicking-off quilt intelligent recognition and prompting method and system

Abstract

The present application is suitable for the field of machine vision intelligent identification, and provides a kind of infant sleep kicking quilt intelligent identification and prompt method and system based on machine vision, method includes: using image acquisition device to collect infant sleep image;Using the instance segmentation model trained to the image collected carries out instance segmentation, and the body parts such as the trunk, arm, leg and head of infant exposed outside quilt are segmented as independent instance, and binary mask data is generated;The exposed area proportion of each part is counted respectively, and different alarm weights are given to different parts to carry out weighted summation calculation comprehensive alarm value;When alarm value exceeds the preset threshold corresponding to temperature, output alarm prompt in combination with indoor temperature sensor real-time data.The present application carries out differentiation evaluation to each part of infant body, and dynamically adjusts alarm threshold in combination with environmental temperature, improves the accuracy and practicability of kicking quilt identification.

Description

Technical Field

[0001] This invention relates to the field of machine vision intelligent recognition, specifically a machine vision-based intelligent recognition and prompting method and system for infants kicking off their blankets during sleep. Background Technology

[0002] Infants and toddlers sleep deeply, their ability to perceive temperature changes is less than that of adults, and their self-care abilities are not yet fully developed, making it common for them to kick off their blankets. At night, because caregivers are also asleep, it is difficult to detect these incidents promptly. Currently, there are three main approaches to addressing the problem of infants kicking off their blankets: First, improve the quilt itself by designing it as a wearable structure, so that infants and young children can wear the quilt like clothes to prevent them from kicking it off mechanically. Although this method alleviates the problem of infants and young children kicking off the quilt to some extent, it restricts their movement and is not suitable for designing a thick quilt, resulting in poor applicability and comfort. Secondly, relevant sensors are deployed on the bed and blankets. When an infant kicks off the blankets, the sensors detect the blanket's state and determine whether kicking has occurred. If kicking occurs, a reminder is issued through relevant technical means. This method uses sensors for proactive sensing and prompting, improving the informatization level of infant care. However, this solution requires deploying sensors on the bed, on the infant, or on the blankets, increasing the deployment difficulty. Furthermore, due to limitations in manufacturing technology, the sensors cannot be seamlessly integrated into the sleep environment, affecting the infant's sleep comfort. Third, visual and infrared sensors are deployed on walls or ceilings, using the blanket coverage area as a criterion. Raw data is collected via image sensors, and a pre-set algorithm or trained model is used to determine if the infant's blanket coverage area is below a certain threshold, thus indicating whether a blanket-kicking incident has occurred. This technology deploys sensors outside the bed, largely replicating the human logic for judging whether an infant has kicked off the blanket, simplifying the bed structure and sensor deployment, and offering significant technological advantages. However, judging blanket-kicking solely based on the infant's body coverage area ignores the infant's heat dissipation needs during sleep and fails to consider the underlying logic that blanket-kicking leads to colds. The main reason infants catch colds from blanket-kicking is that their torso is not covered, causing them to get cold. Existing methods do not differentiate between different parts of the infant's body (such as torso, arms, and legs), treating them all equally, which contradicts common sense and is prone to misjudgment. Therefore, to address these shortcomings, there is an urgent need for a machine vision-based intelligent recognition and prompting method and system for infant blanket-kicking during sleep to overcome the deficiencies in current practical applications. Summary of the Invention

[0003] The purpose of this invention is to provide a machine vision-based intelligent recognition and prompting method and system for infants kicking off their blankets during sleep, effectively solving the problems mentioned in the background art.

[0004] This invention is implemented as follows: a machine vision-based intelligent recognition and prompting method for infants kicking off their blankets during sleep, comprising: Images of infants and young children during sleep are captured using image acquisition equipment; The trained instance segmentation model is used to segment the acquired images, and the body parts of the infant exposed outside the blanket, including at least the torso, arms and legs, are segmented as independent instances, and binary mask data corresponding to each instance is generated. Based on the binary mask data corresponding to each instance, calculate the proportion of exposed area for each body part; Different alarm weights are assigned to the proportion of exposed area of ​​each body part, and the weighted sum is calculated to obtain the alarm value for uncovered body. Collect real-time temperature values ​​of the environment in which infants and young children are located; A preset threshold is determined based on the real-time temperature value. The uncovered alarm value is compared with the preset threshold. When the uncovered alarm value is greater than or equal to the preset threshold, an alarm signal is output.

[0005] As a further aspect of the present invention: The binary mask data is a three-dimensional binary mask tensor (N, H, W), where N represents the number of target instances detected in the current image, and H and W represent the pixel height and width corresponding to the mask, respectively. Each target instance corresponds to a two-dimensional binary mask, with pixels belonging to the instance region having a value of 1 and pixels not belonging to the instance region having a value of 0.

[0006] As a further aspect of the present invention: the calculation of the proportion of exposed area of ​​each body part specifically includes: Sum all mask values ​​for each instance: Where W is the number of pixels in the width direction of the image, and H is the number of pixels in the height direction of the image. This is the mask value for the corresponding pixel position; Calculate the percentage of exposed area for each body part: ,in This is the sum of the instance two-dimensional mask values ​​calculated in the previous step. The sum of the two-dimensional mask values ​​for the corresponding instances of images captured of infants without any blankets; The Obtain it through the following methods: With the infant lying face up and without any blanket, the acquired image is segmented using the instance segmentation model, and the sum of the two-dimensional mask values ​​of each instance is stored as a calibration reference value.

[0007] As a further aspect of the present invention: the alarm weights of the torso, arms, and legs. The values ​​are 0.65, 0.10, and 0.25 respectively, with a head alarm weight of 0; the calculation formula for the uncovered alarm value is: ,in For the first Alarm weight for each instance, For the first The proportion of exposed area in each instance This represents the number of instances.

[0008] As a further aspect of the present invention: the step of determining the corresponding preset threshold based on the real-time temperature value specifically involves: when At ℃, the preset threshold is 80; when At ℃, the preset threshold is 50; when At ℃, the preset threshold is 30; when At ℃, the preset threshold is 15; in, This represents the real-time ambient temperature.

[0009] As a further aspect of the present invention: the instance segmentation model is based on the YOLO architecture instance segmentation network and is obtained through the following steps: Collect sample images of infants and young children in different sleeping positions; The body regions of infants and young children in the sample images are finely annotated, and the infants and young children's bodies are divided into head, torso, arms and legs, generating annotated images and label files; The instance segmentation network is trained iteratively through multiple rounds using the labeled dataset to generate a dedicated instance segmentation model file.

[0010] This invention also provides a machine vision-based intelligent recognition and prompting system for infants kicking off their blankets during sleep, which executes the method described above, including: The image acquisition module is used to acquire images of infants and young children while they are sleeping. The instance segmentation module is used to segment the acquired images using a trained instance segmentation model, segmenting the body parts of infants exposed outside the blanket, including at least the torso, arms, and legs, as independent instances, and generating binary mask data corresponding to each instance. The area ratio calculation module is used to calculate the exposed area ratio of each body part based on the binary mask data corresponding to each instance. The alarm value calculation module is used to assign different alarm weights to the proportion of exposed area of ​​each body part and then perform weighted summation to calculate the alarm value for uncovered body parts. The temperature acquisition module is used to collect the real-time temperature value of the environment in which the infants and young children are located; The judgment and alarm module is used to determine the corresponding preset threshold based on the real-time temperature value, compare the uncovered alarm value with the preset threshold, and output an alarm prompt signal when the uncovered alarm value is greater than or equal to the preset threshold.

[0011] The present invention also provides a computer device, characterized in that it includes a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the method described above are performed.

[0012] The present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the steps of the method described above.

[0013] The present invention also provides a computer program product, including a computer program / instructions, characterized in that the computer program / instructions, when executed by a processor, implement the steps of the method described above.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: This method uses instance segmentation technology to finely divide an infant's body into independent parts such as the torso, arms, and legs, and separately calculates the proportion of exposed area for each part. This overcomes the shortcomings of traditional methods that treat all body parts equally, and can more accurately reflect the infant's actual state of kicking off the blankets. Different alarm weights are assigned to different body parts, with the highest weight given to the torso. This makes alarm judgments more consistent with the medical common sense that a cold torso is the main cause of colds, reducing the false alarm rate. An ambient temperature sensor is introduced to dynamically adjust the alarm threshold according to the indoor temperature. In high-temperature environments, a higher degree of exposure is allowed without triggering an alarm, while in low-temperature environments, an alarm is triggered when the degree of exposure is low. This avoids excessive alarms in high temperatures and ensures timely alerts in low temperatures. The image acquisition device is deployed outside the bed, eliminating the need to deploy any sensors on the bed, on the infant, or on the blankets, thus not affecting the infant's sleep comfort and simplifying deployment. Attached Figure Description

[0015] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0016] Figure 1 This is a flowchart of the method of the present invention.

[0017] Figure 2 This is a schematic diagram of the intelligent recognition result using the method of the present invention.

[0018] Figure 3 This is a schematic diagram of the (N,H,W) binary mask data generated after the instance segmentation technique used in this invention performs instance segmentation on an ellipse. Detailed Implementation

[0019] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] The present invention will be further explained below with reference to specific embodiments.

[0021] Please see Figures 1-3 The present invention provides a machine vision-based intelligent recognition and prompting method for infants kicking off their blankets during sleep, comprising the following steps: 101. Collect images and videos of infants and young children in real sleep states, and extract frames from the video data: Collect image data of infants and young children in sleep states through online collection, photo shooting, video recording, etc., and extract frames from different sleeping positions of infants and young children in the video to obtain sample images of infants and young children in different sleeping positions. Collect 2,000 images of infants and young children in real sleep states in this way. 102. Perform data annotation on the image data and generate data annotation files and corresponding label files: Use data annotation software to annotate the head, torso, arms and legs of infants exposed outside the blanket in the sleeping state of the image data collected in step 101, and generate annotation data files and corresponding label files. Store the generated labeled data files and corresponding label files in the folders corresponding to the training dataset and validation dataset, respectively, in an 8:2 ratio. Write a YAML configuration file to configure the number and names of instances to be identified in the infant kicking the blanket detection scenario; 103. Train the data using the YOLO architecture to generate a dedicated model: Use the YOLO26n-seg model under the YOLO architecture to perform multiple rounds of iterative training on the data in step 102, and use the generated best.pt file as a dedicated instance segmentation model file in the infant kicking the blanket recognition scenario; 104. Convert the model file format and deploy the model to the edge computing device platform: Use software tools to convert the model file generated in step 103 to the TensorRT format used by the NVIDIA CUDA computing platform; and deploy the model file on the NVIDIA edge computing device Jetson Nano.

[0022] 105. Develop software programs on an edge computing platform to collect and analyze data, and calculate the exposure value of infants and young children: Connect the temperature sensor, Bluetooth module, camera, LED indicator, buttons, and corresponding interfaces of the Jetson Nano device, and develop software programs with the following functions: 1) The software program can switch between calibration mode and intelligent recognition mode based on the external button status; 2) When the program enters calibration mode, the user should have the infant lie flat on their back on the bed, allowing the camera to capture a complete image of the infant's head, torso, arms, and legs. This image will then be processed by a 104-step deployment model to segment the infant's head, torso, arms, and legs into instances, and the sum of the two-dimensional mask values ​​for each instance will be calculated. These values ​​are stored separately in the Jetsonnano file system. They are proportional to the size of the infant's body area and serve as an important reference parameter for subsequent assessment of the infant's nakedness.

[0023] 3) When the program enters the intelligent recognition mode, in order to adapt to the infant sleep scenario and reduce energy consumption, the software collects camera data every 1 minute and uses a 104-step deployment model to segment the infant's head, torso, arms, legs and feet. The segmentation results include binary mask data represented by a three-dimensional binary mask tensor (N, H, W) (where N represents the number of target instances detected in the current single frame image, H and W represent the pixel height and width corresponding to the mask, respectively, each target instance corresponds to a two-dimensional binary mask, the pixel value of the instance area is 1, and the pixel value of the non-instance area is 0) and two one-dimensional vectors of length N, one of which is used to store the category identifier ID corresponding to each mask, and the other is used to store the instance box / category / confidence.

[0024] 4) If N is 0, or the confidence level of all instances is less than 0.5, it means there are no infants in the bed and the case will not proceed to the next step. 5) If N is greater than 0, and the number of instances with a confidence level greater than 0.5 is also greater than 0, then it means there is an infant in the bed, and proceed to the next step: For each instance with a confidence level greater than 0.5, sum all mask values: ; in, W This represents the number of pixels along the width of the image. H This represents the number of pixels in the image's height direction. This is the mask value for the corresponding pixel position; Next, calculate the proportion of naked body parts: ; The head, torso, arms, and legs were assigned computational weights of 0, 0.65, 0.10, and 0.25, respectively. According to the formula: Calculate the alarm value for uncovered blankets.

[0025] 106. Combining the exposed temperature value and the indoor temperature data collected by the temperature sensor, a comprehensive judgment is made and an alarm signal is output to the relevant equipment: The temperature sensor value is collected. In conjunction with the alarm value for uncovered clothing, a notification message is output to relevant devices (such as smartwatches, smartphones, and tablets) to alert caregivers. when ℃ Output alarm prompts in real time; when ℃ Output alarm prompts in real time; when ℃ Output alarm prompts in real time; when ℃ Output alarm prompts in real time; in, This represents the real-time ambient temperature.

[0026] In this embodiment of the invention, multiple video and image data of infants and young children in a sleep environment are collected, and frames of different sleeping postures of infants and young children in the videos are extracted to obtain sample images of infants and young children in different sleeping postures. Annotation tools are used to perform fine annotation of the infants and young children's body regions in the sample images, dividing the infants and young children's bodies into multiple parts such as head, torso, arms, legs and feet, generating corresponding annotated images and label files. Using an instance segmentation network based on the YOLO architecture, the above-mentioned annotated dataset is trained in multiple rounds to obtain a dedicated instance segmentation model suitable for the infant kicking the blanket recognition scenario. This model can perform instance segmentation on various body parts of infants in input images or video frames and output standardized segmentation results. The output data includes binary mask data represented by a three-dimensional binary mask tensor (N, H, W), where N represents the number of target instances detected in the current single frame image, and H and W represent the pixel height and width of the mask, respectively. Each target instance corresponds to a two-dimensional binary mask, with pixels belonging to the instance region having a value of 1 and pixels not belonging to the region having a value of 0. The model also outputs a one-dimensional vector of length N to store the category identifier ID corresponding to each mask. This ID can be used to identify the instance type represented by each two-dimensional mask matrix in the mask tensor. First, the confidence level of each instance is determined; if it exceeds a certain threshold, the intelligent recognition result is trusted. Then, the sum of all mask values ​​for each instance that meets the requirements is calculated. ,in W This represents the number of pixels along the width of the image. H This represents the number of pixels in the image's height direction. This represents the mask value corresponding to the pixel location; the calculation result, to a certain extent, represents the uncovered area of ​​different parts of the infant's body. This result is then used to calculate the uncovered proportion of each part of the body. ,in This is the sum of the instance two-dimensional mask values ​​calculated in the previous step. The sum of the two-dimensional mask values ​​for each instance of an image captured of an infant without any blanket; a certain alarm weight for "uncovered" is assigned to each instance. The uncovered alarm value is obtained by weighted summation of all segmented instances: The corresponding preset threshold is determined based on the real-time temperature value. When the alarm value is greater than or equal to the preset threshold, an alarm prompt signal is output. This invention segments the body parts of sleeping infants into instances, calculates the exposure rate of each part, and then performs a weighted calculation to comprehensively assess whether the infant has kicked off the covers. This assessment is combined with real-time data from indoor temperature sensors to provide alerts. This method more accurately reflects the true extent of an infant's blanket-kicking behavior, thus providing more precise alerts.

[0027] This invention also provides a machine vision-based intelligent recognition and prompting system for infants kicking off their blankets during sleep, which executes the method described above, including: The image acquisition module is used to acquire images of infants and young children while they are sleeping. The instance segmentation module is used to segment the acquired images using a trained instance segmentation model, segmenting the body parts of infants exposed outside the blanket, including at least the torso, arms, and legs, as independent instances, and generating binary mask data corresponding to each instance. The area ratio calculation module is used to calculate the exposed area ratio of each body part based on the binary mask data corresponding to each instance. The alarm value calculation module is used to assign different alarm weights to the proportion of exposed area of ​​each body part and then perform weighted summation to calculate the alarm value for uncovered body parts. The temperature acquisition module is used to collect the real-time temperature value of the environment in which the infants and young children are located; The judgment and alarm module is used to determine the corresponding preset threshold based on the real-time temperature value, compare the uncovered alarm value with the preset threshold, and output an alarm prompt signal when the uncovered alarm value is greater than or equal to the preset threshold.

[0028] This invention also provides a computer device, characterized in that it includes a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the method described above are performed.

[0029] This invention also provides a computer-readable storage medium storing a computer program thereon, characterized in that the program, when executed by a processor, implements the steps of the method described above.

[0030] This invention also provides a computer program product, including a computer program / instructions, characterized in that the computer program / instructions, when executed by a processor, implement the steps of the method described above.

[0031] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A machine vision-based intelligent recognition and prompting method for infants kicking off their blankets during sleep, characterized in that, include: Images of infants and young children during sleep are captured using image acquisition equipment; The trained instance segmentation model is used to segment the acquired images, and the body parts of the infant exposed outside the blanket, including at least the torso, arms and legs, are segmented as independent instances, and binary mask data corresponding to each instance is generated. Based on the binary mask data corresponding to each instance, calculate the proportion of exposed area for each body part; Different alarm weights are assigned to the proportion of exposed area of ​​each body part, and the weighted sum is calculated to obtain the alarm value for uncovered body. Collect real-time temperature values ​​of the environment in which infants and young children are located; A preset threshold is determined based on the real-time temperature value. The uncovered alarm value is compared with the preset threshold. When the uncovered alarm value is greater than or equal to the preset threshold, an alarm signal is output.

2. The method according to claim 1, characterized in that, The binary mask data is a three-dimensional binary mask tensor (N, H, W), where N represents the number of target instances detected in the current image, and H and W represent the pixel height and width corresponding to the mask, respectively. Each target instance corresponds to a two-dimensional binary mask, with pixels belonging to the instance region having a value of 1 and pixels not belonging to the instance region having a value of 0.

3. The method according to claim 1, characterized in that, The calculation of the proportion of exposed area for each body part specifically includes: Sum all mask values ​​for each instance: Where W is the number of pixels in the width direction of the image, and H is the number of pixels in the height direction of the image. This is the mask value for the corresponding pixel position; Calculate the percentage of exposed area for each body part: ,in This is the sum of the instance two-dimensional mask values ​​calculated in the previous step. The sum of the two-dimensional mask values ​​for the corresponding instances of images captured of infants without any blankets; The Obtain it through the following methods: With the infant lying face up and without any blanket, the acquired image is segmented using the instance segmentation model, and the sum of the two-dimensional mask values ​​of each instance is stored as a calibration reference value.

4. The method according to claim 1, characterized in that, The alarm weights for the torso, arms, and legs The values ​​are 0.65, 0.10, and 0.25 respectively, with a head alarm weight of 0; the calculation formula for the uncovered alarm value is: ,in For the first Alarm weight for each instance, For the first The proportion of exposed area in each instance This represents the number of instances.

5. The method according to claim 4, characterized in that, The specific steps for determining the corresponding preset threshold based on the real-time temperature value are as follows: when At ℃, the preset threshold is 80; when At ℃, the preset threshold is 50; when At ℃, the preset threshold is 30; when At ℃, the preset threshold is 15; in, This represents the real-time ambient temperature.

6. The method according to claim 1, characterized in that, The instance segmentation model is based on the YOLO architecture instance segmentation network and is trained through the following steps: Collect sample images of infants and young children in different sleeping positions; The body regions of infants and young children in the sample images are finely annotated, and the infants and young children's bodies are divided into head, torso, arms and legs, generating annotated images and label files; The instance segmentation network is trained iteratively through multiple rounds using the labeled dataset to generate a dedicated instance segmentation model file.

7. A machine vision-based intelligent recognition and prompting system for infants kicking off their blankets during sleep, comprising performing the method as described in any one of claims 1 to 6, including: The image acquisition module is used to acquire images of infants and young children while they are sleeping. The instance segmentation module is used to segment the acquired images using a trained instance segmentation model, segmenting the body parts of infants exposed outside the blanket, including at least the torso, arms, and legs, as independent instances, and generating binary mask data corresponding to each instance. The area ratio calculation module is used to calculate the exposed area ratio of each body part based on the binary mask data corresponding to each instance. The alarm value calculation module is used to assign different alarm weights to the proportion of exposed area of ​​each body part and then perform weighted summation to calculate the alarm value for uncovered body parts. The temperature acquisition module is used to collect the real-time temperature value of the environment in which the infants and young children are located; The judgment and alarm module is used to determine the corresponding preset threshold based on the real-time temperature value, compare the uncovered alarm value with the preset threshold, and output an alarm prompt signal when the uncovered alarm value is greater than or equal to the preset threshold.

8. A computer device, characterized in that, It includes a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, it performs the steps of the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1 to 6.

Images