Infant turning-over detection method and device based on thermal infrared image and robot

Abstract

The invention relates to the technical field of image processing, solves the problem that an existing image processing method is low in accuracy in infant turning-over action detection, and provides an infant turning-over detection method and device based on a thermal infrared image and a robot. The method comprises the following steps: acquiring thermal infrared video data and visible light video data in an infant nursing scene; infant target detection is carried out on the visible light video data, and whether an infant exists in a scene or not is judged; when it is judged that the infant exists in the scene, obtaining an infant thermal infrared image according to the thermal infrared video data; carrying out trunk positioning on the infant thermal infrared image to obtain trunk position information; and according to the trunk position information, performing turning-over detection on the trunk area to obtain a turning-over detection result. According to the invention, the turning-over action of the infant can be monitored in real time, and instant turning-over alarm reminding is provided for guardians.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to a baby turning-over detection method and device based on a thermal infrared image and a robot. BACKGROUND

[0002] When a baby turns over in sleep, it may fall or get stuck due to instability in sleep. Monitoring can timely detect the activity state of the baby, and parents can take prompt action to ensure the safety of the baby if an accident is found. Therefore, monitoring the turning-over state of a baby in various sleep environments is an important measure to ensure the safety of the baby.

[0003] In the prior art related to the monitoring field of babies, a common method is to use an image processing algorithm to position and monitor the body of a baby. Segmentation of a human body is an important link for positioning and monitoring. The goal of image segmentation is to divide an image into two categories of a human body and a background. Image processing techniques based on image segmentation mainly include edge detection, region growing and threshold segmentation. Edge detection detects edge information in an image by using a Canny operator and a Sobel operator. The edge is the boundary line of a target or a scene. However, edge detection obtains an image edge by calculating a gradient, and only relies on neighboring pixels. This locality may cause some edges to be blurred or not completely detected, and the obtained edge is discontinuous. Region growing starts from a seed point, and gradually adds pixels similar to the seed point to the same region according to certain growth criteria. According to the feature difference of the region, the image is divided into several regions with independent features. However, in the presence of strong noise or uneven illumination, region growing is prone to overgrowth or breakage, resulting in low image segmentation accuracy. In the baby monitoring scene, the temperature of a human body is quite different from the temperature of the environment. Therefore, compared with the above two segmentation methods, threshold segmentation is more appropriate. This method divides the baby target and the background in an image by setting a static threshold. However, the temperature distribution of a human body is complex, and only relying on a static threshold cannot fully meet the detection needs of baby turning-over.

[0004] Therefore, how to accurately identify the body trunk of a baby according to the temperature characteristics of the baby, effectively monitor the turning-over of the baby, provide technical support for the safety monitoring of the baby, and realize intelligent care and reminders is an urgent problem to be solved. SUMMARY

[0005] Therefore, the embodiments of the present application provide a baby turning-over detection method and device based on a thermal infrared image and a robot, to solve the problem of low accuracy in baby turning-over detection in the prior art.

[0006] In a first aspect, an embodiment of the present invention provides a method for detecting an infant turning over based on thermal infrared images, the method comprising:

[0007] Acquire thermal infrared video data and visible light video data in infant care scenarios;

[0008] Performing infant target detection on the visible light video data to determine whether an infant is present in the scene;

[0009] When it is determined that an infant is present in the scene, obtaining a thermal infrared image of the infant based on the thermal infrared video data;

[0010] Performing torso positioning on the infant thermal infrared image to obtain torso position information;

[0011] According to the trunk position information, a turn-over detection is performed on the trunk area to obtain a turn-over detection result.

[0012] Preferably, when it is determined that an infant exists in the scene, obtaining a thermal infrared image of the infant based on the thermal infrared video data includes:

[0013] Get the preset ambient temperature threshold;

[0014] Decomposing the thermal infrared video data to obtain a thermal infrared video image;

[0015] performing binary segmentation on the thermal infrared video image according to the ambient temperature threshold to obtain a foreground area in the thermal infrared video image;

[0016] The thermal infrared image of the infant is determined according to the foreground area.

[0017] Preferably, performing torso positioning on the infant thermal infrared image to obtain torso position information includes:

[0018] According to the ambient temperature threshold, obtaining an initial temperature value, a target temperature value and a preset temperature increase step value;

[0019] Acquire multiple target temperature thresholds according to the initial temperature value, the target temperature value, and the temperature increase step value, wherein the target temperature threshold is less than or equal to the target temperature value;

[0020] performing binary segmentation on the thermal infrared image of the infant according to each target temperature threshold to obtain area ratios between a first foreground region having a largest area and a second foreground region having a second largest area;

[0021] comparing the area ratios, and taking the first foreground region corresponding to the minimum area ratio as the infant torso region;

[0022] According to the infant torso region, the torso position information is determined.

[0023] Preferably, the binarization segmentation of the infant thermal infrared image according to each target temperature threshold value includes:

[0024] According to each target temperature threshold value, the binarization segmentation of the infant thermal infrared image obtains a plurality of foreground region contour position information in the infant thermal infrared image.

[0025] According to each foreground region contour position information, the area of each foreground region segmented by each binarization is calculated and sorted, to obtain a plurality of first foreground regions with maximum area and a plurality of second foreground regions with sub-maximum area.

[0026] According to each first foreground region and each second foreground region, each area ratio is calculated.

[0027] Preferably, the comparison of each area ratio includes:

[0028] According to the first foreground region and the second foreground region corresponding to the minimum ratio, first pixel point position information in the first foreground region and second pixel point position information in the second foreground region are obtained.

[0029] According to the first pixel point position information and the second pixel point position information, the number of adjacent pixel points between the first foreground region and the second foreground region is calculated, to obtain a target number of adjacent pixel points.

[0030] The target number of adjacent pixel points is compared with a preset pixel point number threshold value, and when the target number of adjacent pixel points is greater than the pixel point number threshold value, the first foreground region is taken as the infant torso region.

[0031] Preferably, the turning-over detection of the torso region according to the torso position information includes:

[0032] The thermal infrared video data is decomposed to obtain a plurality of thermal infrared video images.

[0033] According to each torso position information corresponding to each thermal infrared video image, the aspect ratio of each torso region contour is calculated to obtain an aspect ratio sequence.

[0034] Each aspect ratio in the aspect ratio sequence is compared and calculated to obtain a maximum aspect ratio, a minimum aspect ratio, and an aspect ratio mean.

[0035] a first image frame number difference is calculated according to the first image frame number and the second image frame number;

[0036] a third image frame number is determined according to the first image frame number and the average aspect ratio;

[0037] when the aspect ratio difference between the maximum aspect ratio and the minimum aspect ratio is greater than the minimum aspect ratio, a turning-over detection result is obtained by detecting whether the infant turns over according to the first image frame number and the third image frame number in combination with the first image frame number difference.

[0038] Preferably, when the aspect ratio difference between the maximum aspect ratio and the minimum aspect ratio is greater than the minimum aspect ratio, the turning-over detection result is obtained by detecting whether the infant turns over according to the first image frame number and the third image frame number in combination with the image frame number difference, including:

[0039] a second image frame number difference is calculated according to the first image frame number and the third image frame number;

[0040] a comparison result is obtained by comparing the second image frame number difference with the first image frame number difference;

[0041] according to the comparison result, if the second image frame number difference is less than the first image frame number difference, the turning-over detection result is that the infant turns over;

[0042] if the second image frame number difference is greater than or equal to the first image frame number difference, the turning-over detection result is that the infant does not turn over.

[0043] In a second aspect, an embodiment of the present application provides an infant turning-over detection device based on thermal infrared images, which comprises:

[0044] a video data acquisition module, configured to acquire thermal infrared video data and visible light video data in an infant care scene;

[0045] a target detection module, configured to perform infant target detection on the visible light video data to determine whether there is an infant in the scene;

[0046] a thermal infrared image acquisition module, configured to acquire an infant thermal infrared image according to the thermal infrared video data when it is determined that there is an infant in the scene;

[0047] a torso positioning module, configured to perform torso positioning on the infant thermal infrared image to obtain torso position information;

[0048] The turning-over detection module is configured to perform turning-over detection on the torso region according to the torso position information, and obtain a turning-over detection result.

[0049] In a third aspect, an embodiment of the present application provides an infant care robot, comprising at least one processor, at least one memory, and computer program instructions stored in the memory, wherein when the computer program instructions are executed by the processor, the method of the first aspect in the above-mentioned embodiments is implemented.

[0050] In a fourth aspect, an embodiment of the present application provides a storage medium having computer program instructions stored thereon, wherein when the computer program instructions are executed by a processor, the method of the first aspect in the above-mentioned embodiments is implemented.

[0051] In summary, the beneficial effects of the present application are as follows:

[0052] The embodiment of the present application provides a kind of based on thermal infrared image baby turning over detection method, device and robot, the method comprises: obtaining the thermal infrared video data and visible light video data in baby care scene;The visible light video data is carried out baby target detection, judge whether there is baby in scene;When judging that there is baby in scene, according to the thermal infrared video data, obtain baby thermal infrared image;The trunk location information is obtained by carrying out trunk positioning to the baby thermal infrared image;According to the trunk location information, turning over detection is carried out to trunk area, and turning over detection result is obtained.The present application first obtains the thermal infrared and visible light video data of baby, visible light data is used to assist to determine measurement distance and angle, then infrared temperature data is compensated, the accuracy of temperature measurement is improved.And visible light imaging and thermal infrared imaging two methods are non-contact measurement, avoid the direct contact of sensor and baby skin, reduce the risk of infection.According to the thermal infrared and visible light video data and preset network model, thermal infrared image data is obtained, and the thermal infrared image data includes the body area of the baby, the preset network model is trained and optimized, can adapt to image data processing under different environmental conditions, including illumination variation, background complexity and baby turning over and other factors, enhance the robustness and reliability of monitoring system in practical application;According to the thermal infrared image data, the trunk positioning information of the baby is obtained using dynamic binary segmentation, baby may have turning over action in sleep monitoring process, dynamic binary segmentation can better handle this dynamic changing image situation, can quickly respond and adjust threshold, to keep the accurate segmentation of trunk area, to ensure real-time and reliability;According to the trunk positioning information, turning over detection result is obtained using dynamic analysis, dynamic analysis can feed back turning over monitoring result to the guardian in real time, so as to take necessary measures in time, this kind of timely feedback can greatly improve the response speed and efficiency of monitoring system, guarantee the sleep safety of baby, can effectively monitor the sleep state of baby, improve the safety and comfort of baby care, also provide immediate turning over alarm reminder for guardian. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments of the present application will be briefly introduced as follows, and for those skilled in the art, other drawings can also be obtained without creative labor on the premise of these drawings, which are within the protection scope of the present application.

[0054] Figure 1 It is the whole flow schematic diagram of the baby turning over detection method based on thermal infrared image in the embodiment 1 of the present application;

[0055] Figure 2is a flowchart of a turning-over detection process of a torso region according to torso position information in Embodiment 1 of the present application.

[0056] Figure 3 is a structural diagram of an infant turning-over detection device based on thermal infrared images in Embodiment 2 of the present application.

[0057] Figure 4 is a structural diagram of an infant care robot in Embodiment 3 of the present application. DETAILED DESCRIPTION

[0058] The features and exemplary embodiments of various aspects of the present application will be described below in detail, in order to make the purposes, technical solutions and advantages of the present application more clear and apparent, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only configured to explain the present application, and are not configured to limit the present application. The present application can be implemented without some of these specific details for those skilled in the art. The following description of the embodiments is only to provide a better understanding of the present application by showing examples of the present application.

[0059] It should be noted that, in this paper, relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the elements defined by the statement "include" do not exclude the presence of other identical elements in the process, method, article or equipment including the elements.

[0060] It should be noted that all the actions of obtaining signals, information or data in the present application are carried out in accordance with the corresponding data protection regulations and policies of the place, and with the authorization given by the owner of the corresponding device.

[0061] Embodiment 1

[0062] Please see Figure 1 The embodiment of the present application provides an infant turning-over detection method based on thermal infrared images, which comprises the following steps:

[0063] Obtaining thermal infrared video data and visible light video data in an infant care scene;

[0064] Specifically, in the baby care scene, simultaneously collecting thermal infrared video data and visible light video data is the basic link to realize multi-modal behavior recognition. The thermal infrared video data is an image sequence obtained by an infrared thermal imaging module, which can reflect the distribution of the surface thermal radiation of an object and is suitable for non-contact monitoring at night or in low light conditions. The visible light video data is an image sequence collected by a conventional camera, which has higher resolution and structural detail information and can provide visual features for subsequent target detection and auxiliary analysis. The dual-mode video stream enhances the robustness and adaptability of baby turning detection. In the actual care environment, babies may be in different postures, lighting environments, or partially occluded. Relying only on thermal infrared images may result in a lack of structural information. Relying only on visible light images is difficult to work effectively at night. Therefore, by simultaneously acquiring two video streams, the information is complementary and enhanced, laying the foundation for subsequent positioning, recognition, and dynamic analysis. In the process of realizing this step, an integrated camera assembly, including a thermal infrared module and a visible light camera, needs to be pre-installed above the baby activity area. The assembly should be fixed at an appropriate height from the baby bed, and the two sensors should be vertically aligned in the field of view as much as possible to ensure that the images collected by the two modalities have high spatial consistency. The thermal infrared module can use a low-resolution, small-volume device (such as a resolution of 82x60) to reduce power consumption and hardware cost. The visible light camera needs to have high-definition video output capability to support the operation of the target detection algorithm. By combining infrared imaging and visible light imaging, a non-disturbing data acquisition method that can be used day and night is realized, and the accuracy of subsequent image registration and analysis is improved through integrated and aligned installation of the camera, which improves the adaptability of the system to lighting changes and provides a stable data source for subsequent image fusion, torso recognition, and turning dynamic analysis.

[0065] Perform baby target detection on the visible light video data to determine whether a baby exists in the scene.

[0066] Specifically, by performing infant target detection on the visible light video stream, it is determined whether an infant exists in the current image and the specific location area of the infant is determined. The infant target detection refers to using a deep learning model (such as a target detection framework such as YOLOv8s) to identify the bounding box of the infant in the visible light image (usually represented as Baby(x, y, w, h), that is, the center coordinates and width and height of the infant target); the detection result is not only used to confirm whether the infant is present, but also is an important reference for subsequent association of the trunk area positioning in the thermal infrared image. When there is no infant in the scene, subsequent high-overhead image processing and dynamic analysis algorithms do not need to be performed, thereby saving system resources; when there are multiple high-temperature targets (such as adults, pets, etc.), the infant area can be locked by means of target detection in the visible light image, thereby avoiding misjudgment due to unclear target features in the thermal infrared image; in the implementation process, a pre-trained lightweight target detection model such as YOLOv8s is used, which can quickly output the detection box information after image input; in order to improve the detection accuracy, a model weight fine-tuned on an infant daily scene dataset can be selected; at the same time, in order to ensure the consistency between the detection result and the infrared image, the position coordinates of the detection box are registered or projected and mapped to adapt to the size and resolution of the thermal infrared image; the registration process can use a geometric transformation algorithm based on perspective calibration or an image pyramid matching strategy. Real-time infant target recognition is realized by using a high-precision target detection network, and the position information output by the model inference provides auxiliary guidance for trunk area locking and dynamic analysis of the infrared image.

[0067] When it is determined that an infant exists in the scene, an infant thermal infrared image is acquired according to the thermal infrared video data;

[0068] Specifically, after it is confirmed that an infant exists in the current scene, a thermal infrared image processing process is enabled to further extract the target area of the infant in the thermal infrared image; the infant thermal infrared image refers to an image area containing the heat distribution of the infant's body extracted from the thermal infrared video image, which mainly focuses on the relatively high-temperature part released by the infant's skin or trunk; compared with the environment background, these areas usually show higher gray value or pseudo-color brightness in thermal infrared imaging, and therefore can be separated from the overall image by using an image segmentation method. Under the premise that the infant is known to exist, the foreground area corresponding to the infant heat source is quickly and accurately extracted from the continuous thermal infrared image, thereby providing input for subsequent deep analysis such as trunk recognition and turning over judgment.

[0069] Preferably, when it is determined that an infant exists in the scene, the infant thermal infrared image is acquired according to the thermal infrared video data, and the method comprises the following steps:

[0070] A preset environmental temperature threshold is acquired;

[0071] Specifically, a preset ambient temperature threshold is first obtained, which is a temperature reference for distinguishing the background from the infant body temperature. In the infant care scene, the common indoor temperature is between 22-26℃, while the infant body surface temperature is generally higher than 30℃, therefore, for example, 26℃ is usually selected as the ambient temperature threshold. The ambient temperature threshold can be adjusted or adaptively updated according to the actual room environment.

[0072] The thermal infrared video data is decomposed to obtain a thermal infrared video image;

[0073] According to the ambient temperature threshold, the thermal infrared video image is binarized and segmented to obtain a foreground region in the thermal infrared video image;

[0074] Specifically, the thermal infrared image is binarized and segmented according to the ambient temperature threshold. Binarization refers to dividing the image pixel values according to the preset ambient temperature threshold: the region with a temperature higher than the ambient temperature threshold is marked as foreground (usually set to 1 or 255), and the region with a temperature lower than the ambient temperature threshold is marked as background (set to 0).

[0075] According to the foreground region, the infant thermal infrared image is determined.

[0076] Specifically, according to the foreground region, the connected domains in the foreground region image are further analyzed, such as calculating the maximum thermal area, center point position, shape ratio, etc., to identify the target region most likely representing the infant. Combined with the detection box coordinates of the infant in the visible light image in the previous stage, the matching accuracy can be further enhanced. If the foreground region has a spatial overlap or proximity relationship with the detection box region in the visible light, the foreground region is confirmed as the thermal infrared image of the infant.

[0077] The infant thermal infrared image is located to obtain the torso position information;

[0078] Specifically, after successfully extracting the infant thermal infrared image, the torso region in the infant thermal image is further identified, that is, the thermal area representing the core body parts such as the chest, abdomen or back of the infant is separated from the high-temperature foreground, which is called torso position information. This information is a key input for subsequent roll-over detection, because the roll-over action mainly manifests as a significant change in the torso contour, and only by accurately locking the torso can stable dynamic analysis be carried out. Since the infant's face temperature is usually higher than the torso, and the torso is higher than the background, a single threshold segmentation cannot accurately distinguish different regions of the body. In order to obtain a more stable and clear structure of the torso region, a dynamic multi-threshold segmentation combined with area ratio analysis method is introduced in the torso positioning, taking the change trend of the image structure as the basis for segmentation judgment, which avoids the defects of the traditional fixed threshold scheme that is easily disturbed by posture and environmental fluctuations.

[0079] Preferably, the trunk positioning of the infant thermal infrared image to obtain the trunk position information comprises:

[0080] According to the ambient temperature threshold, an initial temperature value, a target temperature value and a preset temperature increasing step value are obtained;

[0081] Specifically, first, according to the ambient temperature threshold, an initial temperature value, a target temperature value and a preset temperature increasing step value are obtained; the initial temperature value is generally set to a low temperature value slightly higher than the ambient temperature, for example, 26℃, for roughly screening out the background; the target temperature value is close to the upper limit of human body temperature (such as 35℃), indicating the maximum threshold upper limit allowed for separating the complete trunk; and the temperature increasing step value (such as 0.1℃) controls the granularity of the threshold increase, for gradually approaching the optimal segmentation effect. This setting ensures that the temperature segmentation process has sufficient dynamic resolution.

[0082] According to the initial temperature value, the target temperature value and the temperature increasing step value, a plurality of target temperature thresholds are obtained, wherein the target temperature threshold is less than or equal to the target temperature value;

[0083] Specifically, according to the initial temperature value, the target temperature value and the temperature increasing step value, the initial temperature value is increased by the temperature increasing step value successively until the temperature increasing reaches the target temperature, thereby obtaining a plurality of target temperature thresholds, forming an ordered temperature threshold sequence, such as: [26.0℃, 26.1℃, 26.2℃, …, 35.0℃]. This temperature sequence is used to control the conditions of multiple binarization operations. Each temperature value represents the critical condition of an image segmentation operation, so that the system can systematically observe the pattern change of the image structure with the change of the threshold value, and find the optimal segmentation point.

[0084] According to each of the target temperature thresholds, the infant thermal infrared image is binarized and segmented to obtain the area ratio between the first foreground region with the maximum area and the second foreground region with the second maximum area;

[0085] Specifically, according to each of the target temperature thresholds, the infant thermal infrared image is binarized and segmented one by one, the foreground region contour position information under each threshold condition is extracted, and the area of all foreground regions in each segmentation result is calculated. By sorting these region areas, the first foreground region with the maximum area and the second foreground region with the second maximum area are determined in turn, and then the area ratio of the two is calculated to form a temperature threshold-area ratio mapping sequence. This process can effectively depict the separation trend of the thermal image structure with the change of temperature, provide a quantitative basis for subsequent determination of the optimal trunk region, and enhance the adaptability and recognition accuracy of the system under complex sleeping posture, illumination and shielding conditions.

[0086] Preferably, the area ratio between the first foreground region with the maximum area and the second foreground region with the second maximum area obtained by binarization segmentation of the infant thermal infrared image according to each of the target temperature threshold values comprises:

[0087] According to each of the target temperature threshold values, the infant thermal infrared image is binarization segmented to obtain the contour position information of a plurality of foreground regions in the infant thermal infrared image.

[0088] Specifically, according to each of the target temperature threshold values, the infant thermal infrared image is binarization segmented, and the binarization segmentation refers to dividing the pixels in the thermal infrared image into two regions of foreground and background by using a preset temperature threshold value, the foreground is a thermal zone with a temperature higher than the target temperature threshold value, and the background is a region with a temperature lower than or equal to the target temperature threshold value. The foreground region contour position information refers to the boundary shape, position coordinates and connectivity information of each recognized thermal zone (such as the head, trunk, limbs, etc.) in the image. For example, there are two high-temperature regions in the image, one of which surrounds the head of the infant, and the other surrounds the trunk. By identifying the thermal zone contours that may represent the body parts of the infant in the image from different temperature sensitivities, a spatial basis is provided for subsequent area sorting and structure analysis; since the thermal zones in the thermal infrared image may be blurred due to posture, cover or angle problems, they cannot be judged by a single threshold value, but a contour distribution under multiple threshold values should be established. To achieve this, the thermal infrared image is binarization segmented according to a sequence of target temperature threshold values (such as from 26.0℃ to 35.0℃ with a step of 0.1℃); after each operation, a contour extraction algorithm is used to extract all thermal zone contour information, record its bounding rectangle, area, centroid and other attributes; these contour data can be uniformly stored through a data structure (such as a contour array) for subsequent screening, realizing the whole process of contour construction from low-temperature fusion thermal clusters to high-temperature separated regions.

[0089] According to each of the foreground region contour position information, the areas of the foreground regions segmented by each binarization are calculated and sorted respectively to obtain a plurality of first foreground regions with the maximum area and a plurality of second foreground regions with the second maximum area.

[0090] Specifically, according to each of the foreground region contour position information, the areas of the foreground regions segmented by each binarization are calculated and sorted respectively, and the area of the foreground region is the number of pixels occupied by a certain thermal zone in the image; for example, a thermal zone contour contains 1500 pixel points, and its area is 1500. By sorting these area values, the two most prominent regions in the thermal map under different threshold values can be found, i.e. the first foreground region (with the maximum area) and the second foreground region (with the second maximum area).

[0091] The most representative body part hot area is identified by sorting the size of the hot area, especially distinguishing the torso and head of the infant. In the actual scene, the infant's head temperature is higher but the area is smaller, and the torso temperature is slightly lower but the area is larger, so the two can be accurately distinguished by area sorting.

[0092] According to each of the first foreground area and each of the second foreground area, the area ratio is calculated.

[0093] Specifically, according to each of the first foreground area and each of the second foreground area, the area ratio is calculated, where the area ratio refers to the ratio of the largest hot area to the second largest hot area, usually represented by S1 / S2; wherein S1 is the largest foreground area, and S2 is the second largest foreground area. For example, under a certain threshold, the torso area is 1800 pixels, and the head area is 600 pixels, and the area ratio is 3.0. By establishing a structure change index across temperature thresholds, it is determined under which thermal threshold condition the infant's torso region is most obvious, or the separation degree is best; when the head and torso have not been separated, they may form a large area, and the area ratio is large; when the two are clearly separated, and the torso becomes the dominant hot area, the area ratio will have a minimum value. The optimal thermal map segmentation result is selected by the structure evolution trend to avoid misidentification.

[0094] The minimum area ratio corresponds to the first foreground area as the infant's torso region.

[0095] Specifically, the area ratios calculated under all target temperature thresholds are compared one by one to identify the first foreground area and the second foreground area with the smallest area ratio. The minimum area ratio usually corresponds to the state where the head and torso are most clearly separated in the thermal structure of the image, so this is used as the basis for judgment, and the first foreground area with the largest area in this group is selected as the infant's torso region. By this structure optimization selection method, the recognition deviation caused by hot area fusion or missegmentation can be effectively avoided, and the selected region has high reliability in temperature characteristics and spatial structure, thereby providing accurate thermal map basic regions for subsequent roll analysis.

[0096] Preferably, the comparison of each of the area ratios, the first foreground area corresponding to the minimum area ratio as the infant's torso region comprises:

[0097] According to the first foreground area and the second foreground area corresponding to the minimum ratio, the first pixel point position information in the first foreground area and the second pixel point position information in the second foreground area are obtained.

[0098] Specifically, the first pixel point position information and the second pixel point position information refer to a set of spatial coordinates of pixel points respectively belonging to the first foreground region (such as the torso) and the second foreground region (such as the head), and the information is used for subsequent judgment of whether there is spatial contact or adjacent edges between the two hot regions. Although the area ratio is the smallest, it can be initially judged that the first foreground region is likely to be a torso, but if the first and second foreground regions are distributed far apart on the image, they may belong to different objects (such as the head of another person and the body of the baby). Therefore, by extracting the pixel point position information, the system can further verify whether the two have spatial adjacency to enhance the accuracy of identification.

[0099] According to the first pixel point position information and the second pixel point position information, the number of adjacent pixels between the first foreground region and the second foreground region is calculated to obtain a target number of adjacent pixels;

[0100] Specifically, the number of adjacent pixels refers to the number of pixel pairs adjacent on the image between the first foreground region and the second foreground region; for example, if there are multiple groups of pixel points on the two region boundaries of the first foreground region and the second foreground region, and the distance between the groups is only 1 pixel (such as the Euclidean distance is 1 or √2), these points form an adjacent pixel pair. This step is used to measure whether the first foreground region (suspected torso) and the second foreground region (suspected head) have spatial contact or edge continuity; because the torso and the head are usually physically connected in the baby heat map, even if they are separated by a heat threshold, they should have the characteristic of edge adjacency on the image. Therefore, the number of adjacent pixels becomes an important criterion for verifying the region attribution relationship. In implementation, the boundary pixel points of the first foreground region are traversed, and for each point, it is checked whether there is a pixel of the second foreground region in its 8-neighborhood (or 4-neighborhood); fast Boolean mask comparison, spatial KD-tree indexing, or bitmap erosion and expansion method are used to realize this contact judgment; if there are a large number of boundary point pairs between the two regions with a distance of less than 1 pixel, these point pairs are counted as adjacent pixel pairs, and the target number of adjacent pixels is accumulated.

[0101] The target number of adjacent pixels is compared with a preset pixel point number threshold, and when the target number of adjacent pixels is greater than the pixel point number threshold, the first foreground region is taken as a baby torso region.

[0102] Specifically, the pixel point quantity threshold is a preset integer value for determining whether two hot regions are closely connected; for example, if the threshold is set to 100, it is considered that the two regions are in contact if the number of adjacent pixel points exceeds 100. The infant trunk region is the final confirmed target hot region, which is the main body for subsequent roll-over analysis. Based on the strength of the adjacency between the hot regions, it is determined whether the first foreground region segmented optimally can be used as the effective trunk region of the infant. By comparing the actual number of adjacent pixel points with the empirical threshold, it is determined whether the current region segmentation result has a reasonable structure, further improving the accuracy and reliability of the automatic recognition of the system. In the implementation process, first, the number of adjacent pixel points calculated in the previous step is compared with the set threshold. If the actual number of adjacent points is greater than the threshold, it is considered that the first foreground region and the second foreground region have a real structural contact relationship, that is, the head-trunk connection state is established, and thus the first region is marked as the trunk region of the infant. If the condition is not met, the threshold sequence can be re-traced or the current frame can be marked as an uncertain state. This step introduces a structural contact strength threshold judgment mechanism, which combines hot map structure judgment and rule decision logic, enhances the recognition accuracy of the system for the real connected region, and significantly reduces the misjudgment rate caused by the false connection of the head with other irrelevant heat sources.

[0103] According to the infant trunk region, the trunk position information is determined.

[0104] According to the trunk position information, roll-over detection is performed on the trunk region to obtain a roll-over detection result.

[0105] Preferably, referring to Figure 2 , the roll-over detection on the trunk region according to the trunk position information to obtain a roll-over detection result comprises:

[0106] The thermal infrared video data is decomposed to obtain a plurality of thermal infrared video images;

[0107] Specifically, the thermal infrared video data refers to a sequence of images continuously collected by a thermal infrared module, usually stored in a video stream format (such as MJPEG or MP4), and a thermal infrared video image frame refers to each static image in the video stream, for example, at a sampling frequency of 10 frames per second (FPS), a video of 1 second contains 10 image frames. Converting continuous thermal infrared video into a sequence of frames provides raw image data for subsequent frame-by-frame trunk shape analysis and dynamic behavior detection. Through frame-by-frame processing, changes in the trunk contour in the time dimension can be observed, so that rapid posture switching behaviors such as roll-over actions can be captured. In actual operation, a video analysis function is used to extract thermal infrared video images frame by frame. Each frame of image is stored in a buffer or list and is attached with a timestamp or frame number as a unique identifier for subsequent dynamic information synchronization processing.

[0108] According to each trunk position information corresponding to each thermal infrared video image, a length-width ratio value of each trunk region contour is calculated to obtain a length-width ratio value sequence;

[0109] Specifically, the length-width ratio value refers to a ratio value of length and width of a rectangle circumscribed to a contour boundary of the trunk region in each frame of image. For example, the trunk of the baby in the lying state is horizontally expanded, and the length-width ratio is low, while the trunk is narrowed and the length-width ratio is high when the baby turns over. By converting the morphological change of the trunk into a quantifiable numerical sequence, a judgment basis is provided for posture recognition and turning-over determination. By calculating the length-width ratio of the trunk of each frame, a length-width ratio sequence changing with time can be formed, which is used to detect the geometric change trend of the key point. In the implementation process, the located trunk region in each frame of image is read, the bounding box thereof is extracted, and the length-width ratio thereof is calculated. After the calculation is completed, the ratio value is taken as the length-width ratio value of the current frame, and is added to the length-width ratio value sequence list.

[0110] The length-width ratio values in the length-width ratio value sequence are compared and calculated to obtain a maximum length-width ratio value, a minimum length-width ratio value, and a length-width ratio mean value;

[0111] Specifically, the length-width ratio values in the length-width ratio value sequence are compared and calculated to obtain a maximum length-width ratio value, a minimum length-width ratio value, and a length-width ratio mean value. The maximum length-width ratio value represents the most vertical or narrow state of the trunk contour of the baby in a period of time, the minimum length-width ratio value represents the most flat or expanded state of the trunk contour of the baby, and the length-width ratio mean value is the average level of the length-width ratio in the period of time. These values collectively reflect the fluctuation range of the body posture of the baby in the time dimension. By identifying the extreme value moment in the posture change process, and by estimating the regular stable state of the body posture through the average level, the turning-over behavior is usually accompanied by a sharp fluctuation of the length-width ratio, and therefore the difference between the maximum value and the minimum value (especially the multiple relationship thereof) and the deviation from the average value can be used as important features of the turning-over determination criterion. In the implementation, the length-width ratio value sequence is traversed, and the maximum value and the frame number thereof, the minimum value and the frame number thereof, and the total average value are recorded, respectively. The calculation method can adopt a standard array operation or a statistical function.

[0112] According to the first image frame number corresponding to the maximum length-width ratio value and the second image frame number corresponding to the minimum length-width ratio value, a first image frame number difference value is calculated;

[0113] Specifically, the first image frame number difference refers to the frame number difference between the frame number when the maximum aspect ratio appears and the frame number when the minimum aspect ratio appears. For example, if the maximum value appears at the 25th frame and the minimum value appears at the 18th frame, the difference is 7 frames. The frame difference is used to measure the time span of the sudden change of the trunk state. Because the turning-over behavior has the characteristic of short-time mutation, if the time difference between the maximum and minimum aspect ratios is small, that is, the frame difference is low, it can be inferred that it is a short-time sudden change and has a high possibility of turning over. In terms of technical implementation, the frame number difference is calculated by directly subtracting the frame number of the maximum value and the frame number of the minimum value. It should be noted that, in order to ensure the consistency in the time direction, the frame number with the larger value should be placed in front to form the sequence from lying to turning over.

[0114] determine a third image frame number according to the first image frame number and the average aspect ratio;

[0115] Specifically, the third image frame number refers to the frame number near the maximum aspect ratio frame and closest to the normal state (i.e., the average aspect ratio) of the trunk shape, which is used to determine whether the change trend meets the rapid mutation-regression stable mode. This mode is common in the turning-over behavior: the baby appears an extreme value shape at the moment of turning over, and then quickly returns to a relatively stable average state. The purpose of this step is to backtrack from the maximum structural fluctuation point to find the nearest image frame that returns to the normal state, and to construct a dynamic path judgment standard from the extreme value to the stable state. This path can be used as a key behavior sequence in the turning-over process to further verify whether the turning-over has occurred completely. In terms of technical implementation, the maximum aspect ratio frame number is taken as the reference point, and a number of frames are searched forward to find the frame with the aspect ratio value closest to the average value. The search process can use the window scanning method to calculate the difference between the aspect ratio of each candidate frame and the average value, and select the one with the smallest difference as the third image frame number. The window size (such as ±10 frames) is usually set to limit the calculation overhead. By combining the extreme value frame and the average value frame to construct a time clue of shape change, the description ability of the complete turning-over process is improved, and the interference behavior of incomplete change or abnormal posture is effectively identified, providing structural closed-loop support for the subsequent logic judgment of the system whether the baby has turned over.

[0116] When the aspect ratio difference between the maximum aspect ratio value and the minimum aspect ratio value is greater than the minimum aspect ratio value, according to the first image frame number and the third image frame number, and combining the first image frame number difference, the detection of whether the baby has turned over is performed to obtain the turning-over detection result.

[0117] Specifically, first, it is judged whether the morphological change amplitude of the infant's trunk contour in the thermal infrared image sequence is significant, that is, when the difference between the maximum aspect ratio value and the minimum aspect ratio value is greater than the minimum value itself, it indicates that the trunk morphology has changed dramatically, which has the potential to turn over. On this basis, the system combines the first image frame number corresponding to the maximum aspect ratio with the third image frame number of the morphological regression to the mean, and refers to the frame number difference between the maximum and minimum aspect ratio (that is, the first image frame number difference), and further analyzes the timing characteristics of the morphological change. If the change amplitude is large and the time span conforms to the rapid occurrence rule of turning over, the system finally judges that turning over occurs, and outputs the turning over detection result. This method combines structural mutation amplitude and dynamic time constraint, and improves the accuracy and robustness of turning over detection.

[0118] Preferably, when the aspect ratio difference between the maximum aspect ratio value and the minimum aspect ratio value is greater than the minimum aspect ratio value, the first image frame number and the third image frame number are combined with the image frame number difference to detect whether the infant turns over, and obtain the turning over detection result, including:

[0119] According to the first image frame number and the third image frame number, a second image frame number difference is calculated;

[0120] Specifically, the first image frame number refers to the frame number corresponding to the maximum aspect ratio value, reflecting the time when the trunk morphology reaches the most extreme state; the third image frame number is the frame number corresponding to the state when it recovers to the average morphology. The difference between the two is the second image frame number difference, which represents the number of frames the baby experiences from the occurrence of the dramatic turning over action to the recovery of the normal posture, and further calculates the approximate time length, which quantifies the time span of the action convergence to judge whether the turning over behavior is completed quickly. Because a complete turning over behavior usually occurs quickly and is accompanied by a morphological mutation-recovery process, if the frame difference between the two states is small, it is more likely to be turning over rather than slow twisting or posture adjustment. In actual implementation, the third image frame number is subtracted from the first image frame number, and the absolute value is the second image frame number difference. The image morphological change path is converted into a time index, and the recovery trend is introduced as a discrimination element of the turning over behavior.

[0121] The second image frame number difference is compared with the first image frame number difference to obtain a comparison result;

[0122] Specifically, by comparing the first image frame difference value (i.e. the frame difference between the maximum and minimum aspect ratio) with the second image frame difference value (i.e. the frame difference for the morphology to recover from the extreme value to the average value), it is identified whether the morphology change has the timing characteristics of short mutation + rapid stabilization. Compared with each other, it can reveal whether the action mutation is fast enough and whether it has a behavior structure completed in a short period. If the first frame difference value is larger and the second frame difference value is smaller, it means that the baby quickly reaches the posture extreme value and then quickly recovers to the normal posture, which meets the typical timing characteristics of the turning-over behavior; if the two frame difference values are similar or the second frame difference value is larger, it may be only a slow twist or a natural stretching process. Numerical comparison is made between the two frame difference values, and a Boolean result or a difference value is output as the input for subsequent judgment. For example: if ΔT2<ΔT1, it is recorded as "rapid stabilization"; otherwise, it is marked as "slow change" or "no turning-over". This step is a double timing comparison judgment mechanism, which combines the structure mutation time and the recovery time in two dimensions to avoid misjudgment of the behavior nature due to a single frame difference.

[0123] According to the comparison result, if the second image frame difference value is smaller than the first image frame difference value, the turning-over detection result is that the turning-over has occurred;

[0124] If the second image frame difference value is greater than or equal to the first image frame difference value, the turning-over detection result is that the turning-over has not occurred.

[0125] Specifically, if the second image frame difference value is smaller than the first image frame difference value, i.e. the baby quickly completes the process from the normal body posture to the extreme value and then recovers in a short time, "turning-over has occurred" is output; otherwise, it is considered as no turning-over or non-significant behavior. Based on the time characteristics, combined with the previous structure fluctuation data, an automatic action classification standard is established, so as to avoid misjudgment of non-key behaviors such as small twists and turns, hand and foot swings, etc. Such rules constitute the ultimate judgment condition of the turning-over behavior recognition model. The Boolean comparison result is directly combined with the set threshold logic to update the behavior state of the current frame segment. If the turning-over is detected, the system can further trigger the alarm mechanism, data recording or behavior labeling function, etc. The threshold decision method based on the timing dynamic characteristics has better behavior recognition ability than the static structure judgment, greatly improves the accuracy and real-time performance of the turning-over detection, and adapts to the behavior differences of different individuals and different sleeping postures.

[0126] Embodiment 2

[0127] Please refer to Figure 3 The embodiment of the present application provides a baby turning-over detection device based on thermal infrared images, which comprises:

[0128] A video data acquisition module is configured to acquire thermal infrared video data and visible light video data in a baby care scene.

[0129] The target detection module is configured to perform infant target detection on the visible light video data to determine whether an infant exists in a scene.

[0130] The thermal infrared image acquisition module is configured to acquire an infant thermal infrared image according to the thermal infrared video data when it is determined that the infant exists in the scene.

[0131] The torso positioning module is configured to perform torso positioning on the infant thermal infrared image to obtain torso position information.

[0132] The turning-over detection module is configured to perform turning-over detection on a torso region according to the torso position information to obtain a turning-over detection result.

[0133] Specifically, the device comprises: a video data acquisition module, configured to acquire thermal infrared video data and visible light video data in a baby care scene; a target detection module, configured to perform baby target detection on the visible light video data to determine whether there is a baby in the scene; a thermal infrared image acquisition module, configured to acquire baby thermal infrared images according to the thermal infrared video data when it is determined that there is a baby in the scene; a torso positioning module, configured to perform torso positioning on the baby thermal infrared images to obtain torso position information; and a turning detection module, configured to perform turning detection on a torso region according to the torso position information to obtain a turning detection result. The device first acquires thermal infrared and visible light video data of a baby, and the visible light data is used to assist in determining the measurement distance and angle, and then compensates the infrared temperature data, thereby improving the accuracy of temperature measurement. Both visible light imaging and thermal infrared imaging are non-contact measurement methods, which avoids direct contact between the sensor and the baby's skin and reduces the risk of infection. According to the thermal infrared and visible light video data and a preset network model, thermal infrared image data is acquired, the thermal infrared image data includes the body region of the baby, and the preset network model is trained and optimized to adapt to image data processing under different environmental conditions, including light changes, background complexity, and baby turning, thereby enhancing the robustness and reliability of the monitoring system in actual application. According to the thermal infrared image data, torso positioning information of the baby is acquired using dynamic binary segmentation. The baby may have a turning action during sleep monitoring, and dynamic binary segmentation can better handle such dynamically changing image conditions, quickly respond and adjust the threshold to maintain accurate segmentation of the torso region, thereby ensuring real-time performance and reliability. According to the torso positioning information, a turning detection result is acquired using dynamic analysis. Dynamic analysis can provide real-time feedback of the turning monitoring result to the caregiver, so that necessary measures can be taken in time. This timely feedback can greatly improve the response speed and efficiency of the monitoring system, ensuring the sleep safety of the baby, and effectively monitoring the sleep state of the baby, thereby improving the safety and comfort of baby care and providing immediate turning alarm reminders for the caregiver.

[0134] It should be noted that the modules and units in the device correspond one by one to the steps in the method, and therefore the specific embodiments can refer to the embodiments of the method.

[0135] Embodiment 3

[0136] In addition, the device can be combined with Figure 4The infant turning-over detection method based on thermal infrared images of the described embodiments of the present application can be implemented by an infant care robot, which is characterized by being used to implement the method as described in Embodiment 1. Figure 4 A hardware structure schematic diagram of an infant care robot provided by an embodiment of the present application is shown.

[0137] The infant care robot can include a processor and a memory storing computer program instructions.

[0138] Specifically, the processor can include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or can be configured as one or more integrated circuits implementing an embodiment of the present application.

[0139] The memory can include a mass storage for data or instructions. By way of example and not limitation, the memory can include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory can include removable or non-removable (or fixed) media. Where appropriate, the memory can be internal or external to the data processing device. In certain embodiments, the memory is a nonvolatile solid-state memory. In certain embodiments, the memory includes read-only memory (ROM). Where appropriate, this ROM can be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0140] The processor reads and executes the computer program instructions stored in the memory to implement any of the infant turning-over detection methods based on thermal infrared images in the above embodiments.

[0141] In one example, the infant care robot can further include a communication interface and a bus. Wherein, as shown in Figure 4 The processor 401, the memory 402, and the communication interface 403 are connected by the bus 410 and complete communication with each other.

[0142] The communication interface is mainly used to implement the communication between the modules, devices, units and / or equipment in the embodiments of the present application.

[0143] The bus includes hardware, software, or both that couples components of the infant care robot to each other. By way of example, and not limitation, the bus can include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand (IB) interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association local (VLB) bus, or another suitable bus or a combination of two or more of these. Where suitable, the bus can include one or more buses. Although particular buses are described and shown in the embodiments of the present application, the present application contemplates any suitable bus or interconnect.

[0144] Embodiment 4

[0145] In addition, in combination with the infant turning over detection method based on thermal infrared images in one of the above embodiments, the embodiment of the present application can provide a computer readable storage medium to implement. The computer readable storage medium has computer program instructions stored thereon; the computer program instructions are executed by the processor to implement any of the infant turning over detection methods based on thermal infrared images in the above embodiments.

[0146] In summary, the embodiment of the present application provides an infant turning over detection method, device and robot based on thermal infrared images.

[0147] It needs to be clear that the present application is not limited to the specific configurations and processes described above and shown in the drawings. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after understanding the spirit of the present application.

[0148] The functional blocks shown in the structural block diagrams described above can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, functional cards, and the like. When implemented in software, the elements of the present application are program or code segments that are used to perform the required tasks. The program or code segments can be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave over a transmission medium or communication link. A "machine-readable medium" includes any medium that can store or transport information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, and the like. The code segments can be downloaded via computer networks such as the Internet, intranets, and the like.

[0149] It should also be noted that the exemplary embodiments mentioned in the present application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in an order different from that in the embodiments, or several steps can be performed simultaneously.

[0150] The above description is merely a specific implementation of the present application. Those skilled in the art can clearly understand the specific working processes of the above-described system, modules and units for the convenience and brevity of description, which can refer to the corresponding processes in the foregoing method embodiments, which will not be described here. It should be understood that the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application.

Claims

1. A method for detecting infant turning over based on thermal infrared images, characterized in that: The method comprises: Acquire thermal infrared video data and visible light video data in infant care scenarios; Performing infant target detection on the visible light video data to determine whether an infant is present in the scene; When it is determined that an infant is present in the scene, obtaining a thermal infrared image of the infant based on the thermal infrared video data; Performing torso positioning on the infant thermal infrared image to obtain torso position information; According to the trunk position information, a turn-over detection is performed on the trunk area to obtain a turn-over detection result.

2. The method for detecting infant turning over based on thermal infrared images according to claim 1, characterized in that: When it is determined that an infant exists in the scene, obtaining a thermal infrared image of the infant based on the thermal infrared video data includes: Get the preset ambient temperature threshold; Decomposing the thermal infrared video data to obtain a thermal infrared video image; performing binary segmentation on the thermal infrared video image according to the ambient temperature threshold to obtain a foreground area in the thermal infrared video image; The thermal infrared image of the infant is determined according to the foreground area.

3. The method for detecting infant turning over based on thermal infrared images according to claim 2, characterized in that: Positioning the torso of the infant or child using the thermal infrared image to obtain torso position information includes: According to the ambient temperature threshold, obtaining an initial temperature value, a target temperature value and a preset temperature increase step value; Acquire multiple target temperature thresholds according to the initial temperature value, the target temperature value, and the temperature increase step value, wherein the target temperature threshold is less than or equal to the target temperature value; performing binary segmentation on the thermal infrared image of the infant according to each target temperature threshold to obtain area ratios between a first foreground region having a largest area and a second foreground region having a second largest area; comparing the area ratios, and taking the first foreground region corresponding to the minimum area ratio as the infant torso region; The torso position information is determined based on the infant torso region.

4. The method for detecting infant turning over based on thermal infrared images according to claim 3, characterized in that: The step of performing binary segmentation on the infant thermal infrared image according to each target temperature threshold to obtain area ratios between a first foreground region having a largest area and a second foreground region having a second largest area includes: performing binary segmentation on the infant thermal infrared image according to each target temperature threshold to obtain contour position information of a plurality of foreground regions in the infant thermal infrared image; calculating and sorting the areas of the foreground regions obtained by each binarization segmentation according to the position information of the contours of the foreground regions, to obtain a plurality of first foreground regions with the largest areas and a plurality of second foreground regions with the second largest areas; The area ratios are calculated based on the first foreground areas and the second foreground areas.

5. The method for detecting infant turning over based on thermal infrared images according to claim 3, characterized in that: The comparing the area ratios and taking the first foreground area corresponding to the minimum area ratio as the infant torso area includes: Acquire first pixel position information in the first foreground area and second pixel position information in the second foreground area according to the first foreground area and the second foreground area corresponding to the minimum ratio; Calculating the number of adjacent pixels between the first foreground area and the second foreground area based on the first pixel position information and the second pixel position information to obtain a target number of adjacent pixels; The target number of adjacent pixels is compared with a preset pixel number threshold, and when the target number of adjacent pixels is greater than the pixel number threshold, the first foreground area is used as the infant torso area.

6. A method for detecting infant turning over based on thermal infrared images according to any one of claims 1 to 5, characterized in that: The performing turn detection on the torso region according to the torso position information to obtain the turn detection result includes: Decomposing the thermal infrared video data to obtain multiple frames of thermal infrared video images; Calculating the aspect ratio of each torso region contour according to the torso position information corresponding to each thermal infrared video image to obtain an aspect ratio value sequence; Comparing and calculating each aspect ratio value in the aspect ratio value sequence to obtain a maximum aspect ratio value, a minimum aspect ratio value, and an average aspect ratio value; Calculating a difference in the number of first image frames according to the number of first image frames corresponding to the maximum aspect ratio and the number of second image frames corresponding to the minimum aspect ratio; determining a third image frame number according to the first image frame number and the average aspect ratio; When the aspect ratio difference between the maximum aspect ratio value and the minimum aspect ratio value is greater than the minimum aspect ratio value, whether the infant turns over is detected based on the first image frame number and the third image frame number, combined with the first image frame number difference, to obtain the turning over detection result.

7. The method for detecting infant turning over based on thermal infrared images according to claim 6, characterized in that: When the aspect ratio difference between the maximum aspect ratio value and the minimum aspect ratio value is greater than the minimum aspect ratio value, detecting whether the infant has turned over based on the first image frame number and the third image frame number in combination with the image frame number difference, and obtaining the turning over detection result includes: Calculating a second image frame number difference based on the first image frame number and the third image frame number; Comparing the second image frame number difference with the first image frame number difference to obtain a comparison result; According to the comparison result, if the second image frame number difference is smaller than the first image frame number difference, the turning over detection result is that turning over has occurred; If the second image frame number difference is greater than or equal to the first image frame number difference, the turn-over detection result is that no turn-over has occurred.

8. A device for detecting infant turning over based on thermal infrared images, characterized in that: The device comprises: Video data acquisition module, used to acquire thermal infrared video data and visible light video data in infant care scenarios; A target detection module, configured to perform infant target detection on the visible light video data to determine whether an infant is present in the scene; a thermal infrared image acquisition module, configured to acquire a thermal infrared image of an infant based on the thermal infrared video data when it is determined that an infant is present in the scene; a torso positioning module, configured to perform torso positioning on the infant's thermal infrared image to obtain torso position information; The turning detection module is used to perform turning detection on the torso area according to the torso position information to obtain a turning detection result.

9. An infant care robot, characterized in that: include: At least one processor, at least one memory, and computer program instructions stored in the memory, which implement the method according to any one of claims 1 to 7 when the computer program instructions are executed by the processor.

10. A storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images